Glaucoma screening method, system, device and storage medium based on AS-OCT video

Abstract

The present invention discloses a glaucoma screening method, system, device and storage medium based on AS-OCT video. The glaucoma screening method based on AS-OCT video includes: obtaining anterior segment OCT video of a human eye to be tested in a dark-light-dark changing environment, and splitting the anterior segment OCT video into multiple video frame images; aligning the multiple video frame images to obtain multiple aligned images; extracting regions of interest of the multiple aligned images to obtain multiple feature regions; calculating the multiple feature regions according to a preset multi-view spatiotemporal network model to obtain a predicted score; and determining the screening result according to the comparison between the predicted score and the preset score. The present invention obtains anterior segment OCT video of a human eye to be tested in a dark-light-dark changing environment, and detects the state of the chamber angle in the changing environment through the anterior segment OCT video to obtain the screening result of the human eye to be tested, so as to detect the dynamic human eye to be tested, so as to improve the accuracy of primary angle-closure glaucoma detection.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a glaucoma screening method, system, device and storage medium based on AS-OCT video. Background Art

[0002] Glaucoma is an eye disease with extremely complex etiology, ranking second among the four major blinding eye diseases. With the aging of the population, the number of people with glaucoma is increasing year by year. By 2040, it is estimated that 112 million people worldwide will be affected by this disease. Among them, the incidence of primary angle closure glaucoma (PACG) is relatively high. To some extent, PACG is a preventable disease. If angle treatment methods such as laser peripheral iridotomy (LPI), iridoplasty, or cataract extraction can be used in the early stage, the angle closure process can be effectively stopped. Therefore, it is very important to screen for PACG, predict it early, and treat it in time.

[0003] In the related art, screening for primary angle-closure glaucoma is mainly based on several static AS-OCT images to give classification results. However, static anatomical factors alone cannot fully explain the relatively high incidence of PACG, and the dynamic changes in the anterior chamber structure are more convincing for diagnosis. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the present invention proposes a glaucoma screening method based on AS-OCT video, which can improve the accuracy of glaucoma screening.

[0005] The present invention also proposes a glaucoma screening system based on AS-OCT video.

[0006] The invention also provides an electronic control device.

[0007] The present invention also provides a computer-readable storage medium.

[0008] In a first aspect, an embodiment of the present invention provides a glaucoma screening method based on AS-OCT video, comprising:

[0009] Acquire anterior segment OCT video of a human eye to be tested in a dark-bright-dark changing environment, and split the anterior segment OCT video into multiple video frame images;

[0010] Performing alignment processing on the plurality of video frame images to obtain a plurality of aligned images;

[0011] Extracting regions of interest from a plurality of the aligned images to obtain a plurality of feature regions;

[0012] Calculating the plurality of feature regions according to a preset multi-view spatiotemporal network model to obtain a prediction score;

[0013] The screening result is determined based on the comparison between the predicted score and the preset score.

[0014] The glaucoma screening method based on AS-OCT video in the embodiment of the present invention has at least the following beneficial effects: by acquiring the anterior segment OCT video of the human eye to be tested in a dark-light-dark changing environment, the anterior segment OCT video is used to detect the state of the chamber angle in the changing environment to obtain the screening result of the human eye to be tested, so as to detect the dynamic human eye to be tested, so as to improve the accuracy of primary angle-closure glaucoma detection.

[0015] According to some other embodiments of the present invention, the glaucoma screening method based on AS-OCT video, the alignment processing of the plurality of video frame images to obtain a plurality of aligned images includes:

[0016] Set the first video frame image as the reference video frame image;

[0017] Extracting multi-scale surface point features and corner point features of the reference video frame image to obtain reference feature points;

[0018] Extracting multi-scale surface point features and corner point features of a plurality of the video frame images to obtain feature points corresponding to the video frame images;

[0019] Determining conversion parameters of the video frame image according to the feature points, the reference feature points and a preset optimization algorithm;

[0020] The video frame images are converted according to the conversion parameters corresponding to the multiple video frame images to obtain the multiple aligned images.

[0021] According to some other embodiments of the glaucoma screening method based on AS-OCT video of the present invention, the step of extracting regions of interest of the plurality of aligned images to obtain a plurality of feature regions comprises:

[0022] locating scleral protrusion positions of the plurality of aligned images according to a preset derivative network;

[0023] determining anterior chamber angle region and iris region of the aligned image according to the scleral protrusion positions of the plurality of aligned images;

[0024] The anterior chamber angle region and the iris region of the plurality of aligned images are cropped at a preset size to obtain a plurality of the feature regions.

[0025] According to some other embodiments of the present invention, the glaucoma screening method based on AS-OCT video, the preset multi-view spatiotemporal network model includes:Figure 3 D spatial sub-network model, temporal sub-network model.

[0026] According to other embodiments of the present invention, the glaucoma screening method based on AS-OCT video includes: anterior chamber angle area and iris area, and the calculation of the plurality of the characteristic areas according to the preset multi-view spatiotemporal network model to obtain the prediction score includes:

[0027] According to the multi-view Figure 3 D. The spatial sub-network model performs angle state analysis on the plurality of anterior chamber angle regions to obtain a spatial score;

[0028] Performing temporal change calculation on the plurality of iris regions according to the temporal sub-network model to obtain a temporal score;

[0029] The prediction score is determined based on the spatial score and the temporal score.

[0030] According to some other embodiments of the glaucoma screening method based on AS-OCT video, the step of performing temporal change calculation on the plurality of iris regions according to the temporal subnetwork model to obtain a time score includes:

[0031] Inputting the plurality of iris regions into a long short-term memory layer and performing normalization processing;

[0032] Encoding the temporal state of the long short-term memory layer and adding a fully connected layer to the output layer of the long short-term memory layer;

[0033] The time score is obtained according to the output of the long short-term memory layer.

[0034] According to some other embodiments of the glaucoma screening method based on AS-OCT video of the present invention, the preset optimization algorithm includes: an iteratively weighted least squares algorithm.

[0035] In a second aspect, an embodiment of the present invention provides a glaucoma screening system based on AS-OCT video, comprising:

[0036] A video processing module, used for acquiring anterior segment OCT video of a human eye to be tested in a dark-bright-dark changing environment, and splitting the anterior segment OCT video into multiple video frame images;

[0037] An alignment module, used for performing alignment processing on the plurality of video frame images to obtain a plurality of aligned images;

[0038] An extraction module, used for extracting regions of interest of the plurality of aligned images to obtain a plurality of feature regions;

[0039] A calculation module, used for calculating the plurality of feature regions according to a preset multi-view spatiotemporal network model to obtain a prediction score;

[0040] A comparison module is used to determine the screening result based on the comparison between the predicted score and the preset score.

[0041] The glaucoma screening system based on AS-OCT video in the embodiment of the present invention has at least the following beneficial effects: by acquiring the anterior segment OCT video of the human eye to be tested in a dark-light-dark changing environment, the anterior segment OCT video is used to detect the state of the chamber angle in the changing environment to obtain the screening result of the human eye to be tested, so as to detect the dynamic human eye to be tested, so as to improve the accuracy of primary angle-closure glaucoma detection.

[0042] In a third aspect, an embodiment of the present invention provides an electronic control device, comprising:

[0043] at least one processor, and

[0044] a memory communicatively connected to the at least one processor; wherein,

[0045] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the glaucoma screening method based on AS-OCT video as described in the first aspect.

[0046] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are read and executed, the glaucoma screening method based on AS-OCT video as described in the first aspect is performed.

[0047] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained through the structures particularly pointed out in the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] FIG. 1( a ) is a video frame image of a normal human eye in an embodiment of the present invention;

[0049] FIG1( b ) is a video frame image of a normal human eye in an embodiment of the present invention;

[0050] FIG. 2( a ) is a video frame image of a human eye suffering from PACG according to an embodiment of the present invention;

[0051] FIG2( b ) is a video frame image of a human eye suffering from PACG in an embodiment of the present invention;

[0052] Figure 3 It is a flowchart of a specific embodiment of a glaucoma screening method based on AS-OCT video in an embodiment of the present invention;

[0053] Figure 4 is a flowchart of another specific embodiment of a glaucoma screening method based on AS-OCT video in an embodiment of the present invention;

[0054] FIG5( a ) is a video frame image of an original PACG human eye in an embodiment of the present invention;

[0055] FIG5( b ) is another video frame image of the original PACG human eye in an embodiment of the present invention;

[0056] FIG5( c ) is a schematic diagram of two video frames before alignment according to an embodiment of the present invention;

[0057] FIG5( d ) is a schematic diagram of two video frames aligned in an embodiment of the present invention;

[0058] Figure 6 is a flowchart of another specific embodiment of a glaucoma screening method based on AS-OCT video in an embodiment of the present invention;

[0059] Figure 7 is a flowchart of another specific embodiment of a glaucoma screening method based on AS-OCT video in an embodiment of the present invention;

[0060] Figure 8 is a flowchart of another specific embodiment of a glaucoma screening method based on AS-OCT video in an embodiment of the present invention;

[0061] Fig. 9 is a schematic diagram of the structure of the long short-term memory layer in an embodiment of the present invention;

[0062] Fig.10 It is a module block diagram of a specific embodiment of a glaucoma screening system based on AS-OCT video in an embodiment of the present invention.

[0063] Figure numerals: 100, video processing module; 200, alignment module; 300, extraction module; 400, calculation module; 500, comparison module. DETAILED DESCRIPTION

[0064] The following will be combined with the embodiments to clearly and completely describe the concept of the present invention and the technical effects produced, so as to fully understand the purpose, characteristics and effects of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, other embodiments obtained by those skilled in the art without creative work are all within the scope of protection of the present invention.

[0065] In the description of the present invention, if it involves a description of orientation, for example, the orientation or position relationship indicated by "upper", "lower", "front", "back", "left", "right", etc. is based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present invention. If a feature is referred to as being "set", "fixed", "connected", or "installed" on another feature, it can be directly set, fixed, or connected to the other feature, or it can be indirectly set, fixed, connected, or installed on the other feature.

[0066] In the description of the embodiments of the present invention, if "several" is involved, it means more than one, if "multiple" is involved, it means more than two, if "greater than", "less than" or "exceed" is involved, it should be understood as not including the number itself, if "above", "below" or "within" is involved, it should be understood as including the number itself. If "first" or "second" is involved, it should be understood as being used to distinguish technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0067] Glaucoma is an eye disease with a complex etiology and ranks second among the four major blinding eye diseases. Globally, in 2020, the number of people with glaucoma (40-80 years old) is estimated to increase to 66 million to 80 million, and 11 million of these patients will eventually become blind in both eyes. With the aging of the population, the number of people with glaucoma is increasing year by year. By 2040, it is estimated that 112 million people will be affected by this disease worldwide. Among them, the incidence of primary angle closure glaucoma (PACG) is relatively high. To some extent, PACG is a preventable disease. If angle treatment methods such as laser peripheral iridotomy (LPI), iridoplasty, or cataract extraction can be used in the early stage, the angle closure process can be effectively stopped. Therefore, it is very important to screen for PACG, predict it early, and treat it in time.

[0068] Among them, AS-OCT (Anterior Segment Optical Coherence Tomography, AS-OCT) is also called anterior segment OCT. Anterior segment OCT is a fast, efficient, non-contact in vivo imaging technology for ocular structures, and plays an important role in the evaluation of the anterior segment structure. AS-OCT uses infrared scanning during the examination process, which can reduce the impact of light on the pupil diameter and improve the accuracy of the results. In addition, since AS-OCT is a non-contact examination, it can avoid compression of the eyeball during the examination, making the measurement results more accurate and objective, and its operation is simple. AS-OCT can provide clear images of the anterior segment anatomical structure, and is used for quantitative measurement of anterior chamber structure parameters such as the chamber angle, iris, and cornea. At the same time, anterior segment OCT can monitor the eye position in real time through a camera, ensuring good repeatability. It is widely used in clinical work, especially for glaucoma examination.

[0069] In the screening of PACG, shallow anterior chamber is an important risk factor, so ophthalmologists often judge the open or closed state of ACA from AS-OCT. In recent years, some computer-assisted angle closure classification algorithms based on machine learning or convolutional neural network (CNN) have been proposed to reduce the burden on doctors. However, most of the current algorithms give classification results based on several static AS-OCT images. However, static anatomical factors alone cannot fully explain the relatively high incidence of PACG, and the dynamic changes in the anterior chamber structure are more convincing for diagnosis. Among them, referring to Figures 1(a) and 1(b), they are PACG video samples, Figure 1(a) is the 3rd frame, Figure 1(b) is the 55th frame, and the 3rd and 55th frames are the angle states in dark and bright environments, respectively, while Figures 2(a) and 2(b) are sample videos of normal people, Figure 2(a) is the 4th frame, and Figure 2(b) is the 34th frame, and the 4th and 34th frames are the angle states in dark and bright environments, respectively. Figures 1(a) and 1(b) show the video frames of the two samples when the iris state is fully extended in a bright environment. From the comparison between Figures 1 and 2, it can be seen that in a dark environment, the chamber angle states of the two samples are similar, but after illumination, the chamber angles become more open. At this time, it is difficult to distinguish these samples only by static anatomical images of the anterior chamber in bright or dark environments. The current method of classifying the chamber state based on static AS-COT images has certain limitations.

[0070] Based on this, the present application obtains the anterior segment OCT video of the eye to be tested, then processes the OCT video to obtain a predicted score, and then determines the screening result of the eye to be tested based on the predicted score and the preset score. Therefore, by dynamically analyzing the person to be tested, the research on angle closure detection based on OCT video is realized to improve the accuracy of glaucoma screening.

[0071] First, refer to Figure 3 The embodiment of the present invention discloses a glaucoma screening method based on AS-OCT video, comprising:

[0072] S100, obtaining an anterior segment OCT video of a human eye to be tested in a dark-bright-dark changing environment, and splitting the anterior segment OCT video into multiple video frame images;

[0073] S200, performing alignment processing on a plurality of video frame images to obtain a plurality of aligned images;

[0074] S300, extracting regions of interest from a plurality of aligned images to obtain a plurality of feature regions;

[0075] S400, calculating multiple feature areas according to a preset multi-view spatiotemporal network model to obtain prediction scores;

[0076] S500, determining the screening result based on the comparison between the predicted score and the preset score.

[0077] Since the iris of a healthy human eye is spongy and compressible, the iris of a PACG patient's eye is incompressible. In addition, the movement characteristics of the open and closed angles are different. The iris of the closed angle group contracts more slowly when reflecting light, and contracts faster when receiving laser iridotomy. Iris elastic acceleration and pupil block acceleration are related to PACG. Therefore, the present application obtains an anterior segment OCT video of the eye to be tested in a dark-light-dark changing environment, and detects the state of the chamber angle in the changing environment through the anterior segment OCT video to obtain the screening result of the eye to be tested. Among them, the anterior segment OCT video integrates the spatial information and temporal information of multiple views to detect the dynamic eye to be tested, so as to improve the accuracy of primary angle-closure glaucoma detection.

[0078] Reference Figure 4 In some embodiments, step S200 includes:

[0079] S210, setting the first video frame image as a reference video frame image;

[0080] S220, extracting multi-scale surface point features and corner point features of the reference video frame image to obtain reference feature points;

[0081] S230, extracting multi-scale surface point features and corner point features of multiple video frame images to obtain feature points corresponding to the video frame images;

[0082] S240, determining conversion parameters of the video frame image according to the feature points, the reference feature points and the preset optimization algorithm;

[0083] S250 , converting the video frame images according to conversion parameters corresponding to the multiple video frame images to obtain multiple aligned images.

[0084] Since the anterior segment OCT video is obtained by scanning the eyes of the person to be tested in a dark-bright-dark changing environment through the anterior segment OCT technology, the user to be tested may have involuntary eye movement during the test, or the optical axis of the eyeball may be improperly placed, or there may be misalignment between adjacent video frames in the acquired anterior segment OCT video. Therefore, it is necessary to align the video frame images in the anterior segment OCT video to remove the error in the screening result caused by video jitter. Therefore, by selecting the first video frame image at the beginning of multiple video frame images as the reference video frame image, and then aligning multiple video frame images with the reference video frame image, and the alignment method is mainly to obtain multi-scale surface point features and corner point features of the reference video frame image to obtain reference feature points, and then obtain multi-scale surface point features and corner point features of multiple video frame images to obtain feature points corresponding to multiple video frame images. Since multiple video frame images are rotated, translated, and scaled with the reference video frame image as a template, the aligned aligned images are obtained. The conversion parameters of the video frame image are determined according to the benchmark feature points, the feature points corresponding to the video frame image and the preset optimization algorithm, and then the video frame image is converted according to the conversion parameters corresponding to the video frame image to obtain an aligned image aligned with the benchmark video frame image to eliminate the jitter of the anterior segment OCT video.

[0085] Specifically, if the previous OCT video is split into N video frames, and the i-th video frame image is represented by f i (i∈[1,N]), and the first video frame image is the reference video frame image f1. In order to ensure the spatial position consistency of the anterior chamber structure in the anterior segment OCT video, we set the video frame f i (i∈[2,N]) is aligned to the video frame f1, and the converted video frame is cropped so that the aligned images have the same size. First, multi-scale surface point features and corner point features are extracted from the video frame image to obtain feature points p f 、p c , and then determine the conversion parameter according to the feature point, the reference feature point and the preset optimization algorithm, and the conversion parameter is θ, and the calculation formula of the conversion parameter is as follows:

[0086]

[0087] Where, d f (·) is the distance between opposite points, d c (·) is the Euclidean distance of paired corner points. The surface point feature matching set and the corner point feature matching set are denoted as and ωf and ω c is a distance-based weighting factor. The parameter σ c and σ f is the registration error scale (standard deviation) after normalization of the angular feature distance. ρ(·) is the Beaton-Tukey function, defined as follows:

[0088]

[0089] Following standard usage, the constant a is set to 4. Therefore, the conversion parameters required for alignment adjustment of each video frame image can be quickly and accurately calculated by formula (1).

[0090] Among them, the preset optimization algorithm includes: Iteratively-Reweighted Least-Squares (IRLS). The conversion parameters are optimized by the iteratively reweighted least squares algorithm to obtain conversion parameters that are more in line with the video frame image conversion. After aligning multiple video frame images with the reference video frame image, the video frame image needs to be adjusted first. In this embodiment, the video frame image is median filtered and the image size is adjusted to 1 / 2 of the original size, and then the video frame image is aligned to obtain an aligned image that meets the subsequent processing.

[0091] Reference Figure 5(a) to Figure 5(b) , where Figure 5(a) and Figure 5(b) are the original closed-angle video frame images, Figure 5(c) is a schematic diagram of the two video frame images before alignment, and Figure 5(d) is the aligned image after the video frame images are aligned. It can be seen that after the video frame image alignment operation, the video jitter can be removed, thereby improving the accuracy of the screening results.

[0092] Reference Figure 6 In some embodiments, step S300 includes:

[0093] S310, locating scleral protrusion positions of multiple aligned images according to a preset derivative network;

[0094] S320, determining anterior chamber angle regions and iris regions of the aligned images according to scleral protrusion positions of the multiple aligned images;

[0095] S330 , cropping the anterior chamber angle region and the iris region of the multiple aligned images according to a preset size to obtain multiple feature regions.

[0096] Since the anterior chamber angle region and the iris region are the regions of interest for ophthalmologists to diagnose PACG, it is necessary to locate the regions of interest, that is, the anterior chamber angle region and the iris region, based on the aligned image. Among them, the scleral protrusion position is the connecting part of the anterior chamber angle region and the iris region, so the scleral protrusion position is located through a preset derivative network, and then the anterior chamber angle region and the iris region in the aligned image are determined according to the scleral protrusion position, and then the anterior chamber angle region and the iris region in the aligned image are cropped out to obtain multiple feature regions, and the multiple feature regions are cropped into a fixed image size. Therefore, the anterior chamber angle region and the iris region are determined after locating the scleral protrusion position, and then the anterior chamber angle region and the iris region are cropped out from the aligned image, so as to facilitate the subsequent temporal and spatial analysis of the feature regions.

[0097] Specifically, the preset derivative network (UNet++) is an encoder-decoder structure with symmetrical cross-layer connections, which is currently the most popular medical image segmentation network. It can retain the detailed information of feature areas of different resolutions in the encoder, so that the decoder can better repair the target details. UNet++ uses cross-layer connections to enhance the ability of feature propagation based on traditional UNet. Cross-layer connections integrate features and position information of different resolutions and different intensity semantics captured by the network layer. UNet++ makes the semantic level of the encoder feature map closer to the semantic level of the feature map in the decoder through a redesigned dense cross-layer path, reducing the semantic gap between the feature maps of the encoder and the decoder. Among them, the preset derivative network needs to be trained based on multiple video frame images with scleral protrusion positions annotated, and then the trained preset derivative network is tested on a set of video data frames to determine whether the obtained scleral protrusion position is the same as the annotated scleral protrusion position, so as to continuously adjust the relevant parameters of the preset derivative network to obtain an accurate preset derivative network, and then the preset derivative network is used to process multiple aligned images to find the scleral protrusion position to obtain accurate anterior chamber angle area and iris area.

[0098] In some embodiments, the preset multi-view spatiotemporal network model includes: Figure 3 D spatial sub-network model, temporal sub-network model.

[0099] Multi-view Figure 3 The D spatial sub-network model is used to extract the spatial information of the feature area, while the temporal sub-network model is used to extract the temporal information of the feature area. The feature areas include: the anterior chamber angle area and the iris area. Since the anterior chamber angle area of ​​multiple video frame images contains the spatial information of the APCG study, the multi-view Figure 3The D-space sub-network model processes multiple anterior chamber angle regions to obtain spatial information. The dynamic motion characteristics of the iris of the human eye in a dark-bright-dark environment are obtained, and then the temporal sub-network model is used to analyze the dynamic motion characteristics of the iris region to obtain temporal information.

[0100] Reference Figure 7 In some embodiments, step S400 includes:

[0101] S410, based on multi-view Figure 3 The D-space subnetwork model analyzes the angle status of multiple anterior chamber angle regions to obtain spatial scores;

[0102] S420, calculating the temporal changes of multiple iris regions according to the temporal sub-network model to obtain a temporal score;

[0103] S430: Determine a prediction score according to the spatial score and the temporal score.

[0104] By adopting a preset multi-view space-time network model, the preset multi-view space-time network model includes a multi-view Figure 3 D spatial sub-network model and temporal sub-network model, because the screening method of PACG mainly focuses on the anterior chamber angle area and iris area of ​​the human eye, and multi-view Figure 3 The D-space subnetwork model can analyze the fluctuation characteristics of the anterior chamber angle iris in multiple anterior chamber angle regions, and can effectively assist in distinguishing closed angles. Figure 3 The D spatial sub-network model performs angle state analysis based on multiple anterior chamber angle regions to obtain the spatial score of the angle state analysis, and then performs feature extraction on the iris region and analyzes the iris movement based on the temporal sub-network model to obtain the temporal score of the iris changing over time. Therefore, the predicted score of the eye to be tested is determined based on the spatial score and the temporal score. Then, whether the person to be tested suffers from PACG can be determined based on the predicted score and the preset score, thereby improving the accuracy of PACG screening.

[0105] Reference Figure 8 In some embodiments, step S420 includes:

[0106] S421, inputting multiple iris regions into the long short-term memory layer and performing normalization processing;

[0107] S422, encoding the temporal state of the long short-term memory layer, and adding a fully connected layer to the output layer of the long short-term memory layer;

[0108] S423. Obtain a time score according to the output of the long short-term memory layer.

[0109] The iris region input value is passed to the long short-term memory layer, and then the temporal state of the long short-term memory layer is encoded to add a fully connected layer. The time score is obtained according to the output of the long short-term memory layer. The iris temporal change information of the entire anterior segment OCT video is represented by the time score, making the time score calculation operation simple and accurate.

[0110] Among them, for the temporal sub-network model, the dynamic motion characteristics of the iris in a dark-bright-dark environment are mainly used to help classify the state of the chamber angle. The temporal sub-network model includes a feature encoding module, and the feature encoding module uses ResNet50 as a feature extraction network to extract the features of the iris area. Specifically, ResNet has a shortcut mechanism to prevent gradient disappearance and accelerate network convergence. The temporal sub-network proposed in this embodiment retains the first four feature extraction blocks of ResNet50 and removes the average pooling layer and the fully connected layer. Then, the extracted features are input to the long short-term memory (LSTM) layer and batch normalized. The temporal state is encoded through this layer, and the long-term dependency of the feature graph along the time axis is modeled. Finally, a fully connected layer is added to the LSTM output layer for binary classification. The LSTM structure mainly includes an input gate, an output gate, and a forget gate. Among them, the calculation of the LSTM layer can be expressed as follows:

[0111]

[0112] The specific structure of the LSTM layer is referenced Fig. 9 , where h t ,C t is the hidden layer vector, Updates the value for the cell state. x t is the input vector, b f ,b i ,b c ,b o is the bias vector, W f ,W i ,W c ,W o is the parameter matrix, σ,tanh is the activation function. Therefore, by inputting the features of the extracted iris area into the long short-term memory layer, and then obtaining the iris temporal change information of the entire front OCT video through the long short-term memory layer output gate to obtain the time score, the calculation operation of the time score is simple and accurate.

[0113] Among them, multiple anterior chamber angle regions are combined into a volume data of size H*Q*T, where H is the image height, W is the image height, T is the number of video frames, and the volume data provides contextual information of the anterior chamber angle in the time dimension T. Therefore, in the process of volume data analysis, the volume data is rotated 90 degrees along the H axis and referred to Fig.10It can be seen that the image information in the W dimension shows the fluctuation characteristics of the anterior chamber angle iris, so the closed angle can be effectively distinguished based on the fluctuation characteristics of the anterior chamber angle iris. Figure 3 The D-space subnetwork model scores the volume data. In this embodiment, there are 16 video frames, and the anterior chamber angle area of ​​16 frames is input into the multi-view Figure 3 In the D spatial subnetwork model, the fluctuation characteristics of the anterior chamber angle iris are analyzed through the anterior chamber angle area of ​​16 frames to obtain the spatial score, and then the predicted score is determined according to the obtained spatial score and time score, so as to effectively and accurately determine the screening result according to the predicted score and the preset score. If the predicted score is greater than the preset score, the screening result is that the patient does not have PACG. If the predicted score is less than the predicted score, the screening result is that the patient has PACG. Therefore, the spatial score and the time score are calculated according to the anterior segment OCT video to realize dynamic automatic scoring of the human eye, thereby improving the accuracy of PACG screening.

[0114] Refer to Figures 1 to Fig. 9 The glaucoma screening method based on AS-OCT video according to an embodiment of the present invention is described in detail with a specific embodiment. It is worth noting that the following description is only an exemplary description and is not a specific limitation of the invention.

[0115] The anterior segment OCT video is obtained by scanning the human eye to be tested in a dark-bright-dark changing environment through the anterior segment OCT technology, and then the anterior segment OCT video is split into multiple video frame images, the first video frame image in the multiple video frame images is selected as the reference video frame image, and the multi-scale surface point features and corner point features of the reference video frame image are obtained to obtain the reference feature points, and then the multi-scale surface point features and corner point features of the multiple video frame images are obtained to obtain the feature points corresponding to the multiple video frame images, and the conversion parameters are optimized according to the reference feature points and the feature points of the multiple video frame images through the iterative reweighted least squares algorithm to obtain the conversion parameters, and then the multiple video frame images are aligned with the conversion parameters to obtain multiple aligned images. The preset derivative network needs to be trained according to multiple video frame images with the scleral protrusion position marked first, so as to continuously adjust the relevant parameters of the preset derivative network to obtain an accurate preset derivative network, locate the scleral protrusion position through the preset derivative network, and then determine the anterior chamber angle area and iris area in the aligned image according to the scleral protrusion position, and then cut out the anterior chamber angle area and iris area in the aligned image to obtain multiple feature areas. The multiple feature areas include: multiple anterior chamber angle areas and multiple iris areas. Figure 3The D spatial sub-network model performs angle state analysis based on multiple anterior chamber angle regions to obtain the spatial score of the angle state analysis, and then performs feature extraction on the iris region and analyzes the iris movement based on the temporal sub-network model to obtain the temporal score of the iris changing over time. Therefore, the predicted score of the eye to be tested is determined based on the spatial score and the temporal score. Then, whether the person to be tested suffers from PACG can be determined based on the predicted score and the preset score, thereby improving the accuracy of PACG screening.

[0116] Second, refer to Fig.10 The embodiment of the present invention also discloses a glaucoma screening system based on AS-OCT video, including: a video processing module 100, an alignment module 200, an extraction module 300 and a calculation module 400; the video processing module 100 is used to obtain an anterior segment OCT video of a human eye to be tested in a dark-bright-dark changing environment, and split the anterior segment OCT video into multiple video frame images; the alignment module 200 is used to align the multiple video frame images to obtain multiple aligned images; the extraction module 300 is used to extract regions of interest of the multiple aligned images to obtain multiple feature regions; the calculation module 400 is used to calculate the multiple feature regions according to a preset multi-view spatiotemporal network model to obtain a prediction score; the comparison module 500 is used to determine the screening result according to the comparison between the prediction score and the preset score.

[0117] By acquiring the anterior segment OCT video of the human eye to be tested in a dark-bright-dark changing environment, the anterior segment OCT video is used to detect the state of the chamber angle in the changing environment to obtain the screening result of the human eye to be tested. Among them, the anterior segment OCT video integrates the spatial information and temporal information of multiple views to detect the dynamic human eye to be tested, so as to improve the accuracy of primary angle-closure glaucoma detection.

[0118] In a third aspect, an embodiment of the present invention further discloses an electronic control device, comprising: at least one processor, and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the glaucoma screening method based on AS-OCT video as in the first aspect.

[0119] In a fourth aspect, an embodiment of the present invention further discloses a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are read and executed, the glaucoma screening method based on AS-OCT video as described in the first aspect is executed.

[0120] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0121] It will be appreciated by those skilled in the art that all or some of the steps and systems in the disclosed method above may be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that may be used to store desired information and may be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0122] The embodiments of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above embodiments. Various changes can be made within the knowledge of ordinary technicians in the relevant technical field without departing from the purpose of the present invention. In addition, the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict.

Claims

1. A glaucoma screening method based on AS-OCT video, characterized in that: include: Acquire anterior segment OCT video of a human eye to be tested in a dark-bright-dark changing environment, and split the anterior segment OCT video into multiple video frame images; Performing alignment processing on the plurality of video frame images to obtain a plurality of aligned images; Extracting regions of interest from a plurality of the aligned images to obtain a plurality of feature regions; Calculating the plurality of feature regions according to a preset multi-view spatiotemporal network model to obtain a prediction score; Determine the screening result based on the comparison between the predicted score and the preset score; The preset multi-view spatiotemporal network model includes: a multi-view 3D space sub-network model and a temporal sub-network model; the feature area includes: anterior chamber angle area and iris area, and the calculation of the plurality of feature areas according to the preset multi-view spatiotemporal network model to obtain a prediction score includes: Performing angle state analysis on the plurality of anterior chamber angle regions according to the multi-view 3D spatial sub-network model to obtain a spatial score; Performing temporal change calculation on the plurality of iris regions according to the temporal sub-network model to obtain a temporal score; determining the prediction score based on the spatial score and the temporal score; The step of performing temporal change calculation on the plurality of iris regions according to the temporal subnetwork model to obtain a temporal score includes: Inputting the plurality of iris regions into a long short-term memory layer and performing normalization processing; Encoding the temporal state of the long short-term memory layer, and adding a fully connected layer to the output layer of the long short-term memory layer; The time score is obtained according to the output of the long short-term memory layer.

2. The glaucoma screening method based on AS-OCT video according to claim 1, characterized in that: The step of performing alignment processing on the plurality of video frame images to obtain a plurality of aligned images comprises: Set the first video frame image as the reference video frame image; Extracting multi-scale surface point features and corner point features of the reference video frame image to obtain reference feature points; Extracting multi-scale surface point features and corner point features of a plurality of the video frame images to obtain feature points corresponding to the video frame images; Determining conversion parameters of the video frame image according to the feature points, the reference feature points and a preset optimization algorithm; The video frame images are converted according to the conversion parameters corresponding to the multiple video frame images to obtain the multiple aligned images.

3. The glaucoma screening method based on AS-OCT video according to claim 1, characterized in that: The step of extracting the regions of interest of the plurality of aligned images to obtain a plurality of feature regions comprises: locating scleral protrusion positions of the plurality of aligned images according to a preset derivative network; determining anterior chamber angle region and iris region of the aligned image according to the scleral protrusion positions of the plurality of aligned images; The anterior chamber angle region and the iris region of the plurality of aligned images are cropped at a preset size to obtain a plurality of the feature regions.

4. The glaucoma screening method based on AS-OCT video according to claim 2, characterized in that: The preset optimization algorithm includes: iterative reweighted least squares algorithm.

5. A glaucoma screening system based on AS-OCT video, characterized in that: The method for glaucoma screening based on AS-OCT video according to any one of claims 1 to 4, wherein the system comprises: A video processing module, used for acquiring anterior segment OCT video of a human eye to be tested in a dark-bright-dark changing environment, and splitting the anterior segment OCT video into multiple video frame images; An alignment module, used for performing alignment processing on the plurality of video frame images to obtain a plurality of aligned images; An extraction module, used for extracting regions of interest of the plurality of aligned images to obtain a plurality of feature regions; A calculation module, used for calculating the plurality of feature regions according to a preset multi-view spatiotemporal network model to obtain a prediction score; A comparison module is used to determine the screening result based on the comparison between the predicted score and the preset score.

6. An electronic control device, characterized in that: include: at least one processor, and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the glaucoma screening method based on AS-OCT video as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are read and executed, the glaucoma screening method based on AS-OCT video as described in any one of claims 1 to 4 is executed.

Images