Optic cup and optic disc positioning point detection method and device based on eye OCT images

The deep learning detection model extracts the visual cup and visual disc positioning points in the OCT image, which solves the problem of low efficiency and insufficient accuracy of measuring parameters of visual disc morphology evaluation in the prior art, and achieves efficient and accurate automated detection.

CN111311565BActive Publication Date: 2024-11-12PING AN TECH (SHENZHEN) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202010087226.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-02-11
Publication Date
2024-11-12
Estimated Expiration
2040-02-11

AI Technical Summary

Technical Problem

In the prior art, the measurement of the visual disk morphology evaluation parameter based on OCT images depends on manual or semi-automated methods, is inefficient and has insufficient accuracy, and lacks efficient automated detection solutions.

Method used

Using a deep learning-based detection model, multiple feature maps of different scales are extracted through the first network branch, and combined with the second network branch, the accurate detection of the positioning points of the visual cup and the visual disc in the eye OCT image is achieved, providing terminal equipment and computers. Read storage media for automated detection.

Benefits of technology

It greatly improves the efficiency and accuracy of positioning point detection of the view cup and view disc, simplifies the detection process, is suitable for different terminal equipment, and enhances the adaptability and reliability of the detection model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN111311565B_ABST
    Figure CN111311565B_ABST
Patent Text Reader

Abstract

The present application is applicable to the field of image processing technology, and provides a method, device and terminal device for detecting optic cup and optic disc positioning points based on eye OCT images, including: acquiring an eye OCT image; using a preset detection model to detect the eye OCT image to obtain two positioning point coordinates of the optic cup in the eye OCT image, and two positioning point coordinates of the optic disc; the detection model includes a first network branch and a second network branch, the first network branch is used to extract feature maps of multiple different scales of the eye OCT image, and the second network branch is used to extract the two positioning point coordinates of the optic cup in the eye OCT image, and the two positioning point coordinates of the optic disc according to the feature maps of multiple different scales. The present application realizes accurate and efficient positioning of the optic cup and optic disc in the eye OCT image.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and in particular, relates to a method, device, terminal device and computer-readable storage medium for detecting optic cup and optic disc positioning points based on eye OCT images. Background Art

[0002] Optical Coherence Tomography (OCT) is a new type of tomographic imaging technology that has developed rapidly in recent years and has the most promising development prospects, especially in the detection and imaging of living biological tissues. OCT images obtained by OCT technology have the characteristics of non-invasiveness, no radiation, non-intrusion, high resolution, high detection sensitivity, and safe and efficient image acquisition, and are becoming increasingly important in ophthalmic diagnosis.

[0003] Optic disc morphology assessment parameters are very important indicators in ophthalmic diagnosis. Optic disc morphology assessment parameters include but are not limited to optic disc area, cup area, disc rim area, vertical disc rim area, horizontal disc rim volume, average cup to disc ratio (CDR), horizontal and vertical CDR, etc.

[0004] However, at present, the measurement of optic disc morphology evaluation parameters based on OCT images still mostly relies on manual measurement and semi-automatic measurement by machine. Therefore, there is an urgent need for a detection scheme for optic disc morphology evaluation parameters based on eye OCT images. Summary of the invention

[0005] The embodiments of the present application provide a method, apparatus, terminal device and computer-readable storage medium for detecting optic cup and optic disc positioning points based on eye OCT images, and provide a solution for detecting optic cup and optic disc positioning points based on eye OCT images, so as to achieve accurate and efficient detection of optic cup and optic disc positioning points.

[0006] In a first aspect, an embodiment of the present application provides a method for detecting optic cup and optic disc positioning points based on an eye OCT image, comprising:

[0007] Acquire ocular OCT images;

[0008] The eye OCT image is detected using a preset detection model to obtain two positioning point coordinates of the optic cup and two positioning point coordinates of the optic disc in the eye OCT image; the detection model includes a first network branch and a second network branch, the first network branch is used to extract feature maps of multiple different scales of the eye OCT image, and the second network branch is used to extract the two positioning point coordinates of the optic cup and two positioning point coordinates of the optic disc in the eye OCT image based on the feature maps of multiple different scales.

[0009] In a second aspect, the embodiment of the present application provides an optic cup and optic disc positioning point detection device based on an eye OCT image, comprising:

[0010] An acquisition module, used for acquiring an eye OCT image;

[0011] A detection module is used to detect the eye OCT image using a preset detection model to obtain the coordinates of two positioning points of the optic cup and the coordinates of two positioning points of the optic disc in the eye OCT image; the detection model includes a first network branch and a second network branch, the first network branch is used to extract feature maps of multiple different scales of the eye OCT image, and the second network branch is used to extract the coordinates of two positioning points of the optic cup and the coordinates of two positioning points of the optic disc in the eye OCT image based on the feature maps of multiple different scales.

[0012] In a third aspect, an embodiment of the present application provides a terminal device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect when executing the computer program.

[0013] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0014] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on a terminal device, enables the terminal device to execute the method described in the first aspect.

[0015] In an embodiment of the present application, the optic cup and optic disc positioning points of the eye OCT image are detected through a preset detection model. On the one hand, the positioning point detection result can be obtained by directly detecting the eye OCT image through the detection model, which greatly improves the detection efficiency; on the other hand, because the detection model extracts features of multiple scales of the eye OCT image, the positioning point detection of the optic cup and optic disc is achieved more accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0017] Figure 1It is a flow chart of a method for detecting optic cup and optic disc positioning points based on an eye OCT image provided by an embodiment of the present application;

[0018] Figure 2 It is a structural schematic diagram of a detection model used in a method for detecting optic cup and optic disc positioning points based on an eye OCT image provided in an embodiment of the present application;

[0019] Figure 3 It is a schematic diagram of a process of preprocessing an original eye OCT image in a method for detecting optic cup and optic disc positioning points based on an eye OCT image provided by an embodiment of the present application;

[0020] Figure 4 It is a schematic diagram of marking an eye OCT image in a method for detecting optic cup and optic disc positioning points based on an eye OCT image provided by an embodiment of the present application;

[0021] Figure 5 It is a structural schematic diagram of a first network branch used in a method for detecting optic cup and optic disc positioning points based on an eye OCT image provided in an embodiment of the present application;

[0022] Figure 6 It is a structural schematic diagram of module 1 of the first network branch used in the method for detecting optic cup and optic disc positioning points based on eye OCT images provided in one embodiment of the present application;

[0023] Figure 7 It is a structural schematic diagram of module 2 of the first network branch used in the method for detecting optic cup and optic disc positioning points based on eye OCT images provided in one embodiment of the present application;

[0024] Figure 8 It is a structural diagram of module 3 of the first network branch used in the method for detecting optic cup and optic disc positioning points based on eye OCT images provided in one embodiment of the present application;

[0025] Fig. 9 It is a structural diagram of module 4 of the first network branch used in the method for detecting optic cup and optic disc positioning points based on eye OCT images provided in one embodiment of the present application;

[0026] Fig.10 It is a structural schematic diagram of a second network branch used in a method for detecting optic cup and optic disc positioning points based on an eye OCT image provided in an embodiment of the present application;

[0027] Fig.11 It is a structural schematic diagram of a first sub-network of a second network branch used in a method for detecting optic cup and optic disc positioning points based on an eye OCT image provided in an embodiment of the present application;

[0028] Fig.12 It is a structural schematic diagram of a second sub-network of a second network branch used in a method for detecting optic cup and optic disc positioning points based on an eye OCT image provided in an embodiment of the present application;

[0029] Fig.13 It is a structural schematic diagram of the attention module of the second sub-network of the second network branch used in the method for detecting optic cup and optic disc positioning points based on eye OCT images provided in one embodiment of the present application;

[0030] Fig.14 is a schematic diagram of an optic cup ellipse and an optic disc ellipse obtained in a method for detecting optic cup and optic disc positioning points based on an eye OCT image provided in an embodiment of the present application;

[0031] Fig.15 It is a structural schematic diagram of an optic cup and optic disc positioning point detection device based on an eye OCT image provided by an embodiment of the present application;

[0032] Fig.16 It is a schematic diagram of the structure of a terminal device to which the method for detecting optic cup and optic disc positioning points based on eye OCT images provided in an embodiment of the present application is applicable. DETAILED DESCRIPTION

[0033] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures and technologies are provided to facilitate a thorough understanding of the embodiments of the present application.

[0034] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only embodiments of a part of the present application, not all embodiments. Based on the embodiments in the present application, for ordinary technicians in the field, all other embodiments obtained without creative labor should belong to the scope of protection of the present application. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0035] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0036] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0037] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0038] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0039] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0040] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0041] Optic disc morphology evaluation parameters are very important indicators in ophthalmic diagnosis, and positioning detection of the optic cup and optic disc in the eye OCT image is the basis for obtaining the optic disc morphology evaluation parameters. Therefore, the embodiment of the present application provides a method for detecting optic cup and optic disc positioning points based on the eye OCT image, so as to achieve accurate and efficient detection of the optic cup and optic disc positioning points in the eye OCT image.

[0042] Figure 1A flowchart for implementing a method for detecting optic cup and optic disc positioning points based on eye OCT images provided in an embodiment of the present application is shown. The method is applied to a terminal device. The method for detecting optic cup and optic disc positioning points based on eye OCT images provided in an embodiment of the present application can be applied to terminal devices such as ophthalmic OCT devices, mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA), independent servers, distributed servers, server clusters or cloud servers. The embodiment of the present application does not impose any restrictions on the specific type of terminal devices. Figure 1 As shown, the method includes steps S110 to S130. The specific implementation principle of each step is as follows.

[0043] S110, acquiring an eye OCT image.

[0044] The eye OCT image is an object for which optic cup and optic disc positioning point detection is required, and the eye OCT image may be a frame of original eye OCT image.

[0045] When the terminal device is an OCT device, the eye OCT image may be an eye OCT image obtained by the OCT device scanning the eye of the human body to be tested in real time.

[0046] When the terminal device is not an OCT device, the eye OCT image may be an eye OCT image acquired by the terminal device in real time from the OCT device, or may be a pre-stored eye OCT image acquired from an internal or external memory of the terminal device.

[0047] In a non-limiting example, the OCT device collects OCT images of the human eye to be tested in real time, sends the OCT images to the terminal device, and the terminal device obtains the OCT images.

[0048] In another non-limiting example, the OCT device collects the OCT image of the eye of the human body to be tested and sends it to the terminal device. The terminal device first stores the OCT image in a database and then obtains the OCT image of the eye of the human body to be tested from the database.

[0049] In some embodiments of the present application, the terminal device acquires an eye OCT image, and after acquiring the eye OCT image, directly proceeds to the subsequent step S120, ie, detecting the optic cup and optic disc positioning points in the eye OCT image.

[0050] In some embodiments of the present application, a terminal device acquires an eye OCT image. After acquiring the eye OCT image, the eye OCT image is first pre-cropped to a preset size, such as 512*512, and then a subsequent step S120 is performed, i.e., the optic cup and optic disc positioning points in the preprocessed eye OCT image are detected.

[0051] In a non-limiting usage scenario of the present application, when the user wants to perform optic cup and optic disc positioning point detection on a selected frame of eye OCT image, the positioning point detection function of the terminal device is enabled by clicking a specific physical button and / or virtual button of the terminal device. At this time, the terminal device will automatically process the selected frame of eye OCT image according to the process from step S110 to step S120 to obtain the positioning point detection result.

[0052] In another non-limiting usage scenario of the present application, when the user wants to perform optic cup and optic disc positioning point detection on a frame of eye OCT image, the user can enable the positioning point detection function of the terminal device by clicking a specific physical button and / or virtual button, and select a frame of eye OCT image. The terminal device will then automatically process the eye OCT image according to the process from step S110 to step S120 to obtain the positioning point detection result.

[0053] It can be understood here that the order of clicking the button and selecting a frame of eye OCT image can be interchanged, and the embodiments of the present application are applicable to but not limited to these two different usage scenarios.

[0054] S120, using a preset detection model to detect the eye OCT image, and obtain the coordinates of two positioning points of the optic cup and the coordinates of two positioning points of the optic disc in the eye OCT image.

[0055] Step S120 is a step of performing positioning point detection on the eye OCT image using a preset detection model, and determining the coordinates of two positioning points of the optic cup and two positioning points of the optic disc in the eye OCT image.

[0056] Among them, Figure 2 As shown, the detection model includes a first network branch and a second network branch. The first network branch is used to extract feature maps of multiple different scales of the eye OCT image, and the second network branch is used to extract the coordinates of two positioning points of the optic cup and the coordinates of two positioning points of the optic disc in the eye OCT image according to the feature maps of multiple different scales.

[0057] In an embodiment of the present application, the detection model may be a deep learning network model, and the deep learning network model may be a deep learning network model based on machine learning technology in artificial intelligence.

[0058] When the eye OCT image is input into the deep learning network model, the deep learning network model outputs the coordinates of two positioning points of the optic cup and the coordinates of two positioning points of the optic disc in the eye OCT image.

[0059] Among them, the training process of the detection model includes: obtaining a sample data set, the sample data set includes multiple sample images, each sample image is an eye OCT sample image with optic cup and optic disc positioning points marked; using the sample data set to train the key point detection model, and adjusting the weight of the key point detection model during the training process until the output result of the key point detection model after the weight adjustment meets the preset conditions, or the number of iterations of the training process reaches the preset number of iterations, then the training is stopped.

[0060] As a non-limiting example of the present application, a large number of eye OCT images are acquired as sample images to form a sample data set; each sample image is an eye OCT sample image with optic cup and optic disc positioning points marked.

[0061] In order to obtain good labeling accuracy and train a detection model with better performance, in some embodiments of the present application, the sample image is an original eye OCT image that has been preprocessed and then the optic cup and optic disc positioning points have been marked.

[0062] It can be understood that preprocessing includes but is not limited to operations such as interpolation and truncation. For example, see Figure 3 As shown in the figure, the original image of the OCT image usually obtained is 1024 (1 pixel corresponds to 6 mm in reality) * 768 (1 pixel corresponds to 3.01 mm in reality). The OCT image is first interpolated to 1200 * 462, so that 1 pixel represents 5 μm (micrometer), which is very convenient for labeling; then, it is truncated on both sides, 200 pixels on each side, and the pre-processed OCT image resolution is 800 * 462.

[0063] The pre-processed eye OCT images are then annotated, mainly based on the doctors' experience. Combined with the definitions of optic cup and optic disc in clinical applications, multiple doctors accurately annotate the positioning points of optic cup and optic disc in different OCT images, and finally an expert doctor reviews them to ensure the accuracy and consistency of the annotations. For a schematic diagram of the annotation results, please see Figure 4 As shown. Figure 4As shown in the figure, the annotation results include four positioning points: two optic disc positioning point coordinates and two optic cup positioning point coordinates. The coordinates of the two optic disc positioning points are: optic disc positioning point 1, coordinates (x1, y1); optic disc positioning point 2, coordinates (x2, y2). The coordinates of the two optic cup positioning points are: optic cup positioning point 1, coordinates (x3, y3); optic cup positioning point 2, coordinates (x4, y4). The annotation complies with clinical specifications. The optic disc positioning point is the end of the retinal pigment epithelium (RPE). The optic cup line is parallel to the optic disc line and intersects with the internal limiting membrane (ILM) at the optic cup positioning points (x3, y3) and (x4, y4). The distance d between the optic cup line and the optic disc line is 110 μm according to clinical use. 1 pixel represents 5 μm, and the distance can be calculated to be 22 pixels.

[0064] The labeled sample images are stored in a preset database as a sample data set.

[0065] A sample data set is obtained from a preset database, and a sample image is used as input. The annotation results in the sample image are used as target positioning points to establish a cup and disc positioning point detection model. During the model training process, the model weights are adjusted until the output result of the model after the weight adjustment meets the preset accuracy threshold, or the number of iterations reaches the preset iteration threshold, and the model training process is stopped.

[0066] Optionally, the sample images are divided into a training sample set, a verification sample set and a test sample set, and a deep learning network model is trained using a back propagation algorithm based on the training sample set, the verification sample set and the test sample set.

[0067] It should be noted that the process of training the detection model can be implemented locally on the terminal device, and can also be implemented on other devices that are connected to the terminal device for communication. When the successfully trained detection model is deployed on the terminal device side, or other devices push the trained detection model to the terminal device and deploy it successfully, the optic cup and optic disc positioning points of the acquired eye OCT image can be detected on the terminal device. It should be noted that the eye OCT image to be detected obtained in the process of detecting the optic cup and optic disc positioning points can also be used to increase the sample size in the sample data set, and further optimize the detection model on the terminal device or other device side, and deploy the further optimized detection model to the terminal device to replace the previous detection model. In this way, the detection model is optimized and the performance of the detection model is further improved.

[0068] In an embodiment of the present application, the preset detection model includes a first network branch and a second network branch.

[0069] The first network branch is used to extract feature maps of multiple different scales of the eye OCT image. The first network branch is an improved Xception network, which is used to extract image information of different scales in the target image. The structure of the improved Xception network is as follows: Figure 5 As shown. Figure 5 As shown, the first network branch includes cascaded blocks (module) 1a, module2a, module3a and module4a, wherein the output of module4a is concatenated with the output of module2a after being upsampled by 4 times (upsample×4) and input into module2b, the output of module3a is concatenated with the output of module2b and input into module3b, the output of module3b is concatenated with the output of module4a and input into module4b, and the output of module4b is concatenated with the output of module2b after being upsampled by 4 times (upsample×4) and input into module2c; module 1a, module2a, module2b and module2c output four feature maps of different scales, the feature map scale of module 1a output is 256×256, and the number of channels is 8; the feature map scale of module 2a output is 128×128, and the number of channels is 48; The feature map scale of 2b output is 64×64, and the number of channels is 48; the feature map scale of module2c output is 32×32, and the number of channels is 48.

[0070] Among them, the structures of module 1 (including module 1a), module 2 (including module 2a, module 2b and module 2c), module 3 (including module 3a and module 3b), and module 4 (including module 4a and module 4b) are as follows Figures 6 to 9 shown.

[0071] like Figure 6 The following is a schematic diagram of the structure of module 1. Figure 6 As shown in the figure, module 1 includes a convolution layer and a BN (Batch Normalization) layer with an activation function. The activation function after the BN layer is the ReLU function; the convolution kernel of the convolution layer is 3×3, the stride is 2×2, and the number of channels is 8.

[0072] like Figure 7 The following is a schematic diagram of the structure of module 2. Figure 7As shown in the figure, module 2 includes 5 parts. The first to fifth parts and the second to fifth parts have the same network structure. The first part includes three convolutional layers, namely the first convolutional layer, the second convolutional layer and the third convolutional layer, and a fourth convolutional layer. The first and second convolutional layers are followed by a BN layer and an activation function ReLU function. The output of the third convolutional layer and the output of the fourth convolutional layer are first summed and then input into the second part. The second part includes the activation function ReLU function, three cascaded convolutional layers, and the data of the second part is input into the third part after summing the data after passing through three convolutional layers. And so on, until the fifth part gets the output of module 2.

[0073] Figure 8 The following is a schematic diagram of the structure of module 3. Fig. 9 The figure shows the structure of module 4. The overall structure of module 2, module 3, and module 4 is similar, the difference lies in the size of the convolution kernel and the number of times the module is repeated. Figure 8-Figure 9 As shown, no further description is given here.

[0074] In the embodiment of the present application, the first network branch utilizes the structure of the main module of the original Xception, and the changes to the original Xception include reducing the number of channels, increasing the number of module repetitions, and increasing feature cascades (or aggregations). By reducing the number of channels, the amount of calculation is greatly reduced, the system resource usage is reduced, and the computing power cost is reduced; at the same time, in order to balance the accuracy lost due to reducing the number of channels, on the one hand, the number of module repetitions is increased, and on the other hand, feature cascades are increased.

[0075] Specifically, the number of channels of the original Xception is reduced from 64, 128, 256, and 728 to 8, 48, 96, and 192, forming a lightweight Xception network. However, the reduction in the number of channels will lead to insufficient feature extraction, so a feature cascade operation is added. The feature cascade is specifically: the feature extraction network with a smaller number of channels is copied three times. For the convenience of description, it is called a multi-level network. Each network has multiple convolutional layers, and each layer outputs features of different resolutions, called multi-layer features. First, the multi-level network is connected in series, and the features extracted by each level of the network are passed to the next level as input. At the same time, the features of the corresponding level of the previous level are fused together to achieve feature reuse. This cascade operation fuses features of different resolutions multiple times to fully extract effective information.

[0076] The main advantages of this cascade method are:

[0077] 1) Module 1a, module 2a, module 3a, module 4a and module 2b, module 3b, module 4b belong to different levels. There are multiple networks at different levels, which can fully extract image information of different scales.

[0078] 2) This structure uses various methods, such as module2b simultaneously utilizing the features of the output upsampled from module2a and module4a, to fuse features of different resolutions, achieve feature reuse, and effectively utilize network features at different levels.

[0079] The second network branch is used to extract the coordinates of two positioning points of the optic cup and the coordinates of two positioning points of the optic disc in the eye OCT image according to the feature maps of multiple different scales.

[0080] As an example of this application, Fig.10 As shown, the second network branch includes a first subnetwork and a second subnetwork, wherein the first subnetwork is used to roughly detect the optic cup and optic disc positioning points of the eye OCT image; and the second subnetwork is used to precisely detect the optic cup and optic disc positioning points of the eye OCT image.

[0081] Among them, the first sub-network is a global network (GlobalNet) with added feature cascades; the second sub-network is a segmentation network (RefineNet) with added attention mechanism.

[0082] The first sub-network takes the feature maps of different scales output by the first network branch as input and adds feature cascades. Using the global network, simple key points can be located by extracting image features.

[0083] Fig.11 The diagram shows the structure of the first sub-network. Fig.11 As shown, it should be noted that Fig.11Module1a, module 2a, module 2b, and module 2c correspond to the outputs of different scales in the first network branch. The first subnetwork includes 7 convolutional layers, among which the output of module 2c is concatenated with the output of module 2b after passing through the first convolutional layer (convolution kernel is 3×3, number of channels is 256) and upsampled by 2 times as the input of the second convolutional layer (convolution kernel is 3×3, number of channels is 128); the output of the second convolutional layer is concatenated with the output of module2a after being upsampled by 2 times as the input of the third convolutional layer (convolution kernel is 3×3, number of channels is 64); the output of the third convolutional layer is concatenated with the output of module 2c after being upsampled by 2 times as the input of the third convolutional layer (convolution kernel is 3×3, number of channels is 64). The output concatenation of 1a is used as the input of the fourth convolutional layer (convolution kernel is 3×3, number of channels is 64). The outputs of the second convolutional layer, the third convolutional layer and the third convolutional layer are respectively passed through a fifth convolutional layer (convolution kernel is 1×1, number of channels is 4) and then output three outputs of different scales generated by the first sub-network. The three outputs are global_out1, global_out2 and global_out3.

[0084] The second sub-network takes the output of different scales of the first sub-network as input. The features are already highly dense. By adding the attention mechanism, the features can be screened according to their importance, which can effectively improve the reliability of the final result.

[0085] Fig.12 The diagram shows the structure of the second sub-network. Fig.12 As shown in the figure, the three convolutional layers of the second sub-network are connected to the three outputs of the first sub-network, namely the first convolutional layer (convolution kernel is 1×1, the number of channels is 128), the second convolutional layer (convolution kernel is 1×1, the number of channels is 128) and the third convolutional layer (convolution kernel is 1×1, the number of channels is 256); the output of the second convolutional layer is connected to the first attention module; the output of the third convolutional layer is connected to the second attention module; the output of the second attention module passes through the fourth convolutional layer (convolution kernel is 1×1, the number of channels is 128), the third attention module and 4 times upsampling in turn, and then concatenates with the output of the first attention module after 2 times upsampling and the output of the first convolutional layer, and then inputs the fifth convolutional layer (convolution kernel is 1×1, the number of channels is 4) to obtain the output, which is the detection result of the positioning points of the optic cup and optic disc.

[0086] Among them, the attention module includes the first to third attention modules, and its structural diagram is as follows Fig.13As shown in the figure, the attention module includes a global average pooling layer and two fully connected layers (Dense out). The activation function ReLU function is set between the two fully connected layers, and the activation function Sigmoid function is set after the second fully connected layer. The output of the Sigmoid function is multiplied with the input data after structural adjustment as the output of the attention module.

[0087] The global average pooling layer globally averages the feature map and outputs a value, that is, it converts a W*H*D tensor into a 1*1*D tensor. This layer compresses the features along the spatial dimension, so that it has a global receptive field on the feature channel, and the output dimension matches the number of input feature channels.

[0088] The three layers of fully connected layer, activation function and structural adjustment generate weights for each feature channel through parameters, where the parameters are learned to explicitly model the correlation between feature channels.

[0089] The final multiplication operation is a recalibration operation, which regards the output weight as the importance of each feature channel after feature selection, and then weights it to the previous feature channel by channel through multiplication to complete the recalibration of the original feature in the channel dimension. The attention module in the embodiment of the present application only needs to learn one weight and multiply it by the original convolution.

[0090] The attention module uses SE-Net (Squeeze-and-Excitation Networks). SE-Net explicitly models the interdependence between feature channels. It does not introduce a new spatial dimension to fuse feature channels, but adopts a new feature recalibration strategy. Specifically, it automatically obtains the importance of each feature channel through learning, and then enhances useful features and suppresses features that are not very useful for the current task according to this importance. Therefore, screening features according to importance can effectively improve the final result.

[0091] It should be understood that the deep learning network model described herein is merely an exemplary description and should not be construed as a specific limitation to the invention.

[0092] In an embodiment of the present application, the optic cup and optic disc positioning points of the eye OCT image are detected through a preset detection model. On the one hand, the positioning point detection result of the eye OCT image can be obtained directly through the detection model, which greatly improves the detection efficiency; on the other hand, because the detection model extracts features of multiple scales of the eye OCT image, the positioning point detection of the optic cup and optic disc is achieved more accurately.

[0093] Optionally, based on any of the above embodiments, that is, based on obtaining the detection result of the optic cup and optic disc positioning points in an eye OCT image, in some other embodiments of the present application, in the above Figure 1 After step S120, the illustrated embodiment further includes steps S130 to S160.

[0094] S130, obtaining coordinates of two positioning points of the optic cup and two positioning points of the optic disc in the eye OCT images at at least three different angles; the at least three different angles include 0 degree and 90 degrees.

[0095] S140, determining the optic cup length and the optic disc length in each of the eye OCT images according to the coordinates of two positioning points of the optic cup and the coordinates of two positioning points of the optic disc in each of the eye OCT images.

[0096] S150, forming at least three first line segments according to the optic cup lengths at at least three different angles, concentrically projecting the at least three first line segments onto the same plane, and fitting an optic cup ellipse; forming at least three second line segments according to the optic disc lengths at at least three different angles, concentrically projecting the at least three second line segments onto the same plane, and fitting an optic disc ellipse.

[0097] S160, obtaining morphological parameters such as the optic cup area, optic disc area, cup-to-disc area ratio, vertical cup-to-disc ratio, and horizontal cup-to-disc ratio based on the optic cup length at at least three different angles, the optic disc length at at least three angles, the optic cup ellipse, and the optic disc ellipse.

[0098] In the embodiment of the present application, the OCT image includes 0 degrees, 45 degrees, 90 degrees, 135 degrees, etc., so the calculated optic cup length and optic disc length are the lengths at various angles, for example, the vertical direction is the length at 90 degrees, and the horizontal direction is the length at 0 degrees.

[0099] In a non-limiting example, in this example, four different angles are used as examples to illustrate, and the optic cup length and optic disc length at 0 degrees, 45 degrees, 90 degrees, and 135 degrees can be obtained respectively. According to these optic cup lengths and optic disc lengths, 8 line segments can be constructed. These 8 line segments are concentric and are projected onto the same plane, such as Fig.14 Now take the length of the optic cup as an example: project the concentric line segments onto the same plane to obtain 8 points, and fit these 8 points into an ellipse, as shown in Fig.14 The small ellipse inside the ellipse shown in can be obtained, and the ellipse parameters can be obtained, so the area of ​​the optic cup can be calculated, that is, the area of ​​the smaller ellipse (the optic cup ellipse). Fig.14The outer large ellipse shown in the figure can be used to calculate the disc area, that is, the area of ​​the larger ellipse (the disc ellipse). The cup-to-disc area ratio is the ratio of the cup to the disc area, which is the ratio of the smaller ellipse area to the larger ellipse area; the horizontal ratio of the cup to the disc is the ratio of the cup length at 0 degrees to the disc length at 0 degrees; the vertical ratio of the cup to the disc is the ratio of the cup length to the disc length at 90 degrees at 90 degrees.

[0100] In the embodiments of the present application, on the one hand, the detection model can be used to directly obtain the detection results of the positioning points of the eye OCT image, which greatly improves the detection efficiency. On this basis, line segment projection is performed according to the positioning point results of the optic cup and optic disc at multiple angles, and the morphological parameters of the optic cup and optic disc are obtained by fitting. The method is simple, efficient and easy to implement. On the other hand, since the detection model extracts features of multiple scales of the eye OCT image, the positioning point detection of the optic cup and optic disc is more accurately achieved, which improves the accuracy of obtaining the morphological parameters of the optic cup and optic disc. On the other hand, based on the optic cup and optic disc positioning points at multiple different angles, more and richer morphological parameters of the eye OCT image are obtained, so that the solution of the present application can be applied to different scenarios and is more adaptable.

[0101] An embodiment of the present application also provides a glaucoma grading method. After the 4-dimensional morphological parameters of optic cup area, optic disc area, cup-to-disc area ratio, and vertical cup-to-disc ratio are obtained by the aforementioned embodiment, the glaucoma grading method also includes: obtaining 5-dimensional ganglion cell complex (GCC) features and 4-dimensional retinal optic nerve fiber layer (RNFL) thickness features; combining the 4-dimensional morphological parameters, 5-dimensional GCC features, and 4-dimensional RNFL features into 13-dimensional input features, inputting them into a glaucoma grading model trained using a machine learning method, and obtaining a glaucoma grading result.

[0102] Among them, the 5-dimensional GCC features about GCC parameters include: upper GCC thickness, lower GCC thickness, average GCC thickness, local loss volume (FLV) and global loss volume (GLV).

[0103] The 4-dimensional RNFL thickness features regarding RNFL thickness include superior RNFL thickness, inferior RNFL thickness, nasal RNFL thickness, and temporal RNFL thickness.

[0104] The 5-dimensional GCC features and 4-dimensional RNFL thickness features can be directly read from the OCT image acquisition instrument.

[0105] The 4-dimensional morphological parameters, 5-dimensional GCC features and 4-dimensional RNFL features were combined into 13-dimensional input features, which were input into the glaucoma grading model trained using machine learning methods to obtain the glaucoma grading results.

[0106] The glaucoma grading model may be a classification model based on machine learning, such as a decision tree model based on Xgboost.

[0107] Exemplarily, the grading results of the glaucoma grading model include: no glaucoma, low risk, medium risk, high risk. This example is a four-category model, and in addition, it can also be a two-category model, a three-category model, or a classification model with more categories.

[0108] It is understandable that those skilled in the art can select a suitable classification model according to the actual implementation situation under the guidance of the embodiments of the present application, and the classification results of the classification model can also be selected and set according to the actual situation, and the present application does not make specific limitations on this.

[0109] The embodiment of the present application combines multiple parameters to improve the accuracy of classification. In addition, classification based on the glaucoma classification model can complete the decision within seconds, reduce system resource usage, and greatly improve classification efficiency.

[0110] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0111] Corresponding to the method for detecting the optic cup and optic disc positioning points based on the eye OCT image described in the above embodiment, Fig.15 A structural block diagram of an optic cup and optic disc positioning point detection device based on an eye OCT image provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0112] Reference Fig.15 , the device comprises:

[0113] An acquisition module 151 is used to acquire an eye OCT image;

[0114] The detection module 152 is used to detect the optic cup and optic disc positioning points of the eye OCT image through a preset detection model. On the one hand, the positioning point detection result can be obtained by directly detecting the eye OCT image through the detection model, which greatly improves the detection efficiency; on the other hand, because the detection model extracts features of multiple scales of the eye OCT image, the positioning point detection of the optic cup and optic disc is achieved more accurately.

[0115] It should be noted that the information interaction, execution process, etc. between the above-mentioned modules / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0116] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0117] Fig.16 This is a schematic diagram of the structure of a terminal device provided in an embodiment of the present application. Fig.16 As shown, the terminal device 16 of this embodiment includes: at least one processor 160 ( Fig.16 Only one processor is shown in the figure), a memory 161, and a computer program 162 stored in the memory 161 and executable on the at least one processor 160. When the processor 100 executes the computer program 162, the steps in the above-mentioned various method embodiments are implemented. Figure 1 Steps S110 to S120 are shown.

[0118] The terminal device may include but is not limited to a processor 160 and a memory 161. Those skilled in the art will appreciate that Fig.16 It is only an example of the terminal device 16 and does not constitute a limitation of the terminal device 16. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electrocardiograph may also include input and output devices, network access devices, buses, etc.

[0119] The processor 160 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0120] The memory 161 may be an internal storage unit of the terminal device 16, such as a hard disk or memory of the terminal device 16. The memory 161 may also be an external storage device of the terminal device 16, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 16. Further, the memory 161 may also include both an internal storage unit and an external storage device of the terminal device 16. The memory 161 is used to store the computer program and other programs and data required by the terminal device 16. The memory 161 may also be used to temporarily store data that has been output or is to be output.

[0121] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0122] An embodiment of the present application provides a computer program product. When the computer program product runs on a mobile terminal, the mobile terminal can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0123] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0124] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0125] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0126] In the embodiments provided in the present application, it should be understood that the disclosed terminal devices and methods can be implemented in other ways. For example, the terminal device embodiments described above are only illustrative. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0127] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0128] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for calculating optic cup and optic disc morphological parameters based on ocular OCT images, characterized in that: include: Acquire ocular OCT images; The eye OCT image is detected using a preset detection model to obtain two positioning point coordinates of the optic cup and two positioning point coordinates of the optic disc in the eye OCT image; the detection model includes a first network branch and a second network branch, the first network branch is used to extract feature maps of multiple different scales of the eye OCT image, and the second network branch is used to extract the two positioning point coordinates of the optic cup and two positioning point coordinates of the optic disc in the eye OCT image according to the feature maps of multiple different scales; Obtaining coordinates of two positioning points of the optic cup and two positioning points of the optic disc in at least three different angles of the eye OCT images; the at least three different angles include 0 degrees and 90 degrees; Determine the optic cup length and optic disc length in each of the eye OCT images according to the coordinates of two positioning points of the optic cup and the coordinates of two positioning points of the optic disc in each of the eye OCT images; Calculating optic cup and optic disc morphological parameters according to the coordinates of two positioning points of the optic cup and the coordinates of two positioning points of the optic disc in at least three of the eye OCT images; The calculating of the optic cup and optic disc morphological parameters according to the coordinates of two positioning points of the optic cup and two positioning points of the optic disc in at least three of the eye OCT images comprises: At least three first line segments are formed according to the optic cup lengths at at least three different angles, and the at least three first line segments are concentric and projected onto the same plane to fit the optic cup ellipse; at least three second line segments are formed according to the optic disc lengths at at least three different angles, and the at least three second line segments are concentric and projected onto the same plane to fit the optic disc ellipse; Optic cup and optic disc morphology parameters are obtained according to the optic cup length at at least three different angles, the optic disc length at at least three angles, the optic cup ellipse, and the optic disc ellipse.

2. The method for calculating the morphological parameters of the optic cup and optic disc according to claim 1, characterized in that: The training process of the detection model includes: Acquire a sample data set, wherein the sample data set includes a plurality of sample images, each sample image being an eye OCT sample image with optic cup and optic disc positioning points marked; The key point detection model is trained using the sample data set, and the weights of the key point detection model are adjusted during the training process until the output result of the key point detection model after the weight adjustment meets the preset conditions, or the number of iterations of the training process reaches the preset number of iterations, then the training is stopped.

3. The method for calculating the morphological parameters of the optic cup and optic disc according to claim 1 or 2, characterized in that: The second network branch includes a first subnetwork and a second subnetwork, wherein the first subnetwork is used for roughly detecting the optic cup and optic disc positioning points of the eye OCT image; and the second subnetwork is used for precisely detecting the optic cup and optic disc positioning points of the eye OCT image.

4. The method for calculating the morphological parameters of the optic cup and optic disc according to claim 3, characterized in that: The first sub-network is GlobalNet with added feature cascade; the second sub-network is RefineNet with added attention mechanism.

5. The method for calculating the morphological parameters of the optic cup and optic disc according to claim 1, characterized in that: The optic cup and optic disc morphological parameters include: at least one of the optic cup area, optic disc area, cup-disc area ratio, vertical cup-disc ratio and horizontal cup-disc ratio.

6. A device for calculating optic cup and optic disc morphological parameters based on ocular OCT images, characterized in that: include: An acquisition module, used for acquiring an eye OCT image; A detection module, used to detect the eye OCT image using a preset detection model to obtain the coordinates of two positioning points of the optic cup and the coordinates of two positioning points of the optic disc in the eye OCT image; the detection model includes a first network branch and a second network branch, the first network branch is used to extract feature maps of multiple different scales of the eye OCT image, and the second network branch is used to extract the coordinates of two positioning points of the optic cup and the coordinates of two positioning points of the optic disc in the eye OCT image according to the feature maps of multiple different scales; The detection device is also used for: Obtaining coordinates of two positioning points of the optic cup and two positioning points of the optic disc in at least three different angles of the eye OCT images; the at least three different angles include 0 degrees and 90 degrees; Determine the optic cup length and optic disc length in each of the eye OCT images according to the coordinates of two positioning points of the optic cup and the coordinates of two positioning points of the optic disc in each of the eye OCT images; Calculating optic cup and optic disc morphological parameters according to the coordinates of two positioning points of the optic cup and the coordinates of two positioning points of the optic disc in at least three of the eye OCT images; The calculating of the optic cup and optic disc morphological parameters according to the coordinates of two positioning points of the optic cup and two positioning points of the optic disc in at least three of the eye OCT images comprises: At least three first line segments are formed according to the optic cup lengths at at least three different angles, and the at least three first line segments are concentric and projected onto the same plane to fit the optic cup ellipse; at least three second line segments are formed according to the optic disc lengths at at least three different angles, and the at least three second line segments are concentric and projected onto the same plane to fit the optic disc ellipse; Optic cup and optic disc morphology parameters are obtained according to the optic cup length at at least three different angles, the optic disc length at at least three angles, the optic cup ellipse, and the optic disc ellipse.

7. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.