A neural network-based OCT human eye image acquisition and registration method and system

Abstract

The present invention discloses a method and system for collecting and registering OCT human eye images based on a neural network. The offset between two images is directly obtained through a deep neural network model trained in advance, thereby reducing the time it takes for the OCT system to display high-quality images and reducing the displacement deviation between multiple images collected using the OCT system. Compared with the traditional compensation algorithm, the method proposed in the present invention does not require repeated calculation of the optimal offset, which reduces the time consumption of the offset process. At the same time, the present invention adopts a method based on A-scans registration, treating the two-dimensional OCT image as a non-rigid image, thereby improving the accuracy of compensating the displacement deviation of the OCT image.

Description

Technical Field

[0001] The present invention relates to the technical field of OCT system, OCTA system image acquisition and image registration, and in particular to a neural network-based OCT human eye image acquisition and registration method and system. Background Art

[0002] Optical coherence tomography (OCT) is an imaging technology that obtains depth-direction tomography capabilities based on the principle of low-coherence interference of light. This technology has the advantages of non-contact, non-invasive, real-time three-dimensional imaging, and high imaging resolution. Its imaging system consists of five parts: broadband light source, fiber coupler, reference arm, sample arm, and image collector. During the imaging process, each time the broadband light source emits a beam of light, the image collector will receive a one-dimensional data with depth information. In OCT technology, each one-dimensional data is called an A-scan. By collecting A-scans at multiple different positions, a two-dimensional or three-dimensional image of the internal structure of biological tissue or material can be reconstructed. However, when performing tomographic imaging of living tissue or materials in a shaking state. Due to the displacement of the object being detected, the A-scans collected at the same light source incident point will have different depth information, which will lead to the problem that the image display information cannot fully correspond to the actual information when reconstructing the two-dimensional or three-dimensional image from the A-scans data. Therefore, during the OCT imaging acquisition process, using the registration method to compensate for the relative motion of the offset image will make the information collected by OCT technology more reflective of the actual situation.

[0003] Among the existing OCT image registration technologies, ①. A method for registering OCT images based on image vascular feature information is described in the patent with application number 201710080611.5. This patent preprocesses the OCT two-dimensional image through noise reduction, stratification, image projection and other preprocessing methods, and finally obtains the feature points of the blood vessels in the two-dimensional image. Then the two-dimensional feature points are projected into the three-dimensional image, and the correspondence between the feature points is performed to achieve the effect of registering the OCT image. ②. A method for registering OCT images based on image amplitude information is described in the patent with application number 201210109566.9. This patent first uses the amplitude normalized cross-correlation method to roughly register the OCT image, and then uses the amplitude normalized cross-correlation method to register the A-scans information that constitutes the OCT image to achieve the purpose of fine registration. ③. A method for registering OCT images based on image phase information is described in the patent with application number 201580006926.4. This patent uses the phase information between OCT images and certain judgment strategies to achieve the purpose of alignment.

[0004] The disadvantage of these methods is that they need to repeatedly calculate the optimal offset for each frame. For example, in the patent application number 201580006926.4, they solve the problem of human eye jitter by calculating the correlation of phase information between the registration image and the image to be registered, and obtain an optimal offset by continuously adjusting the position between the two frames. This process requires the optimal offset calculation between all OCT images to be registered. This will consume more machine computing resources, and the time consumed is more than ten times longer than the imaging method that does not eliminate eye movement. At the same time, the registration scale in this patent is a two-dimensional image, and the jitter between the A-scans that constitute the two-dimensional image is not taken into account, which will reduce the accuracy of the registration effect. In the patent application number 201210109566.9, the jitter between A-scans is taken into account, but the registration method uses a method based on image intensity information to iteratively solve the optimal value for registration, which will have the problem of excessive registration time consumption. In clinical applications, shorter registration time consumption can better highlight the real-time advantage of OCT imaging. Summary of the invention

[0005] The purpose of the present invention is to propose a neural network-based OCT human eye image acquisition and registration method and system to solve one or more technical problems existing in the prior art and at least provide a beneficial choice or create conditions.

[0006] The present invention directly obtains the offset between two images through a pre-trained deep neural network model, thereby reducing the time it takes for an OCT system to display high-quality images.

[0007] In order to achieve the above object, according to one aspect of the present invention, a method for collecting and registering OCT human eye images based on a neural network is provided, and the method comprises the following steps:

[0008] S100, collecting and calibrating image data through the OCT system;

[0009] S200, training the first neural network model and the second neural network model by the calibrated image data to obtain a first model and a second model respectively;

[0010] S300, compensating for the offset of the image during acquisition by the OCT system using the first model and the second model.

[0011] Further, in S100, the OCT system is composed of five parts: a light source, a fiber optic coupler, a reference arm, a sample arm, and a signal collector. The optical path of the OCT system is that a light source emits a beam of light, which is divided into two beams of light through a fiber optic coupler. One beam of light enters a reference arm composed of a plane mirror, and the other beam of light passes through a sample arm composed of an XY scanning galvanometer and a lens group. By adjusting the position of the XY scanning galvanometer, different retinal fault positions are scanned, and the return light of the reference arm and the sample arm enters the signal collector after passing through the fiber optic coupler, and OCT image data is obtained through the signal collector.

[0012] Further, in S100, the method for collecting and calibrating image data is: collecting a living human eye retinal OCT image through an OCT system, and manually calibrating the lateral offset Dx between two images in the OCT image to be registered and the longitudinal offset t between each A-scans, wherein the longitudinal offsets between n A-scans constitute an offset vector T = [t 1 ,t 2 ,t 3 ,…t i …t n-1 ,t n ], i = [1, n] where t i With t i+1 It represents the longitudinal offset between A-scans at different acquisition times at the same position between two images. Each set of OCT images to be registered is marked with (Dx, T) information, which is the combination of parameters Dx and T.

[0013] Further, in S200, the method of respectively obtaining the first model and the second model by training the first neural network model and the second neural network model with the calibrated image data includes the following steps:

[0014] S201, the input of the first neural network model is two OCT images to be registered at the same position but at different acquisition times, and the output is the lateral offset Dx between the two OCT images;

[0015] The structure of the first neural network model includes: a first residual neural network (using ResNet34 network), a second residual neural network (ResNet34), a dropout layer and three fully connected layers connected in sequence, the three fully connected layers are the first fully connected layer, the second fully connected layer, and the third fully connected layer, the first residual neural network and the second residual neural network are connected to the dropout layer respectively, and the first residual neural network and the second residual neural network are output to the dropout layer; the dropout layer is connected to the three fully connected layers connected in sequence; wherein the parameter of the dropout layer is 0.5, that is, the data input after the first residual neural network and the second residual neural network are merged and input into the dropout layer closes half of the neurons; the input dimension of the first fully connected layer is 1024, and the output dimension is 64, the input dimension of the second fully connected layer is 64, and the output dimension is 64, the input dimension of the third fully connected layer is 64, and the output dimension is 1;

[0016] In the first neural network model, two OCT images of the same position at different acquisition times are respectively input into the first residual neural network and the second residual neural network for feature extraction. Then, the two extracted features are merged into one feature and half of the neurons are shielded through the dropout layer for output. The merged feature is output through three fully connected layers, that is, the lateral offset Dx between the two images is obtained.

[0017] S202, the input of the second neural network model is two OCT images to be registered at the same position but at different acquisition times and a lateral offset Dx, and the output is a longitudinal offset vector T between multiple A-scans of the two images;

[0018] The structure of the second neural network model includes: a third residual neural network (ResNet34), three fully connected layers; the third residual neural network is connected and output to three fully connected layers connected in sequence, the three fully connected layers are a fourth fully connected layer, a fifth fully connected layer, and a sixth fully connected layer, the fourth fully connected layer has an input dimension of 1024 and an output dimension of 512, the fifth fully connected layer has an input dimension of 512 and an output dimension of 512, the sixth fully connected layer has an input dimension of 512 and an output dimension of M;

[0019] The output dimension M is a dynamic parameter, which is generally the number of groups of A-scans that need to be registered;

[0020] The length of the longitudinal offset vector T is the same as the size of the output dimension M;

[0021] The lateral offset Dx is the lateral offset between two OCT images to be registered at the same position but at different acquisition times;

[0022] In the second neural network model, two OCT images of the same position at different acquisition times and the lateral offset Dx are merged into a dual-channel image, and then the dual-channel image is used for feature extraction using a residual neural network. The extracted features are passed through three fully connected layers and finally output as a one-dimensional vector T.

[0023] S203, inputting each set of calibrated OCT images and calibrated (Dx, T) information, i.e., the lateral offset Dx and the vector T, into the first neural network model and the second neural network model to train the neural network parameters to obtain the first model and the second model respectively.

[0024] Further, in S300, the method of compensating the offset of the image during the acquisition process of the OCT system by using the first model and the second model includes the following steps:

[0025] S301, using the first model and the second model as output parts of a signal collector of an OCT acquisition system, and whenever the signal collector of the OCT acquisition system acquires two two-dimensional OCT image information of the same position at different acquisition times, namely, a first image and a second image, the first image and the second image are input into the first model to obtain a lateral offset Dx;

[0026] S302, inputting Dx and the first image and the second image into the second model to obtain a longitudinal offset vector T between the A-scans of the first image and the second image;

[0027] S303, compensating for the offset between the OCT images acquired in real time by the OCT system according to the Dx and T information between the OCT images obtained by the first model and the second model.

[0028] The present invention also provides an OCT human eye image acquisition and registration system based on a neural network, the system comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the computer program running in the following system units:

[0029] An image acquisition and calibration unit, used for acquiring and calibrating image data through the OCT system;

[0030] A neural network training unit, used to train the first neural network model and the second neural network model through the calibrated image data to obtain the first model and the second model respectively;

[0031] The OCT compensation unit is used to compensate for the offset of the image during the acquisition process of the OCT system through the first model and the second model.

[0032] The beneficial effects of the present invention are as follows: the present invention provides a method and system for collecting and registering OCT human eye images based on a neural network, which reduces the displacement deviation between multiple images collected using an OCT system. Compared with the traditional compensation algorithm, the method proposed by the present invention does not require repeated calculation of the optimal offset, which reduces the time consumption of the offset process. At the same time, the present invention adopts a method based on A-scans registration, which regards the two-dimensional OCT image as a non-rigid image, thereby improving the accuracy of compensating the displacement deviation of the OCT image. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The above and other features of the present invention will become more obvious by describing in detail the embodiments shown in the accompanying drawings. The same reference numerals in the accompanying drawings of the present invention represent the same or similar elements. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other accompanying drawings can be obtained based on these accompanying drawings without creative work. In the accompanying drawings:

[0034] Figure 1 Shown is a flow chart of a neural network-based OCT human eye image acquisition and registration method;

[0035] Figure 2 Shown is a block diagram of the OCT system;

[0036] Figure 3 Shown is a structural diagram of the first neural network model;

[0037] Figure 4 Shown is a structural diagram of the second neural network model;

[0038] Figure 5 Shown is a structural diagram of an OCT human eye image acquisition and registration system based on a neural network. DETAILED DESCRIPTION

[0039] The following will be combined with the embodiments and drawings to clearly and completely describe the concept, specific structure and technical effects of the present invention, so as to fully understand the purpose, scheme and effect of the present invention. It should be noted that the embodiments and features in the embodiments of this application can be combined with each other without conflict.

[0040] like Figure 1 The flowchart of the method for collecting and registering OCT human eye images based on a neural network according to the present invention is shown below. Figure 1 To illustrate a neural network-based OCT human eye image acquisition and registration method according to an embodiment of the present invention.

[0041] The present invention proposes a neural network-based OCT human eye image acquisition and registration method, which specifically includes the following steps:

[0042] S100, collecting and calibrating image data through the OCT system;

[0043] S200, training the first neural network model and the second neural network model by the calibrated image data to obtain a first model and a second model respectively;

[0044] S300, compensating for the offset of the image during acquisition by the OCT system using the first model and the second model.

[0045] Further, in S100, if Figure 2 As shown, Figure 2 It is a structural diagram of the OCT system, which consists of five parts: light source, fiber optic coupler, reference arm, sample arm, and signal collector. The optical path of the OCT system is that a light source emits a beam of light, which is divided into two beams of light through a fiber optic coupler. One beam of light enters the reference arm composed of a plane mirror, and the other beam of light passes through the sample arm composed of an XY scanning galvanometer and a lens group. By adjusting the position of the XY scanning galvanometer, different retinal fault positions are scanned, and the return light of the reference arm and the sample arm enters the signal collector after passing through the fiber optic coupler, and the OCT image data is obtained through the signal collector.

[0046] Further, in S100, the method for collecting and calibrating image data is: collecting a living human eye retinal OCT image through an OCT system, and manually calibrating the lateral offset Dx between two images in the OCT image to be registered and the longitudinal offset t between each A-scans, wherein the longitudinal offsets between n A-scans constitute an offset vector T = [t 1 ,t 2 ,t 3 ,…t i …t n-1 ,t n ], natural number i = [1, n], where two adjacent longitudinal offsets in T represent the longitudinal offsets between the same position between two images at different acquisition times A-scans, and each set of OCT images to be registered is marked with (Dx, T) information.

[0047] Further, in S200, the method of respectively obtaining the first model and the second model by training the first neural network model and the second neural network model with the calibrated image data includes the following steps:

[0048] S201, the input of the first neural network model is two OCT images to be registered at the same position but at different acquisition times, and the output is the lateral offset Dx between the two OCT images;

[0049] like Figure 3 As shown, Figure 3 : is a structural diagram of the first neural network model. The structure of the first neural network model includes: a first residual neural network (using ResNet34 network), a second residual neural network (using ResNet34 network), a dropout layer and three fully connected layers connected in sequence. The three fully connected layers are the first fully connected layer, the second fully connected layer, and the third fully connected layer. The first residual neural network and the second residual neural network are connected to the dropout layer respectively, and the first residual neural network and the second residual neural network are output to the dropout layer; the dropout layer is connected to the three fully connected layers connected in sequence; wherein, the parameter of the dropout layer is 0.5, that is, the data input after the first residual neural network and the second residual neural network are merged and input into the dropout layer closes half of the neurons; the input dimension of the first fully connected layer is 1024, and the output dimension is 64, the input dimension of the second fully connected layer is 64, and the output dimension is 64, the input dimension of the third fully connected layer is 64, and the output dimension is 1;

[0050] In the first neural network model, two OCT images of the same position at different acquisition times are respectively input into the first residual neural network and the second residual neural network for feature extraction. Then, the two extracted features are merged into one feature and half of the neurons are shielded through the dropout layer for output. The merged feature is output through three fully connected layers, that is, the lateral offset Dx between the two images is obtained.

[0051] S202, the input of the second neural network model is two OCT images to be registered at the same position but at different acquisition times and a lateral offset Dx, and the output is a longitudinal offset vector T between multiple A-scans of the two images;

[0052] like Figure 4 As shown, Figure 4 : is a structural diagram of the second neural network model. The structure of the second neural network model includes: a third residual neural network (using a ResNet34 network), three fully connected layers; the third residual neural network is connected and output to three fully connected layers connected in sequence, the three fully connected layers are a fourth fully connected layer, a fifth fully connected layer, and a sixth fully connected layer, the fourth fully connected layer has an input dimension of 1024 and an output dimension of 512, the fifth fully connected layer has an input dimension of 512 and an output dimension of 512, the sixth fully connected layer has an input dimension of 512 and an output dimension of M;

[0053] The output dimension M is a dynamic parameter, which is generally the number of groups of A-scans that need to be registered;

[0054] The length of the longitudinal offset vector T is the same as the size of the output dimension M;

[0055] The lateral offset Dx is the lateral offset between two OCT images to be registered at the same position but at different acquisition times;

[0056] In the second neural network model, two OCT images of the same position at different acquisition times and the lateral offset Dx are merged into a dual-channel image, and then the dual-channel image is used for feature extraction using a residual neural network. The extracted features are passed through three fully connected layers and finally output as a one-dimensional vector T.

[0057] S203, inputting each set of calibrated OCT images and calibrated (Dx, T) information, i.e., the lateral offset Dx and the vector T, into the first neural network model and the second neural network model to train the neural network parameters to obtain the first model and the second model respectively.

[0058] Further, in S300, the method of compensating the offset of the image during the acquisition process of the OCT system by using the first model and the second model includes the following steps:

[0059] S301, using the first model and the second model as output parts of a signal collector of an OCT acquisition system, and whenever the signal collector of the OCT acquisition system acquires two two-dimensional OCT image information of the same position at different acquisition times, namely, a first image and a second image, the first image and the second image are input into the first model to obtain a lateral offset Dx;

[0060] S302, inputting Dx and the first image and the second image into the second model to obtain a longitudinal offset vector T between the A-scans of the first image and the second image;

[0061] S303, compensating for the offset between the OCT images acquired in real time by the OCT system according to the Dx and T information between the OCT images obtained by the first model and the second model.

[0062] The embodiment of the present invention provides an OCT human eye image acquisition and registration system based on a neural network, such as Figure 5 The figure shows a structure diagram of a neural network-based OCT human eye image acquisition and registration system of the present invention. The neural network-based OCT human eye image acquisition and registration system of this embodiment includes: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps in the above-mentioned neural network-based OCT human eye image acquisition and registration system embodiment are implemented.

[0063] The system comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to run in the following units of the system:

[0064] An image acquisition and calibration unit, used for acquiring and calibrating image data through the OCT system;

[0065] A neural network training unit, used to train the first neural network model and the second neural network model through the calibrated image data to obtain the first model and the second model respectively;

[0066] The OCT compensation unit is used to compensate for the offset of the image during the acquisition process of the OCT system through the first model and the second model.

[0067] The OCT human eye image acquisition and registration system based on a neural network can be run in computing devices such as desktop computers, notebooks, PDAs and cloud servers. The OCT human eye image acquisition and registration system based on a neural network can be operated on systems including, but not limited to, processors and memories. Those skilled in the art can understand that the example is only an example of an OCT human eye image acquisition and registration system based on a neural network, and does not constitute a limitation on an OCT human eye image acquisition and registration system based on a neural network. It can include more or fewer components than the example, or a combination of certain components, or different components. For example, the OCT human eye image acquisition and registration system based on a neural network can also include input and output devices, network access devices, buses, etc.

[0068] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the OCT human eye image acquisition and registration system based on a neural network, and uses various interfaces and lines to connect the various parts of the entire OCT human eye image acquisition and registration system based on a neural network.

[0069] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the OCT human eye image acquisition and registration system based on neural network by running or executing the computer program and / or module stored in the memory and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0070] Although the description of the present invention has been quite detailed and has been described in particular with respect to several described embodiments, it is not intended to be limited to any of these details or embodiments or any particular embodiment, so as to effectively cover the intended scope of the present invention. In addition, the present invention is described above with the embodiments foreseeable by the inventors, and its purpose is to provide a useful description, and those non-substantial changes to the present invention that are not currently foreseen may still represent equivalent changes of the present invention.

Claims

1. A neural network-based OCT human eye image acquisition and registration method, characterized in that: The method comprises the following steps: S100, collecting and calibrating image data through the OCT system; S200, training the first neural network model and the second neural network model by the calibrated image data to obtain a first model and a second model respectively; S300, compensating for the offset of the image during acquisition by the OCT system using the first model and the second model; In S200, the method of respectively obtaining the first model and the second model by training the first neural network model and the second neural network model with the calibrated image data includes the following steps: S201, the input of the first neural network model is two OCT images to be registered at the same position but at different acquisition times, and the output is the lateral offset Dx between the two OCT images; The structure of the first neural network model includes: a first residual neural network, a second residual neural network, a dropout layer and three fully connected layers connected in sequence, the three fully connected layers are respectively the first fully connected layer, the second fully connected layer and the third fully connected layer, the first residual neural network and the second residual neural network are respectively connected to the dropout layer, the first residual neural network and the second residual neural network are output to the dropout layer; the dropout layer is connected to the three fully connected layers connected in sequence; wherein the parameter of the dropout layer is 0.5, that is, the data input after the first residual neural network and the second residual neural network are merged and input into the dropout layer closes half of the neurons; the input dimension of the first fully connected layer is 1024, and the output dimension is 64, the input dimension of the second fully connected layer is 64, and the output dimension is 64, the input dimension of the third fully connected layer is 64, and the output dimension is 1; In the first neural network model, two OCT images of the same position at different acquisition times are respectively input into the first residual neural network and the second residual neural network for feature extraction. Then, the two extracted features are merged into one feature and half of the neurons are shielded through the dropout layer for output. The merged feature is output through three fully connected layers, that is, the lateral offset Dx between the two images is obtained. S202, the input of the second neural network model is two OCT images to be registered at the same position but at different acquisition times and a lateral offset Dx, and the output is a longitudinal offset vector T between multiple A-scans of the two images; The structure of the second neural network model includes: a third residual neural network, three fully connected layers; the third residual neural network is connected and output to three fully connected layers connected in sequence, the three fully connected layers are a fourth fully connected layer, a fifth fully connected layer, and a sixth fully connected layer, the input dimension of the fourth fully connected layer is 1024, and the output dimension is 512, the input dimension of the fifth fully connected layer is 512, and the output dimension is 512, and the input dimension of the sixth fully connected layer is 512, and the output dimension is M; The output dimension M is a dynamic parameter, which is generally the number of groups of A-scans that need to be registered; The length of the longitudinal offset vector T is the same as the size of the output dimension M; The lateral offset Dx is the lateral offset between two OCT images to be registered at the same position but at different acquisition times; In the second neural network model, two OCT images of the same position at different acquisition times and the lateral offset Dx are merged into a dual-channel image, and then the dual-channel image is used for feature extraction using a residual neural network. The extracted features are passed through three fully connected layers and finally output as a one-dimensional vector T. S203, inputting each set of calibrated OCT images and calibrated (Dx, T) information, i.e., the lateral offset Dx and the vector T, into the first neural network model and the second neural network model to train the neural network parameters to obtain the first model and the second model respectively.

2. The method for collecting and registering OCT human eye images based on a neural network according to claim 1, characterized in that: In S100, the OCT system consists of five parts: a light source, a fiber optic coupler, a reference arm, a sample arm, and a signal collector. The light source emits a beam of light, which is divided into two beams of light through a fiber optic coupler. One beam of light enters a reference arm composed of a plane mirror, and the other beam of light passes through a sample arm composed of an XY scanning galvanometer and a lens group. By adjusting the position of the XY scanning galvanometer, different retinal fault positions are scanned, and the return light of the reference arm and the sample arm enters the signal collector after passing through the fiber optic coupler, and OCT image data is obtained through the signal collector.

3. The method for collecting and registering OCT human eye images based on a neural network according to claim 1, characterized in that: In S100, the method for collecting and calibrating image data is: collecting a living human eye retinal OCT image through an OCT system, and manually calibrating the lateral offset Dx between two images in the OCT image to be registered and the longitudinal offset t between each A-scans, wherein the longitudinal offsets between n A-scans constitute an offset vector T = [t1, t2, t3,…t i … t n-1 , t n ], i=[1,n] where t i With t i+1 It represents the longitudinal offset between A-scans at different acquisition times at the same position between two images. Each set of OCT images to be registered is marked with (Dx, T) information.

4. The method for collecting and registering OCT human eye images based on a neural network according to claim 1, characterized in that: In S300, the method of compensating the offset of the image during the acquisition process of the OCT system by using the first model and the second model includes the following steps: S301, using the first model and the second model as output parts of a signal collector of an OCT acquisition system, and whenever the signal collector of the OCT acquisition system acquires two two-dimensional OCT image information of the same position at different acquisition times, namely, a first image and a second image, the first image and the second image are input into the first model to obtain a lateral offset Dx; S302, inputting Dx and the first image and the second image into the second model to obtain a longitudinal offset vector T between the A-scans of the first image and the second image; S303, compensating for the offset between the OCT images acquired in real time by the OCT system according to the Dx and T information between the OCT images obtained by the first model and the second model.

5. A neural network-based OCT human eye image acquisition and registration system, characterized in that: The system comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to run in the following units of the system: An image acquisition and calibration unit, used for acquiring and calibrating image data through the OCT system; A neural network training unit, used to train the first neural network model and the second neural network model through the calibrated image data to obtain the first model and the second model respectively; An OCT compensation unit, used to compensate for the offset of the image during the acquisition process of the OCT system through the first model and the second model; The method for respectively obtaining a first model and a second model by training a first neural network model and a second neural network model with calibrated image data comprises the following steps: S201, the input of the first neural network model is two OCT images to be registered at the same position but at different acquisition times, and the output is the lateral offset Dx between the two OCT images; The structure of the first neural network model includes: a first residual neural network, a second residual neural network, a dropout layer and three fully connected layers connected in sequence, the three fully connected layers are respectively the first fully connected layer, the second fully connected layer and the third fully connected layer, the first residual neural network and the second residual neural network are respectively connected to the dropout layer, the first residual neural network and the second residual neural network are output to the dropout layer; the dropout layer is connected to the three fully connected layers connected in sequence; wherein the parameter of the dropout layer is 0.5, that is, the data input after the first residual neural network and the second residual neural network are merged and input into the dropout layer closes half of the neurons; the input dimension of the first fully connected layer is 1024, and the output dimension is 64, the input dimension of the second fully connected layer is 64, and the output dimension is 64, the input dimension of the third fully connected layer is 64, and the output dimension is 1; In the first neural network model, two OCT images of the same position at different acquisition times are respectively input into the first residual neural network and the second residual neural network for feature extraction. Then, the two extracted features are merged into one feature and half of the neurons are shielded through the dropout layer for output. The merged feature is output through three fully connected layers, that is, the lateral offset Dx between the two images is obtained. S202, the input of the second neural network model is two OCT images to be registered at the same position but at different acquisition times and a lateral offset Dx, and the output is a longitudinal offset vector T between multiple A-scans of the two images; The structure of the second neural network model includes: a third residual neural network, three fully connected layers; the third residual neural network is connected and output to three fully connected layers connected in sequence, the three fully connected layers are a fourth fully connected layer, a fifth fully connected layer, and a sixth fully connected layer, the input dimension of the fourth fully connected layer is 1024, and the output dimension is 512, the input dimension of the fifth fully connected layer is 512, and the output dimension is 512, and the input dimension of the sixth fully connected layer is 512, and the output dimension is M; The output dimension M is a dynamic parameter, which is generally the number of groups of A-scans that need to be registered; The length of the longitudinal offset vector T is the same as the size of the output dimension M; The lateral offset Dx is the lateral offset between two OCT images to be registered at the same position but at different acquisition times; In the second neural network model, two OCT images of the same position at different acquisition times and the lateral offset Dx are merged into a dual-channel image, and then the dual-channel image is used for feature extraction using a residual neural network. The extracted features are passed through three fully connected layers and finally output as a one-dimensional vector T. S203, inputting each set of calibrated OCT images and calibrated (Dx, T) information, i.e., the lateral offset Dx and the vector T, into the first neural network model and the second neural network model to train the neural network parameters to obtain the first model and the second model respectively.

Images