3D Human Eye Model Construction Method, System, Device and Storage Medium

Abstract

The present invention discloses a method, system, device and storage medium for constructing a three-dimensional human eye model. The method for constructing a three-dimensional human eye model includes: obtaining preoperative human eye OCT images, fundus scan images and intraoperative human eye microscopic images; determining a preoperative fundus retina three-dimensional model according to the three-dimensional coordinate information of the human eye OCT images and the fundus scan images; registering according to the vascular information of the fundus scan images and the human eye microscopic images to determine a first transformation matrix; and transforming the preoperative fundus retina three-dimensional model according to the first transformation matrix to obtain an intraoperative fundus retina three-dimensional model. The present invention constructs an intraoperative fundus retina three-dimensional model through preoperative human eye OCT images, fundus scan images and intraoperative human eye microscopic images, so that doctors can clearly understand the structure of the human eye during the operation according to the intraoperative fundus retina three-dimensional model, thereby improving the level of ophthalmic disease treatment.

Description

Technical Field

[0001] The present invention relates to the technical field of model construction, and in particular, to a method, a system, a device and a storage medium for constructing a three-dimensional human eye model. Background Art

[0002] As a disease with a relatively high incidence rate among the elderly population, the diagnosis and treatment of macular diseases are also issues that are emphasized in current medical technology. Among them, since it takes at least 5 years to train an independent surgeon for macular disease surgery, the training cost and difficulty of fundus retina surgeons are very high.

[0003] In the related art, in the clinical system of ophthalmic surgery, there is a lack of key precise navigation technology. Even though different modal images provide the fundus retina environment for doctors, doctors still cannot accurately master the complex environment inside the eyeball. Summary of the Invention

[0004] The present invention aims to at least solve one of the technical problems existing in the prior art. For this purpose, the present invention proposes a method for constructing a three-dimensional human eye model, which can construct an intraoperative fundus retina model, facilitate doctors to understand the environment of the eyeball, and improve the success rate of ophthalmic surgery.

[0005] The present invention also proposes a system for constructing a three-dimensional human eye model.

[0006] The present invention also proposes an electronic control device.

[0007] The present invention also proposes a computer-readable storage medium.

[0008] In a first aspect, an embodiment of the present invention provides a method for constructing a three-dimensional human eye model, including:

[0009] Obtaining a preoperative human eye OCT image, a fundus scan image, and an intraoperative human eye microscopic image;

[0010] Determining a preoperative fundus retina three-dimensional model according to the three-dimensional coordinate information of the human eye OCT image and the fundus scan image;

[0011] Performing registration according to the blood vessel information of the fundus scan image and the human eye microscopic image to determine a first transformation matrix;

[0012] Converting the preoperative fundus retina three-dimensional model according to the first transformation matrix to obtain an intraoperative fundus retina three-dimensional model.

[0013] The method for constructing a three-dimensional human eye model according to an embodiment of the present invention has at least the following beneficial effects: constructing a three-dimensional model of the fundus retina during surgery through the preoperative human eye OCT image, fundus scan image, and intraoperative human eye microscopic image, so that doctors can clearly understand the structure of the human eye during surgery based on the three-dimensional model of the fundus retina during surgery, thereby improving the level of eye disease treatment.

[0014] According to another embodiment of the present invention, in the method for constructing a three-dimensional human eye model, determining the preoperative fundus retina three-dimensional model according to the three-dimensional coordinate information of the human eye OCT image and the fundus scan image includes:

[0015] Converting the human eye OCT image into a human eye projection image;

[0016] Constructing a corresponding initial human eye model according to the human eye OCT image;

[0017] Performing rigid registration and elastic registration on the human eye projection image and the fundus scan image to obtain a second transformation matrix;

[0018] Combining the fundus scan image and the initial human eye model according to the second transformation matrix to obtain the preoperative fundus retina three-dimensional model.

[0019] According to another embodiment of the present invention, in the method for constructing a three-dimensional human eye model, constructing a corresponding initial human eye model according to the human eye OCT image includes:

[0020] Layering the human eye OCT image to extract corresponding three-dimensional coordinate information;

[0021] Constructing the initial human eye model according to the three-dimensional coordinate information.

[0022] According to another embodiment of the present invention, in the method for constructing a three-dimensional human eye model, performing rigid registration and elastic registration on the human eye projection image and the fundus scan image to obtain a second transformation matrix includes:

[0023] Performing translation transformation registration, rotation transformation registration, and scaling transformation registration on the human eye projection image and the fundus scan image to obtain a second rigid transformation matrix and a second registered image;

[0024] Performing elastic registration on the second registered image and the fundus scan image to obtain the second transformation matrix.

[0025] According to another embodiment of the present invention, the first transformation matrix includes: a first elastic transformation matrix and a first rigid transformation matrix; determining the first transformation matrix according to the vascular information of the fundus scan image and the human eye microscopic image includes:

[0026] Based on the fundus scan image and the human eye microscopic image, perform rigid registration based on vascular information to obtain a first rigid transformation matrix and a first registered image;

[0027] Determine a first elastic change matrix based on the first registered image and the human eye microscopic image using a deep learning algorithm.

[0028] According to another embodiment of the three-dimensional human eye model construction method of the present invention, the converting the preoperative fundus retina three-dimensional model according to the first transformation matrix to obtain an intraoperative fundus retina three-dimensional model includes:

[0029] Convert the preoperative fundus retina three-dimensional model according to the first rigid transformation matrix and the first elastic change matrix to obtain the intraoperative fundus retina three-dimensional model.

[0030] According to another embodiment of the three-dimensional human eye model construction method of the present invention, the constructing the initial human eye model according to the three-dimensional coordinate information includes:

[0031] Perform fitting based on the three-dimensional coordinate information using a preset least squares method to obtain the initial human eye model.

[0032] In a second aspect, an embodiment of the present invention provides a three-dimensional human eye model construction system, including:

[0033] An acquisition module, configured to acquire a preoperative human eye OCT image, a fundus scan image, and an intraoperative human eye microscopic image;

[0034] A first processing module, configured to determine a preoperative fundus retina three-dimensional model according to the three-dimensional coordinate information of the human eye OCT image and the fundus scan image;

[0035] A second processing module, configured to perform registration based on the vascular information of the fundus scan image and the human eye microscopic image to determine a first transformation matrix;

[0036] A conversion module, configured to convert the preoperative fundus retina three-dimensional model according to the first transformation matrix to obtain an intraoperative fundus retina three-dimensional model.

[0037] The three-dimensional human eye model construction system of the embodiment of the present invention has at least the following beneficial effects: constructing an intraoperative fundus retina three-dimensional model through a preoperative human eye OCT image, a fundus scan image, and an intraoperative human eye microscopic image, so that a doctor can clearly understand the structure of the human eye during the operation, thereby improving the level of eye disease treatment.

[0038] In a third aspect, an embodiment of the present invention provides an electronic control device, including:

[0039] At least one processor, and,

[0040] A memory communicatively connected to the at least one processor; wherein,

[0041] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the three-dimensional human eye model construction method as described in the first aspect.

[0042] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium storing computer-executable instructions for causing a computer to execute the three-dimensional human eye model construction method as described in the first aspect.

[0043] Other features and advantages of the present application will be described in the subsequent specification, and in part will be obvious from the specification, or will be understood by implementing the present application. The objectives and other advantages of the present application can be achieved and obtained by the structures specifically pointed out in the specification and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 is a schematic flowchart of a specific embodiment of the three-dimensional human eye model construction method in an embodiment of the present invention;

[0045] Figure 2 is a schematic flowchart of another specific embodiment of the three-dimensional human eye model construction method in an embodiment of the present invention;

[0046] Figure 3 is a schematic flowchart of another specific embodiment of the three-dimensional human eye model construction method in an embodiment of the present invention;

[0047] Figure 4 is a schematic flowchart of another specific embodiment of the three-dimensional human eye model construction method in an embodiment of the present invention;

[0048] Figure 5 is a schematic flowchart of another specific embodiment of the three-dimensional human eye model construction method in an embodiment of the present invention;

[0049] Figure 6 is a schematic flowchart of another specific embodiment of the three-dimensional human eye model construction method in an embodiment of the present invention;

[0050] Figure 7 is a schematic flowchart of another specific embodiment of the three-dimensional human eye model construction method in an embodiment of the present invention;

[0051] Figure 8 is a schematic block diagram of a specific embodiment of the three-dimensional human eye model construction system in an embodiment of the present invention;

[0052] Figure 9 It is a block diagram of a specific embodiment module of the electronic control device in the embodiment of the present invention.

[0053] Reference numerals in the drawings: 100, acquisition module; 200, first processing module; 300, second processing module; 400, conversion module; 500, processor; 600, memory. Specific implementation manner

[0054] The following will clearly and completely describe the concept of the present invention and the technical effects produced in combination with the embodiments, so as to fully understand the purpose, features and effects of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, other embodiments obtained by those skilled in the art without creative efforts shall fall within the scope of protection of the present invention.

[0055] In the description of the present invention, if it involves orientation description, such as "upper", "lower", "front", "rear", "left", "right", etc., the orientation or position relationship indicated is based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, so it cannot be understood as a limitation to the present invention. If a certain feature is described as "set", "fixed", "connected", "installed" on another feature, it can be directly set, fixed, connected, installed on another feature, or indirectly set, fixed, connected, installed on another feature.

[0056] In the description of the embodiments of the present invention, if it involves "several", its meaning is more than one. If it involves "multiple", its meaning is more than two. If it involves "greater than", "less than", "exceeding", it should be understood as not including the number itself. If it involves "above", "below", "within", it should be understood as including the number itself. If it involves "first", "second", it should be understood as being used to distinguish technical features, rather than indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features or implicitly indicating the sequence relationship of the indicated technical features.

[0057] The macula is the position at the center of the retina, located at the posterior pole of the fundus. It is a special area with the most sensitive vision and is mainly related to visual functions such as main vision and color vision. Macular diseases mainly include epiretinal membrane, macular hole, vitreomacular traction syndrome, macular splitting, etc., which cause great damage to central vision and can even lead to blindness in severe cases.

[0058] No effective drugs for treating the above macular diseases have been found yet. The current standardized treatment plan is vitreoretinal surgery combined with internal limiting membrane peeling. However, the macula is the most vulnerable area of the eye, and the inner limiting membrane (ILM) is located in the innermost layer of the retina. It is a 1-2 μm thick unstructured homogeneous membrane with a smooth and transparent surface. It is very difficult for doctors to distinguish its position and edge area under the microscope during the operation, and it is also very difficult to determine its position. Unclear judgment of the target position during the peeling operation is likely to cause damage to the macular area and retina, even blindness. Incomplete peeling is likely to lead to recurrence of the disease. Therefore, the syndrome of vitreoretinal surgery combined with internal limiting membrane peeling makes the treatment of macular diseases a difficult point in clinical ophthalmic surgery. The curved and narrow internal environment of the human eye limits the doctor's perception space during the operation. The size of the patient's eye, the diseased area, and the uneven thickness and texture of the retina vary from person to person. The operation has very high requirements for the doctor's technology and experience.

[0059] The treatment environment of macular diseases is complex, the conditions of patients vary, and the surgical requirements for doctors are very high. Precise and effective intraoperative navigation is the key to disease treatment. However, in the current clinical system of ophthalmic surgery, due to the lack of key precise navigation technology, even though different modalities of images provide the fundus retina environment for doctors, doctors still cannot accurately master the complex environment inside the eye.

[0060] Based on this, the present application discloses a three-dimensional human eye model construction system, which can reconstruct the three-dimensional structure model of the eyeball, so that doctors can clearly master the curved structure of the eyeball during the operation, which is beneficial to improving the doctor's overall perception of the environment, depth, strength, etc. during the operation, so as to reduce the probability of surgical syndrome and improve the level of eye disease treatment.

[0061] In the first aspect, referring to Figure 1 , the embodiments of the present application disclose a method for constructing a three-dimensional human eye model, including:

[0062] S100. Obtain preoperative human eye OCT images, fundus scan images, and intraoperative human eye microscopic images;

[0063] S200. Determine the preoperative fundus retina three-dimensional model according to the three-dimensional coordinate information of the human eye OCT image and the fundus scan image;

[0064] S300. Register according to the blood vessel information of the fundus scan image and the human eye microscopic image to determine the first transformation matrix;

[0065] S400. Convert the preoperative fundus retina three-dimensional model according to the first transformation matrix to obtain the intraoperative fundus retina three-dimensional model.

[0066] Since a human eye image with a large field of view and high clarity can be taken before the operation, while only a two-dimensional microscopic image can be taken during the operation, and the human eye image taken before the operation is of the patient sitting, while the patient lies during the operation, and the insertion of surgical instruments into the interior of the eyeball will cause deformation of the eyeball, there are differences in the retinal structure between during and before the operation. Therefore, by obtaining the preoperative human eye OCT image and fundus scan image, and obtaining the intraoperative human eye microscopic image, a preoperative fundus retinal three-dimensional model is constructed based on the preoperative human eye OCT image and fundus scan image, and then the first transformation matrix is determined according to the vascular information of the fundus scan image and the human eye microscopic image, so as to transform the preoperative fundus retinal three-dimensional model with the first transformation matrix to obtain the intraoperative fundus retinal three-dimensional model. Therefore, the intraoperative fundus retinal three-dimensional model is constructed through the preoperative human eye OCT image, fundus scan image and intraoperative human eye microscopic image, so that the doctor can clearly understand the structure of the human eye during the operation based on the intraoperative fundus retinal three-dimensional model, so as to improve the level of eye disease treatment.

[0067] Among them, the fundus scan image is an ultra-wide-angle fundus scan image (Optomap image) with a 200-degree viewing angle range. The imaging range of the Optomap image is large, and the disease conditions around the equator of the eyeball can be observed. However, the Optomap image is a two-dimensional image, and it is difficult for doctors to correspond it to the spatial structure inside the eyeball. The human eye OCT image belongs to a three-dimensional image, and the human eye OCT image contains information in the depth direction, can obtain the three-dimensional information of the retina, and can clearly observe areas such as the optic disc and macula, and reflect the curvature of the retina. Therefore, a clear and complete preoperative fundus retinal three-dimensional model can be constructed through the Optomap image and the human eye OCT image. The human eye microscopic image is the intraoperative human eye image, and the human eye microscopic image can reflect the changes in the fundus during the operation in real time. However, the human eye microscopic image belongs to a two-dimensional image, and doctors cannot obtain the information in the depth direction of the human eye from the human eye microscopic image, thus affecting the doctor's perception of space. Therefore, the first transformation matrix is determined according to the human eye microscopic image and the Optomap image, so as to transform the preoperative fundus retinal three-dimensional model according to the first transformation matrix to obtain the intraoperative fundus retinal three-dimensional model. Therefore, the three-dimensional structure of the deformed fundus retina during the operation is obtained through image processing, and the doctor can master the curved structure of the eyeball during the operation based on the intraoperative fundus retinal three-dimensional model, so as to improve the doctor's comprehensive perception of the environment, depth, strength, etc. of the eyeball during the operation, thereby reducing the risk of the operation.

[0068] In some embodiments, referring to Figure 2 , step S200 includes but is not limited to the following steps:

[0069] S210. Convert the human eye OCT image into a human eye projection image;

[0070] S220. Construct a corresponding initial human eye model based on the human eye OCT image;

[0071] S230. Perform rigid registration and elastic registration on the human eye projection image and the fundus scan image to obtain a second transformation matrix;

[0072] S240. Combine the fundus scan image and the initial human eye model according to the second transformation matrix to obtain a preoperative three-dimensional fundus retina model.

[0073] Since the human eye OCT image is a volume data composed of multiple scan images, the volume data of the human eye OCT image is projected onto the XY plane to obtain a human eye projection image, that is, the three-dimensional human eye coordinate information is projected onto a two-dimensional plane to obtain a human eye projection image. Since the human eye OCT image is a volume data composed of scan images, a corresponding initial human eye model is constructed according to the human eye OCT image. By performing rigid registration and elastic registration on the preoperative human eye projection image and the fundus scan image to obtain a second transformation matrix, the fundus scan image and the initial human eye model can be combined according to the second transformation matrix to obtain a preoperative three-dimensional fundus retina model. Therefore, a preoperative three-dimensional fundus retina model is constructed through the preoperative human eye OCT image and the fundus scan image to obtain a preoperative three-dimensional fundus retina model that can not only reflect the curvature of the retina before surgery but also observe the disease conditions around the equator of the eyeball.

[0074] Among them, image registration is to find one or a series of spatial transformations between two images so that the spatial coordinates of the two images are consistent. Its purpose is to establish a one-to-one mapping relationship point by point between different image spaces. Image registration can be classified into two categories: rigid registration and non-rigid registration, and non-rigid registration is also called elastic registration. Among them, rigid registration includes translation and rotation transformation, and elastic registration includes affine transformation. Therefore, rigid registration and elastic registration are performed on the human eye projection image and the fundus scan image to make the spatial coordinates of the two images consistent, so as to establish a one-to-one mapping relationship point by point between the spaces of the human eye projection image and the fundus scan image to obtain a second transformation matrix, and the second transformation matrix is also the pixel position matrix corresponding to the human eye projection image and the fundus scan image after registration. Among them, an initial transformation matrix is obtained after rigid registration of the human eye projection image and the fundus scan image, and then elastic registration is performed on the human eye projection image and the fundus scan image to obtain a deformation field, and then the second transformation matrix is determined according to the initial transformation matrix and the deformation field. After determining the second transformation matrix, the fundus scan image and the initial human eye model are combined according to the second transformation matrix to obtain a preoperative three-dimensional fundus retina model. Therefore, the preoperative three-dimensional fundus retina model constructed according to the human eye projection image and the fundus scan image can not only reflect the curvature of the retina but also observe the disease conditions around the equator of the eyeball.

[0075] Refer toFigure 3 , in some embodiments, step S220 includes but is not limited to the following steps:

[0076] S221. Stratify the human eye OCT image to extract the corresponding three-dimensional coordinate information;

[0077] S222. Construct an initial human eye model based on the three-dimensional coordinate information.

[0078] Since the human eye OCT image is a volume data composed of many scanned images, the human eye OCT is stratified to obtain the three-dimensional coordinate information of each layer of the retina, and then an initial human eye model corresponding to the human eye OCT image is constructed based on the three-dimensional coordinate information of each layer of the retina, so as to construct an initial human eye model that can clearly observe the optic disc, macular area and reflect the curvature of the retina.

[0079] Refer to Figure 4 , in some embodiments, step S230 includes but is not limited to the following steps:

[0080] S231. Perform translational transformation registration, rotational transformation registration, and scaling transformation registration on the human eye projection image and the fundus scan image to obtain a second rigid transformation matrix and a second registered image;

[0081] S232. Perform elastic registration on the second registered image and the fundus scan image to obtain a second transformation matrix.

[0082] Among them, image registration mainly matches the human eye projection image and the fundus scan image, and rigid registration includes translational transformation registration, rotational transformation registration, and scaling transformation registration. By performing translational transformation registration, rotational transformation registration, and scaling transformation registration on the human eye projection image and the fundus scan image, a second rigid transformation matrix and a second registered image are obtained, and then elastic registration is performed based on the second registered image and the fundus scan image to obtain a second transformation matrix. Among them, the second rigid transformation matrix is obtained by rigid registration, and the deformation field is obtained by elastic registration. The second transformation matrix is obtained by applying the second rigid transformation matrix and the deformation field to the fundus scan image, so as to facilitate the conversion of the fundus scan image and the initial human eye model according to the second transformation matrix to obtain the three-dimensional model of the preoperative fundus retina.

[0083] In some embodiments, refer to Figure 5 , the first transformation matrix includes: a first elastic transformation matrix and a first rigid transformation matrix; step S300 includes:

[0084] S310. Perform rigid registration on the fundus scan image and the human eye microscopic image based on vascular information to obtain a first rigid transformation matrix and a first registered image;

[0085] S320. Determine the first elastic transformation matrix based on the first registered image and the human eye microscopic image using a deep learning algorithm.

[0086] Refer to Figure 6 , where step S400 includes but is not limited to the following steps:

[0087] S410. Transform the preoperative fundus retina three-dimensional model according to the first rigid transformation matrix and the first elastic transformation matrix to obtain the intraoperative fundus retina three-dimensional model.

[0088] Since the blood vessels, macula, and optic disc regions of the blood vessels in the preoperative fundus scan image and the intraoperative human eye microscopic image are relatively similar, a rigid registration is performed based on the blood vessel information of the fundus scan image and the human eye microscopic image to obtain the first rigid transformation matrix and the first registered image. And because there are large deformations between the preoperative and intraoperative images, the first registered image after rigid registration and the intraoperative human eye microscopic image are used based on a deep learning algorithm to obtain the first elastic transformation matrix. Then, the preoperative fundus retina three-dimensional model is transformed according to the first elastic transformation matrix and the first rigid transformation matrix to obtain the deformed intraoperative fundus retina three-dimensional model during the operation, so as to provide the doctor with effective spatial perception. Then the doctor can perform surgical operations based on the constructed intraoperative fundus retina three-dimensional model, thereby reducing the risk of ophthalmic surgery.

[0089] Refer to Figure 7 , in some embodiments, step S222 includes but is not limited to the following steps:

[0090] S2221. Perform fitting based on the three-dimensional coordinate information using a preset least squares method to obtain an initial human eye model.

[0091] By performing hierarchical processing on the human eye OCT image to obtain three-dimensional coordinate information, multiple three-dimensional coordinate information is subjected to retrograde three-dimensional fitting to obtain an initial human eye model, and the least squares method is used for fitting to obtain an initial human eye model corresponding to the human eye OCT image.

[0092] Next, refer to Figures 1 to 7 A specific embodiment is used to describe in detail the method for constructing a three-dimensional human eye model according to the embodiments of the present invention. It should be understood that the following description is only an exemplary illustration and not a specific limitation of the invention.

[0093] Obtain the preoperative human eye OCT image, fundus scan image, and intraoperative human eye microscopic image. Project the volume data of the human eye OCT image onto the XY plane to obtain the human eye projection image. Perform layer-by-layer processing on the human eye OCT to obtain the three-dimensional coordinate information of each layer of the retina, and then construct an initial human eye model corresponding to the human eye OCT image based on the three-dimensional coordinate information of each layer of the retina. Perform translational transformation registration, rotational transformation registration, and scaling transformation registration on the human eye projection image and the fundus scan image to obtain the second rigid transformation matrix and the second registered image, and then perform elastic registration based on the second registered image and the fundus scan image to obtain the second transformation matrix. Combine the fundus scan image and the initial human eye model according to the second transformation matrix to obtain the preoperative three-dimensional fundus retina model. Perform rigid registration based on the vascular information of the fundus scan image and the human eye microscopic image to obtain the first rigid transformation matrix and the first registered image. Since there will be significant deformations between the preoperative and intraoperative images. Therefore, obtain the first elastic transformation matrix by using the first registered image after rigid registration and the intraoperative human eye microscopic image based on the deep learning algorithm. Then, the preoperative three-dimensional fundus retina model is transformed according to the first elastic transformation matrix and the first rigid transformation matrix to obtain the intraoperative deformed three-dimensional fundus retina model during the operation. The doctor can perform surgical operations based on the constructed intraoperative three-dimensional fundus retina model to improve the spatial perception during the operation, thereby reducing the risk of ophthalmic surgery.

[0094] In a second aspect, with reference to 8, the embodiment of the present invention also discloses a three-dimensional human eye model construction system, including: an acquisition module 100, a first processing module 200, a second processing module 300, and a conversion module 400; the acquisition module 100 is used to acquire the preoperative human eye OCT image, fundus scan image, and intraoperative human eye microscopic image; the first processing module 200 is used to determine the preoperative three-dimensional fundus retina model according to the three-dimensional coordinate information of the human eye OCT image and the fundus scan image; the second processing module 300 is used to perform registration based on the vascular information of the fundus scan image and the human eye microscopic image to determine the first transformation matrix; the conversion module 400 is used to transform the preoperative three-dimensional fundus retina model according to the first transformation matrix to obtain the intraoperative three-dimensional fundus retina model.

[0095] By acquiring preoperative human eye OCT images and fundus scanning images, and acquiring intraoperative microscopic images of the human eye, a preoperative three-dimensional model of the fundus retina is constructed based on the preoperative human eye OCT images and fundus scanning images. Then, a first transformation matrix is determined according to the blood vessel information of the fundus scanning images and the intraoperative microscopic images of the human eye, and the preoperative three-dimensional model of the fundus retina is transformed with the first transformation matrix to obtain an intraoperative three-dimensional model of the fundus retina. Therefore, an intraoperative three-dimensional model of the fundus retina is constructed through the preoperative human eye OCT images, fundus scanning images, and intraoperative microscopic images of the human eye, so that doctors can clearly understand the structure of the human eye during the operation, thereby improving the level of eye disease treatment.

[0096] In a third aspect, referring to Figure 9 , an embodiment of the present invention also discloses an electronic control device, including: at least one processor 500, and a memory 600 communicatively connected to the at least one processor 500; wherein, the memory 600 stores instructions executable by the at least one processor 500, and the instructions are executed by the at least one processor 500 to enable the at least one processor 500 to execute the three-dimensional human eye model construction method as described in the first aspect.

[0097] In a fourth aspect, an embodiment of the present invention also discloses a computer-readable storage medium storing computer-executable instructions for causing a computer to execute the three-dimensional human eye model construction method as described in the first aspect.

[0098] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0099] Those of ordinary skill in the art will appreciate that all or some of the steps and systems disclosed above can be implemented as software, firmware, hardware, and appropriate combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include a computer storage medium (or non-transitory medium) and a communication medium (or transitory medium). As is well known to those of ordinary skill in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and that can be accessed by a computer. In addition, it is well known to those of ordinary skill in the art that a communication medium typically includes computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and can include any information delivery medium.

[0100] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments, and various changes can be made without departing from the spirit of the present invention within the scope of knowledge possessed by those of ordinary skill in the art. In addition, the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict.

Claims

1. A method for constructing a three-dimensional human eye model, characterized in that, Including: Obtaining a preoperative human eye OCT image, a fundus scan image, and an intraoperative microscopic image of the human eye; Determining a preoperative fundus retina three-dimensional model based on the three-dimensional coordinate information of the human eye OCT image and the fundus scan image; Performing registration based on the vascular information of the fundus scan image and the intraoperative microscopic image of the human eye to determine a first transformation matrix; Converting the preoperative fundus retina three-dimensional model according to the first transformation matrix to obtain an intraoperative fundus retina three-dimensional model.

2. The three-dimensional human eye model construction method according to claim 1, wherein The step of determining a preoperative fundus retina three-dimensional model based on the three-dimensional coordinate information of the human eye OCT image and the fundus scan image includes: Converting the human eye OCT image into a human eye projection image; Constructing a corresponding initial human eye model based on the human eye OCT image; Performing rigid registration and elastic registration on the human eye projection image and the fundus scan image to obtain a second transformation matrix; Combining the fundus scan image and the initial human eye model according to the second transformation matrix to obtain the preoperative fundus retina three-dimensional model.

3. The three-dimensional human eye model construction method according to claim 2, characterized in that The step of constructing a corresponding initial human eye model based on the human eye OCT image includes: Layering the human eye OCT image to extract corresponding three-dimensional coordinate information; Constructing the initial human eye model according to the three-dimensional coordinate information.

4. The three-dimensional human eye model construction method according to claim 2, wherein The step of performing rigid registration and elastic registration on the human eye projection image and the fundus scan image to obtain a second transformation matrix includes: Performing translational transformation registration, rotational transformation registration, and scaling transformation registration on the human eye projection image and the fundus scan image to obtain a second rigid transformation matrix and a second registered image; Performing elastic registration on the second registered image and the fundus scan image to obtain the second transformation matrix.

5. The three-dimensional human eye model construction method according to claim 1, characterized in that, The first transformation matrix includes: a first elastic transformation matrix and a first rigid transformation matrix; the step of performing registration based on the vascular information of the fundus scan image and the intraoperative microscopic image of the human eye to determine a first transformation matrix includes: Performing rigid registration on the fundus scan image and the intraoperative microscopic image of the human eye based on vascular information to obtain a first rigid transformation matrix and a first registered image; Determining a first elastic change matrix for the first registered image and the intraoperative microscopic image of the human eye based on a deep learning algorithm.

6. The three-dimensional human eye model construction method according to claim 5, wherein, The step of converting the preoperative fundus retina three-dimensional model according to the first transformation matrix to obtain an intraoperative fundus retina three-dimensional model includes: Converting the preoperative fundus retina three-dimensional model according to the first rigid transformation matrix and the first elastic change matrix to obtain the intraoperative fundus retina three-dimensional model.

7. The three-dimensional human eye model construction method according to claim 3, characterized in that The step of constructing the initial human eye model according to the three-dimensional coordinate information includes: Performing fitting based on the three-dimensional coordinate information using a preset least squares method to obtain the initial human eye model.

8. A three-dimensional human eye model construction system, characterized in that Including: An acquisition module for acquiring a preoperative human eye OCT image, a fundus scan image, and an intraoperative microscopic image of the human eye; A first processing module for determining a preoperative fundus retina three-dimensional model based on the three-dimensional coordinate information of the human eye OCT image and the fundus scan image; A second processing module, configured to perform registration based on the vascular information of the fundus scan image and the human eye microscopic image to determine a first transformation matrix; A conversion module, configured to convert the preoperative three-dimensional fundus retina model according to the first transformation matrix to obtain an intraoperative three-dimensional fundus retina model.

9. An electronic control device, characterized in that, Comprising: At least one processor, and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to execute the three-dimensional human eye model construction method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing a computer to execute the three-dimensional human eye model construction method according to any one of claims 1 to 7.

Images