ICL implantation operation monitoring method and operation microscope system

Abstract

The invention discloses an ICL implantation operation monitoring method and an operation microscope system. The method comprises the following steps: acquiring a real-time eye image; based on the eye image, identifying incision information of the surgical incision knife and incision knife position information; carrying out OCT scanning on a specific position based on the incision information and / or the incision knife position information, and obtaining an OCT image; identifying and segmenting the eye tissue and the incision knife based on the OCT image, and determining at least one kind of relative position relation information of the incision knife and the eye tissue based on the obtained segmentation result; and when the relative position relation information exceeds a preset parameter range, sending adjustment prompt information. The OCT image is obtained based on the automatically recognized incision information and / or the incision knife position information, manual adjustment is not needed, the adjustment prompt information is provided in time, unnecessary operation of a doctor in the operation process is reduced, the operation risk is reduced, and the operation safety and the operation efficiency are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to an ICL implant surgery monitoring method and a surgical microscope system. Background Art

[0002] Currently, ICL (Implantable Collamer Lens) implantation surgery is a mainstream refractive and myopia correction procedure due to its effective results, quick processing, and lack of side effects. During ICL implantation or replacement, an incision is made in the cornea to create a channel for the subsequent syringe to insert the lens. Therefore, during this procedure, the anterior segment of the eye must be observed to assess and analyze the corneal tissue condition.

[0003] In related technologies, during ICL implantation surgery, the incision status is judged by the doctor's experience and the OCT (Optical Coherence Tomography) system. Among them, the OCT system relies on manual adjustment, which increases the doctor's cognitive burden and increases the intraoperative risk, thereby reducing the safety and efficiency of the surgery. Summary of the Invention

[0004] The present invention provides an ICL implant surgery monitoring method and a surgical microscope system to at least solve the technical problems of high intraoperative risk, low surgical safety and low surgical efficiency in related technologies.

[0005] A first embodiment of the present invention provides a method for monitoring ICL implant surgery, comprising:

[0006] Acquire real-time eye images;

[0007] Based on the eye image, identifying incision information and position information of the surgical incision knife;

[0008] Performing an OCT scan at a specific location based on the incision information and / or the incision knife position information and acquiring an OCT image;

[0009] Identifying and segmenting the eye tissue and the incision knife based on the OCT image, and determining at least one piece of relative positional relationship information between the incision knife and the eye tissue based on the obtained segmentation result, wherein the at least one piece of relative positional relationship information includes a first relative distance between the incision knife and the lower border of the cornea, a second relative distance between the incision knife and the lens, and an angle between the incision knife and a horizontal line;

[0010] When the relative position relationship information exceeds a preset parameter range, an adjustment prompt message is issued.

[0011] The ICL implant surgery monitoring method according to the embodiment of the present invention may also have the following additional technical features:

[0012] In one embodiment of the present invention, the incision knife position information includes a rotation angle of the incision knife in a rectangular coordinate system; performing an OCT scan at a specific position and acquiring an OCT image based on the incision information and / or the incision knife position information includes:

[0013] An OCT scan is performed within a first preset imaging depth range and a preset scanning width range with the incision information as the center and / or along the rotation angle direction of the incision knife in a rectangular coordinate system to obtain an OCT image corresponding to the cross-sectional B-scan data.

[0014] In one embodiment of the present invention, the identification and segmentation of eye tissue and incision knife based on the OCT image includes: identifying and segmenting the eye tissue and incision knife in the OCT image through a first convolutional neural network to obtain a segmentation result including the tissue boundary of the cornea and lens and the incision knife.

[0015] In one embodiment of the present invention, determining at least one relative position relationship information between the incision knife and the eye tissue based on the obtained segmentation result includes:

[0016] Determine the maximum distance between the lower border of the cornea and the horizontal line in the vertical direction as the first relative distance between the incision knife and the lower border of the cornea;

[0017] The maximum distance between the horizontal line and the upper border of the lens in the vertical direction is determined as the second relative distance between the incision knife and the lens;

[0018] The inclination angle of the incision knife on the horizontal line is determined as the angle between the incision knife and the horizontal line.

[0019] In one embodiment of the present invention, it is characterized in that the preset parameter range includes at least one of the following:

[0020] The first preset parameter range corresponding to the first relative distance is 2.7 mm to 3.5 mm;

[0021] The second preset parameter range corresponding to the second relative distance is 1.0 mm to 2.0 mm;

[0022] The third preset parameter range corresponding to the included angle between the incision knife and the horizontal line is 45° to 60°.

[0023] In one embodiment of the present invention, the method further comprises:

[0024] Identify the corresponding surgical stage based on real-time eye images;

[0025] The collection of the incision information and the incision knife position information is started based on the surgical stage, and an OCT scan is performed on the specific position.

[0026] In one embodiment of the present invention, the surgical stage includes before incision, during incision, and after incision; and the starting of the collection of the incision information and the incision knife position information based on the surgical stage, and performing OCT scanning on the specific position, includes:

[0027] The surgical stage information is received, the collection of the incision information and the incision knife position information is started, and an OCT scan is performed on the specific position.

[0028] In one embodiment of the present invention, the method further comprises:

[0029] Relative position setting information is superimposed and displayed on the eye image according to the surgical stage and the relative position relationship information, where the relative position setting information is a numerical value or an indication of the relative position relationship information.

[0030] In one embodiment of the present invention, the sending of the adjustment prompt information is achieved by at least one of the following methods:

[0031] Indicating that the relative position relationship information exceeds the preset parameter range by superimposing a change in the displayed relative position setting information;

[0032] The relative position relationship information is indicated to be out of the preset parameter range by means of a sound warning.

[0033] A second embodiment of the present invention provides a surgical microscope system that can be used in ICL implantation surgery, including:

[0034] OCT scanning module, used for acquiring OCT images;

[0035] An eye image acquisition module, used to acquire images of the surgical field of view of a surgical microscope;

[0036] One or more control computing units, used for performing image processing of eye images according to any of the methods described above, and / or calculating at least one relative position relationship information, and / or for identifying eye tissue areas and boundaries, and / or determining the surgical stage based on the eye image.

[0037] The present disclosure also provides an electronic device, including:

[0038] memory for storing computer programs;

[0039] A processor is used to implement the steps of any ICL implant surgery monitoring method provided in the embodiments of the present disclosure when executing a computer program.

[0040] An embodiment of the present disclosure further provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of any ICL implantation surgery monitoring method provided in the embodiment of the present disclosure are implemented.

[0041] The embodiments of the present disclosure also provide a computer program product, including a computer program, which, when executed by a processor, implements the steps of any ICL implant surgery monitoring method provided in the embodiments of the present disclosure.

[0042] The technical solutions provided by the embodiments of the present invention bring at least the following beneficial effects:

[0043] Acquire a real-time eye image; based on the eye image, identify the incision information and incision position information of the surgical incision knife; perform an OCT scan at a specific location based on the incision information and / or the incision knife position information and acquire an OCT image; identify and segment the eye tissue and the incision knife based on the OCT image, and determine at least one relative position relationship information between the incision knife and the eye tissue based on the obtained segmentation result, wherein the at least one relative position relationship information includes a first relative distance between the incision knife and the lower border of the cornea, a second relative distance between the incision knife and the lens, and the angle between the incision knife and the horizontal line; when the relative position relationship information exceeds a preset parameter range, issue an adjustment prompt message. The present invention can perform an OCT scan based on the automatically identified incision information and incision knife position information, determine at least one relative position relationship information between the incision knife and the eye tissue based on the obtained OCT image, and issue an adjustment prompt message when the relative position relationship information exceeds the preset parameter range, eliminating the need for manual adjustment and providing the adjustment prompt message in a timely manner, thereby reducing unnecessary operations by the doctor during the operation, reducing surgical risks, and improving surgical safety and efficiency.

[0044] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0046] Figure 1 1 is a flow chart of a method for monitoring ICL implant surgery according to one embodiment of the present invention;

[0047] Figure 2is a schematic diagram of incision information according to an embodiment of the present invention;

[0048] Figure 3 is a schematic diagram of a segmentation result according to an embodiment of the present invention;

[0049] Figure 4 1 is a flow chart of a method for monitoring ICL implant surgery according to one embodiment of the present invention;

[0050] Figure 5 1 is a flow chart of a method for monitoring ICL implant surgery according to one embodiment of the present invention;

[0051] Figure 6 1 is a flow chart of a method for monitoring ICL implant surgery according to one embodiment of the present invention;

[0052] Figure 7 1 is a flow chart of a method for monitoring ICL implant surgery according to one embodiment of the present invention;

[0053] Figure 8 1 is a flow chart of a method for monitoring ICL implant surgery according to one embodiment of the present invention;

[0054] Figure 9 FIG. 1 is a schematic structural diagram of an ICL implant surgery monitoring system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0055] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments of the present invention can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0056] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0057] The following describes an ICL implant surgery monitoring method and a surgical microscope system according to an embodiment of the present invention with reference to the accompanying drawings.

[0058] Figure 1 FIG. 1 is a flow chart of a method for monitoring ICL implant surgery according to an embodiment of the present invention. Figure 1 As shown, the method may include the following steps:

[0059] Step 101: Acquire a real-time eye image.

[0060] In one embodiment of the present invention, the above-mentioned eye image may be an image formed through the surgical field of a surgical microscope, that is, a front view of the eye.

[0061] Step 102: Identify the incision information and the position information of the surgical incision knife based on the eye image.

[0062] In one embodiment of the present invention, after acquiring the eye image through the above steps, the incision information and the position information of the surgical incision knife can be identified based on the eye image.

[0063] In one embodiment of the present invention, the method for identifying surgical knife incision information and knife position information based on an eye image may include: identifying surgical knife incision information and knife position information based on the eye image using a target convolutional neural network. In one embodiment of the present invention, the target convolutional neural network is obtained through training.

[0064] Furthermore, in one embodiment of the present invention, the above-mentioned incision knife position information can describe the relative position of the incision knife in the field of view (such as Figure 2 As shown), and described by an angle matrix box (θ, x1, y1, w1, h1), where θ is the rotation angle of the incision knife in the rectangular coordinate system, (x1, y1) is the center position coordinate of the rectangular box where the incision knife is located, and (w1, h1) is the width and height of the rectangular box where the incision knife is located; the above incision information can describe the relative position of the incision in the field of view (such as Figure 2 As shown), and described in the form of a rectangular box (x2, y2, w2, h2), where (x2, y2) is the coordinate of the center position of the rectangular box where the incision is located, and (w2, h2) is the width and height of the rectangular box where the incision is located.

[0065] Step 103: Perform an OCT scan at a specific position based on the incision information and / or the incision knife position information and acquire an OCT image.

[0066] In one embodiment of the present invention, after obtaining the incision information and incision knife position information through the above steps, an OCT scan can be performed at a specific position based on the incision information and / or incision knife position information and an OCT image can be obtained, thereby avoiding the need for the doctor to manually adjust the OCT scan during surgery, and having higher accuracy and real-time performance.

[0067] In one embodiment of the present invention, OCT scanning can be performed by emitting low-coherence infrared light (invisible light) with a wavelength of 800-1300nm from a light source and projecting it onto a spectroscope to be divided into two beams: one beam is reflected back as a reference light, and the other beam is projected onto the measured object (eye tissue) as a measurement light. The light is reflected by tissues at different depths and combined with the reference light to form an interference signal, which is converted into depth information through Fourier transform. By measuring optical echoes at different depths (axial z direction), a one-dimensional depth scan (Ascan) is obtained; and by moving the scanning light beam on the surface of the measured sample (lateral xy direction), a two-dimensional cross-section (Bscan) is obtained.

[0068] Furthermore, in one embodiment of the present invention, a method for performing OCT scanning at a specific position and acquiring an OCT image based on incision information and / or incision knife position information may include: performing an OCT scan within a first preset imaging depth range and a preset scanning width range with the incision information as the center and / or along the rotation angle direction of the incision knife in a rectangular coordinate system to obtain an OCT image corresponding to the cross-sectional B-scan data.

[0069] In one embodiment of the present invention, the above-mentioned first preset imaging depth range can be (2mm, 6mm), so that the cornea, anterior chamber and lens can be imaged; the above-mentioned preset scanning width range can be (6mm, 18mm), so that the cornea, chamber angle and entire anterior segment can be scanned.

[0070] Furthermore, in one embodiment of the present invention, the imaging resolution of the OCT scan based on the incision information and / or incision knife position information can be 5-10 μm in the axial direction (depth z direction) and 10-30 μm in the lateral direction (xy direction).

[0071] Step 104 : Identify and segment the eye tissue and the incision knife based on the OCT image, and determine at least one relative position relationship information of the incision knife and the eye tissue based on the obtained segmentation result.

[0072] In one embodiment of the present invention, after obtaining the OCT image through the above steps, the eye tissue and the incision knife can be identified and segmented based on the OCT image, and at least one relative position relationship information between the incision knife and the eye tissue can be determined based on the obtained segmentation result. Therefore, it can be determined based on the at least one relative position relationship information whether to issue an adjustment prompt information, so as to provide timely prompt warnings during the operation.

[0073] In one embodiment of the present invention, the method for identifying and segmenting ocular tissue and a keratin blade based on an OCT image may include: identifying and segmenting the ocular tissue and the keratin blade in the OCT image using a first convolutional neural network, thereby obtaining a segmentation result including the tissue boundaries of the cornea and lens, and the keratin blade. In one embodiment of the present invention, the first convolutional neural network is trained.

[0074] And, in one embodiment of the present invention, Figure 3 A schematic diagram of a segmentation result proposed in an embodiment of the present invention is shown in FIG. Figure 3 As shown, the segmentation result includes the corneal boundary, iris, incision knife, and lens boundary. The horizontal dashed line below the incision knife indicates the horizontal line where the incision knife is located. The horizontal line corresponds to the long axis of the eyeball. Those skilled in the art generally describe the shape of the eyeball as an ellipse, but this method is not limited to describing the horizontal line of the eyeball.

[0075] Furthermore, in one embodiment of the present invention, the at least one relative position relationship information may include a first relative distance between the incision knife and the lower border of the cornea, a second relative distance between the incision knife and the lens, and an angle between the incision knife and the horizontal line.

[0076] Step 105: When the relative position relationship information exceeds a preset parameter range, an adjustment prompt message is issued.

[0077] In one embodiment of the present invention, after the relative position relationship information is obtained through the above steps, an adjustment prompt message may be issued when the relative position relationship information exceeds a preset parameter range, so as to provide a prompt warning in a timely manner during the operation.

[0078] In one embodiment of the present invention, the preset parameter range includes at least one of the following:

[0079] The first preset parameter range corresponding to the first relative distance is 2.7 mm to 3.5 mm;

[0080] The second preset parameter range corresponding to the second relative distance is 1.0 mm to 2.0 mm;

[0081] The third preset parameter range corresponding to the angle between the incision knife and the horizontal line is 45° to 60°.

[0082] In addition, in one embodiment of the present invention, when the first relative distance between the incision knife and the lower border of the cornea exceeds the above-mentioned first preset parameter range, it is determined that the anterior chamber depth is shallow and the surgical space is tight, which may damage the eye tissue; when the second relative distance between the incision knife and the lens exceeds the above-mentioned second preset parameter range, and when the second relative distance is less than 1.0 mm, it is determined that the safety distance between the incision knife and the lens is small, which may damage the eye tissue; when the angle between the incision knife and the horizontal line exceeds the above-mentioned third preset parameter range, and when the angle is greater than 60°, the corneal endothelium may be damaged.

[0083] In one embodiment of the present invention, when the relative position relationship information exceeds the preset parameter range, a sound warning can be used to indicate that the relative position relationship information exceeds the preset parameter range. Specifically, a voice broadcast of "the relative position relationship information exceeds the preset parameter range and the specific exceeded value" can be issued, or a specific prompt sound such as a beep can be issued to prompt that the relative position relationship information exceeds the preset parameter range, so that a warning can be issued in time during the operation to avoid damage to the eye tissue.

[0084] The ICL implant surgery monitoring method of an embodiment of the present invention acquires a real-time eye image; identifies incision information and incision position information of a surgical scalpel based on the eye image; performs an optical coherence tomography (OCT) scan at a specific location and acquires an OCT image based on the incision information and / or the incision position information; identifies and segments the eye tissue and the scalpel based on the OCT image, and determines at least one relative position relationship information between the scalpel and the eye tissue based on the segmentation result, wherein the at least one relative position relationship information includes a first relative distance between the scalpel and the lower border of the cornea, a second relative distance between the scalpel and the lens, and an angle between the scalpel and the horizontal line; and issues an adjustment prompt message when the relative position relationship information exceeds a preset parameter range. The present invention can perform an OCT scan based on the automatically identified incision information and scalpel position information, and determine at least one relative position relationship information between the scalpel and the eye tissue based on the obtained OCT image, and issues an adjustment prompt message when the relative position relationship information exceeds the preset parameter range, eliminating the need for manual adjustment and providing the adjustment prompt message in a timely manner, thereby reducing unnecessary operations by the doctor during the surgery, reducing surgical risks, and improving surgical safety and efficiency.

[0085] In one embodiment of the present invention, as a detailed description of step 104, as shown in FIG. Figure 4 As shown, it may also include:

[0086] Step 201: The maximum distance between the lower border of the cornea and the horizontal line in the vertical direction is determined as the first relative distance between the incision knife and the lower border of the cornea.

[0087] For example, in one embodiment of the present invention, the first relative distance between the incision knife and the lower border of the cornea is Figure 3 H1 in.

[0088] Step 202: The maximum distance between the horizontal line and the upper border of the lens in the vertical direction is determined as the second relative distance between the incision knife and the lens.

[0089] For example, in one embodiment of the present invention, the second relative distance between the incision knife and the lens is Figure 3 H2 in.

[0090] Step 203: determine the inclination angle of the cutting knife on the horizontal line as the angle between the cutting knife and the horizontal line.

[0091] In one embodiment of the present invention, the incision knife needs to be tilted to enter the cornea to form a trapezoidal incision, which helps to seal and heal after surgery. For example, in one embodiment of the present invention, the angle between the incision knife and the horizontal line is Figure 3 The angle a in.

[0092] Among them, in one embodiment of the present invention, after determining at least one relative position relationship information through the above steps, at least one relative position relationship information can be used to determine whether to issue an adjustment prompt information, so that the adjustment prompt information can be provided in time to provide prompt warnings during the operation process.

[0093] In one embodiment of the present invention, Figure 5 As shown, the above method may further include:

[0094] Step 301 : Identify the corresponding surgical stage based on the real-time eye image.

[0095] In one embodiment of the present invention, the above-mentioned method for identifying the corresponding surgical stage based on real-time eye images may include: determining the probability value of the eye image belonging to each category through a second convolutional neural network, and determining the category with the highest confidence among the probability values ​​of each category as the surgical stage corresponding to the eye image.

[0096] In one embodiment of the present invention, the second convolutional neural network may be obtained by training the initial convolutional neural network. In one embodiment of the present invention, the training method of the second convolutional neural network may include the following steps:

[0097] Step 1: obtaining a first training data set consisting of multiple surgical video clips;

[0098] Step 2: annotate the first training data set frame by frame based on a sampling rate of a preset threshold to obtain a corresponding second training data set;

[0099] Step 3: Train the initial convolutional neural network based on the second training data set to obtain a second convolutional neural network.

[0100] In one embodiment of the present invention, a large-scale video dataset (such as AVA ActionsDataset) can be used to obtain multiple (such as 200-400) video clips of a preset duration (such as 15 minutes) to determine the first training dataset.

[0101] Furthermore, in one embodiment of the present invention, the preset threshold can be set as needed. For example, the preset threshold is 1 FPS (frames per second). In one embodiment of the present invention, the actual operation of the doctor and the different stages of the incision in the first training data set can be annotated to obtain the corresponding second training data set.

[0102] Furthermore, in one embodiment of the present invention, the cross entropy loss function can be used in the above training process to measure the gap between the predicted surgical stage of the initial convolutional neural network and the true label, thereby optimizing the initial convolutional neural network to obtain the second convolutional neural network.

[0103] Furthermore, in one embodiment of the present invention, after obtaining the second convolutional neural network through the above steps, the second convolutional neural network can be deployed and run on a GPU (Graphics Processing Unit) to ensure real-time operation.

[0104] In one embodiment of the present invention, the surgical stages may include pre-incision, mid-incision, and post-incision. In one embodiment of the present invention, mid-incision may begin when the scalpel is inserted into the corneal edge and ends when the scalpel is within a specific area of ​​the eye, remains stationary for a preset time, or is withdrawn to the outside of the cornea. The specific area and preset time can be set as needed.

[0105] In one embodiment of the present invention, the incision starts when the knife blade cuts into the cornea. However, during the operation, the insertion or incision process of the knife blade is very rapid. If the judgment point is the time when the knife blade cuts into the cornea, it may cause a delay in recognition, and the incision blade information cannot be obtained in time, resulting in a lack of timely warning. At the same time, some sensitive operations before the incision cannot be identified. Based on this, the second convolutional neural network described above can be combined with the annotated data of the doctor's actual operation needs to identify effective information before the incision, thereby more reasonably dividing the surgical stages and providing timely and reliable auxiliary warning information for different stages.

[0106] Specifically, in one embodiment of the present invention, a second convolutional neural network can be used to determine the physician's usual starting point for incision, thereby effectively positioning and tracking the instrument or related tissue in advance. For example, the instrument's preparatory posture in a relatively short time period before the incision knife is inserted into the eyeball can be identified.

[0107] Step 302 : Start collecting incision information and incision knife position information based on the surgical stage, and perform OCT scanning on a specific position.

[0108] In one embodiment of the present invention, the method for enabling the collection of incision information and incision blade position information based on the surgical stage and performing an OCT scan of a specific location may include: receiving surgical stage information, enabling the collection of incision information and incision blade position information, and performing an OCT scan of the specific location. In one embodiment of the present invention, the collection of incision information and incision blade position information may be enabled, and the OCT scan may be performed on the specific location, when the surgical stage information includes the incision.

[0109] Furthermore, in one embodiment of the present invention, if the surgical stage is after incision, an OCT scan can be performed based on the incision information, the local corneal area scanning range, the second preset imaging depth range and the preset number of sampling voxel points to obtain a corresponding OCT image.

[0110] Specifically, in one embodiment of the present invention, if the surgical stage is after the incision, then after the incision is completed, the incision position information can be used to guide the OCT volume scan in the incision area to obtain the corresponding OCT image, so that the corneal incision can be observed based on the OCT image during the operation, so that the doctor can evaluate and adjust the surgical operation during the operation.

[0111] In one embodiment of the present invention, the scanning range of the above-mentioned local corneal area can be (3×3mm, 6×6mm); the second preset imaging depth range can be (2mm, 3mm); and the preset number of sampling voxel points can be 256×256-512×512.

[0112] Furthermore, in one embodiment of the present invention, after obtaining the second OCT image through the above steps, the doctor can evaluate the incision status based on the second OCT image to evaluate and adjust the surgical operation during the operation.

[0113] In one embodiment of the present invention, the corresponding surgical stage can be identified timely and accurately based on real-time eye images, and the collection of incision information and incision knife position information can be started based on the surgical stage, and OCT scanning can be performed on specific positions, so that effective information before incision can be identified, thereby more reasonably dividing the surgical stages and providing auxiliary warning information in a timely and reliable manner for different stages.

[0114] In one embodiment of the present invention, Figure 6 As shown, the above method may further include:

[0115] Step 401 : Identify the corresponding surgical stage based on the real-time eye image.

[0116] Step 402 : Start collecting incision information and incision knife position information based on the surgical stage, and perform OCT scanning on a specific position.

[0117] For a detailed description of steps 401 to 402 , reference may be made to the above embodiment, which will not be elaborated herein.

[0118] Step 403 : Based on the surgical stage and the relative position relationship information, relative position setting information is superimposed and displayed on the eye image. The relative position setting information is a numerical value or indication of the relative position relationship information.

[0119] In one embodiment of the present invention, the indication of the above-mentioned relative position relationship information may include: displaying information in the relative position relationship information that is greater than or less than the corresponding standard value according to the standard value, so as to facilitate further adjustment; and / or displaying a numerical value, line segment mark and area indicating the relative position relationship information.

[0120] Furthermore, in one embodiment of the present invention, after the relative position setting information is superimposed and displayed on the eye image through the above steps, changes in the superimposed relative position setting information can be used to indicate that the relative position relationship information exceeds the preset parameter range. Specifically, in one embodiment of the present invention, changes in the relative position setting information can be displayed by changing the numerical value of the displayed relative position information, changing the font size of the numerical value, changing the color of the numerical value, changing the color of the line segment marker, or by simultaneously displaying corresponding symbol information.

[0121] In one embodiment of the present invention, relative position setting information is superimposed and displayed on the eye image based on the surgical stage and relative position relationship information, so that the relative position relationship information can be intuitively observed during the operation, allowing the doctor to evaluate and adjust the surgical operation during the operation.

[0122] Based on the above description, the ICL implant surgery monitoring method in the embodiment is illustrated.

[0123] Figure 7: is a flow chart of the ICL implant surgery monitoring method in an embodiment of the present invention, which includes: acquiring real-time eye images; determining the surgical stage corresponding to the eye image through a second convolutional neural network; if the surgical stage is before incision, not starting OCT scanning; if the surgical stage is during incision, starting OCT scanning, and issuing adjustment prompt information for the ICL implant surgery based on the obtained OCT image; if the surgical stage is after incision, starting OCT scanning, and evaluating the incision status based on the obtained OCT image.

[0124] Based on the above description, the ICL implant surgery monitoring method in the embodiment is illustrated.

[0125] Figure 8 This is a flow chart of an ICL implant surgery monitoring method according to an embodiment of the present invention, which includes: acquiring a real-time eye image, and determining the surgical stage corresponding to the image through a second convolutional neural network; if the surgical stage is incision, determining the incision information and the incision knife position information through the eye image; performing an OCT scan based on the incision information and / or the incision knife position information to obtain Bscan scan data; determining a segmentation result based on the Bscan scan data; determining at least one relative position relationship information between the incision knife and the eye tissue based on the segmentation result; and issuing an adjustment prompt message when the relative position relationship information exceeds a preset parameter range.

[0126] Figure 9 FIG. 1 is a schematic structural diagram of a surgical microscope system according to an embodiment of the present invention. Figure 9 As shown, the surgical microscope system 900 may include:

[0127] OCT scanning unit 901, used for acquiring OCT images;

[0128] Eye image acquisition unit 902, used to acquire the surgical field image of the surgical microscope;

[0129] One or more control computing units 903 are used to perform image processing of eye images according to the methods shown in the aforementioned embodiments, and / or calculate at least one relative position relationship information, and / or identify eye tissue areas and boundaries, and / or determine the surgical stage based on the eye image.

[0130] In one embodiment of the present invention, the OCT scanning unit 901 is specifically configured to:

[0131] An OCT scan is performed within a first preset imaging depth range and a preset scanning width range with the incision information as the center and / or along the rotation angle direction of the incision knife in a rectangular coordinate system to obtain an OCT image corresponding to the cross-sectional B-scan data.

[0132] In one embodiment of the present invention, when the control calculation unit 903 is used to identify the eye tissue area and boundaries, it is specifically used to: identify and segment the eye tissue and incision knife in the OCT image through the first convolutional neural network, and obtain the segmentation results including the tissue boundaries of the cornea and lens and the incision knife.

[0133] In one embodiment of the present invention, when the control calculation unit 903 is used to calculate at least one relative position relationship information, it is specifically used to:

[0134] The maximum distance between the lower border of the cornea and the horizontal line in the vertical direction is determined as the first relative distance between the incision knife and the lower border of the cornea;

[0135] The maximum distance between the horizontal line and the upper border of the lens in the vertical direction is determined as the second relative distance between the incision knife and the lens;

[0136] The inclination angle of the incision knife on the horizontal line is determined as the angle between the incision knife and the horizontal line.

[0137] In one embodiment of the present invention, the above-mentioned preset parameter range includes at least one of the following:

[0138] The first preset parameter range corresponding to the first relative distance is 2.7 mm to 3.5 mm;

[0139] The second preset parameter range corresponding to the second relative distance is 1.0 mm to 2.0 mm;

[0140] The third preset parameter range corresponding to the angle between the incision knife and the horizontal line is 45° to 60°.

[0141] In one embodiment of the present invention, when the control calculation unit 903 is used to determine the surgical stage based on the eye image, it is specifically used to:

[0142] Identify the corresponding surgical stage based on real-time eye images;

[0143] The collection of the incision information and the incision knife position information is started based on the surgical stage, and an OCT scan is performed on a specific position.

[0144] In one embodiment of the present invention, when the control calculation unit 903 is used to determine the surgical stage based on the eye image, it is also used to:

[0145] Receive surgical stage information, start collecting incision information and incision knife position information, and perform OCT scanning on specific locations.

[0146] In one embodiment of the present invention, the surgical microscope system may further include an image display unit. After determining the at least one relative position relationship information, the image display unit is configured to:

[0147] According to the surgical stage and the relative position relationship information, the relative position setting information is superimposed and displayed on the eye image, and the relative position setting information is a numerical value or indication of the relative position relationship information.

[0148] In one embodiment of the present invention, the surgical microscope system may further include an information prompting unit, which is configured to send out adjustment prompt information in at least one of the following ways:

[0149] Indicating that the relative position relationship information exceeds the preset parameter range by superimposing the change of the relative position setting information displayed;

[0150] An audible warning is used to indicate that the relative position relationship information exceeds the preset parameter range.

[0151] It should be noted that the explanation of the aforementioned ICL implant surgery monitoring method embodiment is also applicable to the surgical microscope system of this embodiment and will not be repeated here.

[0152] In summary, the system provided by the embodiments of the present disclosure acquires a real-time eye image; identifies incision information and incision position information of a surgical incision knife based on the eye image; performs an OCT scan at a specific location and acquires an OCT image based on the incision information and / or the incision knife position information; identifies and segments the eye tissue and the incision knife based on the OCT image, and determines at least one relative position relationship information between the incision knife and the eye tissue based on the obtained segmentation result, wherein the at least one relative position relationship information includes a first relative distance between the incision knife and the lower border of the cornea, a second relative distance between the incision knife and the lens, and the angle between the incision knife and the horizontal line; and issues an adjustment prompt message when the relative position relationship information exceeds a preset parameter range. The present invention can perform an OCT scan based on the automatically identified incision information and incision knife position information, and determine at least one relative position relationship information between the incision knife and the eye tissue based on the obtained OCT image, and issue an adjustment prompt message when the relative position relationship information exceeds the preset parameter range, without relying on manual adjustment and providing the adjustment prompt message in a timely manner, thereby reducing unnecessary operations by the doctor during the surgery, reducing surgical risks, and improving surgical safety and efficiency.

[0153] The collection, storage, use, processing, transmission, provision and disclosure of user personal information involved in this disclosure are in compliance with relevant laws and regulations and do not violate public order and good morals.

[0154] It is important to note that personal information collected from users should be used for legitimate and reasonable purposes and should not be shared or sold beyond these legitimate uses. Furthermore, such collection / sharing should be conducted only after receiving the user's informed consent, including but not limited to notifying the user to read the user agreement / user notice and sign an agreement / authorization that includes the relevant user information before using the feature. Furthermore, any necessary steps must be taken to safeguard and secure access to such personal information and ensure that others with access to personal information comply with its privacy policy and procedures.

[0155] This disclosure contemplates providing implementations that allow users to selectively block the use or access of personal information data. Specifically, this disclosure contemplates providing hardware and / or software to prevent or block access to such personal information data. Risks can be minimized by limiting data collection and deleting data once it is no longer needed. Furthermore, where applicable, such personal information can be de-identified to protect user privacy.

[0156] In the descriptions of the aforementioned embodiments, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are mutually inconsistent.

[0157] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. Throughout the present disclosure, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0158] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present disclosure includes additional implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present disclosure belong.

[0159] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM) or flash memory, a fiber optic device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.

[0160] It should be understood that various parts of the present disclosure can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement the present invention: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0161] Those skilled in the art will appreciate that all or part of the steps in the method for implementing the above-mentioned embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0162] In addition, the functional units in the various embodiments of the present disclosure may be integrated into a single processing module, each unit may exist physically separately, or two or more units may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or in the form of software functional modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium.

[0163] The storage medium mentioned above may be a read-only memory, a magnetic disk, or an optical disk, etc. Although the embodiments of the present disclosure have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present disclosure. A person of ordinary skill in the art may make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present disclosure.

Claims

1. A method for monitoring ICL implant surgery, characterized in that: include: Acquire real-time eye images; Based on the eye image, identifying incision information and position information of the surgical incision knife; Performing an OCT scan at a specific location based on the incision information and / or the incision knife position information and acquiring an OCT image; Identifying and segmenting the eye tissue and the incision knife based on the OCT image, and determining at least one piece of relative positional relationship information between the incision knife and the eye tissue based on the obtained segmentation result, wherein the at least one piece of relative positional relationship information includes a first relative distance between the incision knife and the lower border of the cornea, a second relative distance between the incision knife and the lens, and an angle between the incision knife and a horizontal line; When the relative position relationship information exceeds a preset parameter range, an adjustment prompt message is issued.

2. The monitoring method according to claim 1, characterized in that: The incision knife position information includes a rotation angle of the incision knife in a rectangular coordinate system; performing OCT scanning at a specific position and acquiring an OCT image based on the incision information and / or the incision knife position information includes: An OCT scan is performed within a first preset imaging depth range and a preset scanning width range with the incision information as the center and / or along the rotation angle direction of the incision knife in a rectangular coordinate system to obtain an OCT image corresponding to the cross-sectional B-scan data.

3. The monitoring method according to claim 1, wherein: The identifying and segmenting of the eye tissue and the incision knife based on the OCT image includes: identifying and segmenting the eye tissue and the incision knife in the OCT image through a first convolutional neural network to obtain a segmentation result including the tissue boundaries of the cornea and lens and the incision knife.

4. The monitoring method according to claim 3, characterized in that: The determining of at least one relative position relationship information between the incision knife and the eye tissue based on the obtained segmentation result includes: Determine the maximum distance between the lower border of the cornea and the horizontal line in the vertical direction as the first relative distance between the incision knife and the lower border of the cornea; The maximum distance between the horizontal line and the upper border of the lens in the vertical direction is determined as the second relative distance between the incision knife and the lens; The inclination angle of the incision knife on the horizontal line is determined as the angle between the incision knife and the horizontal line.

5. The monitoring method according to claim 1, characterized in that: The preset parameter range includes at least one of the following: The first preset parameter range corresponding to the first relative distance is 2.7 mm to 3.5 mm; The second preset parameter range corresponding to the second relative distance is 1.0 mm to 2.0 mm; The third preset parameter range corresponding to the included angle between the incision knife and the horizontal line is 45° to 60°.

6. The monitoring method according to claim 1, characterized in that: The method further comprises: Identify the corresponding surgical stage based on real-time eye images; The collection of the incision information and the incision knife position information is started based on the surgical stage, and an OCT scan is performed on the specific position.

7. The monitoring method according to claim 6, characterized in that: The surgical stages include before incision, during incision, and after incision; and the acquisition of the incision information and the incision knife position information based on the surgical stages, and performing OCT scanning on the specific position, includes: The surgical stage information is received, the collection of the incision information and the incision knife position information is started, and an OCT scan is performed on the specific position.

8. The monitoring method according to claim 6, characterized in that: The method further comprises: Relative position setting information is superimposed and displayed on the eye image according to the surgical stage and the relative position relationship information, where the relative position setting information is a numerical value or an indication of the relative position relationship information.

9. The method according to claim 8, characterized in that The adjustment prompt information is sent out in at least one of the following ways: Indicating that the relative position relationship information exceeds the preset parameter range by superimposing a change in the displayed relative position setting information; The relative position relationship information is indicated to be out of the preset parameter range by means of a sound warning.

10. A surgical microscope system, which can be used in ICL implantation surgery, characterized in that: include: An OCT scanning unit, used for acquiring OCT images; An eye image acquisition unit, used for acquiring images of the surgical field of view of a surgical microscope; One or more control computing units, for performing image processing of eye images according to the method described in any one of claims 1 to 9, and / or calculating at least one relative position relationship information, and / or for identifying eye tissue areas and boundaries, and / or determining the surgical stage based on the eye image.

Images