System for determining an instrument endpoint in ophthalmic surgery

Abstract

This application relates to a system for determining the instrument endpoints during ophthalmic surgery. The system implements a method for determining the instrument endpoints during ophthalmic surgery, comprising: acquiring a first surgical field image captured by an image acquisition device; determining an initial position of the instrument endpoint based on the first surgical field image; determining a first region for performing an OCT scan on the surgical subject based on the initial position of the instrument endpoint; performing an OCT scan on the first region of the surgical subject to obtain first scan data; and determining the current target position of the instrument endpoint based on the first scan data. This method can rapidly locate the instrument endpoint while improving the positional accuracy of the instrument endpoint during ophthalmic surgery.

Description

Technical Field

[0001] This application relates to the field of imaging technology, and in particular to a system for determining the instrument endpoints in ophthalmic surgery. Background Technology

[0002] Currently, the main methods for determining the position of instrument endpoints in ophthalmic surgery include two categories: methods based on camera-captured images and methods based on OCT (Optical Coherence Tomography) scan data.

[0003] One approach involves capturing real-time images of the surgical field of view using a camera, and then using image recognition algorithms to determine the pixel coordinates of the instrument endpoints from these images, thereby estimating their spatial location. However, the camera's positioning accuracy is physically limited by pixel resolution. Under typical ophthalmic surgical imaging conditions, the actual spatial scale corresponding to a single pixel is usually on the order of tens of micrometers, making it difficult to consistently achieve the micrometer-level positioning accuracy required for ophthalmic surgical instrument endpoint navigation.

[0004] Secondly, an OCT scanner is used to perform an OCT scan on the surgical area to obtain scan data, from which the depth position information of the instrument tip is extracted. OCT scanners have micron-level axial resolution, which can meet the spatial accuracy requirements for instrument tip positioning in ophthalmic surgery. However, the imaging mechanism of OCT dictates that if a high-density scan of the entire surgical field is performed to ensure coverage of all possible areas where the instrument tip may be located, a single complete scan can take hundreds of milliseconds, resulting in an update frequency for the instrument tip position that cannot meet the time requirements during the surgical procedure.

[0005] Therefore, there is an urgent need to provide a determination scheme that can simultaneously meet the time and space requirements for instrument endpoint positioning in ophthalmic surgery. Summary of the Invention

[0006] Based on this, this application provides a system for determining the instrument endpoint in ophthalmic surgery, which can simultaneously meet the time and space requirements for instrument endpoint positioning in ophthalmic surgery, and improve the positional accuracy of the instrument endpoint in ophthalmic surgery while enabling rapid instrument endpoint positioning.

[0007] In a first aspect, this application provides a method for determining the instrument tip during ophthalmic surgery, the method comprising:

[0008] Acquire a first surgical field image captured by an image acquisition device, and determine the initial position of the instrument tip based on the first surgical field image;

[0009] Based on the initial position of the instrument tip, determine the first region for performing an OCT scan on the surgical subject;

[0010] An OCT scan was performed on the first region of the surgical subject to obtain the first scan data;

[0011] The current target position of the instrument endpoint is determined based on the first scan data.

[0012] In some embodiments, determining the initial position of the instrument tip based on the first surgical field image includes:

[0013] When the initial position of the instrument tip is determined for the first time, an OCT scan is performed on the area covering the surgical object to obtain initial scan data;

[0014] The initial position of the instrument endpoint is determined based on the initial scan data and the first surgical field image.

[0015] In some embodiments, determining a first region for performing an OCT scan on the surgical subject based on the initial position of the instrument tip includes:

[0016] Obtain the movement speed of the instrument's endpoint;

[0017] The first region for performing an OCT scan on the surgical subject is determined based on the initial position of the instrument tip and the moving speed of the instrument tip.

[0018] In some embodiments, determining a first region for performing an OCT scan on the surgical subject based on the initial position of the instrument tip includes:

[0019] Obtain the confidence level for determining the initial position of the instrument endpoint;

[0020] Based on the confidence level and the initial position of the instrument endpoint, the first region for performing an OCT scan on the surgical subject is determined.

[0021] In some embodiments, it also includes:

[0022] Based on the current target position of the instrument tip, determine the second region for performing an OCT scan on the surgical subject;

[0023] An OCT scan was performed on the second region of the surgical subject to obtain the second scan data;

[0024] The next target location of the instrument tip is determined based on the second scan data;

[0025] Update the current target position of the device endpoint based on the next target position of the device endpoint.

[0026] In some embodiments, determining the next target location of the instrument endpoint based on the second scan data includes:

[0027] If the instrument endpoint is not identified based on the second scan data, the second region of the OCT scan performed on the surgical subject is expanded to obtain a third region;

[0028] Perform an OCT scan on the third region to obtain the third scan data;

[0029] The next target location of the instrument endpoint is determined based on the third scan data.

[0030] In some embodiments, determining the next target location of the instrument endpoint based on third scan data includes:

[0031] If the instrument endpoint is not identified based on the third scan data, a second surgical field image captured by the image acquisition device is obtained, and the next initial position of the instrument endpoint is determined based on the second surgical field image.

[0032] Update the initial position of the instrument endpoint based on the next initial position;

[0033] Update the first region of the surgical subject for OCT scanning based on the updated initial position of the instrument endpoint;

[0034] An OCT scan was performed on the updated first region of the surgical subject to obtain the updated first scan data;

[0035] The next target location of the instrument tip is determined based on the updated first scan data.

[0036] In some embodiments, determining the current target location of the instrument endpoint based on the first scan data includes:

[0037] A signal strength set is determined based on the first scan data. The signal strength set includes multiple signal strength values. The position coordinates corresponding to the largest signal strength value in the signal strength set are determined.

[0038] When the maximum signal strength value in the signal strength set is greater than or equal to the preset strength threshold, the current target position of the instrument endpoint is determined according to the position coordinates corresponding to the maximum signal strength value.

[0039] When all signal strength values ​​in the signal strength set are less than the preset strength threshold, shadow detection is performed on the signal strength set to determine the position coordinates corresponding to the shadow. The current target position of the instrument endpoint is determined based on the position coordinates corresponding to the maximum signal strength value and the position coordinates corresponding to the shadow.

[0040] Secondly, this application provides a system for determining the instrument endpoint during ophthalmic surgery, the system comprising:

[0041] Image acquisition device, used to capture images of the first surgical field of view;

[0042] OCT scanning device, used to perform scanning;

[0043] The processor, connected to the image acquisition device and the OCT scanning device, is configured to perform the following operations: determine the initial position of the instrument tip based on the first surgical field image; determine the first region for performing OCT scanning on the surgical object based on the initial position of the instrument tip; control the OCT scanning device to perform OCT scanning on the first region to obtain first scan data; and determine the current target position of the instrument tip based on the first scan data.

[0044] In some embodiments, the system further includes a display device;

[0045] The processor is configured to perform the following operations: determine a second region for OCT scanning of the surgical subject based on the current target position of the instrument endpoint; perform an OCT scan on the second region of the surgical subject to obtain second scan data; and determine the next target position of the instrument endpoint based on the second scan data.

[0046] When the distance between the next target position of the instrument endpoint and the current target position of the instrument endpoint is greater than or equal to a preset threshold, the OCT image is determined based on the second scan data, and the next target position of the instrument endpoint is marked on the OCT image. The current target position of the instrument endpoint is updated based on the next target position of the instrument endpoint.

[0047] A display device, connected to a processor, is used to display the next target location of the marker in an OCT image.

[0048] In the technical solution provided in this application embodiment, after acquiring a first surgical field image captured by an image acquisition device, the initial position of the instrument tip is first determined based on the first surgical field image for coarse positioning. Then, the first area for OCT scanning is determined based on the initial position, so that the OCT scanning device only needs to perform targeted scanning on the first area. Based on the scanning data, the current target position of the instrument tip is determined for fine positioning. Compared with the solution of positioning the instrument tip based on images captured by a camera, OCT scanning can meet the spatial requirements for instrument tip positioning in ophthalmic surgery. Compared with the solution of positioning the instrument tip based on OCT scanning data of the entire surgical field, only the first area needs to be targeted for scanning, without performing high-density scanning of the entire surgical field, thus meeting the time requirements for instrument tip positioning in ophthalmic surgery. This embodiment of the application, through the cooperation of the image acquisition device and the OCT scanning device, and through the combination of coarse and fine positioning, can simultaneously meet the time and spatial requirements for instrument tip positioning in ophthalmic surgery, improving the positional accuracy of the instrument tip in ophthalmic surgery while rapidly positioning the instrument tip. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 A flowchart illustrating a method for determining the instrument endpoint in ophthalmic surgery, provided in some embodiments;

[0051] Figure 2 A flowchart illustrating a method for determining a first region for performing an OCT scan on a surgical subject based on the initial position of the instrument tip, as provided in the first embodiment;

[0052] Figure 3 A flowchart illustrating a method for determining a first region for performing an OCT scan on a surgical subject based on the initial position of the instrument tip, as provided in the second embodiment;

[0053] Figure 4 A schematic diagram showing the scanning range corresponding to each confidence level range provided in some embodiments;

[0054] Figure 5 A flowchart illustrating a method for updating the current target position as provided in some embodiments;

[0055] Figure 6 A flowchart illustrating the method for determining the next target position of the instrument tip based on second scan data provided in the first embodiment;

[0056] Figure 7 A flowchart illustrating the method for determining the next target position of the instrument tip based on second scan data, provided in the second embodiment;

[0057] Figure 8 A flowchart illustrating a method for determining the current target position of an instrument endpoint based on first scan data, provided for some embodiments;

[0058] Figure 9 A schematic diagram of the structure of a system for determining the instrument endpoint in ophthalmic surgery, provided for some embodiments. Detailed Implementation

[0059] The embodiments of the technical solution of this application will now be described in detail with reference to the accompanying drawings. These embodiments are only used to more clearly illustrate the technical solution of this application and are therefore merely examples, and should not be used to limit the scope of protection of this application.

[0060] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0061] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, "multiple groups" means two or more, and "each" means each of the multiple, unless otherwise explicitly defined.

[0062] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0063] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0064] Unless otherwise specified, the order of execution steps in the embodiments of this application is not limited. It should also be noted that any step in the embodiments of this application can be executed independently, that is, the execution of any step in the above embodiments can be performed without depending on the execution of other steps.

[0065] The OCT scanning device in this application embodiment can generate high-resolution (typically micrometer-level) images non-invasively and without damage by measuring the light signals reflected or scattered within biological tissues or materials. For example, the OCT scanning device can perform OCT scanning on a target area to obtain scan data of the target area. The scan data of the target area includes A-scan data at multiple scanning positions in the target area. The A-scan data at one scanning position includes signal intensity values ​​at different depth positions. Thus, the scan data includes a set of signal intensity values. The position of the instrument endpoint is determined based on the position of the maximum signal intensity value in the signal intensity set.

[0066] The method in this embodiment is applied to a system for determining the instrument endpoint in ophthalmic surgery. This system includes a processor, an image acquisition device, and an OCT scanning device, both of which are communicatively connected to the processor. In some embodiments, the image acquisition device and the OCT scanning device are optically coaxial. In some embodiments, the system may further include a microscope, allowing the operator to observe the surgical object during surgery. Exemplarily, the microscope, image acquisition device, and OCT scanning device are optically coaxial. In some embodiments, the processor's location is not limited; it can be integrated into any one of the image acquisition device, OCT scanning device, or surgical microscope, or it can be a separate component.

[0067] This application uses eye surgery as an example, and the surgical object is the eye. However, this application is not limited to this and is also applicable to other surgical scenarios.

[0068] Figure 1 A flowchart illustrating a method for determining the instrument tip during ophthalmic surgery, as provided in some embodiments, is shown below. Figure 1 As shown, the method includes:

[0069] S101. Acquire the first surgical field image captured by the image acquisition device, and determine the initial position of the instrument end point based on the first surgical field image.

[0070] Step S101 is either the step for determining the initial position of the instrument endpoint for the first time, or the step for determining the initial position of the instrument endpoint for a subsequent time.

[0071] The image acquisition device may include one or more two-dimensional image acquisition devices or one or more three-dimensional image acquisition devices, and this application embodiment does not limit this. This application embodiment uses a two-dimensional surgical field image as an example for illustration.

[0072] The image acquisition device can capture images of the surgical subject to obtain a first surgical field of view image. In some embodiments, the image acquisition device can capture images of the surgical subject in real time at a target frame rate.

[0073] For example, the image acquisition device may include a camera.

[0074] The camera's shooting range can be preset. The surgical field of view image refers to the image acquired by the image acquisition device during surgery within the preset shooting range. The preset shooting range covers the area where the surgical object is located; for example, the preset shooting range can be the area covering the surgical object. For example, acquiring the first surgical field of view image captured by the image acquisition device can include: acquiring the first surgical field of view image captured by the camera. For example, acquiring the first surgical field of view image captured by the image acquisition device can include: acquiring the original image output by the camera at a target frame rate; selecting a frame from multiple consecutive original frame images that meets preset conditions (e.g., the instrument tip is identified, image clarity is higher than a threshold, etc.) as the first surgical field of view image; or, acquiring the processed image obtained after preprocessing (e.g., noise reduction, enhancement, cropping, etc.) the current frame original frame image output by the camera, and using the processed image as the first surgical field of view image.

[0075] For example, determining the initial position of the instrument tip based on the first surgical field of view image includes: determining the pixel coordinates of the instrument tip in the first surgical field of view image; and determining the initial position based on the pixel coordinates. For example, the pixel coordinates of the instrument tip can be determined from the first surgical field of view image using an image recognition algorithm. For example, the pixel coordinates can be used as the initial position. Another example is to determine the coordinates of the instrument tip in a two-dimensional physical coordinate system based on the pixel coordinates, and then use these coordinates as the initial position. This solution does not restrict the image coordinate system corresponding to the initial position; the image coordinate system can be a camera coordinate system or a two-dimensional physical coordinate system. For example, determining the initial position in a camera coordinate system means establishing a two-dimensional coordinate system with the upper left corner of the first surgical field of view image as the origin, the horizontal direction to the right as the positive X-axis, and the vertical direction downwards as the positive Y-axis, using pixels as the unit. The pixel coordinates of the instrument tip in this coordinate system represent the initial position of the instrument tip. Alternatively, the initial position of the instrument tip can be represented by physical coordinates, that is, the physical position of the instrument tip on the plane where the surgical object is located can be used as the initial position. For example, a two-dimensional physical coordinate system is established on the plane where the surgical object is located, with the origin being a preset reference point of the surgical object (such as the corneal apex, pupil center, or the predetermined location of the surgical incision), and the two-dimensional coordinate system is established in millimeters or micrometers.

[0076] S102. Determine the first region for performing OCT scanning on the surgical subject based on the initial position of the instrument tip.

[0077] S102 can be executed by a processor. S102 can be implemented in various ways. For example, S102 may include: mapping the pixel coordinates of the initial position of the instrument tip to the scanning coordinate system of the OCT scanning device to obtain a corresponding position. For example, mapping the pixel coordinates of the initial position to the corresponding position on the scanning coordinate system is used as a first region; another example is using a region where the scanning range is a preset range and the corresponding position is the center of the scanning range as the first region; yet another example is obtaining a position of interest in a first surgical field of view image, mapping the position of interest and the initial position of the instrument tip to the scanning coordinate system of the OCT scanning device to obtain corresponding positions, using the center of the corresponding position of the position of interest as the center of the scanning range, and using the distance between the corresponding position of the position of interest and the corresponding position of the initial position as the radius of the scanning range to determine the first region for performing OCT scanning on the surgical object, so that the coverage of the first region includes both the position of interest and the initial position of the instrument tip.

[0078] In related technologies, the scanning position of an OCT system relies on the surgeon manually setting it via a foot pedal. If the eye moves, the fixed scanning position deviates from the target area, requiring the surgeon to manually readjust it in real time, severely distracting their attention during surgery. Therefore, in this embodiment, the location of interest is determined as the position of the eyeball. Using the eyeball as a scanning anchor point provides a spatial reference for subsequent instrument tracking. Even with eye movement, the effectiveness of the determined scanning area is improved, thereby enhancing the effectiveness of tracking the current target position of the instrument endpoint.

[0079] The embodiments of this application are not limited to this. S102 may have other implementations. For specific steps, please refer to the description below.

[0080] S103. Perform an OCT scan on the first region of the surgical subject to obtain the first scan data.

[0081] S103 can be executed by an OCT scanning device. When the processor determines any region in this embodiment (e.g., any one of the first to fourth regions), it can determine the region's location data. The location data can include at least one of the following: the location of the region's center, the region's length, the location of the region's endpoints, etc. The processor sends the region's location data to the OCT scanning device, enabling the OCT scanning device to perform an OCT scan on the region based on the location data. The OCT scanning device can send scan data to the processor. In this embodiment, the scan data can include A-scan data from multiple scan locations within the region. The A-scan data for one scan location includes the signal strength values ​​at different depths of that scan location. Thus, the scan data includes a set of signal strength values ​​from all scan locations within the scan region, i.e., a signal strength set.

[0082] S104. Determine the current target position of the instrument endpoint based on the first scan data.

[0083] Upon receiving the first scan data, the processor can determine whether the location of the device endpoint is identified from the first scan data. If the location of the device endpoint is identified from the first scan data, the identified location of the device endpoint is determined as the current target location of the device endpoint. If the location of the device endpoint is not identified from the first scan data, one or more of the following processing methods may be used: For example, the first region may be expanded, an OCT scan may be performed on the expanded first region to obtain updated first scan data, and the location of the device endpoint may be identified based on the updated first scan data. This method is suitable for situations where the displacement of the instrument tip is small. The instrument tip can be recaptured by appropriately expanding the scanning range. For example, if the movement speed of the instrument tip is detected to exceed a preset movement speed threshold, S101 can be re-executed to acquire a new surgical field image captured by the image acquisition device as an updated first surgical field image. Then, S102 to S104 can be re-executed in sequence to redetermine the initial position and first region of the instrument tip based on the updated first surgical field image to adapt to significant positional changes of the surgical instrument. For yet another example, if the position of the instrument tip is still not identified from the first scan data after the above-mentioned processing of expanding the scanning range of the first region or re-acquiring the first surgical field image, the first region can be expanded to cover all or most of the scannable range of the surgical object. For example, an OCT scan can be performed on the entire area of ​​the surgical object to finally determine the position of the instrument tip and ensure that the instrument tip can be identified during surgery.

[0084] For example, the first scan data can be volume data. The OCT scanning device performs scans along the X and Y directions within the first region. For instance, the X direction is the fast scan direction, where the X-mirror drives the probe beam to deflect along the arrangement direction of each A-scan within a single B-scan section; the Y direction is the slow scan direction, where the Y-mirror drives the probe beam to deflect between different B-scan sections. Thus, the volume data includes multiple B-scan sections arranged along the Y direction, each B-scan section includes multiple A-scans arranged along the X direction, and each A-scan includes signal intensity values ​​at different depths along the depth direction (Z direction), collectively constituting three-dimensional scan data covering the first region.

[0085] In this case, the current target position of the instrument endpoint is determined based on the first scan data, and one or more of the following representation methods are possible: For example, the current target position is represented by two-dimensional coordinates. First, the B-scan section where the instrument endpoint is located and the corresponding A-scan position along the X direction within the B-scan section are determined from the volume data. Then, the depth position coordinate z corresponding to the instrument endpoint is extracted from the depth direction of the A-scan to obtain the two-dimensional position (x, z), where x is the lateral position coordinate within the B-scan section. This representation method concisely provides the position of the instrument endpoint within a B-scan section when the instrument endpoint is fully covered by that section. For example, it uses three-dimensional coordinates to represent the current target position. Simultaneously, utilizing the spatial information of the volume data in the X and Y directions, the scanning position coordinates (x, y) corresponding to the instrument endpoint are first determined in the XY plane of the scanning coordinate system—where y is used to locate the B-scan section where the instrument endpoint is located, and x is used to locate the lateral position of the A-scan within that B-scan section—and then the depth position coordinates z are extracted from the depth direction of the corresponding A-scan to obtain the three-dimensional position (x, y, z). This representation method fully preserves the spatial integrity of the volume data and can more accurately reflect the actual position of the instrument endpoint in three-dimensional space, making it particularly suitable for situations where the instrument endpoint spans different B-scan sections.

[0086] In the technical solution provided in this application embodiment, after acquiring a first surgical field image captured by an image acquisition device, the initial position of the instrument tip is first determined based on the first surgical field image for coarse positioning. Then, the first area for OCT scanning is determined based on the initial position, so that the OCT scanning device only needs to perform targeted scanning on the first area. Based on the scanning data, the current target position of the instrument tip is determined for fine positioning. Compared with the solution of positioning the instrument tip based on images captured by a camera, OCT scanning can meet the spatial requirements for instrument tip positioning in ophthalmic surgery. Compared with the solution of positioning the instrument tip based on OCT scanning data of the entire surgical field, only the first area needs to be targeted for scanning, without performing high-density scanning of the entire surgical field, thus meeting the time requirements for instrument tip positioning in ophthalmic surgery. This embodiment of the application, through the cooperation of the image acquisition device and the OCT scanning device, and through the combination of coarse and fine positioning, can simultaneously meet the time and spatial requirements for instrument tip positioning in ophthalmic surgery, improving the positional accuracy of the instrument tip in ophthalmic surgery while rapidly positioning the instrument tip.

[0087] Image recognition algorithms can be used to track the initial position of continuously captured multi-frame surgical field images. However, when using image recognition algorithms to track the initial position, camera detection errors may occur, such as false positives, false negatives, and systematic biases. If the OCT scanning device is guided solely by the initial position tracked from the surgical field images, these camera detection errors will accumulate, eventually leading to tracking drift or failure. Therefore, when determining the scanning position of the instrument tip using scan data each time, the initial position tracked from the surgical field images can be corrected using the scanning position of the instrument tip. This ensures that the initial position for the next tracking iteration is based on the corrected initial position, improving the accuracy of tracking the initial position from the surgical field images.

[0088] In some embodiments, when the initial position of the instrument tip is first determined, an OCT scan is performed on the area covering the surgical object to obtain initial scan data; the initial position of the instrument tip is determined based on the initial scan data and the first surgical field image.

[0089] For example, the initial position of the instrument tip is obtained by weighting the position identified from the initial scan data and the position identified from the first surgical field image.

[0090] For example, if the distance between two positions (i.e., the position identified from the initial scan data and the position identified from the first surgical field image) is greater than or equal to a preset distance, it indicates that the position identified from the initial scan data has a large error, and the position identified from the first surgical field image is determined as the initial position of the instrument tip; if the distance between two positions is less than the preset distance, it indicates that the position identified from the initial scan data has a small error, and in order to improve the accuracy of the determined initial position, a weighted average is performed on the position identified from the initial scan data and the position identified from the first surgical field image to obtain the initial position of the instrument tip.

[0091] In this approach, by performing an OCT scan on the area covering the surgical subject to obtain initial scan data when initially determining the initial position of the instrument tip, and combining the first surgical field image with the initial scan data to determine the three-dimensional initial position of the instrument tip, the rapid positioning capability of the image acquisition device in the XY plane and the tomographic imaging capability of the OCT scanning device in the depth direction can be fully utilized to achieve multimodal fusion three-dimensional initial positioning. Specifically, the image acquisition device provides two-dimensional position information of the instrument tip in the XY plane, and the full-area OCT scan provides depth direction structural information of the instrument tip and its surrounding tissues. The three-dimensional initial position obtained by fusing the two is significantly better in spatial accuracy than the two-dimensional initial position relying solely on the image acquisition device, improving the accuracy of subsequent determination of the first region based on the initial position, thereby increasing the probability of successfully identifying the instrument tip from the first scan data.

[0092] In other embodiments, when the initial position of the instrument endpoint is first determined, an OCT scan is performed on the area covering the surgical object to obtain initial scan data, and the position of the instrument endpoint identified from the initial scan data is determined as the initial position of the instrument endpoint.

[0093] For example, when the position of the instrument tip is identified from the initial scan data, the identified position of the instrument tip is determined as the initial position of the instrument tip. For example, when the position of the instrument tip is not identified from the initial scan data, a first surgical field image captured by an image acquisition device is acquired, and the initial position of the instrument tip is determined based on the first surgical field image.

[0094] In other embodiments, when the initial position of the instrument endpoint is first determined, a first surgical field image captured by an image acquisition device is obtained, and the position of the instrument endpoint identified in the first surgical field image is determined as the initial position of the instrument endpoint.

[0095] The following explains the method for determining the first region to be scanned by OCT on a surgical subject. Figure 2 A flowchart illustrating a method for determining a first region for performing an OCT scan on a surgical subject based on the initial position of the instrument tip, provided in the first embodiment, the method including steps S1021 and S1022.

[0096] S1021. Obtain the movement speed of the instrument endpoint.

[0097] For example, the movement distance is calculated based on the initial positions of the instrument endpoints determined in the two most recent determinations, and the time interval is calculated based on the determination times of the two most recent initial positions. The movement speed of the instrument endpoints is then determined based on the movement distance and the time interval. As another example, the movement speed of the instrument endpoints is determined based on the current target positions of the instrument endpoints determined in the two most recent determinations, and the time interval is calculated based on the determination times of the two most recent current target positions. The movement speed of the instrument endpoints is then determined based on the movement distance and the time interval.

[0098] For example, when the initial position / current target position of the instrument tip is obtained for the first time, since there is no previous position to refer to, the movement speed of the instrument tip can be set to 0, or set to a preset speed based on the experience value of similar surgical operations.

[0099] S1022. Determine the first region for performing OCT scanning on the surgical subject based on the initial position of the instrument tip and the moving speed of the instrument tip.

[0100] In some embodiments, S1022 may include: determining the positional offset of the instrument endpoint based on the moving speed of the instrument endpoint and the time elapsed between the determination of the initial position of the instrument endpoint and the start of the OCT scan of the first region of the surgical subject; and determining the first region based on the initial position of the instrument endpoint and the positional offset. By introducing a positional offset compensation mechanism, the position estimation deviation caused by the time delay between determining the initial position of the instrument endpoint and the actual start of the OCT scan can be effectively overcome. Specifically, when the instrument endpoint is in motion, the initial position corresponds to the position of the instrument endpoint at the time of image acquisition. However, when the OCT scan begins, the instrument endpoint has already undergone a certain degree of displacement. If the first region is determined directly based on the initial position, the first region may fail to cover the true position of the instrument endpoint at the start of the OCT scan, thereby increasing the probability of failing to identify the instrument endpoint in the first scan data. By estimating the positional offset of the instrument endpoint at the start of the OCT scan based on the movement speed of the instrument endpoint and the aforementioned duration, and determining the first region with the compensated position, the first region can more accurately cover the actual position of the instrument endpoint at the start of the OCT scan, increasing the probability of successfully identifying the instrument endpoint from the first scan data, thereby reducing repeated scans caused by positional deviations and improving the real-time performance and reliability of surgical instrument endpoint tracking.

[0101] In some embodiments, determining the first region based on the initial position and position offset of the instrument endpoint can be achieved by: shifting the initial position of the instrument endpoint by the position offset along the movement direction of the instrument endpoint to determine the estimated position of the instrument endpoint; mapping the pixel coordinates of the estimated position on the image coordinate system to the scanning coordinate system of the OCT scanning device to obtain the corresponding position; determining the fifth region based on the corresponding position mapped from the estimated position to the scanning coordinate system; and determining the first region based on the fifth region.

[0102] For example, determining the first region based on the fifth region can include: defining the fifth region as the first region. The following describes how the fifth region is determined: For example, the estimated position can be mapped to its corresponding position on the scanning coordinate system as the fifth region. Since the pixel resolution of the image acquisition device is lower than the spatial resolution of the OCT scanning device, a single pixel coordinate, after being mapped to the scanning coordinate system, covers a physical area, which can be directly used as the fifth region for OCT scanning. Another example is that the estimated position mapped to its corresponding position on the scanning coordinate system can be expanded by a preset range to obtain an expanded region, which is then used as the fifth region. Yet another example is that the estimated position mapped to its corresponding position on the scanning coordinate system can be used as a reference, combined with a preset scanning range, to determine the fifth region. For example, the center of the estimated position mapped to its corresponding position on the scanning coordinate system can be used as the center of the fifth region, and the fifth region can be obtained by expanding along the X and Y directions according to a preset scanning range. In this way, the center position of the fifth region is the corresponding position mapped to the estimated position on the scanning coordinate system, and the range of the fifth region is the preset scanning range.

[0103] For example, the shape of the preset scanning range can be a square or a circle. In other embodiments, the preset scanning range can be other shapes, and this application embodiment does not limit this.

[0104] For example, determining the first region based on the fifth region can include: adjusting the shape of the fifth region asymmetrically according to the movement speed and direction of the instrument tip, so that the fifth region extends along the movement direction and relatively contracts in the direction perpendicular to the movement direction, thus obtaining the first region. This covers the positions the instrument tip may reach during OCT scanning in advance, while avoiding unnecessary expansion in other directions that would lead to redundant scan data. For instance, when the instrument tip moves along the positive X-axis, the fifth region can be transformed into an elliptical region with its center position unchanged and stretched along the X-direction. This elliptical region is the first region, where the major axis of the ellipse is longer than the minor axis, and the major axis is parallel to the movement direction of the instrument tip. The length of the major axis of the ellipse can be determined based on the product of the movement speed of the instrument tip and the time required for the OCT scan; the length of the minor axis of the ellipse can be maintained at a preset length, such as the diameter or side length of a preset scan range, or slightly smaller than the length of the major axis to reduce redundant scan range. Compared to using a symmetrical shape, such as a square or a circle, for the first region, stretching the fifth region asymmetrically into an ellipse along the direction of movement can cover the location that the instrument tip may move to during OCT scanning in advance with a smaller scan range increment, increasing the probability of successfully identifying the instrument tip from the first scan data while maintaining scanning efficiency.

[0105] For example, based on the movement speed and direction of the instrument tip, the center of the estimated position mapped to the corresponding position on the scanning coordinate system, offset by a preset distance along the movement direction, can be determined as the center of the first region. The preset distance is determined by the product of the movement speed of the instrument tip and the time required to perform the OCT scan. This method compensates for the movement of the surgical instrument during the scanning process, thereby increasing the probability of identifying the position of the instrument tip from the first scan data and improving the effectiveness of the determined current target position of the instrument tip.

[0106] Considering the time delay between the OCT scanning device performing the scan and the camera capturing the image (approximately on the order of tens of milliseconds), and the fact that the instrument tip can move at speeds of several millimeters per second during surgery, the instrument tip can move on the order of hundreds of micrometers within this delay. If the center of the initial pixel coordinates is mapped to the center of the corresponding position on the scanning coordinate system as the center of the scanning area, the instrument tip is likely to be outside the scanning area of ​​the OCT scanning device, leading to a high risk of tracking failure. Therefore, determining the center of the scanning area based on the instrument tip's movement speed and initial position allows for compensation of the time delay using the initial position of the instrument tip. This increases the probability of the instrument tip falling within the scanning area of ​​the OCT scanning device, improving the effectiveness of tracking the current target position of the instrument tip.

[0107] Figure 3 A flowchart illustrating a method for determining a first region for performing an OCT scan on a surgical subject based on the initial position of the instrument tip, provided for a second embodiment, the method including steps S102A and S102B.

[0108] S102A, Obtain the confidence level for determining the initial position of the instrument endpoint.

[0109] The confidence level is used to characterize the degree of match between the initial position and the actual position of the instrument endpoint. The higher the confidence level, the more reliable the initial position.

[0110] By using an image recognition algorithm to identify the instrument endpoint in the first surgical field image, the initial position of the instrument endpoint and the confidence level of the initial position can be obtained. Thus, S102A includes: obtaining the confidence level of the initial position of the instrument endpoint obtained by identifying the instrument endpoint in the first surgical field image.

[0111] S102B: Determine the first region for performing an OCT scan on the surgical subject based on the confidence level and the initial position of the instrument endpoint.

[0112] For example, a mapping relationship between multiple confidence ranges and multiple scanning ranges can be obtained. For instance, there can be a one-to-one mapping between multiple confidence ranges and multiple scanning ranges. Based on this mapping relationship and the confidence level of the initial position of the instrument endpoint, a preset scanning range is determined among the multiple scanning ranges. The scheme for determining the center of the first region based on the initial position of the instrument endpoint has already been described above and will not be repeated here. The first region is obtained by expanding along the X and Y directions from its center according to the preset scanning range. Thus, the center of the first region is determined based on the initial position of the instrument endpoint, the range of the first region is the preset scanning range, and the preset scanning range is determined based on the confidence level of the initial position of the instrument endpoint.

[0113] By configuring the mapping relationship between the confidence range and the scanning range in the above manner, the scanning range is increased accordingly when the confidence is low to compensate for the impact of the low reliability of the initial position of the instrument endpoint. In this way, the mechanism of dynamically adjusting the scanning range based on the confidence of the initial position of the instrument endpoint can reduce unnecessary scanning time overhead caused by blindly expanding the scanning range, and can also avoid insufficient scanning data caused by blindly shrinking the scanning range, thereby improving the effectiveness of the determined current target position of the instrument endpoint.

[0114] Figure 4 The diagram illustrates the scanning range corresponding to various confidence levels provided in some embodiments. When the confidence level is greater than 0.8, the scanning range is a circle with a radius of 0.5 mm; when the confidence level is between 0.5 and 0.8, the scanning range is a circle with a radius of 1 mm; and when the confidence level is less than 0.5, the scanning range is a circle with a radius of 2 mm.

[0115] The foregoing explanation states that the preset scan range is determined based on the confidence level of the initial position of the instrument tip. In other embodiments, the preset scan range can be determined based on the movement speed of the instrument tip. For example, the greater the movement speed of the instrument tip, the larger the area of ​​the preset scan range.

[0116] In this embodiment, considering that if the preset scanning range is too small, the device may exceed the OCT scanning range when moving rapidly, resulting in tracking loss; if the preset scanning range is too large, the scanning time will increase and the real-time performance will decrease, the preset scanning range is dynamically adjusted by the confidence level of the initial position of the device endpoint, thereby improving the effectiveness of the determined OCT scanning range.

[0117] After obtaining the current target location, the current target location can also be updated. Figure 5 This is a flowchart illustrating a method for updating the current target location provided in some embodiments, such as... Figure 5As shown, the method is performed after S104, and includes steps S105 to S108.

[0118] S105. Based on the current target position of the instrument tip, determine the second region for performing OCT scanning on the surgical subject.

[0119] In some embodiments, the center position of the second region is determined based on the current target position of the instrument endpoint; the second region is then expanded along the X and Y directions within a specified scanning range, with the center position of the second region as the center.

[0120] The process of determining the center position of the second region based on the current target position of the instrument endpoint is similar to the process of determining the center position of the first region based on the initial position of the instrument endpoint, and will not be elaborated here.

[0121] In some embodiments, the specified scan range can be a pre-set scan range. In other embodiments, the specified scan range is the smallest scan range among a plurality of scan ranges.

[0122] S106. Perform an OCT scan on the second region of the surgical subject to obtain the second scan data.

[0123] The implementation method of S106 is similar to that of S103, and will not be described again here.

[0124] S107. Determine the next target position of the instrument endpoint based on the second scan data.

[0125] The implementation of S107 is similar to that of S104, and will not be described again here.

[0126] S108. Update the current target position of the instrument endpoint based on the next target position of the instrument endpoint.

[0127] In this embodiment of the application, after obtaining the current target position of the device endpoint, it is necessary to continuously update the current target position of the device endpoint in order to achieve tracking of the device endpoint.

[0128] In some embodiments, S108 may include: overwriting the current target position of the instrument endpoint with the next target position of the instrument endpoint, such that the current target position of the latest instrument endpoint is the next target position of the instrument endpoint.

[0129] In this embodiment of the application, when the current target position of the device endpoint is identified from the first scan data, the system enters the tracking mode. In the tracking mode, S105 to S108 are repeatedly executed, and the next target position of the device endpoint is identified from the second scan data each time, thereby realizing the effective tracking of the device endpoint position.

[0130] In some embodiments, S108 may include: if the distance between the next target position of the device endpoint and the current target position of the device endpoint is greater than or equal to a preset threshold, overwriting the current target position of the device endpoint with the next target position of the device endpoint, such that the latest current target position of the device endpoint is the next target position of the device endpoint. In some embodiments, S108 may include: if the distance between the next target position of the device endpoint and the current target position of the device endpoint is less than a preset threshold, keeping the current target position of the device endpoint unchanged, i.e., not updating the current target position of the device endpoint. Thus, by setting a preset threshold as a judgment condition, the instability caused by small position fluctuations in the tracking of the device endpoint can be suppressed. When the deviation of the next target position of the device endpoint from the current target position of the device endpoint is small, the tracking position of the device endpoint remains the current target position of the device endpoint, thereby avoiding the tracking trajectory jitter caused by position fluctuations. At the same time, when the deviation is greater than or equal to the preset threshold, the tracking position of the device endpoint is updated to the next target position of the device endpoint in a timely manner, ensuring the sensitivity of the response to the real motion of the device endpoint, and achieving an adaptive balance between the smoothness of the tracking position of the device endpoint and the response speed.

[0131] After S108, since the latest current target position has been determined, steps S105 to S108 can be re-executed.

[0132] In this embodiment, when the current target position of the instrument endpoint is obtained, the area for OCT scanning of the surgical subject is redefined based on the current target position of the instrument endpoint. Thus, the new current target position of the instrument endpoint is determined based on the scanning data of the OCT scanning device for that area. This eliminates the step of determining the initial position of the instrument endpoint based on the surgical field image, reducing the computational load when updating the current target position of the instrument endpoint.

[0133] In some embodiments, the processor outputs the current target position of the instrument endpoint each time it determines or updates the current target position of the instrument endpoint; that is, it outputs the latest obtained current target position of the instrument endpoint. In other embodiments, the processor may output the current target position of the instrument endpoint to the display device in the instrument endpoint determination system during ophthalmic surgery only if the current target position of the instrument endpoint can be determined through the second scan data N times consecutively, so that the display device displays the current target position of the instrument endpoint. N is an integer greater than or equal to 2. For example, the value of N can be 2, 3, 4, 5, 6, or 7.

[0134] In the technical solution provided in this application embodiment, the corresponding position is obtained by mapping the pixel coordinates of the initial position of the instrument endpoint to the scanning coordinate system of the OCT scanning device. This is achieved through precise marking of the mapping relationship between the camera coordinate system and the scanning coordinate system, thereby solving the problem that the positions of the camera coordinate system and the scanning coordinate system cannot be associated due to the different physical principles of the camera coordinate system and the scanning coordinate system. Furthermore, the current target position of the instrument endpoint will only be output to the display device in the instrument endpoint determination system in ophthalmic surgery if the current target position of the instrument endpoint can be determined through the second scanning data N times consecutively. This allows the display device to display the current target position of the instrument endpoint, thereby avoiding the problem of the mapping relationship between the camera coordinate system and the scanning coordinate system shifting due to changes in camera focal length or magnification, and improving the accuracy of the display device in displaying the current target position of the instrument endpoint.

[0135] In some embodiments, when a device endpoint is identified based on the second scan data, the position of the device endpoint identified in the second scan data is determined as the next target position of the device endpoint.

[0136] The following describes another method for determining the next target location of the instrument endpoint based on the second scan data. Figure 6 A flowchart illustrating a method for determining the next target position of an instrument endpoint based on second scan data, provided in the first embodiment, is shown. The method includes steps S1071 to S1073.

[0137] S1071. If the instrument endpoint is not identified based on the second scan data, the second region of the surgical subject is expanded to obtain a third region.

[0138] For example, the scanning range of the second region can be obtained, and then expanded to obtain the scanning range of the third region; the third region is determined based on the center position of the second region and the scanning range of the third region. For example, the third region is obtained by expanding along the X and Y directions according to the scanning range of the third region, with the center position of the second region as the center. For example, the radius corresponding to the scanning range of the third region is the radius corresponding to the scanning range of the second region multiplied by a first coefficient, or the side length corresponding to the scanning range of the third region is the side length corresponding to the scanning range of the second region multiplied by the first coefficient. For example, the first coefficient can be a real number greater than 1. For example, the value range of the first coefficient can be a real number in the range of 1.1-1.5. For example, the first coefficient can be 1.1, 1.2, 1.3, 1.4, or 1.5.

[0139] S1072. Perform an OCT scan on the third region to obtain the third scan data.

[0140] The implementation of S1072 is similar to that of S103, and will not be described again here.

[0141] S1073. Determine the next target position of the instrument endpoint based on the third scan data.

[0142] In some embodiments, after S1073, the following steps may also be performed: update the current target position of the device endpoint according to the next target position of the device endpoint, and proceed to step S105.

[0143] In some embodiments, after S1073, the following steps may also be performed: update the current target position of the instrument endpoint according to the next target position of the instrument endpoint; then steps A1 to A3 may be performed cyclically.

[0144] Step A1: Based on the current target position of the instrument tip, determine the second region for performing OCT scanning on the surgical subject, and expand the second region for performing OCT scanning on the surgical subject to obtain the third region.

[0145] The method for determining the third region can refer to step S1071.

[0146] Step A2: Perform an OCT scan on the third region to obtain the third scan data.

[0147] Step A3: Based on the third scan data, determine the next target position of the instrument endpoint, and update the current target position of the instrument endpoint based on the next target position of the instrument endpoint.

[0148] In some embodiments, when steps A1 to A3 are executed cyclically, if the position of the instrument endpoint can be identified from the third scan data in each of M consecutive steps A1 to A3, then the position of the instrument endpoint identified from the third scan data each time is determined as the next target position of the instrument endpoint. After M consecutive steps A1 to A3, the process proceeds to step S105. Exemplarily, M is an integer greater than or equal to 2. For example, the value of M can be 2-8. For example, the value of M can be 2, 3, 4, 5, 6, 7, or 8. Where M is an integer greater than N.

[0149] For example, in order to reduce the inaccuracy of recognition results caused by noise and other problems, if the position of the instrument endpoint can be identified from the third scan data in M ​​consecutive steps A1 to A3, the current target position of the instrument endpoint is not output to the display device in the instrument endpoint determination system in ophthalmic surgery. Instead, after executing step S105, the current target position of the instrument endpoint is only output to the display device in step S108 after updating the current target position of the instrument endpoint, so that the display device displays the current target position of the instrument endpoint.

[0150] In some embodiments, if the position of the instrument endpoint is not identified from the third scan data in any of the cyclic executions of steps A1 to A3, the process proceeds to step S101.

[0151] In other embodiments, if the position of the instrument endpoint is not identified from the third scan data in any of the cyclic execution of steps A1 to A3, then steps B1 to B3 are executed.

[0152] Step B1: Based on the current target position of the instrument tip, determine the second region for performing OCT scanning on the surgical subject, expand the second region for performing OCT scanning on the surgical subject to obtain a fourth region, and the scanning range of the fourth region is larger than that of the third region.

[0153] Step B2: Perform an OCT scan on the fourth region to obtain the fourth scan data.

[0154] Step B3: Based on the fourth scan data, determine the next target position of the instrument endpoint, and update the current target position of the instrument endpoint based on the next target position of the instrument endpoint.

[0155] For example, the scanning range of the second region can be obtained, and then expanded to obtain the scanning range of the fourth region; the fourth region is determined based on the center position of the second region and the scanning range of the fourth region. For example, the fourth region is obtained by expanding along the X and Y directions according to the scanning range of the fourth region, with the center position of the second region as the center. For example, the radius corresponding to the scanning range of the fourth region is the radius corresponding to the scanning range of the second region multiplied by a second coefficient, or the side length corresponding to the scanning range of the fourth region is the side length corresponding to the scanning range of the second region multiplied by a second coefficient. For example, the second coefficient can be a real number greater than 1. For example, the second coefficient is greater than the first coefficient. For example, the value range of the second coefficient can be a real number greater than 1.1. For example, the value range of the second coefficient can be a real number in the range of 1.2-1.8. For example, the second coefficient can be 1.2, 1.4, 1.5, 1.6, 1.7, or 1.8.

[0156] In some embodiments, when steps B1 to B3 are executed cyclically, if the position of the instrument endpoint can be identified from the fourth scan data in each of K consecutive steps B1 to B3, then the position of the instrument endpoint identified from the fourth scan data each time is determined as the next target position of the instrument endpoint. After K consecutive steps B1 to B3, the process proceeds to step S105. Exemplarily, K is an integer greater than or equal to 2. For example, K can be between 2 and 8. For example, K can be 2, 3, 4, 5, 6, 7, or 8. Where K is an integer greater than N.

[0157] In this embodiment, when the system is in tracking mode, if the device endpoint is not identified based on the second scan data, the system enters alert mode. In alert mode, steps S1071-S1073 are executed and steps A1-A3 are executed repeatedly. If the device endpoint can be identified from the third scan data for M consecutive steps A1 to A3, the system continues to enter tracking mode. Alternatively, steps S1071-S1073 are executed, steps A1-A3 are executed at least once, and steps B1-A3 are executed repeatedly. If the device endpoint can be identified from the fourth scan data for K consecutive steps B1 to B3, the system continues to enter tracking mode.

[0158] For example, in order to reduce the inaccuracy of recognition results caused by noise and other problems, if the position of the instrument tip can be identified from the fourth scan data in K consecutive steps B1 to B3, the current target position of the instrument tip is not output to the display device in the instrument tip determination system in ophthalmic surgery. Instead, after executing step S105, the current target position of the instrument tip is updated in S108, and then the current target position of the instrument tip is output to the display device so that the display device displays the current target position of the instrument tip.

[0159] If the position of the instrument endpoint is not identified from any fourth scan data while executing steps B1 to B3 in a loop, proceed to step S101.

[0160] Figure 7 This is a flowchart illustrating the method for determining the next target position of the instrument tip based on second scan data, as provided in the second embodiment. Figure 7 Compared to the example Figure 6 The difference in the embodiment is that S1073 includes steps S1073A to S1073F.

[0161] S1073A. When the instrument endpoint is identified based on the third scan data, the position of the instrument endpoint identified in the third scan data is determined as the next target position of the instrument endpoint.

[0162] S1073B: When the instrument endpoint is not identified based on the third scan data, acquire the second surgical field image captured by the image acquisition device, and determine the next initial position of the instrument endpoint based on the second surgical field image.

[0163] In S1073B, the acquisition of the second surgical field image and the determination of the next initial position of the instrument endpoint are similar to those in S101, and will not be described again here.

[0164] S1073C. Update the initial position of the instrument endpoint based on the next initial position.

[0165] The implementation of S1073 is similar to that of S108, and will not be described again here.

[0166] S1073D: Update the first region of the surgical subject to be scanned using OCT based on the updated initial position of the instrument endpoint.

[0167] S1073E: Perform an OCT scan on the updated first region of the surgical subject to obtain the updated first scan data.

[0168] The implementation of S1073E is similar to that of S103, and will not be described again here.

[0169] S1073F: Determine the next target location of the instrument endpoint based on the updated first scan data.

[0170] The implementation of S1073F is similar to that of S104, and will not be described again here.

[0171] In this embodiment of the application, when the device endpoint is not identified based on the third scan data, the process proceeds to S101, thereby enabling the current target position of the device endpoint to be determined at a faster speed.

[0172] In this embodiment of the application, in the alert mode, if the position of the device endpoint is not continuously identified in the scan data obtained from the expanded scanning area, the system enters the search mode. In the search mode, steps S1073B-S1073F are executed, and if the position of the device endpoint is continuously identified, the system enters the tracking mode. If the position of the device endpoint is not continuously identified, the system enters the alert mode or the search mode.

[0173] The method for determining the instrument tip position during ophthalmic surgery in this embodiment can determine a B-scan tomographic image based on scan data and the current target position of the instrument tip, thereby displaying the B-scan tomographic image corresponding to the position of the automatically tracked instrument tip (i.e., the current target position of the tracked instrument tip) in real time on a display device. It should be noted that manually fixing the B-scan tomographic image corresponding to the position of the instrument tip is still the highest priority. For example, during any step, if a user is detected performing a first action on the positioning mechanism in the system, such as pressing down on the foot pedal, the real-time display of the B-scan tomographic image corresponding to the position of the automatically tracked instrument tip stops. Instead, the newly determined B-scan tomographic image corresponding to the instrument tip position is fixedly displayed on the display device. Thus, the position of the instrument tip displayed on the display device no longer changes according to changes in the position of the instrument tip. If, while the newly determined position of the instrument tip is fixedly displayed, a second action is detected on the positioning mechanism, such as releasing the foot pedal, the method in this embodiment continues, i.e., the B-scan tomographic image corresponding to the position of the automatically tracked instrument tip is displayed in real time.

[0174] In this way, by detecting the user's first action on the positioning mechanism in the system, the display device fixes the display of the B-scan tomographic image corresponding to the newly determined instrument tip position. This avoids continuous changes in the display screen caused by the movement of the instrument tip when the user needs to carefully observe the tomographic image of a specific area. During ophthalmic surgery, the operator may need to statically examine the deep tissue at the location of the instrument tip. Real-time tracking displays update the image with the movement of the instrument, which is not conducive to stable observation of details in a fixed area. With the above solution, the user can pause tracking and obtain a B-scan tomographic image with the position locked by performing the first action. When the system detects the user performing the second action, it resumes real-time tracking display. This method of switching between tracking and fixed states based on different actions of the same mechanism allows the user to flexibly control the display mode according to actual needs during surgery, completing the transition between fine observation and dynamic tracking of local tissues without interrupting the operation process. In this embodiment, the method continues even when the user performs the first action on the positioning mechanism in the system, except that the processor no longer outputs the current target position of the instrument tip.

[0175] The following explains S104 above. Figure 8 A flowchart illustrating a method for determining the current target location of an instrument endpoint based on first scan data, provided for some embodiments, the method including steps S1041 to S1043.

[0176] S1041. Determine a signal intensity set based on the first scan data. The signal intensity set includes multiple signal intensity values. Determine the position coordinates corresponding to the largest signal intensity value in the signal intensity set.

[0177] For example, determining the signal intensity set based on the first scan data may include: determining the signal intensity values ​​at different depth positions in the A-scan data of each scan position included in the first scan data as the signal intensity set. For example, the signal intensity values ​​may include reflection intensity values.

[0178] For example, the location corresponding to the maximum signal strength value is the location of the device.

[0179] S1042. When the maximum signal strength value in the signal strength concentration is greater than or equal to the preset strength threshold, the current target position of the instrument endpoint is determined according to the position coordinates corresponding to the maximum signal strength value.

[0180] In this embodiment of the application, since metal instruments have high reflectivity, their signal strength is much higher than the noise level, and the noise is difficult to reach the preset intensity threshold. Therefore, the interference from noise is not obvious. If at least one signal strength value in the signal strength concentration is greater than or equal to the preset intensity threshold, it indicates that the instrument is a metal instrument.

[0181] The current target position of the instrument endpoint can be determined by fitting the position coordinates corresponding to the maximum signal strength value.

[0182] S1043. When all signal strength values ​​in the signal strength set are less than the preset strength threshold, perform shadow detection on the signal strength set, determine the position coordinates corresponding to the shadow, and determine the current target position of the instrument endpoint based on the position coordinates corresponding to the maximum signal strength value and the position coordinates corresponding to the shadow.

[0183] In some embodiments, the location coordinates corresponding to signal strength values ​​that are less than a specified strength threshold are determined as the location coordinates corresponding to the shadow. For example, the specified strength threshold is much smaller than a preset strength threshold.

[0184] In this embodiment, the coordinates of the connected region between the device and the shadow are determined based on the position coordinates corresponding to the maximum signal strength value and the position coordinates corresponding to the shadow; the current target position of the device endpoint is determined based on the position coordinates corresponding to the maximum signal strength value in the connected region.

[0185] In this embodiment, due to the low reflectivity of non-metallic instruments, if all signal intensity values ​​in the signal intensity set are less than a preset intensity threshold, it indicates that the instrument is a non-metallic instrument. In this case, the method of spatially fitting the position coordinates corresponding to the maximum signal intensity value to obtain the current target position of the instrument endpoint is easily affected by noise and other interference. In this embodiment, shadow detection is performed on the signal intensity set to determine the position coordinates corresponding to the shadow. The current target position of the instrument endpoint is determined based on the position coordinates corresponding to the maximum signal intensity value and the position coordinates corresponding to the shadow. Thus, the current target position of the instrument endpoint can be determined by combining the position coordinates corresponding to the maximum signal intensity value and the position coordinates corresponding to the shadow. This allows for the determination of the spatial overlap boundary between the instrument and the surgical object. Even when some signals are blocked or high-intensity feature points are missing, the current target position of the instrument endpoint can still be reliably determined.

[0186] In clinical practice, when the instrument tip cannot be identified due to being outside the imaging field of view, being obstructed, or having insufficient image quality, performing OCT sampling will result in the acquisition of useless data that does not contain valid information about the instrument tip. This not only wastes sampling resources and storage space but also increases the unnecessary burden on subsequent signal processing. Based on this, in this embodiment, S101 may include: acquiring a first surgical field of view image captured by an image acquisition device; if the instrument tip is identified as being outside the surgical object from the first surgical field of view image, or if the instrument tip cannot be identified from the first surgical field of view image (e.g., the instrument has left the field of view or is obstructed), or if the instrument tip cannot be identified from multiple consecutive scans (e.g., air bubble interference or severe bleeding), determining the next surgical field of view image as the new first surgical field of view image, and proceeding to step S101 until the current target position of the instrument tip can be determined by the OCT scanning device.

[0187] In cases where the instrument tip is identified as being outside the surgical object from the first surgical field image, or where the instrument tip cannot be identified from the first surgical field image (e.g., the instrument has left the field of view or is obstructed), or where the instrument tip cannot be identified from multiple consecutive scans (e.g., air bubble interference or severe bleeding), considering that no effective management mechanism has been established in related technologies, in this embodiment, the next surgical field image is determined as the new first surgical field image, and the process proceeds to step S101 until the current target position of the instrument tip can be determined by the OCT scanning device. This reduces the risk of outputting incorrect instrument tip positions and improves the reliability of the output current target position of the instrument tip.

[0188] Based on the same concept, this application also provides a system for determining the instrument endpoints in ophthalmic surgery to implement the method for determining the instrument endpoints in ophthalmic surgery described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations of one or more embodiments of the system for determining the instrument endpoints in ophthalmic surgery provided below can be found in the limitations of the method for determining the instrument endpoints in ophthalmic surgery described above, and will not be repeated here.

[0189] In one exemplary embodiment, Figure 9 A schematic diagram of the structure of a system for determining the instrument endpoint in ophthalmic surgery provided in some embodiments, such as Figure 9 As shown, the instrument tip determination system in this ophthalmic surgery includes:

[0190] Image acquisition device, used to capture images of the first surgical field of view;

[0191] OCT scanning device, used to perform scanning;

[0192] The processor, connected to the image acquisition device and the OCT scanning device, is configured to perform the following operations: determine the initial position of the instrument tip based on the first surgical field image; determine the first region for performing OCT scanning on the surgical object based on the initial position of the instrument tip; control the OCT scanning device to perform OCT scanning on the first region to obtain first scan data; and determine the current target position of the instrument tip based on the first scan data.

[0193] For example, the instrument endpoint determination system in ophthalmic surgery can be an ophthalmic imaging system.

[0194] In some embodiments, the system further includes a display device;

[0195] The processor is configured to perform the following operations: determine a second region for OCT scanning of the surgical subject based on the current target position of the instrument endpoint; perform an OCT scan on the second region of the surgical subject to obtain second scan data; and determine the next target position of the instrument endpoint based on the second scan data.

[0196] When the distance between the next target position of the instrument endpoint and the current target position of the instrument endpoint is greater than or equal to a preset threshold, the OCT image is determined based on the second scan data, and the next target position of the instrument endpoint is marked on the OCT image. The current target position of the instrument endpoint is updated based on the next target position of the instrument endpoint.

[0197] A display device, connected to a processor, is used to display the next target location of the marker in an OCT image.

[0198] In this embodiment of the application, in order to avoid trajectory jitter, the position of the instrument endpoint on the OCT image is updated only when the instrument endpoint moves a large range.

[0199] In some embodiments, the system further includes a microscope. The microscope, OCT scanning device, and image acquisition device are optically coaxial. For example, the microscope is positioned facing the surgical subject, and the OCT scanning device and image acquisition device are also positioned facing the surgical subject to capture images via an optical path conversion device within the system.

[0200] Figure 9 The surgical object shown is the eye. In other embodiments, the surgical object can be other parts of the body, and this application does not limit this.

[0201] The description of the system embodiments above is similar to that of the method embodiments above, and has similar beneficial effects. For technical details not disclosed in the system embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0202] The various devices in the instrument endpoint determination system described above for ophthalmic surgery can be implemented entirely or partially through software, hardware, or a combination thereof. These devices can be embedded in the system's processor in hardware form or independent of it, or stored in the system's memory in software form, so that the processor can call and execute the corresponding operations of each device.

[0203] In one embodiment, a computer-readable storage medium is provided, wherein a computer program, when executed by a processor, implements the steps of the method provided in any of the above embodiments.

[0204] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the method provided in any of the above embodiments.

[0205] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the methods described above.

[0206] The processor, functional devices, or functional units in any embodiment of this application may include an integration of one or more of the following: a general-purpose processor, an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field-programmable gate array (FPGA), a central processing unit (CPU), a graphics processing unit (GPU), an embedded neural network processing unit (NPU), a controller, a microcontroller, a microprocessor, a programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, a quantum computing-based data processing logic device, an artificial intelligence (AI) processor, etc. Exemplarily, the general-purpose processor may be a microprocessor or any conventional processor.

[0207] The memory or computer-readable storage medium in any embodiment of this application may include at least one of non-volatile memory and volatile memory. Non-volatile memory includes integration of one or more of the following: Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Ferromagnetic Random Access Memory (FRAM), Flash Memory, Magnetic Surface Memory, Optical Disc, Compact Disc Read-Only Memory (CD-ROM), Magnetic Tape, Floppy Disk, Flash Memory, Optical Memory, High-Density Embedded Non-Volatile Memory, Resistive Random Access Memory (ReRAM), Magnetoresistive Random Access Memory (MRAM), Ferroelectric Random Access Memory (FRAM), Phase Change Memory (PCM), Graphene Memory, Volatile Memory, etc. Volatile memory includes one or more of the following: Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0208] The acquisition, transmission, storage, use, and processing of data in this application comply with relevant national laws and regulations. It should be noted that certain software, components, models, and other existing industry solutions may be mentioned in the embodiments of this application. These should be considered exemplary, intended only to illustrate the feasibility of implementing the technical solution of this application, and do not imply that the applicant has already used or necessarily used such solutions.

[0209] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0210] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A system for determining the instrument tip during ophthalmic surgery, characterized in that, The system includes: an image acquisition device, an OCT scanning device, and a processor, wherein the processor is connected to the image acquisition device and the OCT scanning device; the system is used to implement a method for determining the instrument endpoint during ophthalmic surgery, the method comprising: Acquire a first surgical field image captured by an image acquisition device, and determine the initial position of the instrument tip based on the first surgical field image; Based on the initial position of the instrument tip, determine the first region for performing an OCT scan on the surgical subject; An OCT scan was performed on the first region of the surgical subject to obtain first scan data; The current target position of the instrument endpoint is determined based on the first scan data; Based on the current target position of the instrument endpoint, determine the second region for which an OCT scan will be performed on the surgical subject; An OCT scan was performed on the second region of the surgical subject to obtain second scan data; The next target location of the instrument tip is determined based on the second scan data; Update the current target position of the instrument endpoint based on the next target position of the instrument endpoint.

2. The system according to claim 1, characterized in that, Determining the initial position of the instrument tip based on the first surgical field image includes: When the initial position of the instrument tip is determined for the first time, an OCT scan is performed on the area covering the surgical object to obtain initial scan data; The initial position of the instrument endpoint is determined based on the initial scan data and the first surgical field image.

3. The system according to claim 1 or 2, characterized in that, Determining the first region for performing an OCT scan on the surgical subject based on the initial position of the instrument tip includes: Obtain the movement speed of the instrument endpoint; Based on the initial position of the instrument tip and the moving speed of the instrument tip, a first region for performing an OCT scan on the surgical subject is determined.

4. The system according to claim 1 or 2, characterized in that, Determining the first region for performing an OCT scan on the surgical subject based on the initial position of the instrument tip includes: Obtain the confidence level for determining the initial position of the instrument endpoint; Based on the confidence level and the initial position of the instrument endpoint, a first region for performing an OCT scan on the surgical subject is determined.

5. The system according to claim 1 or 2, characterized in that, Determining the next target location of the instrument endpoint based on the second scan data includes: If the instrument endpoint is not identified based on the second scan data, the second region of the surgical subject subjected to OCT scanning is expanded to obtain a third region; An OCT scan was performed on the third region to obtain third scan data; The next target location of the instrument endpoint is determined based on the third scan data.

6. The system according to claim 5, characterized in that, Determining the next target location of the instrument endpoint based on the third scan data includes: If the instrument endpoint is not identified based on the third scan data, a second surgical field image captured by the image acquisition device is obtained, and the next initial position of the instrument endpoint is determined based on the second surgical field image. Update the initial position of the instrument endpoint based on the next initial position; Update the first region of the surgical subject to perform an OCT scan based on the updated initial position of the instrument endpoint; An OCT scan is performed on the updated first region of the surgical subject to obtain updated first scan data; The next target location of the instrument endpoint is determined based on the updated first scan data.

7. The system according to claim 1 or 2, characterized in that, Determining the current target position of the instrument endpoint based on the first scan data includes: A signal strength set is determined based on the first scan data. The signal strength set includes multiple signal strength values. The position coordinates corresponding to the largest signal strength value in the signal strength set are determined. When the maximum signal strength value in the signal strength set is greater than or equal to a preset strength threshold, the current target position of the instrument endpoint is determined according to the position coordinates corresponding to the maximum signal strength value. When all signal strength values ​​in the signal strength set are less than a preset strength threshold, shadow detection is performed on the signal strength set to determine the position coordinates corresponding to the shadow. The current target position of the device endpoint is determined based on the position coordinates corresponding to the maximum signal strength value and the position coordinates corresponding to the shadow.

8. The system according to claim 1 or 2, characterized in that, The image acquisition device is used to capture images of the first surgical field of view; the OCT scanning device is used to perform scanning. The processor is used to control the OCT scanning device to perform an OCT scan on the first region to obtain the first scan data.

9. The system according to claim 1 or 2, characterized in that, The system also includes a display device; The processor is configured to perform the following operations: determine a second region for performing an OCT scan on the surgical subject based on the current target position of the instrument endpoint; An OCT scan is performed on the second region of the surgical subject to obtain second scan data; the next target position of the instrument tip is determined based on the second scan data; When the distance between the next target position of the instrument endpoint and the current target position of the instrument endpoint is greater than or equal to a preset threshold, an OCT image is determined based on the second scan data, and the next target position of the instrument endpoint is marked on the OCT image. The current target position of the instrument endpoint is updated based on the next target position of the instrument endpoint. The display device, connected to the processor, is used to display the next target location marked in the OCT image.

Images