Intraoperative OCT imaging position tracking device, surgical microscope system and storage medium

Abstract

The present disclosure discloses a tracking device for intraoperative OCT imaging position, a surgical microscope system and a storage medium. The method comprises: determining a predicted position of the surgical instrument in a second coordinate system at a second moment according to a historical position of the surgical instrument in a first coordinate system at a first moment, wherein the first coordinate system is used for OCT imaging and the second coordinate system is used for microscope imaging; determining a detected position of the surgical instrument in the second coordinate system at a second moment according to the predicted position; and determining a mapped position of the surgical instrument in the first coordinate system at the second moment according to the detected position, so as to track the intraoperative OCT imaging position.

Description

Technical Field

[0001] The present disclosure relates to technical fields such as OCT image processing, camera image processing, surgical microscopes, and ophthalmic surgical microscopes, and in particular to a tracking device for intraoperative OCT imaging position, a surgical microscope system, and a storage medium. Background Art

[0002] Taking ophthalmic surgical microscopes as an example, some surgical microscopes have both microscope imaging and optical coherence tomography (OCT) imaging functions. During surgery, these two imaging functions of the surgical microscope can be combined to observe the surgical site.

[0003] In the related art, due to design issues, an additional doctor is required to manually adjust the OCT scanning position of the surgical microscope during use. This manual adjustment method is limited by the doctor's experience and may have a significant impact on the operation time. Summary of the invention

[0004] The embodiments of the present disclosure provide a device for tracking the intraoperative OCT imaging position, a surgical microscope system, and a storage medium, which can realize automatic tracking of the intraoperative OCT imaging position, thereby avoiding the impact of the doctor's experience on the operation time.

[0005] According to a first aspect of the present disclosure, a method for tracking an intraoperative OCT imaging position is provided, comprising:

[0006] Determine a predicted position of the surgical instrument in a second coordinate system at a second moment according to a historical position of the surgical instrument in a first coordinate system at a first moment, wherein the first coordinate system is used for OCT imaging and the second coordinate system is used for microscope imaging;

[0007] Determining a detected position of the surgical instrument in the second coordinate system at a second moment according to the predicted position; and

[0008] According to the detected position, the mapping position of the surgical instrument in the first coordinate system at the second moment is determined to track the intraoperative OCT imaging position.

[0009] According to a second aspect of the present disclosure, a device for tracking intraoperative OCT imaging position is provided, comprising:

[0010] A first determination module is configured to determine a predicted position of the surgical instrument in a second coordinate system at a second moment based on a historical position of the surgical instrument in a first coordinate system at a first moment, wherein the first coordinate system is used for OCT imaging and the second coordinate system is used for microscope imaging;

[0011] A second determination module is configured to determine a detection position of the surgical instrument in the second coordinate system at a second moment according to the predicted position; and

[0012] The third determination module is configured to determine the mapping position of the surgical instrument in the first coordinate system at the second moment according to the detected position, so as to track the intraoperative OCT imaging position.

[0013] According to a third aspect of the present disclosure, there is provided a surgical microscope system, comprising:

[0014] A surgical microscope configured with a microscope imaging module and an OCT imaging module; and

[0015] The control device is configured to track and control the imaging position of the OCT imaging module according to the intraoperative OCT imaging position tracking method provided by any embodiment of the first aspect of the present invention.

[0016] According to a fourth aspect of the present disclosure, a control device is provided, which may include:

[0017] at least one processor; and

[0018] a memory communicatively connected to at least one processor; wherein,

[0019] The memory stores a computer program that can be executed by at least one processor, and the computer program is executed by at least one processor so that when the at least one processor executes, the method for tracking the intraoperative OCT imaging position provided by any embodiment in the first aspect of the present disclosure is implemented.

[0020] According to a fifth aspect of the present disclosure, a computer-readable storage medium is provided, on which computer instructions are stored, and the computer instructions are used to enable a processor to implement the method for tracking the intraoperative OCT imaging position provided by any embodiment in the first aspect of the present disclosure when executed. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The following is a brief introduction to the drawings required for describing the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention.

[0022] Figure 1 is a flow chart of a method for tracking intraoperative OCT imaging position according to an embodiment of the present disclosure;

[0023] Figure 2a is a schematic diagram of microscope imaging provided according to an embodiment of the present disclosure;

[0024] Figure 2b is a schematic diagram of lateral imaging of an OCT cross scan provided according to an embodiment of the present disclosure;

[0025] Figure 2c is a schematic diagram of longitudinal imaging of an OCT cross scan provided according to an embodiment of the present disclosure;

[0026] Figure 3a is a schematic diagram of determining a scanning position corresponding to OCT cross scanning imaging according to an embodiment of the present disclosure;

[0027] Figure 3b is another schematic diagram of determining a scanning position corresponding to OCT cross scanning imaging according to an embodiment of the present disclosure;

[0028] Figure 4 It is a schematic diagram of comparison before and after the rotation of the first coordinate system in intraoperative OCT imaging position tracking provided according to an embodiment of the present disclosure;

[0029] Figure 5 is a flowchart of another intraoperative OCT imaging position tracking method provided according to an embodiment of the present disclosure;

[0030] Figure 6 is a flowchart of another intraoperative OCT imaging position tracking method provided according to an embodiment of the present disclosure;

[0031] Figure 7 is a flowchart of an optional example of another intraoperative OCT imaging position tracking method provided according to an embodiment of the present disclosure;

[0032] Figure 8 It is a structural block diagram of a device for tracking intraoperative OCT imaging position according to an embodiment of the present disclosure;

[0033] Fig. 9 is a structural block diagram of a surgical microscope system provided according to an embodiment of the present disclosure;

[0034] Fig.10 A schematic diagram of a control device for implementing the method for tracking the intraoperative OCT imaging position provided by an embodiment of the present disclosure is exemplarily shown. DETAILED DESCRIPTION

[0035] The technical solutions in the embodiments of the present invention will be described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0036] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. The situations of "target", "original", etc. are similar and will not be repeated here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0037] As mentioned above, the surgical microscope provided in the related art cannot automatically locate the OCT scanning position of the OCT module due to design problems, resulting in the need for additional doctors to assist in manually adjusting the OCT scanning position of the OCT module in the surgical microscope when using this type of surgical microscope. Adding additional doctors is not only not conducive to the hospital's coordination of doctor resources, but also increases the hospital's medical care costs and the patient's medical costs. In addition, the manual adjustment method is limited by the doctor's experience. If the doctor who assists in adjusting the OCT scanning position of the OCT module is not skilled in the operation, it will have a greater impact on the duration of the operation, and may even increase the risk and difficulty of the operation, which may endanger the safety of the patient.

[0038] The disclosed embodiment proposes an improved method for tracking intraoperative OCT imaging position, which can automatically adjust the OCT scanning position of the OCT module without the need for additional doctor assistance for manual adjustment.

[0039] In the process of implementing the embodiments of the present disclosure, the inventors found that in some embodiments, the OCT scanning position of the OCT module in the surgical microscope at a later moment can be located based on the OCT imaging of the surgical microscope at a previous moment, that is, the intraoperative OCT imaging position is tracked only based on the OCT imaging without combining the microscope imaging. The inventors also found that the OCT scanning speed of the surgical microscope is slow, generally slower than the microscope imaging speed of the surgical microscope. Therefore, if the OCT scanning position of the OCT module at a later moment is tracked and located based only on the OCT imaging (three-dimensional image) at a previous moment, it may be difficult to obtain more accurate positioning information.

[0040] In the process of implementing the embodiments of the present disclosure, the inventors also found that in other embodiments, the OCT scanning position of the OCT module in the surgical microscope can be tracked and located based on the OCT imaging of the surgical microscope and combined with its microscope imaging (two-dimensional image). The positioning method provided in this embodiment can obtain more accurate positioning information. The embodiments of the present disclosure will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0041] This embodiment provides a method for tracking an intraoperative OCT imaging position.

[0042] Figure 1 It is a flowchart of a method for tracking intraoperative OCT imaging position provided according to an embodiment of the present disclosure.

[0043] The method can be executed by the intraoperative OCT imaging position tracking device provided by the embodiment of the present invention. The device can be implemented by software and / or hardware. The device can be integrated on a control device, which can be various user terminals or servers.

[0044] The method provided in this embodiment can be used to automatically locate and accurately track the OCT scanning position (OCT imaging position) of an OCT module in a surgical microscope, such as an ophthalmic surgical microscope.

[0045] See also Figure 1 In the embodiment of the present disclosure, the method for tracking the intraoperative OCT imaging position may include operations S110-S130:

[0046] In operation S110, a predicted position of the surgical instrument in a second coordinate system at a second moment is determined based on a historical position of the surgical instrument in a first coordinate system at a first moment, wherein the first coordinate system is used for OCT imaging and the second coordinate system is used for microscope imaging.

[0047] In the process of implementing the embodiments of the present disclosure, the inventors found that, taking ophthalmic surgery as an example, the surgeon during the operation usually pays more attention to the ocular structure directly below the surgical instrument performing the operation, especially directly below the top of the surgical instrument. Therefore, the surgeon during the operation prefers that the scanning position of the OCT module in the surgical microscope can be positioned directly below the surgical instrument, especially directly below the top of the surgical instrument. Therefore, in the embodiments of the present disclosure, in operation S110, the OCT scanning position of the OCT module in the surgical microscope is positioned and tracked according to the position of the surgical instrument performing the operation during the operation.

[0048] In the disclosed embodiment, different coordinate systems can be defined for OCT imaging and microscope imaging. For example, a first coordinate system can be defined for OCT imaging, and a second coordinate system can be defined for microscope imaging, wherein the first coordinate system is different from the second coordinate system, but there is a certain mapping relationship between the first coordinate system and the second coordinate system. In this embodiment, the mapping relationship between the first coordinate system and the second coordinate system can be determined based on the microscope imaging and OCT imaging of the surgical microscope. Exemplary, Figure 2a A schematic diagram of microscope imaging corresponding to the second coordinate system is shown; Figure 2b and Figure 2c A schematic diagram of OCT cross scanning imaging corresponding to the first coordinate system is shown. Specifically, Figure 2b This is a schematic diagram of the lateral imaging of the OCT cross scan. Figure 2c This is a schematic diagram of longitudinal imaging of OCT cross scanning.

[0049] In the disclosed embodiment, the first moment is the earlier moment, and the second moment is the later moment. Therefore, in operation S110, the "historical position" of the surgical instrument in the first coordinate system at the first moment is a relative concept, relative to the "second moment". In some embodiments, when the "first moment" is a historical moment that has already occurred, the "historical position" can be understood as the position of the surgical instrument in the first coordinate system at a historical moment. In other embodiments, when the "first moment" is the current moment that is occurring, the "historical position" can be understood as the position of the surgical instrument in the first coordinate system at the current moment. In other embodiments, when the "first moment" is a future moment that has not yet occurred, the "historical position" can be understood as the position of the surgical instrument in the first coordinate system at a future moment.

[0050] It should be understood that the position of the surgical instrument during the operation will shift at the surgical site as the operation progresses. In this embodiment, the movement speed of the surgical instrument at each moment can be predicted according to the motion tracking algorithm during the operation.

[0051] Taking ophthalmic surgery (surgery on the patient's treatment eye) as an example, in operation S110, the OCT imaging of the treatment eye at the previous moment (recorded as t1) can be obtained, and the position information of the surgical instrument performing the surgical operation in the OCT imaging at time t1 can be determined based on the OCT imaging, and the position information can represent the historical position of the surgical instrument in the first coordinate system at time t1. Then, based on the historical position, the mapping relationship between the first coordinate system and the second coordinate system, and the movement speed of the surgical instrument at time t1-t2 obtained by the motion tracking algorithm, the position information of the surgical instrument in the microscope imaging at the next moment (recorded as t2) is predicted, and the position information can represent the predicted position of the surgical instrument in the second coordinate system at time t2.

[0052] In operation S120, a detected position of the surgical instrument in the second coordinate system at a second moment is determined according to the predicted position.

[0053] In this embodiment, the predicted position is a prediction result obtained based on the historical position obtained in operation S110 and a pre-set algorithm, which may be different from the actual position of the surgical instrument in the second coordinate system at the second moment. Therefore, in this embodiment, in operation S120, the position of the surgical instrument is further detected based on the predicted position in the microscope imaging formed by the surgical microscope at the second moment, so as to obtain the detected position of the surgical instrument in the second coordinate system at the second moment. The detected position is slightly different from the actual position of the surgical instrument in the second coordinate system at the second moment, or even has no difference.

[0054] In this way, based on the predicted position, a corresponding detection position is further identified in the corresponding detection area for tracking the OCT scanning position of the OCT module of the surgical microscope, so that the OCT scanning position of the OCT module at the second moment can be more accurately located.

[0055] In operation S130, a mapping position of the surgical instrument in the first coordinate system at a second moment is determined according to the detected position, so as to track the intraoperative OCT imaging position.

[0056] In this embodiment, the position of the surgical instrument in the OCT imaging at the second moment is determined based on the detected position and the mapping relationship between the first coordinate system and the second coordinate system. This position is the mapping position of the surgical instrument in the first coordinate system at the second moment. This position can then be used as the OCT scanning position of the OCT module to achieve tracking of the intraoperative OCT imaging position.

[0057] As described above, the inventors have discovered that in the disclosed embodiments, since the OCT imaging speed of the surgical microscope is relatively slow, it may be difficult to obtain accurate prediction results by directly predicting the position of the surgical instrument in the OCT imaging formed by the surgical microscope at a later moment based on the OCT imaging formed by the surgical microscope at a previous moment.

[0058] In this embodiment, the surgical instrument is positioned by combining the OCT imaging of the surgical microscope and the microscope imaging, thereby tracking the OCT scanning position. In this way, the defect of inaccurate positioning results caused by the slow OCT imaging speed when only using OCT imaging for OCT scanning position positioning can be avoided, thereby obtaining more accurate OCT scanning position positioning.

[0059] The automatic positioning method for OCT scanning position provided in the embodiment of the present disclosure does not require additional doctors to assist in manual positioning. Therefore, compared with the manual positioning method, the automatic positioning method is more conducive to the hospital to coordinate doctor resources, and will not increase the hospital's medical care costs and the patient's medical costs. In addition, the automatic positioning method is not limited by the doctor's experience, which can avoid the great impact on the operation time due to the doctor's unskilled operation, thereby reducing the surgical risk and difficulty, and reducing the harm to the patient's safety. In addition, as mentioned above, since OCT imaging has a slower imaging speed than microscope imaging, if only OCT imaging is used to track the intraoperative OCT imaging position, it is easy to cause inaccurate positioning. In the embodiment of the present disclosure, OCT imaging and microscope imaging are cleverly combined for intraoperative OCT imaging position tracking, which can overcome the defects caused by tracking only using OCT imaging, thereby achieving more accurate intraoperative OCT imaging position tracking.

[0060] In an optional embodiment, when the microscope imaging is updated, the intraoperative OCT imaging position tracking can be performed according to operations S110-S130, that is, in response to the update of the microscope imaging, the operations of determining the predicted position of the surgical instrument in the second coordinate system at the second moment based on the historical position of the surgical instrument in the first coordinate system at the first moment, and subsequent operations S120 and S130 are performed in sequence, thereby completing the intraoperative OCT imaging position tracking.

[0061] In one embodiment, if the position of the surgical instrument changes between a later moment and a previous moment, it is considered that the microscope image formed at the later moment is updated compared to the microscope image formed at the previous moment. If the position of the surgical instrument does not change between a later moment and a previous moment, it is considered that the microscope image formed at the later moment is not updated compared to the microscope image formed at the previous moment. Based on this, in the case where the microscope image is updated, tracking the intraoperative OCT imaging position according to operations S110-S130 can ensure the accuracy of surgical instrument tracking, and further ensure the accuracy of intraoperative OCT imaging position tracking.

[0062] In another embodiment, no matter whether the position of the surgical instrument changes at a later moment compared to a previous moment, as long as there is a microscope imaging output at the later moment, it can be considered that the microscope imaging formed at the later moment is updated compared to the microscope imaging formed at the previous moment, and if there is no microscope imaging output at the later moment, it can be considered that the microscope imaging formed at the later moment is not updated compared to the microscope imaging formed at the previous moment. Based on this, when the microscope imaging is updated, tracking the intraoperative OCT imaging position according to operations S110-S130 can ensure the accuracy of surgical instrument tracking, and further ensure the accuracy of intraoperative OCT imaging position tracking.

[0063] In another optional embodiment, the inventors found that if the microscope image formed at a later moment has no change in the position of the surgical instrument compared to the microscope image formed at a previous moment (the microscope image is not updated), the predicted position of the surgical instrument in the second coordinate system at the second moment is substantially consistent with the detected position of the surgical instrument in the second coordinate system at the second moment determined based on the predicted position. Therefore, in this embodiment, the intraoperative OCT imaging position tracking can be performed according to the following operation when the microscope image is not updated.

[0064] Based on the historical position of the surgical instrument in the first coordinate system at the first moment, the predicted position of the surgical instrument in the second coordinate system at the second moment is determined; in response to the microscope imaging not being updated, the predicted position is directly used as the detected position of the surgical instrument in the second coordinate system at the second moment; and based on the detected position, the mapping position of the surgical instrument in the first coordinate system at the second moment is determined to track the intraoperative OCT imaging position.

[0065] Through the embodiments of the present disclosure, since there is no need to delineate the detection area according to the predicted position and determine the corresponding detection position within the detection area according to a specific algorithm, the predicted position is directly used as the detection position, thereby eliminating detection-related operations and improving the tracking efficiency of the intraoperative OCT imaging position while ensuring accurate tracking.

[0066] In the disclosed embodiment, when performing surgery using a surgical microscope, the tracking of the OCT imaging position can be started when the surgical instrument enters the visual field of the surgical site, and the tracking of the OCT imaging position can be paused or turned off before the surgical instrument enters the visual field of the surgical site or after the surgical instrument leaves the visual field of the surgical site. In this way, the computing power of the device can be saved and the operating efficiency and quality of other functions of the device can be improved.

[0067] Therefore, in an optional embodiment of the present disclosure, in response to the appearance of a surgical instrument in the microscope imaging, operation S110 can be initiated to determine the predicted position of the surgical instrument in the second coordinate system at the second moment based on the historical position of the surgical instrument in the first coordinate system at the first moment, as well as subsequent operations S120 and S130.

[0068] In the process of implementing the embodiments of the present disclosure, the inventors also found that the peripheral area where the top of the surgical instrument is located during the operation is usually the location where the doctor focuses on the surgical site. Therefore, in an optional embodiment of the present disclosure, the predicted position can be defined to include: the predicted position of the top of the surgical instrument in the second coordinate system at the second moment , wherein the surgical instrument vertex is the vertex of the surgical instrument; then, according to the predicted position, determining the detection position of the surgical instrument in the second coordinate system at the second moment, including: according to the predicted position of the surgical instrument vertex , determining a detection area of ​​the surgical instrument in the second coordinate system at a second moment; and within the detection area, identifying a detection position of the surgical instrument in the second coordinate system at the second moment. .

[0069] In this embodiment, the vertex position of the surgical instrument is used to represent the instrument position of the surgical instrument. This is because in ophthalmic surgery, doctors are more concerned about the eye structure directly below the surgical instrument. Therefore, the OCT imaging position is obtained by tracking the vertex of the surgical instrument, and OCT imaging is performed at this imaging position. This is in line with the actual needs of the doctor.

[0070] In other embodiments, for example, for a smaller surgical instrument, the middle position of the surgical instrument may also be used to represent the position of the surgical instrument. In this case, the intraoperative OCT imaging position may be tracked based on the predicted position of the middle position of the surgical instrument in the second coordinate system at the second moment.

[0071] In the embodiment of the present disclosure, the detection area can be understood as an area defined according to the predicted position in the second coordinate system at the second moment and used to detect the actual position of the surgical instrument. In combination with the application scenarios that may be involved in the embodiment of the present disclosure, for example, the detection area can be understood as an area in the eye image (eye microscope imaging) corresponding to the second moment for detecting the actual position of the surgical instrument. In one embodiment of the present disclosure, according to the predicted position of the vertex of the surgical instrument , determine the detection area, illustratively, based on the predicted position of the apex of the surgical instrument , will be As the vertex, and with length and width both ( ) as the detection area of ​​the surgical instrument in the second coordinate system at the second moment. In another embodiment of the present disclosure, the predicted position The detection area is a circular area with a center of the circle and a preset value as the radius.

[0072] Further, within the detection area, the detection position of the surgical instrument in the second coordinate system at the second moment is identified. , for example, it can be the detection position of the vertex of the surgical instrument .

[0073] In the disclosed embodiment, the approximate area where the detection position is located, i.e., the detection area, is roughly estimated by predicting the position, and then the detection position is further accurately located within the detection area. This can ensure the accuracy of positioning and improve the processing efficiency of the positioning operation.

[0074] In an optional embodiment of the present disclosure, the mapping position may be defined to include: the mapping position of the surgical instrument vertex in the first coordinate system at the second moment , where the surgical instrument vertex is the vertex of the surgical instrument.

[0075] Based on this, the above intraoperative OCT imaging position tracking method may also include: determining the mapping position of the surgical instrument vertex in the first coordinate system at the second moment After that, map the location Centered on the OCT scanning module of the surgical microscope to perform OCT cross scanning imaging; or, to map the position The first coordinate system is rotated by a predetermined angle and then the OCT scanning module of the surgical microscope is controlled to perform OCT cross scanning imaging.

[0076] In this embodiment, when determining the mapping position of the surgical instrument vertex in the first coordinate system at the second moment Afterwards, based on the mapping location , control the OCT scanning module of the surgical microscope to perform OCT cross scanning imaging, thereby obtaining the mapping position based on the OCT cross-scan imaging. In the disclosed embodiment, the mapping position of the apex of the surgical instrument at the second moment in the first coordinate system is used as the position of the apex of the surgical instrument in the first coordinate system at the second moment, and OCT cross-scan imaging is performed based on the position. Taking ophthalmic surgery as an example, it is convenient for doctors to observe the surgical site of the eye tissue in both the horizontal and vertical imaging directions based on the OCT image, and smoothly carry out ophthalmic surgery. In the disclosed embodiment, the mapping position of the apex of the surgical instrument in the first coordinate system at the second moment can also be used as the historical position of the apex of the surgical instrument in the first coordinate system at the next moment, so as to continuously track the intraoperative OCT imaging position.

[0077] In this embodiment, the mapping position can be As the center, the OCT scanning module controlling the surgical microscope performs OCT cross scanning imaging based on the center. Figure 3a As shown in the figure, repeat (such as repeat = 2*N+1) indicates the number of times the OCT cross scan is repeated at the same position, and N is a positive integer, that is, the mapping position With the image center at the center, (2*N+1) OCT cross scanning images are performed in the horizontal direction (transverse imaging direction) and the vertical direction (longitudinal imaging direction).

[0078] In some surgical scenarios, as the surgery progresses, the position of the surgical instrument may change, for example, the direction of the surgical instrument may rotate. Therefore, in other embodiments, the mapping position may also be As the center, the first coordinate system is first rotated by a predetermined angle, and then the OCT scanning module is controlled to perform OCT cross scanning imaging based on the center. The predetermined angle of rotation of the first coordinate system can be the same as the direction angle of rotation of the surgical instrument. In the embodiment of the present disclosure, by rotating the first coordinate system by a predetermined angle, one coordinate axis in the first coordinate system can be parallel to the surgical instrument, and the other coordinate axis can be perpendicular to the surgical instrument. For example, Figure 4 As shown, the left figure shows the relative position relationship between the surgical instrument and the first coordinate system before rotation, and the right figure shows the relative position relationship between the surgical instrument and the first coordinate system after rotation. In this embodiment, after the direction of the surgical instrument changes, OCT cross scanning imaging is performed based on the rotated first coordinate system, and the imaging direction of the OCT cross scanning imaging can be changed according to the change in the direction of the surgical instrument, that is, the direction of the horizontal and vertical coordinate axes in the new first coordinate system can be used as the OCT imaging direction to obtain the corresponding OCT cross-sectional image, so as to meet the actual observation needs of the doctor, thereby achieving a more flexible intraoperative navigation effect.

[0079] In another optional embodiment of the present disclosure, the mapping position may be defined to include: the mapping position of the surgical instrument vertex in the first coordinate system at the second moment , where the surgical instrument vertex is the vertex of the surgical instrument.

[0080] Based on this, the above intraoperative OCT imaging position tracking method further includes: determining the mapping position of the surgical instrument vertex in the first coordinate system at the second moment After that, map the location Centered on the is a step length, a plurality of horizontal scanning position points are determined in the horizontal direction, and a plurality of vertical scanning position points are determined in the vertical direction; the OCT scanning module of the surgical microscope is controlled to perform one OCT horizontal scan at each horizontal scanning position point to obtain a plurality of OCT horizontal scan images; the OCT scanning module of the surgical microscope is controlled to perform one OCT vertical scan at each vertical scan position point to obtain a plurality of OCT vertical scan images; and the corresponding OCT cross scan imaging is determined according to the plurality of OCT horizontal scan images and the plurality of OCT vertical scan images.

[0081] See also Figure 5 , the intraoperative OCT imaging position tracking method may include the following operations:

[0082] In operation S210, a predicted position of the surgical instrument in a second coordinate system at a second moment is determined based on a historical position of the surgical instrument in a first coordinate system at a first moment, wherein the first coordinate system is used for OCT imaging and the second coordinate system is used for microscope imaging.

[0083] In operation S220, a detected position of the surgical instrument in the second coordinate system at a second moment is determined according to the predicted position.

[0084] In operation S230, based on the detected position, a mapping position of the surgical instrument vertex in the first coordinate system at the second moment is determined. , to track the intraoperative OCT imaging position, where the surgical instrument vertex is the vertex of the surgical instrument.

[0085] Operations S210 to S230 are the same or similar to operations S110 to S130 in the aforementioned embodiment, and are not described in detail in this embodiment.

[0086] In operation S240, the mapping position Centered on the A plurality of horizontal scanning position points are determined in the horizontal direction, and a plurality of vertical scanning position points are determined in the vertical direction.

[0087] For example, see Figure 3b , to map the location Centered at a predetermined scanning interval is the step length, and 5 vertical scanning position points are determined in the horizontal direction ,in, , in the diagram , and determine 5 horizontal scanning position points in the vertical direction ,in, .

[0088] Figure 3a The OCT cross scan in the example shown has a smaller scanning range. Figure 3b The scan range of the OCT cross scan in the example shown is relatively large. Taking ophthalmic surgery as an example, Figure 3b The OCT cross scan shown can scan the apex of the surgical instrument and the surrounding eye tissue, which is more conducive to the effective tracking of the subsequent OCT imaging position.

[0089] In operation S250, the OCT scanning module of the surgical microscope is controlled to perform one OCT horizontal scan at each horizontal scanning position point to obtain a plurality of OCT horizontal scanning images.

[0090] In this embodiment, the OCT scanning module can be controlled to perform one OCT horizontal scan at each horizontal scanning position point, thereby obtaining an OCT horizontal scanning image corresponding to each horizontal scanning position point.

[0091] In operation S260, the OCT scanning module of the surgical microscope is controlled to perform one OCT vertical scan at each vertical scanning position point to obtain a plurality of OCT vertical scanning images.

[0092] In this embodiment, the OCT scanning module can be controlled to perform one OCT vertical scan at each vertical scanning position point, thereby obtaining an OCT vertical scanning image corresponding to each vertical scanning position point.

[0093] In one embodiment, after determining each scanning position point, the corresponding OCT horizontal scanning and OCT vertical scanning can be directly performed based on each scanning position point. Figure 4 In the example shown, the first coordinate system is first rotated at a predetermined angle according to the change in the direction of the surgical instrument, and then an OCT cross scan is performed based on the directions indicated by the two coordinate axes in the rotated first coordinate system.

[0094] In operation S270, corresponding OCT cross scanning images are determined according to the plurality of OCT horizontal scanning images and the plurality of OCT vertical scanning images.

[0095] Exemplarily, at least one OCT horizontal scanning image may be screened out from a plurality of OCT horizontal scanning images, and at least one OCT vertical scanning image may be screened out from a plurality of OCT vertical scanning images, and then the corresponding OCT cross scanning image may be determined based on the screening results.

[0096] As another example, multiple OCT horizontal scanning images can be fused to obtain an OCT horizontal fusion image, and multiple OCT vertical scanning images can be fused to obtain an OCT vertical fusion image, and the corresponding OCT cross scanning image can be determined based on the OCT horizontal fusion image and the OCT vertical fusion image. In this example, when the surgical instrument is within the scanning range, image fusion can ensure that the surgical instrument must appear in the OCT horizontal fusion image and the OCT vertical fusion image, which is conducive to the effective tracking of the subsequent OCT imaging position.

[0097] In the embodiment of the present disclosure, by mapping the position As the center, OCT cross scanning is performed at multiple horizontal scanning positions and multiple vertical scanning positions. This scanning method can cover a larger scanning range, thereby increasing the possibility of OCT scanning to the apex of the surgical instrument.

[0098] In an optional embodiment of the present disclosure, the historical position may be defined to include: the historical position of the surgical instrument vertex in the first coordinate system at the first moment , wherein the surgical instrument vertex is the vertex of the surgical instrument; the predicted position can be defined to include: the predicted position of the surgical instrument vertex in the second coordinate system at the second moment ; Determine the predicted position of the surgical instrument in the second coordinate system at the second moment according to the historical position of the surgical instrument in the first coordinate system at the first moment, including: , and use Kalman filtering to estimate the optimal predicted position of the surgical instrument vertex in the first coordinate system at the second moment ; Based on the optimal predicted position Perform coordinate mapping to determine the predicted position of the surgical instrument vertex in the second coordinate system at the second moment .

[0099] See also Figure 6 , the method of this embodiment may include the following operations:

[0100] Operation S310, based on the historical position of the surgical instrument vertex in the first coordinate system at the first moment , and use Kalman filtering to estimate the optimal predicted position of the surgical instrument vertex in the first coordinate system at the second moment , wherein the first coordinate system is used for OCT imaging, the second coordinate system is used for microscope imaging, and the surgical instrument vertex is the vertex of the surgical instrument.

[0101] Exemplary, reference Figure 3b , based on historical locations , determining multiple OCT horizontal scanning positions and multiple OCT vertical scanning positions at the first moment, and performing OCT scanning to obtain multiple OCT horizontal scanning images and multiple OCT vertical scanning images, fusing the multiple OCT horizontal scanning images into one OCT horizontal image, and fusing the multiple OCT vertical scanning images into one OCT vertical image, and based on the two OCT images, the position coordinates of the surgical instrument in the OCT imaging coordinate system at this time can be detected As the surgical operation progresses, the position of the surgical instrument vertex will change. Therefore, when obtaining the historical position of the surgical instrument vertex in the OCT image at the previous moment, Then, the Kalman filter can be used to predict the optimal predicted position of the surgical instrument vertex in the OCT image at the next moment. .

[0102] In one embodiment, the optimal predicted position It can be obtained by: obtaining a motion prediction model based on the Kalman filter algorithm, and converting the historical position The information is input into the motion prediction model, and the output result of the motion prediction model is used as the optimal predicted position at the next moment. .

[0103] In operation S320, according to the optimal predicted position Perform coordinate mapping to determine the predicted position of the surgical instrument vertex in the second coordinate system at the second moment .

[0104] In operation S330, a detected position of the surgical instrument in the second coordinate system at the second moment is determined according to the predicted position.

[0105] In operation S340, a mapping position of the surgical instrument in the first coordinate system at a second moment is determined according to the detected position to track the intraoperative OCT imaging position.

[0106] In this embodiment, OCT imaging and surgical microscope imaging are combined, and Kalman filtering is used to estimate the apex position and movement speed of the surgical instrument, which can improve the positioning accuracy of the apex of the surgical instrument and thereby improve the accuracy of OCT imaging position tracking.

[0107] The present disclosure is described in detail below with reference to the accompanying drawings and specific embodiments.

[0108] For example, see Figure 7 The method for tracking intraoperative OCT imaging position provided by the embodiment of the present disclosure may include the following operations:

[0109] In operation S710 , a surgical instrument position is estimated in a fundus image at an initial moment and converted into an OCT scanning position.

[0110] Operation S720: performing an OCT scan at an initial time based on the OCT scanning position.

[0111] Operation S730: Process the OCT image obtained by the OCT scan to obtain the three-dimensional coordinate position of the surgical instrument under the OCT imaging coordinates at the initial moment.

[0112] Operation S740, performing Kalman filtering on the three-dimensional coordinate position of the surgical instrument in the OCT imaging coordinates at the initial moment, obtaining the optimal estimated position and movement speed of the surgical instrument in the OCT imaging coordinates at the next moment, and outputting the optimal estimated position and movement speed at that moment.

[0113] Operation S750, determining whether the fundus image at the current moment is updated? If updated, executing operation S760; if not updated, executing operation S770.

[0114] Operation S760: predict the position of the surgical instrument in the fundus image obtained by the next microscope imaging by using the optimal estimate, and convert it into a corresponding OCT scanning position.

[0115] In operation S770 , a Kalman filter is used to predict the next OCT scanning position, and an OCT scan is performed.

[0116] In this embodiment, in the preoperative preparation stage before performing the above operations S710-S770, the mapping relationship between OCT imaging and microscope imaging can be determined first. In one embodiment, the mapping relationship between OCT imaging and microscope imaging of each surgical microscope can be determined by performing OCT imaging and microscope imaging on the target eye before the surgical microscope leaves the factory. In other embodiments, the mapping relationship between OCT imaging and microscope imaging of the surgical microscope used in the surgery can also be determined by performing OCT imaging and microscope imaging on the treatment eye before each surgery after the surgical microscope leaves the factory.

[0117] Exemplarily, determining the mapping relationship between OCT imaging and microscope imaging includes the following operations: performing microscope imaging on the treated eye before surgery to obtain a corresponding two-dimensional eye image (in anterior segment surgery scenario, the eye image is an anterior segment plane image; in a posterior segment surgery scenario, the eye image is a fundus image); performing OCT volume scanning on the treated eye before surgery to obtain corresponding three-dimensional OCT volume data; performing axial projection on the three-dimensional OCT volume data to obtain a corresponding OCT axial projection image; matching the OCT axial projection image with the eye image to obtain a mapping relationship between the two: ,in, and It represents the coordinates of the position point in the OCT axial projection image under the first coordinate. and Indicates the coordinates of the position point in the eye map in the second coordinate system.

[0118] In this embodiment, in operation S710, it can be determined In one embodiment, at the beginning of the operation, the treated eye can be imaged in real time with a microscope to obtain an eye map, and the position of the surgical instrument vertex can be determined by image recognition of the eye map, and then the OCT scanning position at this time can be further determined based on the vertex position and the above mapping relationship. Exemplarily, when the surgical instrument is detected to appear in the eye map, the tracking mode is entered to track the OCT scanning position, and this moment is marked as , and at the same time identify the position of the surgical instrument vertex in the eye diagram at this time , and determine the OCT scanning position at this time according to the above mapping relationship .

[0119] Further, in this embodiment, At this moment, operation S750 is performed. If the eye map is updated (microscope imaging is updated), the predicted position of the surgical instrument vertex in the updated eye map is estimated first. , and then As the vertex and with length and width both ( Accurate position recognition is performed within the detection area to obtain the detection position of the apex of the surgical instrument . Or in At this moment, if the eye diagram is not updated, it is recorded as = , that is, the predicted position of the surgical instrument vertex in the updated eye map is directly used as its detected position in the eye map.

[0120] In this embodiment, the following operations can be performed to estimate the position of the surgical instrument vertex in the updated eye diagram ( The predicted position in the eye diagram at the moment :Will Detection position at the moment pass Map to the OCT imaging coordinate system to obtain the mapping position , and see Figure 3b ,exist Perform OCT vertical scanning at the position to obtain multiple OCT vertical scanning images, and Perform OCT horizontal scanning at the position to obtain multiple OCT horizontal scanning images. Further, fuse the multiple OCT vertical scanning images into one OCT vertical image (such as averaging one OCT vertical image), and fuse the multiple OCT horizontal scanning images into one OCT horizontal image (such as averaging one OCT horizontal image), and then use these two OCT images to identify the coordinate positions of the apex of the surgical instrument in the vertical direction, horizontal direction and axial direction respectively. . Further, the coordinate position is analyzed using Kalman filtering. exist The coordinate position of the surgical instrument at the moment is estimated and predicted. The optimal prediction estimate (i.e. the optimal prediction position) at time is . Further, the coordinate position Transform the coordinate position to the microscope coordinate system and get .

[0121] Following the above example, get the detected position Afterwards, you can Mapping is performed to obtain the mapping position in the OCT imaging coordinate system (first coordinate system) , and see Figure 3b,exist Perform OCT vertical scanning at the position to obtain multiple OCT vertical scanning images, and Perform OCT horizontal scanning at the position to obtain multiple OCT horizontal scanning images. Further, fuse multiple OCT vertical scanning images into one OCT vertical image, and fuse multiple OCT horizontal scanning images into one OCT horizontal image, and then use these two images to identify the position coordinates of the surgical instrument vertex in the vertical direction, horizontal direction and axial direction at this time. . Further, the coordinate position is analyzed using Kalman filtering. At the next moment ( The coordinate position of the surgical instrument vertex at this moment is estimated and predicted, and the optimal prediction estimate (i.e., the optimal prediction position) is obtained as . Further, the coordinate position Transformed to the microscope coordinate system (second coordinate system), we get , and use this as the basis for tracking the apex of the surgical instrument at the next moment.

[0122] In the above example, a real-time tracking solution for the apex position of surgical instruments is proposed by combining microscope imaging and OCT imaging, which is used for automatic tracking and imaging of the OCT scan of the surgical microscope during ophthalmic surgery. In this way, the three-dimensional coordinates and movement speed estimation of the surgical instrument can be given in real time, and the intraoperative OCT imaging position can be accurately given. As a result, the operating difficulty of doctors using surgical microscopes can be greatly reduced, thereby improving the user experience.

[0123] Figure 8 This is a structural block diagram of a device for tracking intraoperative OCT imaging positions provided in an embodiment of the present invention. The device is used to execute the method for tracking intraoperative OCT imaging positions provided in any of the above embodiments. The device and the method for tracking intraoperative OCT imaging positions in the above embodiments belong to the same inventive concept. For details not described in detail in the embodiment of the device for tracking intraoperative OCT imaging positions, please refer to the embodiment of the method for tracking intraoperative OCT imaging positions. Figure 8 , the device may specifically include: a first determination module 410 , a second determination module 420 and a third determination module 430 .

[0124] The first determination module 410 is configured to determine a predicted position of the surgical instrument in a second coordinate system at a second moment based on a historical position of the surgical instrument in a first coordinate system at a first moment, wherein the first coordinate system is used for OCT imaging and the second coordinate system is used for microscope imaging;

[0125] A second determination module 420 is configured to determine a detection position of the surgical instrument in the second coordinate system at a second moment according to the predicted position; and

[0126] The third determination module 430 is configured to determine the mapping position of the surgical instrument in the first coordinate system at the second moment according to the detected position, so as to track the intraoperative OCT imaging position.

[0127] Optionally, the first determination module 410 may be specifically configured to determine, in response to an update of the microscope imaging, a predicted position of the surgical instrument in the second coordinate system at a second moment based on a historical position of the surgical instrument in the first coordinate system at a first moment.

[0128] Optionally, the above-mentioned intraoperative OCT imaging position tracking device further includes:

[0129] a fourth determination module, configured to directly use the predicted position as a detected position of the surgical instrument in the second coordinate system at a second moment in response to the microscope imaging not being updated; and

[0130] The fifth determination module is configured to determine the mapping position of the surgical instrument in the first coordinate system at the second moment according to the detected position, so as to track the intraoperative OCT imaging position.

[0131] Optionally, the first determination module 410 is specifically configured to determine the predicted position of the surgical instrument in the second coordinate system at the second moment based on the historical position of the surgical instrument in the first coordinate system at the first moment in response to the surgical instrument appearing in the microscope imaging.

[0132] Optionally, the historical position includes: the historical position of the surgical instrument vertex in the first coordinate system at the first moment , where the surgical instrument vertex is the vertex of the surgical instrument;

[0133] The predicted position includes: the predicted position of the surgical instrument vertex in the second coordinate system at the second moment ;

[0134] The first determining module 410 may include:

[0135] The optimal predicted position estimation unit is configured to estimate the position of the surgical instrument vertex according to the historical position of the surgical instrument vertex in the first coordinate system at the first moment. , and use Kalman filtering to estimate the optimal predicted position of the surgical instrument vertex in the first coordinate system at the second moment ;as well as

[0136] The predicted position determination unit is configured to determine the optimal predicted position according to the Perform coordinate mapping to determine the predicted position of the surgical instrument vertex in the second coordinate system at the second moment .

[0137] Optionally, the predicted position includes: the predicted position of the surgical instrument vertex in the second coordinate system at the second moment , where the surgical instrument vertex is the vertex of the surgical instrument;

[0138] The second determining module 420 may include:

[0139] A detection area determination unit is configured to determine the predicted position of the apex of the surgical instrument according to the predicted position of the apex of the surgical instrument. , determining a detection area of ​​the surgical instrument in the second coordinate system at a second moment; and

[0140] A detection position recognition unit is configured to recognize a detection position of the surgical instrument in the second coordinate system at a second moment within the detection area. .

[0141] On this basis, optionally, the detection area determination unit is specifically configured to determine the predicted position of the apex of the surgical instrument according to the predicted position of the apex of the surgical instrument. , will be As the vertex, with length and width both being ( ) as the detection area of ​​the surgical instrument in the second coordinate system at the second moment.

[0142] Optionally, the mapping position includes: the mapping position of the surgical instrument vertex in the first coordinate system at the second moment , where the surgical instrument vertex is the vertex of the surgical instrument;

[0143] The above-mentioned intraoperative OCT imaging position tracking device further includes:

[0144] The control module is configured to determine the mapping position of the surgical instrument vertex in the first coordinate system at the second moment. After that, map the location Centered on the OCT scanning module of the surgical microscope to perform OCT cross scanning imaging; or, to map the position The first coordinate system is rotated by a predetermined angle and then the OCT scanning module of the surgical microscope is controlled to perform OCT cross scanning imaging.

[0145] Optionally, the mapping position includes: the mapping position of the surgical instrument vertex in the first coordinate system at the second moment , where the surgical instrument vertex is the vertex of the surgical instrument;

[0146] The above-mentioned intraoperative OCT imaging position tracking device further includes:

[0147] The sixth determination module is configured to determine the mapping position of the surgical instrument vertex in the first coordinate system at the second moment. After that, map the location Centered on the is a step length, a plurality of horizontal scanning position points are determined in the horizontal direction, and a plurality of vertical scanning position points are determined in the vertical direction;

[0148] A first obtaining module is configured to control the OCT scanning module of the surgical microscope to perform one OCT horizontal scan at each horizontal scanning position point to obtain a plurality of OCT horizontal scanning images;

[0149] A second obtaining module is configured to control the OCT scanning module of the surgical microscope to perform one OCT vertical scanning at each vertical scanning position point to obtain a plurality of OCT vertical scanning images;

[0150] The seventh determination module is configured to determine the corresponding OCT cross scanning images according to the multiple OCT horizontal scanning images and the multiple OCT vertical scanning images.

[0151] The intraoperative OCT imaging position tracking device provided by the embodiment of the present invention can determine the predicted position of the surgical instrument in the second coordinate system for microscope imaging at the second moment according to the historical position of the surgical instrument in the first coordinate system for OCT imaging at the first moment through the first determination module; then, in order to more accurately determine the instrument position of the surgical instrument in the second coordinate system at the second moment, considering that the accuracy of the instrument position calculated by microscope imaging is higher than the accuracy of the instrument position calculated by OCT imaging, the second determination module is used to determine the detection position of the surgical instrument in the second coordinate system at the second moment according to the predicted position; and then, the third determination module is used to determine the mapping position of the surgical instrument in the first coordinate system at the second moment according to the detection position, so as to realize the automatic tracking of the intraoperative OCT imaging position based on the mapping position. The above device, in ophthalmic surgery, realizes the automatic and accurate tracking of the surgical instrument in the OCT imaging process by combining microscope imaging and OCT imaging, and further realizes the automatic and accurate tracking of the intraoperative OCT imaging position on this basis, which helps to improve the doctor's experience of using the surgical microscope and shorten the operation time.

[0152] The intraoperative OCT imaging position tracking device provided in the embodiment of the present invention can execute the intraoperative OCT imaging position tracking method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0153] It is worth noting that in the above-mentioned embodiment of the tracking device for intraoperative OCT imaging position, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0154] Fig. 9 This is a structural block diagram of the surgical microscope system provided by an embodiment of the present invention. Fig. 9 , the system may include: a surgical microscope 510 and a control device 520, wherein the surgical microscope 510 is configured to have a microscope imaging module 5101 and an OCT imaging module 5102; and the control device 520 is configured to track and control the imaging position of the OCT imaging module 5102 according to the intraoperative OCT imaging position tracking method provided in any of the above embodiments. The system and the intraoperative OCT imaging position tracking method of each of the above embodiments belong to the same inventive concept, and the details not described in detail in the embodiment of the surgical microscope system may refer to the embodiment of the intraoperative OCT imaging position tracking method.

[0155] The surgical microscope system provided in the embodiment of the present invention realizes automatic and accurate tracking of surgical instruments during OCT imaging in ophthalmic surgery through the mutual cooperation of the surgical microscope and the control device, and further realizes automatic and accurate tracking of the intraoperative OCT imaging position on this basis, which helps to improve the doctor's experience of using the surgical microscope and shorten the operation time.

[0156] Fig.10 A schematic diagram of a control device 10 that can be used to implement an embodiment of the present invention is shown. The control device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The control device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0157] like Fig.10As shown, the control device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the control device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0158] A number of components in the control device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the control device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0159] The processor 11 may be a variety of general and / or dedicated processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a method for tracking intraoperative OCT imaging positions.

[0160] In some embodiments, the method for tracking the intraoperative OCT imaging position may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the control device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the method for tracking the intraoperative OCT imaging position described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the method for tracking the intraoperative OCT imaging position by any other appropriate means (e.g., by means of firmware).

[0161] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0162] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0163] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0164] To provide interaction with a user, the systems and techniques described herein may be implemented on a control device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the control device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0165] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0166] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0167] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0168] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A device for tracking intraoperative OCT imaging position, comprising: A first determination module is configured to determine a predicted position of the surgical instrument in a second coordinate system of microscope imaging at a second moment based on a historical position of the surgical instrument in a first coordinate system of OCT imaging at a first moment; a second determination module configured to determine a detection position of the surgical instrument in the second coordinate system at the second moment according to the predicted position, wherein the detection position is obtained by performing position detection on the surgical instrument based on the predicted position in combination with a microscope imaging formed at the second moment; and a third determination module, configured to determine, based on the detected position, a mapping position of the surgical instrument in the first coordinate system at the second moment, so as to track an intraoperative OCT imaging position; The mapping position includes: the mapping position of the surgical instrument vertex in the first coordinate system at the second moment , wherein the surgical instrument vertex is the vertex of the surgical instrument; The device further comprises: a control module; wherein the control module is configured to determine the mapping position of the surgical instrument vertex in the first coordinate system at the second moment After that, the mapping position As the center, control the OCT scanning module of the surgical microscope to perform OCT cross scanning imaging; or, with the mapping position As the center, the first coordinate system is rotated by a predetermined angle, and then the OCT scanning module of the surgical microscope is controlled to perform OCT cross scanning imaging; Alternatively, the device further comprises: a sixth determining module, a first obtaining module, a second obtaining module and a seventh determining module; wherein the sixth determining module is configured to determine the mapping position of the surgical instrument vertex in the first coordinate system at the second moment After that, the mapping position Centered at a predetermined scanning interval is a step size, a plurality of horizontal scanning position points are determined in the horizontal direction, and a plurality of vertical scanning position points are determined in the vertical direction; the first obtaining module is configured to control the OCT scanning module of the surgical microscope to perform one OCT horizontal scan at each horizontal scanning position point to obtain a plurality of OCT horizontal scanning images; the second obtaining module is configured to control the OCT scanning module of the surgical microscope to perform one OCT vertical scan at each vertical scanning position point to obtain a plurality of OCT vertical scanning images; the seventh determining module is configured to determine the corresponding OCT cross scanning imaging according to the plurality of OCT horizontal scanning images and the plurality of OCT vertical scanning images.

2. The intraoperative OCT imaging position tracking device according to claim 1, wherein: The first determination module is configured to execute the operation of determining the predicted position of the surgical instrument in the second coordinate system at the second moment based on the historical position of the surgical instrument in the first coordinate system at the first moment in response to an update of the microscope imaging.

3. The intraoperative OCT imaging position tracking device according to claim 1, further comprising: a fourth determination module, configured to directly use the predicted position as the detected position of the surgical instrument in the second coordinate system at the second moment in response to the microscope imaging not being updated; The fifth determination module is configured to determine the mapping position of the surgical instrument in the first coordinate system at the second moment according to the detected position, so as to track the intraoperative OCT imaging position.

4. The intraoperative OCT imaging position tracking device according to claim 1, wherein: The first determination module is configured to execute the operation of determining the predicted position of the surgical instrument in the second coordinate system at the second moment based on the historical position of the surgical instrument in the first coordinate system at the first moment in response to the surgical instrument appearing in the microscope imaging.

5. The intraoperative OCT imaging position tracking device according to claim 1, wherein: The historical position includes: the historical position of the surgical instrument vertex in the first coordinate system at the first moment , wherein the surgical instrument vertex is the vertex of the surgical instrument; The predicted position includes: the predicted position of the surgical instrument vertex at the second moment in the second coordinate system ; The first determining module includes: The optimal predicted position estimation unit is configured to estimate the position of the surgical instrument vertex according to the historical position of the surgical instrument vertex in the first coordinate system at the first moment. , and using Kalman filtering to estimate the optimal predicted position of the surgical instrument vertex in the first coordinate system at the second moment ; A predicted position determining unit is configured to determine the optimal predicted position according to the optimal predicted position. Perform coordinate mapping to determine the predicted position of the surgical instrument vertex in the second coordinate system at the second moment .

6. The intraoperative OCT imaging position tracking device according to claim 1, wherein: The predicted position includes: the predicted position of the surgical instrument vertex in the second coordinate system at the second moment , wherein the surgical instrument vertex is the vertex of the surgical instrument; The second determining module includes: A detection area determination unit is configured to determine the predicted position of the apex of the surgical instrument according to the predicted position of the apex of the surgical instrument. , determining a detection area of ​​the surgical instrument in the second coordinate system at the second moment; a detection position recognition unit configured to recognize, within the detection area, a detection position of the surgical instrument in the second coordinate system at the second moment; .

7. The intraoperative OCT imaging position tracking device according to claim 6, wherein: The detection area determination unit is configured to determine the predicted position of the apex of the surgical instrument according to the predicted position of the apex of the surgical instrument. , will be As the vertex, with length and width both being ( ) as the detection area of ​​the surgical instrument in the second coordinate system at the second moment.

8. A surgical microscope system, comprising: A surgical microscope configured to have a microscope imaging module and an OCT imaging module; as well as The control device is configured to track and control the imaging position of the OCT imaging module according to the tracking method performed by the intraoperative OCT imaging position tracking device according to any one of claims 1 to 7.

9. A control device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor executes the tracking method performed by the intraoperative OCT imaging position tracking device as described in any one of claims 1-7.

10. A computer-readable storage medium, wherein: The computer-readable storage medium stores computer instructions, and the computer instructions are configured to enable a processor to implement the tracking method performed by the intraoperative OCT imaging position tracking device according to any one of claims 1 to 7 when executed.

Images