Method and apparatus for tracking intra-operative oct imaging position, surgical microscope system, and storage medium

Abstract

Disclosed in the present application are a method and apparatus for tracking an intra-operative OCT imaging position, a surgical microscope system, and a storage medium. The method for tracking an intra-operative OCT imaging position comprises: according to a historical position of a surgical instrument in a first coordinate system at a first moment, determining a predicted position of the surgical instrument in a second coordinate system at a second moment, wherein the first coordinate system is used for OCT imaging, and the second coordinate system is used for microscope imaging; according to the predicted position, determining a detected position of the surgical instrument in the second coordinate system at the second moment; and according to the detected position, determining a mapped position of the surgical instrument in the first coordinate system at the second moment, so as to track the intra-operative OCT imaging position.

Description

Intraoperative OCT imaging position tracking method and device, surgical microscope system, and storage medium

[0001] This application claims priority to the Chinese patent application No. 202411238472.0, filed on September 5, 2024, with the Chinese Patent Office, the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the technical fields of OCT image processing, camera image processing, surgical microscopes, ophthalmic surgical microscopes, etc., for example, to an intraoperative OCT imaging position tracking method, device, surgical microscope system, and storage medium. BACKGROUND

[0003] Taking an ophthalmic surgical microscope as an example, some surgical microscopes have microscope imaging functions and optical coherence tomography (OCT) imaging functions. During surgery, the two imaging functions of the surgical microscope can be combined to observe the surgical site.

[0004] In related technologies, due to design problems, the OCT scanning position of the surgical microscope needs to be manually adjusted by additional doctors during use. This manual adjustment method is limited by the experience of doctors and may have a great impact on the operation time. SUMMARY

[0005] The present application provides an intraoperative OCT imaging position tracking method, device, surgical microscope system, and storage medium, which can realize automatic tracking of the intraoperative OCT imaging position, thereby avoiding the impact of the experience of doctors on the operation time.

[0006] The present application provides an intraoperative OCT imaging position tracking method, comprising: determining a predicted position of a surgical instrument in a second coordinate system at a second time according to a historical position of the surgical instrument in a first coordinate system at a first time, wherein the first coordinate system is used for OCT imaging, and the second coordinate system is used for microscope imaging; determining a detected position of the surgical instrument in the second coordinate system at the second time according to the predicted position; and determining a mapping position of the surgical instrument in the first coordinate system at the second time according to the detected position, so as to track the intraoperative OCT imaging position.

[0007] The application provides a tracking device for an intraoperative OCT imaging position, comprising: a first determining module configured to determine a predicted position of a surgical instrument in a second coordinate system at a second time according to a historical position of the surgical instrument in a first coordinate system at a first time, wherein the first coordinate system is used for OCT imaging, and the second coordinate system is used for microscope imaging; a second determining module configured to determine a detected position of the surgical instrument in the second coordinate system at the second time according to the predicted position; and a third determining module configured to determine a mapped position of the surgical instrument in the first coordinate system at the second time according to the detected position, so as to track the intraoperative OCT imaging position.

[0008] The application provides a surgical microscope system, comprising: a surgical microscope configured to have a microscope imaging module and an OCT imaging module; and a control device configured to track and control an imaging position of the OCT imaging module according to the above-mentioned tracking method for an intraoperative OCT imaging position.

[0009] The application provides a control device, which can comprise: at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores a computer program which can be executed by the at least one processor, and the computer program is executed by the at least one processor, so that the at least one processor is implemented to execute the above-mentioned tracking method for an intraoperative OCT imaging position.

[0010] The application provides a computer readable storage medium, which stores computer instructions for making a processor execute the above-mentioned tracking method for an intraoperative OCT imaging position. BRIEF DESCRIPTION OF DRAWINGS

[0011] Fig. 1 is a flow chart of a tracking method for an intraoperative OCT imaging position according to an embodiment of the application;

[0012] Fig. 2a is a schematic diagram of microscope imaging according to an embodiment of the application;

[0013] Fig. 2b is a schematic diagram of transverse imaging of OCT cross scanning according to an embodiment of the application;

[0014] Fig. 2c is a schematic diagram of longitudinal imaging of OCT cross scanning according to an embodiment of the application;

[0015] Fig. 3a is a schematic diagram of determining a scanning position corresponding to OCT cross scanning imaging according to an embodiment of the application;

[0016] Fig. 3b is another schematic diagram of determining a scanning position corresponding to OCT cross scanning imaging according to an embodiment of the application;

[0017] FIG. 4 is a contrast diagram of a first coordinate system before and after rotation in intraoperative OCT imaging position tracking according to an embodiment of the present application;

[0018] FIG. 5 is a flowchart of another method of intraoperative OCT imaging position tracking according to an embodiment of the present application;

[0019] FIG. 6 is a flowchart of yet another method of intraoperative OCT imaging position tracking according to an embodiment of the present application;

[0020] FIG. 7 is a flowchart of an optional example in yet another method of intraoperative OCT imaging position tracking according to an embodiment of the present application;

[0021] FIG. 8 is a structural block diagram of a device for intraoperative OCT imaging position tracking according to an embodiment of the present application;

[0022] FIG. 9 is a structural block diagram of a surgical microscope system according to an embodiment of the present application;

[0023] FIG. 10 is a schematic diagram of a control device for implementing a method of intraoperative OCT imaging position tracking according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] The terms "first", "second", and the like in the description and claims of the present application and above-described drawings are used to distinguish similar objects, and do not necessarily indicate a particular order or a sequence. It should be understood that the data thus used can be interchanged, where appropriate, so that the embodiments of the present application described herein can be carried out in other than the order shown or described herein. The same applies to the cases of "target", "original", and the like, which are not described here again. In addition, the terms "comprise" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or device comprising a series of steps or units shown in the embodiments of the present application can also include other processes, methods, systems, products, and devices not clearly listed in the series of steps or units, or other steps or units inherent to these processes, methods, systems, products, or devices.

[0025] As described above, the surgical microscope provided in the prior art cannot automatically position the OCT scanning position of the OCT module due to design problems, resulting in the need for additional doctors to assist in manually adjusting the OCT scanning position of the OCT module in the surgical microscope when using the surgical microscope. The additional doctors not only disadvantage the hospital in coordinating doctor resources, but also increase the cost of medical care in the hospital and the cost of medical treatment for patients. Moreover, the manual adjustment method is limited by the experience of doctors, and if the doctor assisting in adjusting the OCT scanning position of the OCT module is not skilled, it will have a greater impact on the length of the surgery, and may even increase the risk and difficulty of the surgery, thereby possibly endangering the safety of the patient.

[0026] The embodiment of the present application provides an intraoperative OCT imaging position tracking method, which can automatically adjust the OCT scanning position of the OCT module without additional doctors assisting in manual adjustment.

[0027] In some embodiments, the OCT scanning position of the OCT module in the surgical microscope at the next moment can be positioned according to the OCT imaging of the surgical microscope at the previous moment, that is, the intraoperative OCT imaging position is tracked only according to the OCT imaging without combining the microscope imaging. The OCT scanning speed of the surgical microscope is slower than the microscope imaging speed of the surgical microscope, so if the OCT scanning position of the OCT module at the next moment is tracked and positioned only based on the OCT imaging (three-dimensional image) at the previous moment, it may be difficult to obtain more accurate positioning information.

[0028] In other embodiments, the OCT scanning position of the OCT module in the surgical microscope can also be tracked and positioned based on the OCT imaging of the surgical microscope and combined with the microscope imaging (two-dimensional image) thereof. The positioning method provided in this embodiment can obtain more accurate positioning information. The embodiment of the present application will be described in detail below in combination with the drawings and specific embodiments.

[0029] The embodiment provides an intraoperative OCT imaging position tracking method.

[0030] FIG. 1 is a flowchart of an intraoperative OCT imaging position tracking method according to an embodiment of the present application.

[0031] The method can be performed by an intraoperative OCT imaging position tracking device provided by the embodiment of the present application. The device can be realized by software and / or hardware, and the device can be integrated on a control device. The control device can be a user terminal or a server.

[0032] The method provided by the embodiment can be used to automatically position and accurately track the OCT scanning position (OCT imaging position) of the OCT module in the surgical microscope, such as an ophthalmic surgical microscope.

[0033] Referring to FIG. 1, in the embodiment of the present application, the method for tracking the OCT imaging position during surgery can include operations S110-S130.

[0034] In operation S110, a predicted position of the surgical instrument in the second coordinate system at a second time is determined according to a historical position of the surgical instrument in the first coordinate system at a first time, wherein the first coordinate system is used for OCT imaging, and the second coordinate system is used for microscope imaging.

[0035] Taking eye surgery as an example, the surgeon in charge during surgery usually pays more attention to the eye structure directly below the surgical instrument, especially the eye structure directly below the top end of the surgical instrument, so the surgeon in charge during surgery hopes that the scanning position of the OCT module in the surgical microscope can be positioned directly below the surgical instrument, especially directly below the top end of the surgical instrument. Therefore, in the embodiment of the present application, in operation S110, the OCT scanning position of the OCT module in the surgical microscope is positioned and tracked according to the position of the surgical instrument performing the surgical operation during surgery.

[0036] In the embodiment of the present application, different coordinate systems can be defined for OCT imaging and microscope imaging. For example, a first coordinate system can be defined for OCT imaging, and a second coordinate system can be defined for microscope imaging, wherein the first coordinate system is different from the second coordinate system, but there is a certain mapping relationship between the first coordinate system and the second coordinate system. In this embodiment, the mapping relationship between the first coordinate system and the second coordinate system can be determined according to the microscope imaging and the OCT imaging of the surgical microscope. For example, FIG. 2a shows a schematic diagram of microscope imaging corresponding to the second coordinate system; FIG. 2b and FIG. 2c show schematic diagrams of OCT cross-scan imaging corresponding to the first coordinate system, FIG. 2b is a schematic diagram of horizontal imaging of OCT cross-scan, and FIG. 2c is a schematic diagram of vertical imaging of OCT cross-scan.

[0037] In the embodiment of the present application, the first time is an earlier time, and the second time is a later time. Therefore, in operation S110, the "historical position" of the surgical instrument in the first coordinate system at the first time is a relative concept, which is relative to the "second time". In some embodiments, in the case that the "first time" is a historical time that has occurred, the "historical position" can be understood as the position of the surgical instrument in the first coordinate system at the historical time. In other embodiments, in the case that the "first time" is a current time that is occurring, the "historical position" can be understood as the position of the surgical instrument in the first coordinate system at the current time. In other embodiments, in the case that the "first time" is a future time that has not occurred, the "historical position" can be understood as the position of the surgical instrument in the first coordinate system at the future time.

[0038] The position of the surgical instrument in the operation can be displaced in the operation site as the operation proceeds. In this embodiment, the motion tracking algorithm can be used to predict the motion speed of the surgical instrument at each time point.

[0039] Taking an ophthalmic operation (operation on the treated eye of a patient) as an example, in operation S110, the OCT imaging of the treated eye at a previous time point (denoted as t1) can be acquired, and the position information of the surgical instrument performing the operation in the OCT imaging at the time t1 can be determined according to the OCT imaging. The position information can represent the historical position of the surgical instrument in the first coordinate system at the time t1. Further, according to the historical position, the mapping relationship between the first coordinate system and the second coordinate system, and the motion speed of the surgical instrument at the time t1-t2 obtained by the motion tracking algorithm, the position information of the surgical instrument in the microscope imaging at a subsequent time point (denoted as t2) can be predicted. The position information can represent the predicted position of the surgical instrument in the second coordinate system at the time t2.

[0040] In operation S120, the detected position of the surgical instrument in the second coordinate system at the second time point can be determined according to the predicted position.

[0041] In this embodiment, the predicted position is a prediction result obtained according to the historical position acquired in operation S110 and the pre-set algorithm, which can be different from the actual position of the surgical instrument in the second coordinate system at the second time point. Therefore, in operation S120, the position of the surgical instrument is detected in the microscope imaging formed by the surgical microscope at the second time point according to the predicted position, so as to obtain the detected position of the surgical instrument in the second coordinate system at the second time point. The detected position is less different from, or even identical to, the actual position of the surgical instrument in the second coordinate system at the second time point.

[0042] In this way, the corresponding detected position is further identified in the corresponding detection region based on the predicted position, so as to track the OCT scanning position of the OCT module of the surgical microscope, and the OCT scanning position of the OCT module at the second time point can be more accurately located.

[0043] In operation S130, the mapping position of the surgical instrument in the first coordinate system at the second time point can be determined according to the detected position, so as to track the intraoperative OCT imaging position.

[0044] In this embodiment, the position of the surgical instrument in the OCT imaging at the second time point can be determined according to the detected position and the mapping relationship between the first coordinate system and the second coordinate system. The position is the mapping position of the surgical instrument in the first coordinate system at the second time point, and the position can be used as the OCT scanning position of the OCT module, so as to track the intraoperative OCT imaging position.

[0045] In the embodiments of the present application, because the OCT imaging speed of the surgical microscope is slow, it is difficult to obtain accurate prediction results by directly predicting the position of the surgical instrument in the OCT imaging formed by the surgical microscope at the next time instant according to the OCT imaging formed by the surgical microscope at the previous time instant.

[0046] In the embodiments, the OCT scanning position is tracked by jointly using the OCT imaging and the microscope imaging to position the surgical instrument. In this way, the defect that the positioning result is not accurate enough due to the slow OCT imaging speed when the OCT scanning position is positioned only by using the OCT imaging can be avoided, so that more accurate OCT scanning position positioning can be obtained.

[0047] The OCT scanning position automatic positioning method provided in the embodiments of the present application does not need additional manual positioning by doctors, so compared with the manual positioning method, the automatic positioning method is more conducive to the coordination of doctor resources in the hospital, and does not increase the medical cost of the hospital and the medical cost of the patient. Moreover, the automatic positioning method is not limited by the experience of doctors, and can avoid the great influence on the operation time caused by the unskilled operation of doctors, so as to reduce the operation risk and the operation difficulty, and reduce the harm to the safety of patients. Moreover, as described above, because the OCT imaging speed is slower than the microscope imaging, if the intraoperative OCT imaging position is tracked only by using the OCT imaging, the positioning is not accurate. However, in the embodiments of the present application, the OCT imaging and the microscope imaging are combined to track the intraoperative OCT imaging position, so that the defect caused by tracking only by using the OCT imaging can be overcome, and more accurate tracking of the intraoperative OCT imaging position can be achieved.

[0048] In an optional embodiment, the tracking of the intraoperative OCT imaging position can be performed according to operations S110-S130 when the microscope imaging is updated, that is, in response to the update of the microscope imaging, the operations of determining the predicted position of the surgical instrument in the second coordinate system at the second time instant according to the historical position of the surgical instrument in the first coordinate system at the first time instant, and subsequent operations S120 and S130 are sequentially performed, so that the tracking of the intraoperative OCT imaging position is completed.

[0049] In one embodiment, if there is a change in the position of the surgical instrument at the later time compared to the earlier time, then the microscope imaging at the later time is considered to be updated compared to the microscope imaging at the earlier time. If there is no change in the position of the surgical instrument at the later time compared to the earlier time, then the microscope imaging at the later time is considered to be not updated compared to the microscope imaging at the earlier time. Based on this, in the case of microscope imaging update, the tracking of the intraoperative OCT imaging position is performed according to operation S110-S130, which can ensure the accuracy of the tracking of the surgical instrument, and thus can ensure the accuracy of the tracking of the intraoperative OCT imaging position.

[0050] In another embodiment, regardless of whether there is a change in the position of the surgical instrument at the later time compared to the earlier time, as long as there is microscope imaging output at the later time, then the microscope imaging at the later time is considered to be updated compared to the microscope imaging at the earlier time, and if there is no microscope imaging output at the later time, then the microscope imaging at the later time is considered to be not updated compared to the microscope imaging at the earlier time. Based on this, in the case of microscope imaging update, the tracking of the intraoperative OCT imaging position is performed according to operation S110-S130, which can ensure the accuracy of the tracking of the surgical instrument, and thus can ensure the accuracy of the tracking of the intraoperative OCT imaging position.

[0051] If the microscope imaging at the later time compared to the microscope imaging at the earlier time, in which there is no change in the position of the surgical instrument (no microscope imaging update), then the predicted position of the surgical instrument at the second time in the second coordinate system and the detected position of the surgical instrument at the second time in the second coordinate system determined according to the predicted position are substantially consistent. Therefore, in this embodiment, the tracking of the intraoperative OCT imaging position can be performed according to the following operations in the case of no microscope imaging update.

[0052] According to the historical position of the surgical instrument at the first time in the first coordinate system, a predicted position of the surgical instrument at the second time in the second coordinate system is determined; in response to no microscope imaging update, the predicted position is directly taken as a detected position of the surgical instrument at the second time in the second coordinate system; and according to the detected position, a mapped position of the surgical instrument at the second time in the first coordinate system is determined to track the intraoperative OCT imaging position.

[0053] Through the embodiments of the present application, since the detection related operations can be omitted by directly taking the predicted position as the detected position instead of demarcating the detection area according to the predicted position and determining the corresponding detected position in the detection area according to a specific algorithm, the tracking efficiency of the intraoperative OCT imaging position can be improved on the basis of ensuring the tracking accuracy.

[0054] In the embodiments of the present application, when the surgical microscope is used to carry out surgery, the tracking of the OCT imaging position can be started when the surgical instrument enters the field of view of the surgical site, and the tracking of the OCT imaging position can be paused or stopped before the surgical instrument enters the field of view of the surgical site or after the surgical instrument leaves the field of view of the surgical site. In this way, the computing power of the device can be saved, and the operation efficiency and operation quality of other functions of the device can be improved.

[0055] Therefore, in an optional embodiment of the present application, the operation S110 of determining the predicted position of the surgical instrument in the second coordinate system at the second time according to the historical position of the surgical instrument in the first coordinate system at the first time, and the subsequent operation S120 and operation S130 can be started in response to the appearance of the surgical instrument in the microscopic imaging.

[0056] The peripheral region where the tip of the surgical instrument is located in the surgery is usually the position that the doctor focuses on in the surgical site. Therefore, in an optional embodiment of the present application, the predicted position can be defined as: the predicted position of the top point of the surgical instrument in the second coordinate system at the second time Then, according to the predicted position, the detected position of the surgical instrument in the second coordinate system at the second time is determined, including: according to the predicted position of the top point of the surgical instrument The detected region of the surgical instrument in the second coordinate system at the second time is determined; and the detected position of the surgical instrument in the second coordinate system at the second time (u t , v t ).

[0057] In the embodiments, the top point position of the surgical instrument is used to represent the instrument position of the surgical instrument, because in ophthalmic surgery, the doctor pays more attention to the eye structure directly below the surgical instrument, and therefore, when the OCT imaging position is obtained by tracking the top point of the surgical instrument and OCT imaging is performed at the imaging position, it is consistent with the actual needs of the doctor.

[0058] In other embodiments, for example, for smaller surgical instruments, the middle position of the surgical instrument can also be used to represent the position of the surgical instrument. In this case, the intraoperative OCT imaging position can be tracked according to the predicted position of the middle position of the surgical instrument in the second coordinate system at the second time.

[0059] In the embodiments of the present application, the detected region can be understood as a region defined according to the predicted position in the second coordinate system at the second time and used to detect the actual position of the surgical instrument. In combination with the application scenarios that can be involved in the embodiments of the present application, the detected region can be understood as a region used to detect the actual position of the surgical instrument in the eye diagram (ophthalmic microscopic imaging) corresponding to the second time, for example. In an embodiment of the present application, according to the predicted position of the top point of the surgical instrument The detection region is determined, for example, according to the predicted position of the vertex of the surgical instrument The detection region is determined, for example, according to the predicted position of the vertex of the surgical instrument The detection region is determined, for example, according to the predicted position of the vertex of the surgical instrument The detection region is determined, for example, according to the predicted position of the vertex of the surgical instrument

[0060] In the detection region, the detection position (u t ,v t ) of the surgical instrument at the second time in the second coordinate system is identified, for example, the detection position (u t ,v t ) of the vertex of the surgical instrument.

[0061] In the embodiments of the present application, the detection region in which the detection position is roughly estimated by the predicted position is determined, and then the detection position is further accurately positioned in the detection region, so that the accuracy of positioning is ensured and the processing efficiency of positioning operation is improved.

[0062] In an optional embodiment of the present application, the mapping position of the vertex of the surgical instrument at the second time in the first coordinate system can be defined as (x

[0063] Based on this, the above-mentioned tracking method of intraoperative OCT imaging position can further include: after determining the mapping position (x of the vertex of the surgical instrument at the second time in the first coordinate system, controlling the OCT scanning module of the surgical microscope to perform OCT cross scanning imaging with the mapping position (x as the center; or first rotating the first coordinate system by a predetermined angle, and then controlling the OCT scanning module of the surgical microscope to perform OCT cross scanning imaging with the mapping position (x as the center.

[0064] In the embodiments of the present application, after determining the mapping position (x of the vertex of the surgical instrument at the second time in the first coordinate system, the OCT scanning module of the surgical microscope can be controlled to perform OCT cross scanning imaging based on the mapping position (x , so as to obtain the OCT cross scanning image based on the mapping position (x .OCT cross scan imaging. In the embodiments of the present application, the mapping position of the vertex of the surgical instrument at the second time in the first coordinate system is taken as the position of the vertex of the surgical instrument at the second time in the first coordinate system, and OCT cross scan imaging is performed based on the position. Taking ophthalmic surgery as an example, the surgeon can conveniently observe the surgical site of the eye tissue in the horizontal and vertical imaging directions based on the OCT image, and smoothly carry out ophthalmic surgery. In the embodiments of the present application, the mapping position of the vertex of the surgical instrument at the second time in the first coordinate system can also be taken as the historical position of the vertex of the surgical instrument at the next time in the first coordinate system, so as to continuously track the intraoperative OCT imaging position.

[0065] In the embodiments, the mapping position is taken as the center, and the OCT scanning module of the surgical microscope is controlled to perform OCT cross scan imaging based on the center. For example, as shown in FIG. 3a, where repeat (such as repeat = 2*N+1) represents the number of times of repeating OCT cross scan on the same position, and N represents a positive integer, that is, taking the mapping position as the center, (2*N+1) OCT cross scan imaging is performed in the horizontal direction (horizontal imaging direction) and the vertical direction (vertical imaging direction).

[0066] In some surgical scenarios, the pose of the surgical instrument can change as the surgery progresses, for example, the direction of the surgical instrument can rotate. Therefore, in other embodiments, the mapping position is taken as the center, and the OCT scanning module is controlled to perform OCT cross scan imaging based on the center after the first coordinate system is rotated by a predetermined angle. The predetermined angle of rotation of the first coordinate system can be the same as the direction angle of rotation of the surgical instrument. In the embodiments of the present application, by rotating the first coordinate system by a predetermined angle, one coordinate axis in the first coordinate system can be parallel to the surgical instrument, and the other coordinate axis can be perpendicular to the surgical instrument. For example, as shown in FIG. 4, the left side of the figure shows the relative position relationship between the surgical instrument and the first coordinate system before rotation, and the right side of the figure shows the relative position relationship between the surgical instrument and the first coordinate system after rotation. In the embodiments, after the direction of the surgical instrument changes, OCT cross scan imaging is performed based on the rotated first coordinate system, which can change the imaging direction of OCT cross scan imaging according to the direction change of the surgical instrument, that is, the direction of the horizontal and vertical coordinate axes in the new first coordinate system can be taken as the OCT imaging direction, and the corresponding OCT cross-sectional image can be obtained to meet the actual observation requirements of the surgeon, thereby realizing more flexible intraoperative navigation effect.

[0067] In another alternative embodiment of the present application, the mapping position can be defined as: the mapping position of the vertex of the surgical instrument at the second time in the first coordinate system

[0068] Based on this, the aforementioned intraoperative OCT imaging position tracking method further includes: determining the mapped position of the apex of the surgical instrument in the first coordinate system at a second time. Then, using the mapped position Centered on a predetermined scanning interval δ, multiple horizontal scanning positions are determined in the horizontal direction and multiple vertical scanning positions are determined in the vertical direction. The OCT scanning module of the surgical microscope is controlled to perform one OCT horizontal scan at each horizontal scanning position, resulting in multiple OCT horizontal scan images. The OCT scanning module of the surgical microscope is controlled to perform one OCT vertical scan at each vertical scanning position, resulting in multiple OCT vertical scan images. Based on the multiple OCT horizontal scan images and the multiple OCT vertical scan images, the corresponding OCT cross-scan imaging is determined.

[0069] Referring to Figure 5, the method for tracking the position of intraoperative OCT imaging may include the following operations:

[0070] In operation S210, based on the historical position of the surgical instrument in the first coordinate system at the first moment, the predicted position of the surgical instrument in the second coordinate system at the second moment is determined, wherein the first coordinate system is used for OCT imaging and the second coordinate system is used for microscope imaging.

[0071] During operation S220, the detection position of the surgical instrument in the second coordinate system is determined based on the predicted position.

[0072] During operation S230, based on the detected position, the mapped position of the apex of the surgical instrument in the first coordinate system at the second moment is determined. To track the position of intraoperative OCT imaging.

[0073] Operations S210 to S230 correspond to or are similar to operations S110 to S130 in the previous embodiments, and will not be described again in this embodiment.

[0074] In operation S240, to map the position Centered on a predetermined scanning interval δ, multiple horizontal scanning position points are determined in the horizontal direction, and multiple vertical scanning position points are determined in the vertical direction.

[0075] For example, see Figure 3b for a mapping position. Centered on a predetermined scanning interval δ, five vertical scanning position points x are determined in the horizontal direction, where, In the diagram, N=2, and five horizontal scanning positions y are determined in the vertical direction.

[0076] The scanning range of the OCT cross scan in the example shown in FIG. 3a is small, and the scanning range of the OCT cross scan in the example shown in FIG. 3b is relatively large. Taking ophthalmic surgery as an example, the OCT cross scan shown in FIG. 3b can scan the apex of the surgical instrument and the eye tissue around the apex, thereby being more conducive to subsequent effective tracking of the OCT imaging position.

[0077] In operation S250, the OCT scanning module of the surgical microscope is controlled to perform one OCT horizontal scan through each horizontal scan position point, to obtain a plurality of OCT horizontal scan images.

[0078] In this embodiment, the OCT scanning module can be controlled to perform one OCT horizontal scan through each horizontal scan position point respectively, thereby obtaining the OCT horizontal scan images corresponding to the plurality of horizontal scan position points respectively.

[0079] In operation S260, the OCT scanning module of the surgical microscope is controlled to perform one OCT vertical scan through each vertical scan position point, to obtain a plurality of OCT vertical scan images.

[0080] In this embodiment, the OCT scanning module can be controlled to perform one OCT vertical scan through each vertical scan position point respectively, thereby obtaining the OCT vertical scan images corresponding to the plurality of vertical scan position points respectively.

[0081] In one embodiment, after each scan position point is determined, the corresponding OCT horizontal scan and OCT vertical scan can be directly performed based on each scan position point. In another embodiment, referring to the example shown in FIG. 4, the first coordinate system can be first rotated by a predetermined angle according to the change in the direction of the surgical instrument, and then the OCT cross scan can be performed based on the directions indicated by the two coordinate axes in the rotated first coordinate system.

[0082] In operation S270, the corresponding OCT cross scan imaging is determined according to the plurality of OCT horizontal scan images and the plurality of OCT vertical scan images.

[0083] For example, at least one OCT horizontal scan image can be selected from the plurality of OCT horizontal scan images, and at least one OCT vertical scan image can be selected from the plurality of OCT vertical scan images, and then the corresponding OCT cross scan imaging is determined based on the selection results.

[0084] As another example, a plurality of OCT horizontal scanning images can be fused to obtain an OCT horizontal fusion image, and a plurality of OCT vertical scanning images can be fused to obtain an OCT vertical fusion image, and based on the OCT horizontal fusion image and the OCT vertical fusion image, corresponding OCT cross scanning imaging is determined. In this example, in the case where the surgical instrument is located within the scanning range, through image fusion, it can be ensured that the surgical instrument necessarily appears in the OCT horizontal fusion image and the OCT vertical fusion image, which is beneficial for effective tracking of the subsequent OCT imaging position.

[0085] In the embodiments of the present application, the mapping position As the center, OCT cross scanning is performed on a plurality of horizontal scanning position points and a plurality of vertical scanning position points, and this scanning mode can cover a larger scanning range, so as to improve the possibility of OCT scanning to the vertex of the surgical instrument.

[0086] In an optional embodiment of the present application, the historical position can be defined to include: a historical position (x0, y0) of the vertex of the surgical instrument in the first coordinate system at the first time; and the predicted position can be defined to include: a predicted position (x1, y1) of the vertex of the surgical instrument in the second coordinate system at the second time. According to the historical position of the surgical instrument in the first coordinate system at the first time, the predicted position of the surgical instrument in the second coordinate system at the second time is determined, including: according to the historical position (x0, y0) of the vertex of the surgical instrument in the first coordinate system at the first time, and using Kalman filtering to estimate an optimal predicted position (x1, y1) of the vertex of the surgical instrument in the first coordinate system at the second time. According to the optimal predicted position Coordinate mapping is performed to determine the predicted position (x1, y1) of the vertex of the surgical instrument in the second coordinate system at the second time.

[0087] Referring to FIG. 6, the method of the present embodiment can include the following operations:

[0088] Operation S310, according to the historical position (x0, y0) of the vertex of the surgical instrument in the first coordinate system at the first time, and using Kalman filtering to estimate an optimal predicted position (x1, y1) of the vertex of the surgical instrument in the first coordinate system at the second time. The first coordinate system is used for OCT imaging, and the second coordinate system is used for microscope imaging.

[0089] For example, referring to FIG. 3b, based on the historical position (x0, y0), a plurality of OCT horizontal scanning positions and a plurality of OCT vertical scanning positions at the first time are determined, and OCT scanning is performed to obtain a plurality of OCT horizontal scanning images and a plurality of OCT vertical scanning images. The plurality of OCT horizontal scanning images are fused into one OCT horizontal image, and the plurality of OCT vertical scanning images are fused into one OCT vertical image. According to the two OCT images, the position coordinates (x o ,y o ,z o ) of the surgical instrument in the OCT imaging coordinate system at this time can be detected. As the surgical operation proceeds, the position of the vertex of the surgical instrument changes. Therefore, after obtaining the historical position (x o ,y o ,z o ) of the vertex of the surgical instrument in the OCT image at the previous time, the optimal predicted position

[0090] In one embodiment, the optimal predicted position can be obtained by: obtaining a motion prediction model built based on a Kalman filtering algorithm, inputting the historical position (x o ,y o ,z o ) as input information into the motion prediction model, and taking the output result of the motion prediction model as the corresponding optimal predicted position

[0091] In operation S320, coordinate mapping is performed according to the optimal predicted position to determine the predicted position of the vertex of the surgical instrument in the second coordinate system at the second time.

[0092] In operation S330, the detected position of the surgical instrument in the second coordinate system at the second time is determined according to the predicted position.

[0093] In operation S340, the mapping position of the surgical instrument in the first coordinate system at the second time is determined according to the detected position, so as to track the intraoperative OCT imaging position.

[0094] In this embodiment, the OCT imaging and the surgical microscope imaging are combined, and the Kalman filtering is used to estimate the vertex position and the motion speed of the surgical instrument, so that the positioning accuracy of the vertex of the surgical instrument is improved, and the accuracy of the OCT imaging position tracking is improved.

[0095] The present application will be described in detail below in conjunction with the drawings and specific embodiments.

[0096] For example, referring to FIG. 7, the method for tracking the intraoperative OCT imaging position provided by the embodiment of the present application can include the following operations:

[0097] In operation S710, the position of the surgical instrument in the fundus image at the initial time is estimated and converted into the OCT scanning position.

[0098] In operation S720, OCT scanning is performed at the initial time based on the OCT scanning position.

[0099] In operation S730, the OCT image obtained by performing OCT scanning is processed to obtain the three-dimensional coordinate position of the surgical instrument at the initial time in the OCT imaging coordinate.

[0100] In operation S740, Kalman filtering is performed on the three-dimensional coordinate position of the surgical instrument at the initial time in the OCT imaging coordinate to obtain the optimal estimation position and motion velocity of the surgical instrument at the next time in the OCT imaging coordinate, and the optimal estimation position and motion velocity at the time are output.

[0101] In operation S750, it is determined whether the fundus image at the current time is updated. If the fundus image is updated, operation S760 is performed, and if the fundus image is not updated, operation S770 is performed.

[0102] In operation S760, the position of the surgical instrument in the fundus image obtained by the next time of microscope imaging is predicted by using the optimal estimation, and the corresponding OCT scanning position is converted.

[0103] In operation S770, the next OCT scanning position is predicted by using Kalman filtering, and OCT scanning is performed.

[0104] In the preoperative preparation stage before performing the above-mentioned operations S710-S770, the mapping relationship between the OCT imaging and the microscope imaging can be determined first. In one embodiment, the mapping relationship between the OCT imaging and the microscope imaging of each surgical microscope can be determined by performing OCT imaging and microscope imaging on a target eye before the surgical microscope is shipped. In other embodiments, the mapping relationship between the OCT imaging and the microscope imaging of the surgical microscope used in surgery can also be determined by performing OCT imaging and microscope imaging on a treatment eye before each surgery after the surgical microscope is shipped.

[0105] For example, determining the mapping relationship between the OCT imaging and the microscope imaging includes the following operations: preoperatively performing microscope imaging on the treatment eye to obtain a corresponding two-dimensional eye diagram (in the anterior segment surgery scenario, the eye diagram is an anterior segment plan; in the posterior segment surgery scenario, the eye diagram is a fundus diagram); preoperatively performing OCT volume scanning on the treatment eye to obtain corresponding three-dimensional OCT volume data; performing axial projection on the three-dimensional OCT volume data to obtain a corresponding OCT axial projection diagram; matching the OCT axial projection diagram with the eye diagram to obtain the mapping relationship therebetween: (x, y) = f(u, v), wherein x and y represent the coordinates of a position point in the OCT axial projection diagram in a first coordinate system, and u and v represent the second coordinates of the position point in a second coordinate system.

[0106] In this embodiment, at operation S710, the OCT scanning position at t = 0 can be determined and OCT scanning is performed. In one embodiment, at the beginning of the surgery, real-time microscope imaging can be performed on the treatment eye to obtain an eye diagram, image recognition is performed on the eye diagram to determine the position of the vertex of the surgical instrument, and then the OCT scanning position at this time is further determined according to the vertex position and the above mapping relationship. For example, when it is detected that the surgical instrument appears in the eye diagram, the tracking mode is entered to track the OCT scanning position, and the time t = 0 is marked, and the position (u0, v0) of the vertex of the surgical instrument in the eye diagram at this time is identified, and the OCT scanning position (x0, y0) at this time is determined according to the above mapping relationship.

[0107] In this embodiment, at t > 0, operation S750 is performed, and if the eye diagram is updated (microscope imaging is updated), the predicted position of the vertex of the surgical instrument in the updated eye diagram is estimated , and then the accurate position recognition is performed in the detection region with the vertex and the length and width of (2Δ + 1), to obtain the detection position (u t ,v t ) of the vertex of the surgical instrument. Or at t > 0, if the eye diagram is not updated, the predicted position of the vertex of the surgical instrument in the updated eye diagram is directly taken as the detection position of the vertex in the eye diagram.

[0108] In this embodiment, the predicted position of the vertex of the surgical instrument in the updated eye diagram (the eye diagram at t) can be estimated by the following operations: The detection position (u0, v0) at t = 0 is mapped to the OCT imaging coordinate system through f to obtain the mapping position , and referring to FIG. 3b, the predicted position of the vertex of the surgical instrument in the updated eye diagram is ​OCT vertical scanning is performed at t to obtain a plurality of OCT vertical scanning images, and at OCT horizontal scanning is performed at t to obtain a plurality of OCT horizontal scanning images. The plurality of OCT vertical scanning images are fused into one OCT vertical image (e.g., averaged into one OCT vertical image), and the plurality of OCT horizontal scanning images are fused into one OCT horizontal image (e.g., averaged into one OCT horizontal image), and then the two OCT images are used to identify the coordinate positions (x o ,y o ,z o ) of the apex of the surgical instrument in the vertical direction, the horizontal direction, and the axial direction, respectively. The coordinate positions (x o ,y o ,z o ) at time t are estimated and predicted using Kalman filtering, and the optimal prediction estimate (i.e., the optimal prediction position) of the apex of the surgical instrument at time t is obtained as The coordinate positions (x are transformed into coordinate positions in the microscope coordinate system (the second coordinate system) to obtain

[0109] In the above example, after the detection positions (u t ,v t ) are obtained, they can be mapped through f to obtain the mapping positions Referring to FIG. 3b, at OCT vertical scanning is performed at t to obtain a plurality of OCT vertical scanning images, and at OCT horizontal scanning is performed at t to obtain a plurality of OCT horizontal scanning images. The plurality of OCT vertical scanning images are fused into one OCT vertical image, and the plurality of OCT horizontal scanning images are fused into one OCT horizontal image, and then the two images are used to identify the coordinate positions (x t ,y t ,z t ) of the apex of the surgical instrument at this time in the vertical direction, the horizontal direction, and the axial direction, respectively. The coordinate positions (x t ,y t ,z t ) at the next time (at time t+1) are estimated and predicted using Kalman filtering, and the optimal prediction estimate (i.e., the optimal prediction position) of the apex of the surgical instrument at this time is obtained as The coordinate positions (x are transformed into coordinate positions in the microscope coordinate system (the second coordinate system) to obtain and this is used as the tracking basis for the apex of the surgical instrument at the next time.

[0110] The above example, which combines microscope imaging and OCT imaging, proposes a real-time tracking scheme for the tip position of a surgical instrument, for automatic tracking and imaging of OCT scanning of a surgical microscope during an ophthalmic surgery. In this way, three-dimensional coordinates and motion speed estimates of the surgical instrument can be given in real time, and the intraoperative OCT imaging position can be accurately given. As a result, the operation difficulty of the surgeon using the surgical microscope can be greatly reduced, thereby improving the use experience.

[0111] FIG. 8 is a structural block diagram of an intraoperative OCT imaging position tracking device provided by an embodiment of the present application, which is configured to perform the intraoperative OCT imaging position tracking method provided by any of the above embodiments. The device and the intraoperative OCT imaging position tracking method of the above embodiments belong to the same concept, and details not described in the embodiment of the intraoperative OCT imaging position tracking device can be referred to the embodiment of the intraoperative OCT imaging position tracking method. Referring to FIG. 8, the device can specifically include a first determining module 410, a second determining module 420, and a third determining module 430.

[0112] The first determining module 410 is configured to determine a predicted position of the surgical instrument in the second coordinate system at the second time according to the historical position of the surgical instrument in the first coordinate system at the first time, wherein the first coordinate system is used for OCT imaging, and the second coordinate system is used for microscope imaging; the second determining module 420 is configured to determine a detected position of the surgical instrument in the second coordinate system at the second time according to the predicted position; and the third determining module 430 is configured to determine a mapped position of the surgical instrument in the first coordinate system at the second time according to the detected position, so as to track the intraoperative OCT imaging position.

[0113] Optionally, the first determining module 410 can be configured to determine the predicted position of the surgical instrument in the second coordinate system at the second time according to the historical position of the surgical instrument in the first coordinate system at the first time in response to an update of the microscope imaging.

[0114] Optionally, the second determining module 420 can be configured to directly take the predicted position as the detected position of the surgical instrument in the second coordinate system at the second time in response to no update of the microscope imaging.

[0115] Optionally, the first determining module 410 can be configured to perform the operation of determining the predicted position of the surgical instrument in the second coordinate system at the second time according to the historical position of the surgical instrument in the first coordinate system at the first time in response to the appearance of the surgical instrument in the microscope imaging.

[0116] Optionally, the historical position comprises a historical position (x0, y0) of the vertex of the surgical instrument in the first coordinate system at the first time point; and the predicted position comprises a predicted position of the vertex of the surgical instrument in the second coordinate system at the second time point The first determination module 410 can comprise: an optimal predicted position estimation unit configured to estimate an optimal predicted position of the vertex of the surgical instrument in the first coordinate system at the second time point according to the historical position (x0, y0) of the vertex of the surgical instrument in the first coordinate system at the first time point and by using Kalman filtering , and a predicted position determination unit configured to determine the predicted position of the vertex of the surgical instrument in the second coordinate system at the second time point according to the optimal predicted position , and perform coordinate mapping to determine the predicted position of the vertex of the surgical instrument in the second coordinate system at the second time point

[0117] Optionally, the predicted position comprises a predicted position of the vertex of the surgical instrument in the second coordinate system at the second time point The second determination module 420 can comprise: a detection region determination unit configured to determine a detection region of the surgical instrument in the second coordinate system at the second time point according to the predicted position of the vertex of the surgical instrument , and a detection position recognition unit configured to recognize a detection position (u, v) of the surgical instrument in the second coordinate system at the second time point in the detection region t t

[0118] Optionally, the detection region determination unit is configured to determine the detection region of the surgical instrument in the second coordinate system at the second time point according to the predicted position of the vertex of the surgical instrument , and the detection region is a region with the vertex of , and a length and a width of (2Δ+1)

[0119] Optionally, the mapping position comprises a mapping position of the vertex of the surgical instrument in the first coordinate system at the second time point The in-operation OCT imaging position tracking device described above further comprises: a control module configured to determine a mapping position of the vertex of the surgical instrument in the first coordinate system at the second time point , and then control the OCT scanning module of the surgical microscope to perform OCT cross-scan imaging with the mapping position as the center; or, first rotate the first coordinate system by a predetermined angle, and then control the OCT scanning module of the surgical microscope to perform OCT cross-scan imaging with the mapping position as the center.

[0120] Optionally, the mapping position comprises a mapping position of the vertex of the surgical instrument in the first coordinate system at the second time point ​​The aforementioned intraoperative OCT imaging position tracking device further includes: a sixth determining module, configured to determine the mapped position of the apex of the surgical instrument in the first coordinate system at a second time. Then, using the mapped position Centered on a predetermined scanning interval δ, multiple horizontal scanning position points are determined in the horizontal direction, and multiple vertical scanning position points are determined in the vertical direction. A first obtaining module is configured to control the OCT scanning module of the surgical microscope to perform one OCT horizontal scan at each horizontal scanning position point, thereby obtaining multiple OCT horizontal scanning images. A second obtaining module is configured to control the OCT scanning module of the surgical microscope to perform one OCT vertical scan at each vertical scanning position point, thereby obtaining multiple OCT vertical scanning images. A seventh determining module is configured to determine the corresponding OCT cross-scan imaging based on the multiple OCT horizontal scanning images and the multiple OCT vertical scanning images.

[0121] The intraoperative OCT imaging position tracking device provided in this application, through a first determining module, determines the predicted position of the surgical instrument in a second coordinate system for microscopic imaging at a second moment based on the historical position of the surgical instrument in a first coordinate system for OCT imaging at a first moment. Then, to more accurately determine the instrument position in the second coordinate system at the second moment, considering that the accuracy of the instrument position calculated by microscopic imaging is higher than that calculated by OCT imaging, a second determining module determines the detection position of the surgical instrument in the second coordinate system at the second moment based on the predicted position. Furthermore, a third determining module determines the mapped position of the surgical instrument in the first coordinate system at the second moment based on the detected position, so as to achieve automatic tracking of the intraoperative OCT imaging position based on the mapped position. In ophthalmic surgery, this device, by combining microscopic imaging and OCT imaging, achieves automatic and accurate tracking of the surgical instrument during OCT imaging, and further achieves automatic and accurate tracking of the intraoperative OCT imaging position. This helps improve the surgeon's experience using the surgical microscope and shortens the surgical time.

[0122] The intraoperative OCT imaging position tracking device provided in this embodiment can execute the intraoperative OCT imaging position tracking method provided in any embodiment of this application, and has the corresponding functional modules and beneficial effects of the execution method.

[0123] In the embodiments of the intraoperative OCT imaging position tracking device described above, the multiple units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of the multiple functional units are only for easy differentiation and are not used to limit the scope of protection of this application.

[0124] Figure 9 is a structural block diagram of a surgical microscope system provided by embodiments of the present application. Referring to Figure 9, the system can include a surgical microscope 510, which is configured to have a microscope imaging module 5101 and an OCT imaging module 5102, and a control device 520, which is configured to track and control the imaging position of the OCT imaging module 5102 according to the intraoperative OCT imaging position tracking method provided by any of the above embodiments. The system and the intraoperative OCT imaging position tracking method of the above embodiments belong to the same concept, and the details not described in the embodiments of the surgical microscope system can be referred to the embodiments of the intraoperative OCT imaging position tracking method.

[0125] The surgical microscope system provided by the embodiments of the present application realizes automatic and accurate tracking of the surgical instrument during the OCT imaging process in the ophthalmic surgery through the cooperation of the surgical microscope and the control device, and further realizes automatic and accurate tracking of the intraoperative OCT imaging position on this basis, which helps to improve the use experience of the surgeon for the surgical microscope and shorten the operation time.

[0126] Figure 10 shows a structural schematic diagram of the control device 10 that can be used to implement the embodiments of the present application. The control device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The control device can also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices (such as headsets, glasses, watches, etc.), and other similar computing devices. The components shown here, their connections, and their functions, as well as their relationships with each other, are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0127] As shown in FIG. 10, the control device 10 includes at least one processor 11, and a memory, such as a Read-Only Memory (ROM) 12, a Random Access Memory (RAM) 13, etc., communicatively connected to the at least one processor 11, where the memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer programs stored in the ROM 12 or loaded into the RAM 13 from the storage unit 18. Various programs and data required for the operation of the control device 10 can also be stored in the RAM 13. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An Input / Output (I / O) interface 15 is also connected to the bus 14.

[0128] Various components in the control device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc., an output unit 17, such as various types of displays, speakers, etc., a storage unit 18, such as a magnetic disk, an optical disk, etc., and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the control device 10 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0129] The processor 11 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the processor 11 include a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various special-purpose Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, a Digital Signal Processing (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs various methods and processes described above, such as the method of tracking intraoperative OCT imaging locations.

[0130] In some embodiments, the method of tracking an intraoperative OCT imaging position can be implemented as a computer program tangibly embodied in a computer readable storage medium, e.g., storage unit 18. In some embodiments, parts or all of the computer program can be loaded and / or installed onto control device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded onto RAM 13 and executed by processor 11, one or more steps of the method of tracking an intraoperative OCT imaging position described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the method of tracking an intraoperative OCT imaging position by other means, e.g., with the aid of firmware.

[0131] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a complex programmable logic device (CPLD), a system on a chip (SOC), a complex programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0132] Computer programs used to implement the methods of the present application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the computer program running on the processor implements the functions / operations specified in the flowcharts and / or the block diagrams.

[0133] In the context of this application, a computer-readable storage medium can be a tangible medium that can contain or store computer programs for use by or in connection with an instruction execution system, apparatus, or device. Computer-readable storage media can include electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of a machine-readable storage medium will include one or more lines of a program of instructions in a transitory signal, a portable computer diskette, a hard disk, a RAM, a ROM, an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0134] To provide for interaction with a user, the systems and techniques described here can be implemented on a control device having a display device (e.g., a Cathode-Ray Tube (CRT) or Liquid Crystal Display (LCD) monitor) configured to display information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the control device. Other kinds of devices can be configured to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0135] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a Local Area Network (LAN), a Wide Area Network (WAN), a blockchain network, and the Internet.

[0136] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and virtual private server (VPS) services.

[0137] The steps described above can be reordered, added to, or deleted from using the various forms of flow shown above. For example, the steps described in the present application can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions of the present application can be achieved, and the present application does not limit this.

Claims

1. A method for tracking an intraoperative optical coherence tomography (OCT) imaging position, comprising: determining a predicted position of a surgical instrument in a second coordinate system at a second time according to a historical position of the surgical instrument in a first coordinate system at a first time, wherein the first coordinate system is used for OCT imaging and the second coordinate system is used for microscope imaging; determining a detected position of the surgical instrument in the second coordinate system at the second time according to the predicted position; and determining a mapped position of the surgical instrument in the first coordinate system at the second time according to the detected position, so as to track the intraoperative OCT imaging position.

2. The method of tracking intraoperative OCT imaging locations according to claim 1, wherein, The determining of the predicted position of the surgical instrument in the second coordinate system at the second time according to the historical position of the surgical instrument in the first coordinate system at the first time comprises: in response to microscope imaging being updated, determining the predicted position of the surgical instrument in the second coordinate system at the second time according to the historical position of the surgical instrument in the first coordinate system at the first time.

3. The method of tracking intraoperative OCT imaging location according to claim 1, wherein, The determining of the detected position of the surgical instrument in the second coordinate system at the second time according to the predicted position comprises: in response to microscope imaging being not updated, directly taking the predicted position as the detected position of the surgical instrument in the second coordinate system at the second time.

4. The method of tracking intraoperative OCT imaging location according to claim 1, wherein, The determining of the predicted position of the surgical instrument in the second coordinate system at the second time according to the historical position of the surgical instrument in the first coordinate system at the first time comprises: in response to the surgical instrument appearing in microscope imaging, determining the predicted position of the surgical instrument in the second coordinate system at the second time according to the historical position of the surgical instrument in the first coordinate system at the first time. 5.The method for tracking the intraoperative OCT imaging position according to claim 1, wherein the historical position comprises a historical position (x 0, y 0) of an apex of the surgical instrument in the first coordinate system at the first time; the predicted position comprises a predicted position of an apex of the surgical instrument in the second coordinate system at the second time instant The determining of the predicted position of the surgical instrument in the second coordinate system at the second time according to the historical position of the surgical instrument in the first coordinate system at the first time comprises: estimating an optimal predicted position of the vertex of the surgical instrument in the first coordinate system at the second time instant using Kalman filtering, based on the historical position of the vertex of the surgical instrument in the first coordinate system at the first time instant According to the optimal predicted position performing coordinate mapping to determine a predicted position of an apex of the surgical instrument in the second coordinate system at the second time instant 6.The method for tracking the intraoperative OCT imaging position according to claim 1, wherein the predicted position comprises a predicted position of an apex of the surgical instrument in the second coordinate system at the second time instant The determining of the detected position of the surgical instrument in the second coordinate system at the second time according to the predicted position comprises: According to the predicted position of the apex of the surgical instrument determining a detected region of the surgical instrument in the second coordinate system at the second time; Within the detection region, a detection position (u t ,v t ) of the surgical instrument in the second coordinate system at the second time point is identified.

7. The method of tracking intraoperative OCT imaging locations according to claim 6, wherein, the predicted position of the apex of the surgical instrument determining a detection region of the surgical instrument in the second coordinate system at the second time point, comprising: According to the predicted position of the apex of the surgical instrument will be for the apex, taking a region with a length and a width both being (2Δ+1) as the detected region of the surgical instrument in the second coordinate system at the second time. 8.The method for tracking the intraoperative OCT imaging position according to claim 1, wherein the mapped position of the vertex of the surgical instrument in the first coordinate system at the second time instant The method further comprises: determining a mapping position of an apex of the surgical instrument in the first coordinate system at the second time instant Subsequently, the mapped position for centering, controlling the OCT scan module of the surgical microscope to perform OCT cross-scan imaging; or, with the mapping position for the center, first rotating the first coordinate system by a predetermined angle, and then controlling an OCT scanning module of a surgical microscope to perform OCT cross scanning imaging. 9.The method for tracking the intraoperative OCT imaging position according to claim 1, wherein the mapped position of the vertex of the surgical instrument in the first coordinate system at the second time instant The method further comprises: determining a mapping position of an apex of the surgical instrument in the first coordinate system at the second time instant Subsequently, the mapped position a plurality of horizontal scanning position points are determined in a horizontal direction and a plurality of vertical scanning position points are determined in a vertical direction, with the center as a starting point and with a predetermined scanning interval δ as a step size; an OCT scanning module of the surgical microscope is controlled to perform one OCT horizontal scan through each of the horizontal scanning position points, to obtain a plurality of OCT horizontal scan images; the OCT scanning module of the surgical microscope is controlled to perform one OCT vertical scan through each of the vertical scanning position points, to obtain a plurality of OCT vertical scan images; corresponding OCT cross-scan imaging is determined according to the plurality of OCT horizontal scan images and the plurality of OCT vertical scan images.

10. An intraoperative optical coherence tomography (OCT) imaging position tracking apparatus, comprising: a first determining module configured to determine a predicted position of a surgical instrument in a second coordinate system at a second time according to a historical position of the surgical instrument in a first coordinate system at a first time, wherein the first coordinate system is used for OCT imaging and the second coordinate system is used for microscope imaging; a second determining module configured to determine a detected position of the surgical instrument in the second coordinate system at the second time according to the predicted position; and a third determining module configured to determine a mapped position of the surgical instrument in the first coordinate system at the second time according to the detected position, to track an intraoperative OCT imaging position.

11. A surgical microscope system, comprising: a surgical microscope configured to have a microscope imaging module and an optical coherence tomography (OCT) imaging module; and a control apparatus configured to track and control an imaging position of the OCT imaging module according to the intraoperative OCT imaging position tracking method of any one of claims 1-9.

12. A control apparatus, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to cause the at least one processor to execute the intraoperative optical coherence tomography (OCT) imaging position tracking method of any one of claims 1-9.

13. A computer readable storage medium, the computer readable storage medium storing computer instructions, the computer instructions being configured to cause a processor to implement the intraoperative optical coherence tomography (OCT) imaging position tracking method of any one of claims 1-9 when executed by the processor. ​ ​

Images