Articulated microscope and method of operating articulated microscope

Abstract

A first aspect of the present disclosure relates to an articulated microscope with auto-focus configured to: be mounted in a first position; acquiring information that the microscope has been moved from the first position to the second position; -adjusting the focus to observe the object based on the information that the microscope has been moved to the second position.

Description

Technical Field

[0001] This disclosure relates to an apparatus for autofocusing a hinged microscope and a method for operating a hinged microscope. Background Technology

[0002] Articulated microscopes can be mounted on a boom or similar positioning support system for flexible positioning within their workspace and / or sharing their workspace with other objects or users. Boom-mounted microscopes, for example, may feature an articulated boom that allows for flexible positioning of the microscope head, enabling the user to view objects from various angles and distances. Microscopes mounted on a positioning support system can be tilted, rotated, and moved to different positions, providing a wide range of motion for examining objects of different sizes and shapes without having to move the objects themselves. This flexibility is particularly beneficial for viewing large or irregularly shaped samples. Furthermore, articulated microscopes ensure accurate and consistent observation, making them ideal for applications requiring detailed examination, such as biological or medical analysis. Improvements to articulated microscopes are necessary. Summary of the Invention

[0003] The purpose of this disclosure is to improve the operation of using articulated microscopes.

[0004] This objective is achieved through the disclosed embodiments, which are particularly defined by the subject matter of the independent claims. The dependent claims provide information for further embodiments. Various aspects and embodiments thereof are also disclosed in the following summary and description, providing additional features and advantages.

[0005] The first aspect of this disclosure relates to a hinged microscope with autofocus.

[0006] Configured as:

[0007] - Installed in the first position;

[0008] - Obtain information that the microscope has been moved from the first position to the second position;

[0009] - Adjust the focus based on the information that the microscope has been moved to the second position to observe the object.

[0010] An articulated microscope can be any microscope whose workspace extends beyond its autofocus range. The workspace is the physical space in which the articulated microscope can be positioned. Autofocus can be configured to adjust the focus point until it is accurate. Alternatively or additionally, autofocus can be configured to adjust to previously adjusted focuses and / or predefined focuses, which can be acquired, for example, via a communication device.

[0011] Articulated microscopes can be mounted on or include devices that allow them to be moved within the workspace. For example, an articulated microscope can be mounted on an operating table via an articulated arm. Further examples are provided in the remainder. Articulated microscopes can also be mounted in a fixed position. In this case, the microscope needs to be associated with a movable sample holder. The sample holder can be movable, particularly mounted on or including one of the aforementioned devices. Combinations of movable microscopes and movable sample holders are also feasible. In particular, the microscope or sample holder can subsequently be mounted on or include one of the aforementioned devices.

[0012] Articulated microscopes can also be handheld microscopes. These microscopes offer greater mobility and are suitable for fieldwork and viewing hard-to-reach areas.

[0013] Typically, position can be any information that can represent a location within the microscope workspace. For example, position can be a Cartesian position involving one or more degrees of freedom, particularly three translational degrees of freedom, such as x, y, z-coordinates, and / or three rotational degrees of freedom, such as roll, pitch, and yaw. Position can be absolute and / or relative.

[0014] The first location can be a parking spot. The parking spot can be during a mission or between different missions.

[0015] The parking position during a task can be where the user has moved the microscope because he / she does not need the microscope for one or more steps of the current task, but the user needs the microscope for one or more other steps of the current task. For example, during surgery, a surgeon may move the microscope away to perform a part of the surgery without the microscope. In this case, the surgeon can move the microscope to an area in the workspace where the microscope will not obstruct the surgeon and / or other personnel. Therefore, the parking position can be any location within the given workspace of a hinged microscope.

[0016] The storage location during different tasks can be any location where the microscope is moved when it will not be used for an extended period of time. This storage location can be a place for storage and / or a location that protects the microscope in some way. For example, the storage location could be a position on a crane, particularly near the ceiling. The storage location could also be a location where the microscope is automatically moved. During the storage period, the microscope may lose its previously adjusted focus.

[0017] The first position could also be a position where the microscope has been used. For example, a position from another perspective on an object, a position of another part of an object, or the position of another object. In this type of first position, the focus of the microscope may have been adjusted to be different from the focus required for the second position.

[0018] Information that the microscope has been moved from a first position to a second position can be based on determining that the microscope has stopped at a (second) position different from the first position. Alternatively, the information that the microscope has been moved from the first position to the second position can include the second position, and in particular, can be equal to the second position itself.

[0019] The second position can be the position where the microscope is used or intended to be used. Therefore, the focus will be adjusted based on the second position. The second position can include an absolute position. Alternatively, the second position can include a relative position. For example, the second position can include translational position information relative to the first parking position and rotational position information including absolute values ​​of the microscope's roll, pitch, and yaw. Alternatively, the second position can be entirely relative to the first position or entirely represented as an absolute position including a positional system (e.g., a Cartesian coordinate system) encompassing the microscope's workspace.

[0020] The second position can be determined based on the previous (second) position where the focus has been adjusted. Alternatively, the second position can be based on the distance from the sample. The sample's position may have already been detected by a sensor, such as based on a camera image that is automatically analyzed for the sample's position. The second position can also be based on user input. For example, a surgeon can move the microscope to the position where she / he wants to analyze the sample, and if the microscope has reached that position, the surgeon presses a button to instruct the microscope to move to the second position. The microscope can then begin the autofocus process.

[0021] The second position may also include a set of positions, such as a set of positions that are at the same distance from the reference position (e.g., marked on the object to be observed).

[0022] According to the articulated microscope of the first aspect of this disclosure, after the microscope has been moved from its actual working position or after the microscope has been moved to another working position, especially when different focal points have been adjusted, faster autofocus can be provided for the object to be observed.

[0023] An embodiment of the first aspect of this disclosure relates to an articulated microscope with autofocus.

[0024] The microscope is mounted on one or more of the following:

[0025] - Boom-type support;

[0026] - Ceiling brackets;

[0027] - Operating table or bedside support.

[0028] Articulated microscopes can be mounted, for example, on a boom. The boom can be passive (i.e., non-actuated boom) or active (i.e., motorized boom). By using the boom, the microscope can be repositioned within a workspace defined by the boom's kinematics. An active boom can be, for example, a robotic boom.

[0029] Articulated microscopes can be repositioned in several ways, not just with arm-mounted setups. For example, the microscope can be mounted on a track base. A track base allows the microscope to slide horizontally and / or vertically along a fixed path, suitable for sequentially examining large samples or multiple items. In another example, the microscope can be mounted on a boom base. A boom base allows the horizontal arm to move laterally, which is particularly useful for viewing objects from various angles.

[0030] The microscope can also be equipped with or include a manual stage. With a manual stage, the user can move the microscope relative to the sample along the Cartesian X and Y axes. In another example, the microscope can be equipped with or include a tilting stage. A tilting stage allows for adjustment of the microscope's angle relative to the sample for multi-view observation. In particular, this can be advantageous for analyzing 3D objects.

[0031] Articulated microscopes can also be mounted on or incorporate a lifting frame system. With a lifting frame system, the microscope can cover a larger workspace, much like when mounted on a robotic arm.

[0032] For example, based on one of these position support systems, the microscope can be moved quickly away from the surgeon.

[0033] An embodiment of the first aspect of this disclosure relates to an articulated microscope with autofocus, wherein the microscope includes sensors, particularly a gyroscope, for observing its position; and wherein a second position is obtained based on information from the sensors.

[0034] Sensors can measure position directly or indirectly. Direct position measurement can be based on electrical or magnetic measurement principles, such as those used in MR sensors, which are converted into position information. Indirect position measurement involves observing velocity or acceleration and differentiating the measured values ​​once or twice to obtain the position value. Acceleration can be measured using a gyroscope.

[0035] Mounting sensors (such as gyroscopes) to a portable microscope enables precise tracking of its position and orientation. Various mounting methods can include direct attachment to the microscope frame and / or attaching the gyroscope to an articulated arm to track its movement. Integration of sensors (such as gyroscopes) with the sample stage allows monitoring of arbitrary tilt or displacement during observation. Modular sensor platforms can combine gyroscopes with other sensors for comprehensive motion tracking. Wireless sensor modules offer flexibility by eliminating cables, while custom housings protect the sensors and facilitate installation. Mounting on a portable base allows for tracking of the entire assembly's movement and integration of the sensor with the control system to provide real-time feedback for automatic focus adjustment.

[0036] Sensor information related to the first and / or second positions can also come from external devices and can subsequently be acquired via a corresponding communication interface. However, using its own sensor information, the microscope can effectively use the information to determine the first and / or second positions.

[0037] An embodiment of the first aspect of this disclosure relates to an articulated microscope with autofocus.

[0038] Configured as:

[0039] - Attached to the articulated boom; and - obtain information about the second position from the articulated boom.

[0040] The boom may include passive and / or active (i.e. motorized) joints. Alternatively, the boom may include tandem and / or parallel kinematics.

[0041] The boom may include angle sensors in one or more of its joints. The angle information can then be converted into a first / second Cartesian position. Alternatively, the boom may be configured to directly measure the Cartesian position, particularly the Cartesian position at the machine interface between the boom and the microscope. If the microscope includes a boom, the reference point for position measurement can also be closer to the lens, specifically on the microscope, or even on the lens.

[0042] When the boom has redundant kinematics, additional decisions can be made regarding the position of one or more redundant joints. In particular, this decision can be made to optimize the placement of the boom, for example, to optimize the distance between personnel or objects in the workspace of the articulated microscope.

[0043] Based on these examples, first and / or second location information can be provided quickly and with high accuracy.

[0044] An embodiment of the first aspect of this disclosure relates to an articulated microscope with autofocus.

[0045] Includes one or more markers observable by an external position sensing device; and

[0046] It is configured to obtain information about the second position from an external position sensing device.

[0047] To determine the first and / or second position of a microscope, various types of markers can be used. These can include optical markers, such as one or more QR codes, barcodes, reflective markers (laser-scannable markers), and LED markers. These marker types can provide precise position data when scanned or detected by an optical system. Alternatively, magnetic markers can be used to observe the first and / or second position. One or more magnetic markers can be formed as magnetic strips and magnetic dots. Alternatively, RFID markers (such as RFID tags) can be used for wireless position tracking. Alternatively, radio frequency markers (such as beacons and UWB tags) utilize radio frequency for position measurement.

[0048] One or more markers may be attached to the microscope and / or to the equipment on which the microscope is mounted. Alternatively, one or more markers may also be located on the sample holder, and their position is provided to the microscope.

[0049] The advantage of using these markers is that they can determine the absolute position of the microscope and the object with high precision.

[0050] An embodiment of the first aspect of this disclosure relates to an articulated microscope with autofocus.

[0051] Configured as:

[0052] - Determine if the current focus is valid for the second position.

[0053] Focus verification can be performed based on the second position before or after focus adjustment.

[0054] Focus can be verified by comparing it to a previously adjusted focus. In some embodiments, a focus is assigned to an initial position that is equal to or close to a second position and is stored for later use. In the next step, the microscope is moved to a first position, such as a short-term parking position during the surgical procedure. The movement to the first position is detected as the distance from the patient increases to a predefined threshold. The microscope is then moved to a second position for the surgeon to use again. This position should be substantially the same as the initial position, and the focus at the stored initial position can be adjusted. However, before adjusting the focus at the initial position, it is determined whether it is the correct focus. Therefore, the second position is compared to the initial position, and the focus at the second position is verified only if the second position is equal to or within a predefined threshold distance from the initial position.

[0055] Alternatively, focus can be authenticated through user confirmation.

[0056] An embodiment of the first aspect of this disclosure relates to an articulated microscope with autofocus.

[0057] The adjustment based on the second position is a coarse adjustment; and

[0058] Configured as:

[0059] - Adjusting the focus without relying on a second position.

[0060] In particular, when the focus should be adjusted for a second position, which is the same as or at least similar to the previous position where the focus was adjusted, the focus adjustment may include coarse adjustment and fine adjustment.

[0061] Coarse adjustments can be made by adjusting the previously adjusted focus, particularly the most recently adjusted operating focus (where the operating focus is the focus used to analyze the object and is therefore different from the focus at the parking position). If the microscope has reached a second position (which can be determined by the microscope stopping away from the parking position), the coarse focus is adjusted. Alternatively, if it is detected that the microscope has been moved toward the object to be analyzed and / or another reference point, then the coarse focus has already been adjusted.

[0062] However, after coarse focus adjustment, it cannot be assumed that the focus has been adequately adjusted, because the object being observed may have changed / moved and / or the second position may (at least slightly) differ from the position where the previous focus was acquired. To address this issue, fine-tuning can be performed in a step following focus adjustment based on the second position to precisely adjust the focus. Fine-tuning is independent of the second position and may depend only on the object being analyzed.

[0063] These examples demonstrate that focusing time can be reduced to achieve accurate focusing.

[0064] An embodiment of the first aspect of this disclosure relates to an articulated microscope with autofocus.

[0065] If the speed of the microscope relative to the object is essentially zero, the focus is adjusted based on the second position.

[0066] The second position can be a position where the microscope is stationary. Specifically, this can be the position from which the object should be observed. This embodiment ensures that autofocus is performed only from positions from which the object can be observed (positions where the microscope is still moving—at least at a speed exceeding a predefined threshold—may not be positions from which observation can be performed).

[0067] An embodiment of the first aspect of this disclosure relates to an articulated microscope with autofocus.

[0068] If the time-based derivative of the second position is substantially different from zero, then the focus is adjusted based on the second position.

[0069] The second position can be, for example, defined by the distance from the object to be observed, and when passing through the second position (i.e., at a non-zero speed), a predefined autofocus can be adjusted. Subsequently, when the microscope stops at the position on which the object should be observed, the microscope may already have a (coarsely) adjusted focus. Fine focusing can then be performed if necessary.

[0070] An embodiment of the first aspect of this disclosure relates to an articulated microscope with autofocus.

[0071] Configured as:

[0072] - Get the focus position, and

[0073] The focus is adjusted based on the second position and the direction of movement that causes the second position to move toward the focus position.

[0074] The focal point can be a reference point, such as the location of the operating table or a reference point provided by the user through the GUI. The focal point can also be the location where microscopic analysis should be performed.

[0075] When the microscope reaches the second position, assess whether it has come from, for example, the parking position or from a position where the operating focus for the microscope has been adjusted. If it has come from the parking position, the focus can be adjusted to reach the operating position with at least a properly adjusted focus. If the microscope has come from the operating position and passed through the second position, it can be assumed that the microscope has been moved to the parking position. In this case, no focus adjustment is required.

[0076] Based on these examples, if the microscope is moved away from the object, autofocus does not need to adjust the focus. This saves unnecessary focusing. Autofocus can adjust the focus only when the microscope is moved toward the object. This reduces focusing time.

[0077] An embodiment of the first aspect of this disclosure relates to an articulated microscope with autofocus.

[0078] Configured as:

[0079] - Get the focus position; and

[0080] The focus is adjusted based on whether the second position is within a predefined threshold distance from the object.

[0081] This situation might occur if the second position is defined solely by a stop far from the first position. In this case, check if the second position is sufficiently close to the focus position (i.e., the reference position explained above). If so, adjust the focus using the autofocus system. If not, do not perform autofocus.

[0082] In other words, autofocus can be performed only when the second position is close enough to the object being observed.

[0083] An embodiment of the first aspect of this disclosure relates to an articulated microscope with autofocus.

[0084] The focus is adjusted based on the second position and the first position, and specifically based on whether the Euclidean distance between the first position and the second position exceeds a predefined threshold.

[0085] If the first position is the parking position, autofocus should only begin if the distance between the parking position and the second position is sufficiently large.

[0086] However, if the first position is where the microscope has been used, autofocus can be adjusted only if the Euclidean distance between the parking position and the second position is less than a predefined threshold. This ensures that autofocus is only performed when the microscope has reached the approximate position where microscopic analysis should be performed.

[0087] An embodiment of the first aspect of this disclosure relates to an articulated microscope with autofocus.

[0088] Configured as:

[0089] - Determine whether the microscope movement type is parking movement or using movement; and

[0090] - Adjustments are made only based on the second movement focus when it is determined that movement will be used.

[0091] The type of movement can be determined using machine learning algorithms, such as neural networks. It can then be determined whether the microscope is moving towards its parking position or towards any other location where the microscope is not in use. For these locations, autofocus adjustment is not required. Alternatively, it can be determined whether the microscope is moving towards a location where it should be used. In this case, autofocus may already be activated and adjusted to the best possible focus or a previously adjusted focus, particularly as a coarse focus.

[0092] Therefore, the system can automatically determine which movements require focus adjustment and which do not.

[0093] The second aspect relates to a method for controlling the autofocus of a microscope.

[0094] Includes the following steps:

[0095] - Obtain information about the initial position of the articulated microscope;

[0096] - Obtain information that the microscope has been moved from the first position to the second position;

[0097] - Determine the autofocus information for the articulated microscope based on the information that the microscope has been moved to the second position;

[0098] - Controlling the focus of the articulated microscope based on autofocus information; and / or providing autofocus information for the articulated microscope.

[0099] The method according to the second aspect may include one or more steps described with respect to the first aspect. This may be independent of other steps described together with the corresponding steps.

[0100] Furthermore, the method according to the second aspect of this disclosure can be executed by a processor in a microscope according to the first aspect of this disclosure. Alternatively, the method according to the second aspect can be executed by a network device that acquires information from the microscope according to the first aspect of this disclosure and provides that information to control the autofocus of the microscope (therefore, autofocus information can be provided to an articulated microscope). By executing the method on a network device, the focus of multiple microscopes can be adjusted.

[0101] A third aspect of this disclosure relates to a device for controlling the autofocus of a microscope.

[0102] Configured as:

[0103] - Obtain information about the initial position of the articulated microscope;

[0104] - Obtain information that the microscope has been moved from the first position to the second position;

[0105] - Determine the autofocus information for the articulated microscope based on the information that the microscope has been moved to the second position;

[0106] - Provides information for automatic focusing of the microscope.

[0107] The device according to the fourth aspect can be a network-based controller. Such a device may include one or more structures and / or functions related to the first aspect. Furthermore, such a device is equipped with a communication interface for communicating with a microscope according to the first aspect of this disclosure.

[0108] This device can be configured to operate independently of a microscope and / or serve multiple microscopes. Attached Figure Description

[0109] Further advantages and features arise from the following embodiments, some of which are illustrated in the accompanying drawings. The drawings are not always to scale. Dimensions of various features may be enlarged or reduced, particularly for clarity of description. For this purpose, the drawings are shown at least partially schematically.

[0110] Figure 1 The figure illustrates an articulated microscope system 100 with a passive arm according to an embodiment of the present disclosure.

[0111] Figure 2 The figure illustrates an articulated microscope system 200 with focus adjustment based on target space according to an embodiment of the present disclosure.

[0112] Figure 3 The figure illustrates an articulated microscope system 300 with reference point-based focus adjustment according to an embodiment of the present disclosure.

[0113] Figure 4 The figure illustrates an articulated microscope system 400 with a lifting frame 410 according to an embodiment of the present disclosure.

[0114] Figure 5 The figure illustrates a hinged microscope system 500 according to an embodiment of the present disclosure.

[0115] Reference Symbol List

[0116] 100 Articulated Microscope System

[0117] 110 Microscope

[0118] 112 Focusing lens system

[0119] 114 Parking Location

[0120] 116 Rolling joint

[0121] 120 Passive Boom

[0122] 121 Fixture

[0123] 122 Yaw joint

[0124] 124 First pitch joint

[0125] 126 Second pitch joint

[0126] 128 Third pitch joint

[0127] 130 samples

[0128] 200 Microscope System

[0129] 202 First Position

[0130] 204 Work Location

[0131] 210 Articulated Boom

[0132] 212a Yaw Joint

[0133] 212b Pitch joint

[0134] 212c Pitch joint

[0135] 212d Pitch joint

[0136] 220 microscope

[0137] 222 Rolling joint

[0138] 230 patients on the operating table

[0139] 232 patients

[0140] 240 Target Sphere

[0141] 300 Microscope System

[0142] 310 Laser Scanner

[0143] Ceiling of the 312 operating room

[0144] 314 laser

[0145] 320 Work Location

[0146] 322 The direction away from the work location

[0147] 324 Direction towards the work position

[0148] 330 Second work position

[0149] 410 Crane Frame

[0150] 412 Horizontal moving device

[0151] 414 Vertical moving device

[0152] 420 microscope

[0153] 422 Work Location

[0154] Parking location 424

[0155] 430 is the predefined distance from the operating table.

[0156] 500 Microscope System

[0157] 510 Microscope

[0158] 520 Computer System Detailed Implementation

[0159] In the following description, reference is made to the accompanying drawings, which form part of this disclosure, and the drawings illustrate specific aspects of this disclosure. The same reference numerals denote the same or at least functionally or structurally similar features.

[0160] Generally, the disclosure of the described methods also applies to corresponding devices (or apparatuses) for performing the methods, or to corresponding systems comprising one or more devices, and vice versa. For example, if specific method steps are described, the corresponding device may include features for performing the described method steps, even if such features are not explicitly described or represented in the figures. Conversely, for example, if a specific device is described based on functional units, the corresponding method may include one or more steps to perform the described function, even if such steps are not explicitly described or shown in the figures. Similarly, a system may be provided with corresponding device features or features for performing specific method steps. Unless otherwise explicitly stated, features of the respective exemplary aspects and embodiments described above or below may be combined.

[0161] Figure 1 The figure illustrates an articulated microscope system 100 with a passive arm according to an embodiment of the present disclosure. The microscope system 100 includes a microscope 110. The microscope 110 is mounted on an articulated arm 120. The arm 120 is a passive arm without actuation. Movement of the arm 120 is performed by a user (e.g., a surgeon). The microscope system 100 is configured to observe a sample 130 from different positions and from different angles.

[0162] The microscope 110 includes a lens system that allows observation of the sample 130 at a focus position 112. Furthermore, the microscope 110 includes a rotation joint 116, through which the microscope is connected to a passive arm 120. The rotation joint 116 provides rotational freedom about the microscope's scroll axis.

[0163] The passive boom 120 includes a fastening device 121 (e.g., a threaded clamp) for flexibly mounting the microscope 110 onto a suitable base. Such a base may be, for example, an operating table (not shown). The boom 120 includes a linkage connected by multiple joints. A first joint 122 provides rotational freedom of the system 100 about a yaw axis. The first joint 122 further provides a connection between the fastening device 121 and the first linkage of the boom 120. A second joint 124 provides rotational freedom of the system 100 about a first pitch axis. The second joint 124 is directly coupled to the first joint 122. A third joint 126 provides rotational freedom of the system 100 about a second pitch axis. A fourth joint 128 provides rotational freedom of the system 100 about a third pitch axis.

[0164] The microscope system 100 can be positioned within a workspace defined by the mounting location, the joints (122, 124, 126, 128), the linkages between the joints of the boom 120, and the joint 116 of the microscope 110. Within the workspace, the microscope 110 can be flexibly arranged to observe the sample 130. All joints are equipped with position sensors. Based on the position information, the microscope knows its exact location. Alternatively, the system can be equipped with a gyroscope or force sensor to determine position and / or motion information.

[0165] When the microscope is not in use, for example, when the user wants to view the sample directly (i.e., without using the microscope), the microscope 110 can be moved to a position where the microscope will not obstruct the user and / or where the microscope is safe. Such a position is illustrated by the parking position 114 (the dashed portion of the microscope system 100). The parking position can be considered as the first position of the articulated microscope system 100.

[0166] During the analysis of a sample (such as a sample of a patient's organ during surgery), there may be several stages using a microscope and several stages not using a microscope. If a microscope is not used, it can be moved to a storage location.

[0167] When the microscope is moved to parking position 114, autofocus may change the microscope's focus. Subsequently, if the microscope is moved back to working position 112, autofocus needs to readjust the focus, which can be time-consuming. One reason for this is that autofocus is unaware of the microscope's position relative to the sample. In this situation, autofocus may need to search for the focus by changing autofocus in any direction and analyzing the results, essentially based on trial and error.

[0168] For efficient operation, the focus of the microscope should be adjusted as soon as possible. Therefore, the focus can be readjusted according to working position 112, which can be the second position as described above in the first or second aspect of this disclosure.

[0169] For example, if the microscope system is moved from parking position 114 to working position 112, it can be analyzed whether the working position is closer to the sample than the previously adopted working position (e.g., by operating on the position of the sample support). Based on this information, the autofocus knows in which direction the focus needs to be adjusted to achieve accurate focusing. This can be implemented as a function of the microscope or as a function of the device controlling the microscope's autofocus.

[0170] In another example, the microscope is moved from parking position 114 (first position) to another position (second position). If the system learns that it is approaching the sample, the microscope's autofocus is activated because it infers that the microscope has been moved to working position 112. This can be implemented as an alternative to or supplement to the previously described function. Advantageously, with this function, autofocus does not need to be constantly active to focus the microscope. If the system learns from its position information that the microscope has not been moved to working position 112, but rather, for example, only moved to another parking position 114 (because the old parking position may be used for other purposes), then autofocus is not activated.

[0171] Figure 2 The figure illustrates an articulated microscope system 200 with focus adjustment based on target space according to an embodiment of the present disclosure. The microscope system may be based on... Figure 1 Microscope system 100. Microscope system 200 includes an articulated arm 210, such as a passive arm or an active robotic arm. The arm includes four joints 212a, 212b, 212c, 212d for adjusting the yaw and pitch of microscope 220. Microscope 220 is mounted on the end of arm 210 and includes a joint 222 for controlling the rolling axis of the microscope.

[0172] In this case, a microscope is used during surgery. A microscope system 200 is mounted on the patient's operating table 230 to analyze the organs of the patient 232 during surgery.

[0173] To allow for flexible use of the microscope during surgery, the microscope can be moved from working position 204 to parking position 202. The parking position (shown in gray) can be a predefined location where the microscope system 200 is automatically moved. This requires an active articulated arm 210, such as a robotic arm. Alternatively, the system may include other automated parking positions. This enables flexible use of the workspace. Alternatively, the system may include a mode in which the user can move the system to a parking position. This can be any parking position unrelated to one or more automated parking positions.

[0174] The system in this embodiment also has a general understanding of the location of working position 204. The system knows this because of previously adopted working positions, particularly through machine learning or statistical inference. Alternatively, the user may have provided this information through a user interface. In this case, the system also knows the focus configuration at the working position. The focus position can also be determined through machine learning or statistical inference. Alternatively, the focus configuration can be recorded as the focus adjusted when the user provided information about the working position. Based on the working position, the system can determine when the user moves the microscope from parking position 202 (first position) to working position 204 (second position).

[0175] Furthermore, the system can determine a predefined distance from the working position. This distance is illustrated by a target sphere 240 (displayed as a circle in a two-dimensional diagram). Once the system has passed the sphere / circle 240 toward the working position 204, it begins adjusting the focus assigned to the working position (as described in the previous paragraph). Subsequently, when the microscope reaches the working position 204, the autofocus has at least coarsely adjusted the focus of the working position 204. Depending on the precision of the focus, the autofocus can still perform fine-tuning of the focus once the microscope reaches the working position. This is particularly advantageous if the working position is not exactly the same as the previous working position. In any case, faster focusing can be expected.

[0176] Figure 3 The figure illustrates an articulated microscope system 300 with reference point-based focus adjustment according to an embodiment of the present disclosure. The microscope system 300 may be based on... Figure 1 The microscope system 100 or according to Figure 2 The microscope system 200. The arm 210 of the microscope system 300 can be a passive or active arm. The microscope 220 is attached to the end of the arm 210. The microscope system is configured to operate during surgery to analyze the patient 232 on the operating table 230. The microscope system 300 can be placed in multiple parking positions 202 and multiple working positions 204 within its workspace.

[0177] The focal-related position of microscope 220 is determined by laser scanner 310 mounted on the operating room ceiling 312. The laser scanner is configured to track markings (not shown) located on microscope 220, such as those on the lens and / or on the top of the microscope frame. Therefore, the laser scanner and markings are arranged such that the focal-related position can be determined by laser beam 314 in both the parking and working positions.

[0178] One or more working positions 320 are determined by the location where surgery or at least microscopic observation should be performed. This location can also be indicated by markings that can be tracked by a laser scanner using a laser beam 314. Information from the laser beam is transmitted to the microscope 220 and / or to a device controlling the microscope, and this device can also perform the method according to the second aspect of the invention.

[0179] Based on information about the working position 320, it can be determined whether the microscope 220 is located at or has been moved toward the parking position 202. This can be achieved by knowing whether the microscope has moved or has moved away from the working position 320. At this stage, autofocus can be deactivated, and the microscope lens can be protected from contamination.

[0180] Furthermore, based on information about the working position 320, it can be determined whether the microscope 220 is located at or has been moved toward the working position 320. This can be achieved by knowing whether the microscope has moved or has moved in a direction 324 away from the working position 320. At this stage, autofocus can be activated and can be tuned by the autofocus system to a predetermined and / or previously adjusted focus. In this case, autofocus is used only when needed, and the time required to refocus the microscope after it has been moved to the parking position (and lost its focus) 202 can be reduced.

[0181] In another embodiment not shown (which may be combined with other embodiments of this disclosure), the microscope 320 approaches the working position 320 not from the parking position, but from another working position 330. This could be, for example, another perspective of the surgical cavity. The second working position 330 can be determined in the same manner as the first working position 320. As shown, the two working positions can relate to different perspectives of the knee of the human being undergoing surgery.

[0182] The surgeon can then switch between two different work positions, and for each work position 320, 330, a predefined focus configuration can be implemented. Each predefined focus configuration can be based on a previously adjusted focus configuration and / or on machine learning or statistical inference.

[0183] Figure 4 The figure illustrates an articulated microscope system 400 with a lifting frame 410 according to an embodiment of the present disclosure. The lifting frame includes a horizontal moving device 412 and a vertical moving device 414. Thus, the mounted microscope 420 can be moved to a working position 422 and a parking position 424 (in gray).

[0184] Based on a predefined distance from the operating table 230, the system knows whether the microscope 420 has been moved toward the working position 422 or the parking position 424. For each possible position of the microscope relative to the operating table, the position of the microscope at the predefined distance from the operating table is indicated by plane 430 (illustrated as a dashed line in a two-dimensional diagram).

[0185] When the microscope 420 is moved to the working position and crosses the plane / line 430, autofocus can be activated and the predefined focus can be adjusted. Conversely, if the microscope moves away from the patient 232 and crosses the plane / line 430 in another direction, autofocus can be turned off.

[0186] One concept of the embodiments described in this disclosure is to integrate position sensing components (such as gyroscopes, sensor-equipped arms, or stereoscopic cameras) into the microscope's autofocus system. This feature enables real-time adjustment of the focus based on the movement or repositioning of the microscope. By estimating position changes and pre-adjusting the focus accordingly, the embodiments can minimize the need for manual fine-tuning after the microscope is at rest.

[0187] Such embodiments can produce further advantages, such as:

[0188] 1. Reduced manual intervention: By using position sensing to assist autofocus, the need for manual focus adjustment during microscope repositioning can be significantly reduced, thereby reducing interruptions during surgery.

[0189] 2. Provides real-time adjustment: The system can provide real-time adjustment of the focus point as the microscope moves, thereby improving the efficiency and speed of focusing, which is especially important during complex surgical procedures.

[0190] 3. Improved accuracy: The system can provide precise focus adjustment based on accurate position data, reducing user error and the possibility of potential inaccuracies in focusing.

[0191] Some embodiments relate to a microscope that includes... Figures 1 to 4 One or more related systems described in the text. Alternatively, the microscope can be... Figures 1 to 4 It is part of or connected to one or more related descriptions of the system.

[0192] Figure 5A schematic diagram of a system 500 configured to perform the methods described herein is shown. System 500 includes a microscope 510 and a computer system 520. The microscope 510 is configured to capture images and is connected to the computer system 520. The computer system 520 is configured to perform at least a portion of the methods described herein. The computer system 520 may be configured to execute machine learning algorithms. The computer system 520 and the microscope 510 may be separate entities, or they may be integrated together in a common housing. The computer system 520 may be part of the central processing system of the microscope 510 and / or the computer system 520 may be part of a sub-component of the microscope 510, such as a sensor, actuator, camera, or illumination unit of the microscope 510.

[0193] Computer system 520 may be a local computer device (e.g., a personal computer, laptop, tablet, or mobile phone) having one or more processors and one or more storage devices, or it may be a distributed computer system (e.g., a cloud computing system having one or more processors and one or more storage devices distributed across various locations, such as local clients and / or one or more remote server farms and / or data centers). Computer system 520 may include any circuitry or combination of circuitry. In one embodiment, computer system 520 may include one or more processors, which may be of any type. As used herein, a processor may refer to any type of computing circuitry, such as, but not limited to, a microprocessor, microcontroller, complex instruction set computing (CISC) microprocessor, reduced instruction set computing (RISC) microprocessor, very long instruction word (VLIW) microprocessor, graphics processor, digital signal processor (DSP), multi-core processor, field-programmable gate array (FPGA), computing circuitry for a microscope or microscope component (e.g., a camera), or any other type of processor or processing circuitry. Other types of circuitry that may be included in computer system 520 may be custom circuitry, application-specific integrated circuits (ASICs), etc., such as one or more circuits (e.g., communication circuitry) for wireless devices such as mobile phones, tablets, laptops, two-way radios, and similar electronic systems. Computer system 520 may include one or more storage devices, which may include one or more storage elements suitable for a particular application, such as main memory in the form of random access memory (RAM), one or more hard disk drives, and / or one or more drives that process removable media (e.g., optical discs (CDs), flash memory cards, digital video discs (DVDs), etc.). Computer system 520 may also include a display device, one or more speakers, and a keyboard and / or controller, which may include a mouse, trackball, touchscreen, voice recognition device, or any other device that allows system users to input and receive information from computer system 520.

[0194] Some or all of the method steps can be performed by (or using) hardware devices, such as, for example, a processor, microprocessor, programmable computer, or electronic circuit. In some embodiments, such devices can perform one or more of the most important method steps.

[0195] Depending on certain implementation requirements, embodiments of the present invention can be implemented in hardware or software. This implementation can be performed using a non-transient storage medium, such as a digital storage medium like a floppy disk, DVD, Blu-ray, CD, ROM, PROM, EPROM, EEPROM, or flash memory, storing electronically readable control signals that cooperate (or are capable of cooperating with) a programmable computer system to perform the corresponding methods. Therefore, the digital storage medium can be computer-readable.

[0196] Some embodiments of the invention include a data carrier having electronically readable control signals, which is capable of cooperating with a programmable computer system to perform one of the methods described herein.

[0197] Typically, embodiments of the present invention can be implemented as a computer program product having program code operable to perform one of the methods when the computer program product is run on a computer. For example, the program code may be stored on a machine-readable medium.

[0198] Other embodiments include a computer program stored on a machine-readable medium for performing one of the methods described herein.

[0199] In other words, therefore, one embodiment of the present invention is a computer program having program code for performing one of the methods described herein when the computer program is run on a computer.

[0200] Therefore, another embodiment of the invention is a storage medium (or data carrier, or computer-readable medium) including a computer program stored thereon for performing one of the methods described herein when executed by a processor. Data carriers, digital storage media, or recording media are typically tangible and / or non-transitory. Another embodiment of the invention is an apparatus as described herein, including a processor and a storage medium.

[0201] Therefore, another embodiment of the invention represents a data stream or signal sequence for performing one of the methods described herein. For example, the data stream or signal sequence may be configured to be transmitted via a data communication connection (e.g., via the Internet).

[0202] Another embodiment includes a processing device, such as a computer or programmable logic device, configured or adapted to perform one of the methods described herein.

[0203] Another embodiment includes a computer on which a computer program for performing one of the methods described herein is installed.

[0204] Another embodiment of the invention includes an apparatus or system configured to transmit a computer program (e.g., electronic or optical) for performing one of the methods described herein to a receiver. The receiver may be, for example, a computer, a mobile device, a storage device, etc. For example, the apparatus or system may include a file server for transmitting the computer program to the receiver.

[0205] In some embodiments, a programmable logic device (e.g., a field-programmable gate array) may be used to perform some or all of the functions of the methods described herein. In some embodiments, the field-programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. Generally, these methods are preferably performed by any hardware device.

[0206] As used herein, the term “and / or” includes any and all combinations of one or more of the related listed items and may be abbreviated as “ / ”.

[0207] Although some aspects are described in the context of an apparatus, it is clear that these aspects also represent a description of the corresponding method, where a box or device corresponds to a method step or a feature of a method step. Similarly, aspects described in the context of a method step also represent a description of an item or feature of the corresponding box or corresponding apparatus.

[0208] Implementation examples can be based on the use of machine learning models or machine learning algorithms. Machine learning can refer to algorithms and statistical models that a computer system can use to perform a specific task without the use of explicit instructions, relying instead on models and inference. For example, in machine learning, data transformations inferred from the analysis of historical and / or training data can be used instead of rule-based data transformations. For example, machine learning models or machine learning algorithms can be used to analyze the content of images. To use a machine learning model to analyze the content of an image, the machine learning model can be trained using training images as input and training content information as output. By training the machine learning model with a large number of training images and / or training sequences (e.g., words or sentences) and associated training content information (e.g., labels or annotations), the machine learning model “learns” to recognize the content of images, thus enabling the machine learning model to identify image content not included in the training data. The same principle can be applied to other categories of sensor data: by training a machine learning model using training sensor data and a desired output, the machine learning model “learns” the transformation between sensor data and output, which can be used to provide output based on non-training sensor data provided to the machine learning model. The provided data (e.g., sensor data, metadata, and / or image data) can be preprocessed to obtain feature vectors, which are used as input to the machine learning model.

[0209] Machine learning models can be trained using training input data. The example detailed above uses a training method called "supervised learning." In supervised learning, a machine learning model is trained using multiple training samples, where each sample can include multiple input data values ​​and multiple desired output values; that is, each training sample is associated with a desired output value. By specifying training samples and desired output values, the machine learning model "learns" which output value to provide based on input samples similar to those provided during training. Besides supervised learning, semi-supervised learning can also be used. In semi-supervised learning, some training samples lack corresponding desired output values. Supervised learning can be based on supervised learning algorithms (such as classification algorithms, regression algorithms, or similarity learning algorithms). Classification algorithms are used when the output is restricted to a finite set of values ​​(categorical variables), where the input is classified into one of a finite set of values. Regression algorithms are used when the output can have any numerical value (within a range). Similarity learning algorithms can be similar to classification and regression algorithms, but are based on learning from examples using a similarity function that measures the similarity or relevance of two objects. In addition to supervised or semi-supervised learning, unsupervised learning can also be used to train machine learning models. In unsupervised learning, input data can be provided (only), and unsupervised learning algorithms can be used to find structure in the input data (e.g., by grouping or clustering the input data to find commonalities). Clustering is the process of assigning input data, which includes multiple input values, into subsets (clusters) such that input values ​​within the same cluster are similar according to one or more (predefined) similarity criteria, while being different from input values ​​included in other clusters.

[0210] Reinforcement learning is a third type of machine learning algorithm. In other words, reinforcement learning can be used to train machine learning models. In reinforcement learning, one or more software actors (called "software agents") are trained to take actions in an environment. Rewards are calculated based on the actions taken. Reinforcement learning is based on training one or more software agents to select actions, thereby increasing the cumulative reward, making the software agents better at a given task (as demonstrated by the reward increase).

[0211] Furthermore, certain techniques can be applied to some machine learning algorithms. For example, feature learning can be used. In other words, machine learning models can be trained, at least in part, using feature learning, and / or machine learning algorithms can include feature learning components. Feature learning algorithms, also known as representation learning algorithms, can preserve information from their inputs but can also transform them in a way that makes them useful, often as a preprocessing step before performing classification or prediction. For example, feature learning can be based on principal component analysis or cluster analysis.

[0212] In some examples, anomaly detection (i.e., outlier detection) can be used, the purpose of which is to provide identification of suspicious input values ​​by making them significantly different from most of the input or training data. In other words, machine learning models can be trained using anomaly detection at least in part, and / or machine learning algorithms can include anomaly detection components.

[0213] In some examples, machine learning algorithms can use decision trees as predictive models. In other words, machine learning models can be based on decision trees. In a decision tree, observations about an item (e.g., a set of input values) are represented by branches of the decision tree, and the output value corresponding to that item is represented by the leaves of the decision tree. Decision trees can support both discrete and continuous values ​​as output values. If discrete values ​​are used, the decision tree can be approximated as a classification tree; if continuous values ​​are used, it can be approximated as a regression tree.

[0214] Association rules are another technique that can be used in machine learning algorithms. In other words, a machine learning model can be based on one or more association rules. Association rules are created by identifying relationships between variables in a large amount of data. Machine learning algorithms can identify and / or utilize one or more relationship rules that represent knowledge derived from the data. For example, these rules can be used to store, manipulate, or apply knowledge.

[0215] Machine learning algorithms are typically based on machine learning models. In other words, the term "machine learning algorithm" can broadly refer to a set of instructions that can be used to create, train, or use machine learning models. The term "machine learning model" can broadly refer to a data structure and / or set of rules that represent (e.g., based on training performed by a machine learning algorithm). In embodiments, using a machine learning algorithm may mean using an underlying machine learning model (or multiple underlying machine learning models). Using a machine learning model may mean that the machine learning model and / or the data structure / rule set used as the machine learning model were trained by a machine learning algorithm.

[0216] For example, a machine learning model can be an artificial neural network (ANN). An ANN is a system inspired by biological neural networks, such as those found in the retina or brain. An ANN consists of multiple interconnected nodes and multiple connections between nodes, known as edges. There are typically three types of nodes: input nodes that receive input values, hidden nodes that are connected to other nodes (only) and output nodes that provide output values. Each node can represent an artificial neuron. Each edge can send information from one node to another. The output of a node can be defined as a (non-linear) function of its inputs (e.g., the sum of its inputs). The node's inputs can be used in the function based on the "weights" of the edges or the nodes that provide the inputs. The weights of the nodes and / or edges can be adjusted during the learning process. In other words, training an artificial neural network can involve adjusting the weights of the artificial neural network's nodes and / or edges to achieve the desired output for a given input.

[0217] Alternatively, the machine learning model can be a Support Vector Machine (SVM), a Random Forest model, or a Gradient Boosting model. A Support Vector Machine (SVM, or Support Vector Network) is a supervised learning model with associated learning algorithms that can be used to analyze data (e.g., in classification or regression analysis). An SVM can be trained by providing inputs with multiple training input values ​​belonging to one of two categories. An SVM can be trained to assign new input values ​​to one of the two categories. Alternatively, the machine learning model can be a Bayesian network, a probabilistic directed acyclic graph model. A Bayesian network uses a directed acyclic graph to represent a set of random variables and their conditional dependencies. Alternatively, the machine learning model can be based on a genetic algorithm, a search algorithm and heuristic technique that mimics the process of natural selection.

Claims

1. A hinged microscope (110) with autofocus, Configured as: - Installed in the first position (102); - Obtain information that the microscope has been moved from the first position to the second position (106); - The focus is adjusted to observe the object (104) based on the information that the microscope has been moved to the second position.

2. The microscope according to the preceding claims, The microscope is mounted on one or more of the following: - Boom-type support (120); - Ceiling brackets; - Operating table or bedside support (120).

3. The microscope according to any one of the preceding claims, The microscope includes sensors, particularly a gyroscope, for observing the position of the microscope; and The second position is obtained based on information from the sensor.

4. The microscope according to any one of the preceding claims, Configured as: -Attached to an articulated boom (120); and - Obtain information about the second position from the articulated booms (122, 124, 126, 128).

5. The microscope according to any one of the preceding claims, Includes one or more markers observable via an external position sensing device (310); and It is configured to obtain information about the second position from the external position sensing device.

6. The microscope according to any one of the preceding claims, Configured as: Determine whether the current focus is valid for the second position.

7. The microscope according to any one of the preceding claims, The adjustment based on the second position (204) is a coarse adjustment; and Configured as: - The focus is adjusted without relying on the second position.

8. The microscope according to any one of the preceding claims, If the speed of the microscope relative to the object is substantially zero, the focus is adjusted based on the second position (204).

9. The microscope according to any one of the preceding claims, If the time-based derivative of the second position is substantially different from zero, then the focus is adjusted based on the second position (204).

10. The microscope according to any one of the preceding claims, Configured as: - Get the focus position, and The focus is adjusted based on the second position (204) and the direction of movement that causes the second position to move toward the focus position.

11. The microscope according to any one of the preceding claims, Configured as: - Get the focus position; and The focus is adjusted based on the second position and whether the second position is within a predefined threshold distance (430) from the object.

12. The microscope according to any one of the preceding claims, The focus is adjusted based on the second position and the first position, specifically based on whether the Euclidean distance between the first position and the second position exceeds a predefined threshold.

13. The microscope according to any one of the claims, Configured as: - Determine whether the movement type of the microscope is parking movement or using movement; and - The focus is adjusted based on the second movement only when it is determined that movement is to be used.

14. A method for controlling autofocus in a hinged microscope, comprising the following steps: - Obtain information about the initial position of the articulated microscope; - Obtain information that the microscope has been moved from the first position to the second position; - Determine the autofocus information of the articulated microscope based on the information that the microscope has been moved to the second position; - Control the focal point of the articulated microscope based on the autofocus information; And / or provide the autofocus information for the articulated microscope.

15. A device for controlling the autofocus of an articulated microscope. Configured as: - Obtain information about the initial position of the articulated microscope; - Obtain information that the microscope has been moved from the first position to the second position; - Determine the autofocus information of the articulated microscope based on the information that the microscope has been moved to the second position; - Provides the microscope with information for autofocus.

Images