Method and medical device for setting optical focus

Abstract

A first aspect of the present disclosure relates to a computer-based method for setting an optical focus, comprising the steps of: obtaining a first threshold for a drift of the optical focus; obtaining a first drift value, where the first drift value is based on measurements of at least one focus parameter at different moments in time; comparing the first drift value with a first threshold value; the first drift value is output when the first drift value does not permissively exceed the first threshold value.

Description

Technical Field

[0001] The present disclosure relates to a method for setting an optical focus. In addition, a medical device, such as a microscope, is disclosed, the optical focus of which can be operated by a corresponding method. Background Art

[0002] Autofocus (e.g. in a microscope) is a function that enables the focus of an objective to be automatically set to provide a clear and sharp image. However, autofocus systems can be affected by a phenomenon known as drift, and this phenomenon involves a gradual displacement or movement of the focus plane over time. Drift can negatively impact the accuracy and reliability of the autofocus function.

[0003] In-house experience has shown that drift can be caused by various factors, such as mechanical vibrations, temperature changes and thermal expansion / contraction of microscope components. These factors can lead to movements of the microscope stage, objectives or other components, which can cause a shift in the focal plane. Changes in the sample can also contribute to a loss of the initially set focus again.

[0004] To address problems due to drift in focus based systems, improvements are desired. Summary of the invention

[0005] Embodiments of the present disclosure are directed to improvements in autofocus systems.

[0006] This object is solved by the embodiments disclosed herein, which are particularly defined by the subject matter of the independent claims. Dependent claims relate to further embodiments. Different aspects and embodiments of these aspects are also disclosed in the following summary and description providing additional features and advantages.

[0007] A first aspect of the present disclosure relates to a computer-based method for setting optical focus, comprising the following steps:

[0008] - obtaining a first threshold value for the drift of the optical focus;

[0009] - obtaining a first drift value, wherein the first drift value is based on at least one focusing

[0010] Measurement of parameters;

[0011] - comparing the first drift value with a first threshold value;

[0012] - When the first drift value is not allowed When the first threshold is exceeded, a first drift value is output.

[0013] Drift is usually an error specific to autofocus systems operating in specific situations. Drift is particularly noticeable if the sample is observed over a longer period of time, for example if a series of images are to be generated every 5 minutes to observe the evolution of the sample. The initially sharp set viewing angle loses sharpness over time. This drift is usually corrected by optical focusing, in that the optical focusing tracks the focus accordingly.

[0014] Optical focusing can be included by various devices. Optical focusing can be a focusing system that works with electromagnetic radiation, such as light, UV radiation. Optical focusing can be included by medical and / or diagnostic equipment, such as a microscope. Optical focusing can also be included by an endoscope system or an ophthalmoscope system. Laser systems, such as laser surgery systems, can also have optical focusing. Optical focusing can also be included by a camera or video camera system.

[0015] A threshold value, in particular a first threshold value, may be received or extracted, for example, from the cloud, in particular when the method is not performed by a computer with optical focusing (this will be explained in detail later). A threshold value of an autofocus range refers to a predetermined value or range, which determines how far the autofocus system changes a new focus value relative to an old focus value. The threshold value may be given absolutely and / or relatively. It helps to determine a criterion for determining whether an image is sharp or not based on specific measurements or parameters. The threshold value may be an absolute threshold value, such as a maximum value that is not allowed to be exceeded. In this case, each exceedance of the threshold value is an impermissible exceedance. The threshold value may also be a complex threshold value, such as a maximum value and / or an average value, which are allowed to be exceeded temporarily and / or by a specific amount. In this case, additional parameters (duration of the exceedance, amount of the exceedance) determine whether the drift value exceeds the threshold value in an impermissible manner. The threshold value is used to check whether the identified new focus should be set. The threshold value can be used, for example, for credibility control of the newly identified focus.

[0016] The drift value, in particular the first drift value, can be received or extracted (from the focusing system) similarly to the threshold value. The drift value represents the autofocus, which must be set in order to reproduce the image (again) sharply. Similar to the threshold value, the drift value can be given absolutely or relatively (i.e. relative to another focus and / or drift value). The first drift value can consist of a single value. Alternatively, the drift value can consist of a plurality of values, which are in particular recorded directly one after the other.

[0017] The measured focusing parameters may be, for example, the focal length, the distance of the sample from the objective and / or the set objective aperture. Additionally or alternatively, the parameters may be reflection-based parameters, such as the position, offset or wavelength of the reflected light beam.

[0018] The comparison of the first drift value with the first threshold value can be carried out in particular according to the type of the threshold value or drift value. The comparison can be carried out, for example, by forming a difference. Additionally or alternatively, the comparison can also be carried out by the correlation of the two values. If the threshold value and the drift value are given in different dimensions, at least one of the two values ​​can be transformed accordingly in order to be able to compare the drift value with the threshold value.

[0019] The control of whether the first drift value should be set and therefore whether the previously set drift value should be replaced is based on a comparison of the first drift value with a first threshold value. If the first drift value is less than or equal to the first threshold value, the first drift value can be converted by the autofocus system. Here, determining that the first drift value is "less than or equal to" the threshold value should be an implementation-specific design scheme. It can also be required that the first drift value is less than the first threshold value and that the first drift value is not output when the first drift value is the same as the first threshold value. In both cases, the threshold value is a maximum threshold value. Alternatively, it can be determined that the drift value is allowed to exceed the first threshold value by a certain amount. In this case, the threshold value represents the average drift value.

[0020] Here, the output of the first drift value can be performed so that the optical focusing can convert the first drift value. For example, the first drift value can be sent to a system (e.g., a microscope) running the optical focusing. Additionally or alternatively, the first drift value can also be output so that the system running the optical focusing can obtain the first drift value. For example, the first drift value can be stored in a shared memory location accessible to the system running the optical focusing, such as stored on the cloud.

[0021] Embodiments of the first aspect relate to a computer-based method for setting optical focus, wherein the first threshold is predetermined based on at least one or more of the following parameters:

[0022] - contains information about the manufacturer of the device being focused;

[0023] - Information from the focused environment;

[0024] - Sample specific information.

[0025] This information can be provided, for example, by the manufacturer of the microscope. Additionally or alternatively, this information can be temperature information. The temperature information can be obtained in particular from a temperature sensor included in the focusing system that should be set. Additionally or alternatively, the temperature can be obtained from a temperature sensor operating in the environment (e.g., a room) in which the focusing system works. The information used to determine the threshold value can also be the exposure time of the sample. In particular, if the sample is adversely affected by exposure, the threshold value can be derived, for example, within the range of the maximum exposure time or exposure rate. Additionally or alternatively, the characteristics of the sample to be analyzed can represent the following information, based on which a threshold value for drift can be derived, which can be, for example, viscosity. Thus, a specific threshold determination is feasible. When device-specific and / or sample-specific values ​​are provided from a database, it is feasible to determine a situation-specific threshold value in an effective manner.

[0026] Embodiments of the first aspect relate to a computer-based method for setting an optical focus, wherein the first drift value is based on image-based measurements of the focus performed successively in time.

[0027] The image-based measurement can in particular be based on images recorded with optical focusing. Additionally or alternatively, the image-based measurement can be based on images recorded by a separate imaging system. This is in particular because, as described above, the drift can be based on changes in the focusing system and / or the sample. If these changes are known or have been measured, the drift can be determined based on these changes.

[0028] Thus, image-based drift measurements that rely on the hardware of the optical focusing system can be performed solely electronically or by software. Image-based drift measurements can also be drift measurements in 2 or 3 dimensions. Image-based measurements can also be passive measurements, where the sensor alone measures environmental information, rather than based on a signal generated by the sensor.

[0029] Embodiments of the first aspect relate to a computer-based method for setting optical focus, wherein a first drift value is based on a laser-based measurement of the focus.

[0030] Laser-based drift measurement uses a laser beam to measure motion or position changes with high accuracy. In a laser interferometer, the laser beam is split into two separate paths, one of which is used as a reference and the other interacts with the object to be measured (e.g., a sample). The resulting interference pattern is analyzed to determine the drift of the object.

[0031] In drift measurement based on laser tracking systems, a laser beam and an optical sensor are used to measure the position and / or movement of a sample in three-dimensional space. Here, the laser beam is directed either to the sample or to a system including optical focusing. The optical sensor detects the reflection of the emitted laser beam and analyzes it in order to determine the drift of the object.

[0032] Laser-based drift measurement is an example of a reflection measurement. Furthermore, a laser-based drift measurement is an example of a one-dimensional drift measurement, as long as the drift measurement only includes a single laser beam. In contrast to passive (image-based) drift measurements, reflection-based drift measurements are denoted active drift measurements. Instead of a laser, other light sources, such as LEDs, can also be used.

[0033] Since laser-based drift measurements usually operate at high repetition rates, they can be used in particular for focus holding devices, in which the focus position is adjusted by means of the reflected laser beam. In contrast, the previously described software-based drift measurements can be used as focus finding devices, in particular since more information is provided about the scanned two or three dimensions, with which the correct focus can be found.

[0034] Embodiments of the first aspect relate to a computer-based method for setting an optical focus, wherein measurements of the focus are recorded at different positions.

[0035] The drift measurement can be performed at different parts of the sample. To this end, measurements can be performed simultaneously at different parts and / or successively in time, in particular so that the same part is measured repeatedly. A respective drift value can then be determined for each of these parts and / or a unique drift value can be determined for multiple parts. If images are to be recorded at different positions of the sample, it may happen that the sample is not (anymore) present at one of these positions. In drift correction, the autofocus then attempts to focus on an empty environment. This may result in subsequent images at least at this position having an erroneous focus. If the focus of other positions is also determined based on the focus determined at this position, these focuses will also be erroneous.

[0036] Alternatively, this problem may also occur if the autofocus focuses on the wrong value for another reason, for example due to contamination or damage of the sample or the storage of the sample (e.g., a break in a petri dish). The method according to the first aspect can in particular reject untrustworthy focuses that are generated due to one or more of these reasons.

[0037] An embodiment of the first aspect relates to a computer-based method for setting optical focus, comprising the following steps:

[0038] -When the first drift value is equal to or exceeds the first threshold, discarding the first drift value.

[0039] The drift value obtained may exceed the threshold value for various reasons. On the one hand, the drift value may be unreasonable because it is definitely or with a high probability derived from an erroneous situation. Then, the drift value can be completely, that is, always discarded.

[0040] However, the drift value may also exceed the threshold value, because the threshold value is predetermined too small, and changes in the sample or in the microscope, for example, can lead to drift values ​​greater than the threshold value. This can occur in particular when rapid focusing is required, for example to avoid phototoxicity of the sample. The term "phototoxicity" is understood as damage to the sample caused by electromagnetic radiation, for example by exposure to infrared, visible light or UV radiation. The thus reduced focus search range, in which the drift is determined and which may be accompanied by a reduced threshold value, may result in, for example, detected drift at the edge of the focus search range, although in fact it is discarded as unreal. In particular in this case, although the drift value cannot be output at first, because the drift value is greater than the threshold value. However, further checks, for example using another optical focusing system (software focusing, hardware focusing), may show that the drift value is still real. Therefore, the drift value can also be only temporarily discarded and in particular stored for later use.

[0041] An embodiment of the first aspect relates to a computer-based method for setting optical focus, comprising the following steps:

[0042] - obtaining a second drift value;

[0043] - comparing the second drift value with the first threshold;

[0044] - When the second drift value does not exceed the first threshold, output the second drift value.

[0045] The second drift value can in particular be detected after the first drift value, in particular at the same position as the first drift value. Additionally or alternatively, the second drift value can be detected in parallel, for example with another focusing system (for example with hardware-based focusing when the focus to be set is software-based focusing). The second drift value can be detected at the same position, in particular in order to obtain an integrated drift value with greater safety. Additionally or alternatively, the second drift value can be detected at a different position.

[0046] The comparison of the second drift value with the first threshold value can realize an easy-to-implement and fast credibility check. Then, the second drift value corresponding to the first drift value can be output, or the second drift value can be completely or temporarily discarded. Alternatively, the second drift value can be stored for further use.

[0047] An embodiment of the first aspect relates to a computer-based method for setting optical focus, comprising the following steps:

[0048] - obtaining a second threshold value;

[0049] - obtaining a second drift value;

[0050] - comparing the second drift value with a second threshold value;

[0051] - When the second drift value does not exceed the second threshold, outputting the second drift value.

[0052] The comparison of the second drift value with the second threshold value can realize a dynamic plausibility check. The second threshold value can be predetermined based on the parameters already mentioned above and in particular at least temporarily replaces the first threshold value. The second threshold value can also be predetermined over a predetermined time period and / or specifically according to different locations.

[0053] Embodiments of the first aspect relate to a computer-based method for setting an optical focus, wherein the second threshold value is based on the first drift value, in particular when the first drift value is smaller than a first threshold value.

[0054] It may happen that the first threshold value selected at first is too large and that unrealistic, erroneous drift values ​​are therefore also verified as actual drift values. In order to avoid this, a second threshold value can be predetermined based on the first drift value. This can be done in particular in such a way that the second threshold value exceeds the magnitude of a predetermined absolute value or relative value. For example, the second threshold value can be selected to exceed the first drift value by 5%, 10%, 20%, 50%, 100% or 1000%. Thus, a threshold value that matches the current situation can always be selected.

[0055] In particular, if the first drift value is reliably verified, the second threshold value can be predetermined as a function of the first drift value. For example, a reliable verification can be performed by measuring the first drift value several times and then forming an average value therefrom. The several measurements of the first drift value can be measured successively, in parallel, with the same optical focus and / or with different optical focuses. This can in particular be done only at certain moments in time in order to save resources and reduce phototoxicity. The second threshold value can then also be updated at these moments.

[0056] Embodiments of the first aspect relate to a computer-based method for setting optical focus, wherein the second threshold is based on a laser-based measurement of the focus; and the second drift value is determined based on a measurement of the focus, which measurement is image-based, or vice versa.

[0057] The second threshold value can in particular be provided by a laser-based measurement which is performed with a higher repetition frequency than the image-based measurement of the second drift value. A current value can then always be provided for the threshold value, in particular when the repetition frequency of the image-based focus determination is low in order to minimize phototoxicity. Instead of a laser-based measurement, other types of focus measurements can also be used for the measurement of the second threshold value. Instead of an image-based measurement, other types of focus measurements can also be used for the measurement of the second drift value.

[0058] Embodiments of the first aspect relate to a computer-based method for setting optical focus, wherein a laser-based measurement of the focus is performed with a higher repetition frequency than an image-based focus measurement.

[0059] The repetition frequency of the threshold determination that does not cause phototoxicity can in particular be <10 kHz. This applies in particular to laser-based or LED-based threshold determination. Image-based measurements, in particular image-based measurements that cause phototoxicity, can be repeated for <1 s, <10 s, <100 s, <1 min, <5 min or <10 min.

[0060] Embodiments of the first aspect relate to a computer-based method for setting an optical focus, wherein the first threshold value and / or the second threshold value is determined by machine learning and based on information in a database.

[0061] The threshold value can be automatically determined based on various parameters (e.g., microscope type, ambient temperature, sample type) based on a machine learning algorithm, such as a neural network. A fully connected single-layer or multi-layer neural network can be used to determine the threshold value. The neural regression network can be a feedforward neural network, such as a multi-layer perceptron. The parameters for training or running the network can be extracted from a database.

[0062] A second aspect of the present disclosure relates to a computer-based method for setting a search range for optical focusing,

[0063] The following steps are involved:

[0064] - obtaining a first search range for focus measurement;

[0065] - obtaining a first drift value and / or a first focus value;

[0066] - determining a second search range based on at least one of the following information:

[0067] --First drift value;

[0068] --First focus value.

[0069] The focus search range is that range in which the focus is analyzed to determine the current focus and thus the drift that is currently predominantly present. In this case, the first search range can be predetermined in particular based on a threshold value.

[0070] By determining the second search range based on the first drift value and / or based on the first focus value, the search range for the optical focus of the automatic operation can be automatically adapted. In particular, the second search range can be selected to be greater than the first drift value and / or the first focus value by an absolute value or a relative value. The second search range can also be replaced by a continuous or quasi-continuous sequence of search ranges, which are directed to the obtained, in particular current, drift value or focus value. As a result, the search range can always be tracked so that it corresponds to the current drift characteristics and in particular no time is wasted unnecessarily focusing on a range that does not contain structures or only contains structures that are not of interest.

[0071] In order to avoid that no new drift value is detected when the second search range is set too small, the second search range can also be expanded again. This can be done, for example, after a fixed predetermined time. Additionally or alternatively, the focus search range can be enlarged when the new drift value is at the edge of the currently set focus search range. In particular, the second search range can be expanded when the new drift value is below a predetermined absolute distance or relative distance to the edge of the focus range. Additionally or alternatively, the focus search range can be expanded when a threshold value for the detected drift is exceeded one or more times.

[0072] Embodiments of a second aspect relate to a computer-based method of setting a search range for optical focusing,

[0073] The following steps are involved:

[0074] - determining a threshold value for a drift of the optical focus, wherein the threshold value is based on a second search range;

[0075] - obtaining drift values, wherein the drift values ​​are based on focus measurements at different moments in time;

[0076] - compare the drift value with the threshold;

[0077] -When the drift value does not exceed the threshold, output the drift value.

[0078] A third aspect of the present disclosure relates to a medical device with automatic focusing, in particular a microscope, which is configured to:

[0079] - execute the method according to any one of the preceding embodiments; or - communicate with a device, in particular via an application user interface, which implements the method according to any one of the preceding embodiments.

[0080] The medical device may also be a laboratory device and / or a diagnostic device. The application user interface may be configured in particular for the medical device to exchange information with a central server, for example a central server on the cloud. In particular, the method according to the first aspect or the second aspect may be run on the cloud and may in particular cooperate with a plurality of medical devices to set their optical focus. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] Other advantages and features are derived from the following embodiments, which are partially referenced to the accompanying drawings. The drawings show embodiments that are not always drawn to scale. For clarity of description, the sizes of different features may be enlarged or reduced accordingly. For this reason, these drawings are at least partially schematic.

[0082] Figure 1 Validation of a new drift value according to one embodiment of the present disclosure is shown.

[0083] Figure 2 The matching of the focus search range according to one embodiment of the present disclosure is shown.

[0084] Figure 3 A block diagram of a method according to an embodiment of the present disclosure is shown.

[0085] Figure 4 A schematic diagram of a system for performing the method of the present disclosure is shown.

[0086] In the following description, reference is made to the accompanying drawings which form a part of the present disclosure and which illustrate certain aspects and embodiments that may be understood of the present disclosure. Like reference numerals indicate features that are identical or at least functionally or structurally similar.

[0087] In general, the disclosure of the described method also applies to the corresponding device or corresponding system for performing the method, which system includes one or more devices, and vice versa. If, for example, a specific method step is described, the corresponding device may include features for performing the described method step, even if the feature is not explicitly described or shown in the drawings. On the contrary, if, for example, a specific device is described based on a functional unit, the corresponding method may include one or more steps for performing the described function, even if these steps are not explicitly described or shown in the drawings. Similarly, a system may include device features or corresponding features for performing specific method steps. The features of the various exemplary aspects and embodiments described above or below may be combined unless otherwise explicitly stated. DETAILED DESCRIPTION

[0088] Figure 1 The verification of the new drift value according to one embodiment of the present disclosure is illustrated with the aid of two scenes: The scenes abstractly illustrate the setting of the optical focus of a microscope.

[0089] In the first scene 100a, the verification of the newly detected drift value is shown. At time t=0, the sample 102a is located on the first focal plane. In addition, a threshold value 104 is shown relative to the focal plane 102a. The threshold value 104 indicates how strong the expected maximum drift of the sample is. The currently measured drift 106 is also shown relative to the sample 102. This drift can be achieved, in particular, using a non-image-based focusing system (e.g., laser-based focusing) to reduce / avoid the phototoxicity of the analyzed sample. The comparison of the threshold value 104 with the detected drift value 106 shows that the drift value 106 is less than the expected maximum drift 104. In this case, the drift value can be verified because there is a reasonable drift value. The parameters corresponding to the drift value 108 are then output to the autofocus of the microscope.

[0090] At time t=1, the verified drift value 106 is transferred by the optical focusing system of the microscope into a new focal plane 102c. The aforementioned focal plane of the sample is shown by a rectangle 102b with a dashed border. The change of focus within the range of the drift 106 is represented by an arrow 108. The same threshold 104 is also applied to the refocused sample 102c to verify further drift movement. Therefore, in the first scene 100a, the drift is within the expected drift. Therefore, by applying the verified drift, the sample is shifted to a new focal plane. The threshold 104 of the expected drift is also shifted accordingly.

[0091] In the second scenario 100b, the newly detected drift value is not verified. At time t=0, the sample 102 is again located on the first focal plane. In addition, a threshold value 104 is shown relative to the sample 102, as in scenario 100a. Furthermore, the currently measured drift 110 is shown relative to the sample 102 or relative to its focal plane. The comparison of the threshold value 104 with the detected drift value 110 shows that the drift value 110 is greater than the expected maximum drift 104. In this case, the drift value cannot be verified. There is an unreasonable drift value. Then, the drift value 106 is not output to the autofocus of the microscope. At time t=1, the focal plane of the optical focus of the microscope is still located at the same position. If another drift value is provided, it is compared with the threshold value 104 again. In scenario 100b, the measured drift 106 is greater than the expected drift 104. Therefore, the drift is not applied and the sample remains in its position.

[0092] Figure 2 A matching 200 of a focus search range according to one embodiment of the present disclosure is shown. At time t=0, a sample in a first focal plane 202a is shown. A new drift value is searched in a first focus search range 204a having a first width 206a.

[0093] At time t=1, a new drift value 208a is detected in the first focus search range 204a, and a new focal plane 202b for the sample is set for this purpose. In accordance with the new drift value 208a, a new value for the focus search range is also output.

[0094] At time t=2, the microscope uses a new focus search range 204b. The new focus search range 204b has a width 206b, which is smaller than the width 206a of the initially set focus search range. The width 206b of the new focus search range 204b is determined based on the drift value 208a and is 700% of the drift value 208a. As a result, a faster drift value determination than with the focus search range 204a is possible. In addition, the new focus search range 204b is still so large that larger drifts can also be detected. In addition, the center of the new focus search range 204b is located at a height 210, at which the second focal plane 202b is aligned due to the detected drift 208a.

[0095] In the new focus search range 204b, a further drift 208b is detected and the third focus plane 202c is set accordingly for the sample.

[0096] In an extension of this embodiment, a threshold value may also be predetermined based on the detected drift value, such as in Figure 1 In particular, the focus search range can be predetermined based on a predetermined threshold value.

[0097] Figure 3 A block diagram of a method 300 according to various embodiments of the present disclosure is shown.

[0098] First, in a first step 310, a first threshold value is determined. The first threshold value can be determined, for example, based on the ambient temperature and / or based on drift annotations of the optical focus manufacturer. The determination of the threshold value can be performed in particular automatically by an algorithm based on machine learning. This can be, for example, a neural network, which is continuously provided with information from theory and / or practice in order to be able to predict drift values ​​in specific cases. Such a learning algorithm can be run on a central server and obtain information about different optical focuses that are operated decentralized, for example in optical devices in a hospital or laboratory. Here, input and environmental information as well as actually measured drift values ​​can be provided together so that the algorithm can learn and extrapolate relationships for drift formation. A threshold value for drift, for example a maximum value, can then be determined based on this.

[0099] In a second step 320, a first drift value is determined. This can be done by a hardware- and / or software-based focus measurement. Here, the currently set focus can be compared with the focus to be reset. If they are not equal or if the current image is not sharp, there is a drift. The drift measurement can be arranged in a device in which the optical focus to be set is operated, for example in a microscope. Additionally or alternatively, the drift measurement can be determined by an external device (e.g. a camera or a laser interferometer) operating near the optical focus.

[0100] In the next step 330, the measured first drift value is compared with a first threshold value. This can be done in a simple manner by differentiating the drift value and the threshold value. If the threshold value exists as a statistical variable, for example as a distribution, then the correlation can also be performed, for example, based on a convolution. Depending on the type of threshold value (maximum drift value, average drift value), it can be required for the effective drift value to be smaller or that the effective drift value does not exceed the threshold value by a certain value.

[0101] In a fourth step 340, the first drift value is output when the first drift value inadmissibly exceeds the first threshold value. An admissible exceeding can be performed in particular if the drift value is realized by a plurality of values ​​recorded successively in time and only one of these values ​​exceeds the threshold value. An admissible exceeding can also be performed when the threshold value is a statistical value, which in particular can only be exceeded by a predetermined amount. For example, the threshold value can be a drift mean value, and the measured drift value is also considered to be reasonable if it exceeds the drift mean value by 10%, 50% or 100%.

[0102] In a further step 350, the initially set focus search range for drift matching is changed. This step can be carried out before step 340, in parallel with step 340 or after step 340. In this case, the focus search range can depend on different parameters similar to the threshold value. The focus search range can be determined, for example, based on the ambient temperature and / or based on the specifications of the optical focus manufacturer. The determination of the focus search range can be carried out in particular automatically by an algorithm based on machine learning. This can be carried out in principle exactly the same as the determination of the threshold value.

[0103] Here, a new focus search range is predetermined depending on the currently identified drift value. The focus search range can in particular be 10%, 50%, 100% or 500% larger than the drift. In particular, the focus search range can be predetermined only based on a verified drift value, i.e. a drift value that is positively compared with a threshold value. Alternatively, the new focus search range can also be predetermined based on a threshold value. By means of a focus search range that is predetermined based on the current drift value and / or the current threshold value, a new focus can be determined more quickly and the phototoxicity of the sample can be reduced or kept within acceptable limits.

[0104] Some embodiments relate to a microscope comprising: Figure 1-3 Alternatively, the microscope may be a system such as a combination of Figure 1-3 Part of or connected to one or more of the described systems.

[0105] Figure 4 A schematic diagram of a system 400 configured to perform the methods described herein is shown. System 400 includes a microscope 410 and a computer system 420. Microscope 410 is configured to record images and is connected to computer system 420. Computer system 420 is configured to implement at least a portion of the methods described herein. Computer system 420 can be configured to implement a machine learning algorithm. Computer system 420 and microscope 410 can be separate units, but can also be integrated in a common housing. Computer system 420 can be part of a central processing system of microscope 410, and / or computer system 420 can be part of a subcomponent of microscope 410, such as a sensor, actuator, camera or lighting unit of microscope 410.

[0106] The computer system 420 may be a local computing device (e.g., a personal computer, a laptop, a tablet computer, or a mobile phone) having one or more processors and one or more memory devices, or may be a distributed computer system (e.g., a cloud computer system having one or more processors and one or more memory devices distributed at different locations (e.g., at a local client and / or one or more remote server farms and / or data centers)). The computer system 420 may include any circuit or combination of circuits. In one embodiment, the computer system 420 may include one or more processors that may be of any type. The term "processor" as used herein may refer to any type of computing circuit, such as a microprocessor, microcontroller, complex instruction set computing (CISC) microprocessor, reduced instruction set computing (RISC) microprocessor, VLIW (very long instruction word) microprocessor, graphics processor, digital signal processor (DSP), multi-core processor, FPGA (field programmable gate array), or any other type of microscope (e.g., camera) or any other type of processor or processing circuit, such as a microscope or microscope component (e.g., camera). Other types of circuits that may be included in the computer system 420 may be custom circuits, application specific integrated circuits (ASICs), and the like, such as one or more circuits (e.g., communication circuits) used in wireless devices such as mobile phones, tablet computers, laptop computers, two-way radios, and similar electronic systems. The computer system 420 may include one or more memory devices, which may include one or more memory elements suitable for the respective application, such as, for example, main memory in the form of random access memory (RAM), one or more hard disk drives, and / or one or more drives for handling removable media such as compact disks (CDs), flash memory cards, digital video disks (DVDs), and the like. The computer system 420 may also include a display device, one or more speakers, a keyboard, and / or a controller, which may include a mouse, a trackball, a touch screen, a voice recognition device, or other devices that allow a system user to input information to and receive information from the computer system 420.

[0107] Some or all of the method steps may be implemented by (or using) a hardware device, such as, for example, a processor, a microprocessor, a programmable computer or an electronic circuit. In some embodiments, one or more of the most important method steps may be implemented by such a device.

[0108] According to specific implementation requirements, embodiments of the present invention can be implemented in hardware or software. Non-transient storage media can be used to implement, such as digital storage media (e.g., floppy disks, DVDs, Blu-rays, CDs, ROMs, PROMs, EPROMs, EEPROMs, or flash memory), electronically readable control signals are stored on digital storage media, which collaborate with programmable computer systems (or can collaborate with programmable computer systems) to implement the corresponding methods. Therefore, digital storage media can be computer readable. Some embodiments of the present invention include data carriers with electronically readable control signals, which can collaborate with programmable computer systems to perform one of the methods described herein.

[0109] Generally, the embodiments of the present invention can be implemented as a computer program product with a program code, and when the computer program product runs on a computer, the program code is used to perform one of the methods. The program code can be stored on a machine-readable carrier, for example.

[0110] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

[0111] In other words, one embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0112] Therefore, another embodiment of the present invention is a storage medium (or a data carrier or a computer-readable medium) on which a computer program is stored, which, when the computer program is executed by a processor, is used to perform one of the methods described herein. The data carrier, the digital storage medium or the recorded medium is generally tangible and / or non-transferable. Another embodiment of the present invention is a device as described herein having a processor and a storage medium.

[0113] Therefore, another embodiment of the present invention is a data stream or signal sequence representing a computer program for executing one of the methods described herein. A data stream or signal sequence can be designed, for example, so that it or they can be transmitted via a data communication connection, for example, via the Internet.

[0114] A further embodiment comprises a processing device, for example a computer or a programmable logic device, configured or adapted to perform one of the methods described herein.

[0115] A further embodiment comprises a computer on which is installed the computer program for performing one of the methods described herein.

[0116] Another embodiment of the invention comprises an apparatus or system configured to transmit (e.g., by electronic or optical path) a computer program for performing one of the methods described herein to a receiver. The receiver may be, for example, a computer, a mobile device, a memory device, etc. The apparatus or system may, for example, comprise a file server for transmitting the computer program to the receiver.

[0117] In some embodiments, a programmable logic device (e.g., a field programmable gate array) can be used to implement some or all of the functions of the methods described herein. In some embodiments, a field programmable gate array can collaborate with a microprocessor to perform one of the methods described herein. Generally, these methods are preferably performed by any hardware device.

[0118] As used herein, the term "and / or" includes all combinations of one or more of the listed aspects and may be abbreviated as " / ".

[0119] Although some aspects are described in the context of an apparatus, it is clear that these aspects also represent a description of a corresponding method, where a block or device corresponds to a method step or a feature of a method step. Similarly, aspects described in the context of a method step also represent a description of a corresponding block or element or feature of a corresponding apparatus.

[0120] Implementations may be based on the use of machine learning models or machine learning algorithms. Machine learning may refer to algorithms and statistical models that can use computer systems to implement specific tasks without explicit instructions and instead rely on models and reasoning. For example, in machine learning, instead of rule-based data transformations, transformations of data derived from analysis of historical and / or training data may be used. For example, image content may be analyzed with the aid of a machine learning model or a machine learning algorithm. In order for a machine learning model to analyze the content of an image (or other parameter), a machine learning model may be trained using training images as input and training content information as output. The machine learning model is trained by using a large number of training images and / or training sequences (e.g., words or phrases) and associated training content information (e.g., labels or annotations) to "learn" the machine learning model to identify the content of an image, so that the content of an image not included in the training data may be identified with the aid of a machine learning model. The same principle may also be used for other types of sensor data: by training a machine learning model with training sensor data and desired outputs, the machine learning model "learns" the transformation between sensor data and output, which may be used to provide output based on non-training sensor data provided to the machine learning model. The provided data (e.g., sensor data, metadata, and / or image data) may be preprocessed to obtain a feature vector used as input to a machine learning model.

[0121] The machine learning model can be trained with the help of training data. In the above example, a training method called "supervised learning" is used. In supervised learning, the machine learning model is trained with the help of multiple training samples, each of which can include multiple input data values ​​and multiple expected output values, that is, each training sample is associated with an expected output value. By indicating both the training mode and the expected output value, the machine learning model "learns" what output value it will provide based on an input mode similar to the mode provided during training. In addition to supervised learning, semi-supervised learning can also be used. In semi-supervised learning, some of the training patterns lack corresponding expected output values. Supervised learning can be based on supervised learning algorithms (e.g., classification algorithms, regression algorithms, or similarity learning algorithms). If the output value is limited to a limited set of values ​​(classification variables), that is, the input is mapped to one of a limited set of values, a classification algorithm can be used. When the output can have any value (within a certain range), a regression algorithm can be used. Similarity learning algorithms can be similar to classification and regression algorithms, but based on learning from examples using similarity functions, which measure how similar or related two objects are. In addition to supervised learning or semi-supervised learning, unsupervised learning can also be used to train machine learning models. In unsupervised learning, input data may be (only) provided, and an unsupervised learning algorithm may be used to find structure in the input data (e.g. by grouping or clustering the input data, locating commonalities in the data). Clustering is the mapping of input data comprising multiple input values ​​into subsets (clusters) such that input values ​​within the same cluster are similar according to one or more (predefined) similarity criteria, while being different from input values ​​included in other clusters.

[0122] Reinforcement learning is the third group of machine learning algorithms. In other words, reinforcement learning can be used to train machine learning models. In reinforcement learning, one or more software agents (so-called "software agents") are trained to perform actions in an environment. Rewards are calculated based on the actions performed. In reinforcement learning, one or more software agents are trained to choose actions such that the cumulative rewards increase, which results in the software agents getting better at the tasks they are tasked with (which is reflected in increasing rewards).

[0123] In addition, some techniques can be applied to some machine learning algorithms. For example, feature learning can be used. In other words, the machine learning model can be trained at least in part with the help of feature learning, and / or the machine learning algorithm can include a feature learning component. Feature learning algorithms (also known as representation learning algorithms) can retain information in their inputs, but can also transform them to make them useful, usually as a preprocessing step before performing classification or prediction. The learning of features can be based on, for example, principal component analysis or cluster analysis.

[0124] In some examples, anomaly (i.e., outlier) identification can be used, which aims to identify input values ​​that are suspicious because they are significantly different from the majority of inputs or training data. In other words, the machine learning model can be trained at least in part with the help of anomaly identification, and / or the machine learning algorithm can include an anomaly identification component.

[0125] In some examples, a machine learning algorithm may use a decision tree as a prediction model. In other words, a machine learning model may be based on a decision tree. In a decision tree, observations about an element (e.g., a series of input values) may be represented by a branch of the decision tree, and an output value corresponding to the element may be represented by a leaf of the decision tree. A decision tree may support both discrete and continuous values ​​as output values. If discrete values ​​are used, a decision tree may be referred to as a classification tree, and if continuous values ​​are used, a decision tree may be referred to as a regression tree.

[0126] Association rules are another technique that can be used in machine learning algorithms. In other words, a machine learning model can be based on one or more association rules. Association rules are created by identifying relationships between variables in a large amount of data. A machine learning algorithm can identify and / or use one or more relationship rules that represent knowledge derived from the data. Rules can be used, for example, to store, manipulate, or apply knowledge.

[0127] Machine learning algorithms are typically based on machine learning models. In other words, the term "machine learning algorithm" may refer to a series of instructions that can be used to create, train, or use a machine learning model. The term "machine learning algorithm" may refer to a data structure and / or rule set that represents learned knowledge (e.g., based on training performed by a machine learning algorithm). In some embodiments, the use of a machine learning algorithm may imply the use of an underlying (or multiple underlying) machine learning model. Using a machine learning model may mean that the machine learning model and / or the data structure / rule set representing the machine learning model has been trained by a machine learning algorithm.

[0128] The machine learning model can be, for example, an artificial neural network (ANN). ANN is a system that refers to biological neural networks, such as those found in the retina or brain. ANN consists of a plurality of interconnected nodes and a plurality of connections, so-called edges, between which the edges are located. Typically, there are three types of nodes: input nodes that receive input values, hidden nodes that are (only) connected to other nodes, and output nodes that provide output values. Each node can represent an artificial neuron. Each edge can transfer information from one node to another. The output of a node can be defined as a (nonlinear) function of its input (e.g., the sum of its inputs). The input of a node can be used in a function based on the "weight" of an edge or a node that provides an input. The weights of nodes and / or edges can be matched during the learning process. In other words, training an artificial neural network can include matching the weights of nodes and / or edges of an artificial neural network, i.e., to achieve a desired output for a given input.

[0129] Alternatively, the machine learning model can be a support vector machine, a random forest model or a gradient enhancement model.Support vector machine (i.e., support vector network) is a supervised learning model with an associated learning algorithm, which can be used to analyze data (e.g., in classification or regression analysis).Support vector machine can be trained by adding multiple training input values ​​belonging to one of two categories to the input.Support vector machine can be trained to map new input values ​​to one of two categories.Alternatively, the machine learning model can also be a Bayesian network, which is a probabilistic directed acyclic graphical model.The Bayesian network can represent a series of random variables and conditional dependencies thereof by directed acyclic graphs.Alternatively, the machine learning model can be based on genetic algorithms, search algorithms, and heuristic techniques that mimic the natural selection process.

[0130] Reference numerals list

[0131] 100a First scenario for verification

[0132] 100b Second scenario for verification

[0133] 102 Sample at initial focal plane

[0134] 102a Sample at initial focal plane

[0135] 102b Focal plane before drift

[0136] 102c Drift-corrected focal plane

[0137] 104 Threshold

[0138] 106 First drift value

[0139] 108 Drift due to changing focal plane

[0140] 110 First drift value

[0141] 200 Matches focus search range

[0142] 202a First focal plane

[0143] 202b Second focal plane

[0144] 202c Third focal plane

[0145] 204a First focus search range

[0146] 204b Second focus search range

[0147] 206a Width of the first focus search range

[0148] 206b Width of the second focus search range

[0149] 208a First drift value

[0150] 208b Second Drift

[0151] 210 Reference height

[0152] 300 Methods for focusing

[0153] 310 Determination of the First Threshold

[0154] 320 Measure the first drift value

[0155] 330 Compare the first drift value with the first threshold

[0156] 340 Output effective drift value

[0157] 350 Update focus search scope

[0158] 400 Microscope System

[0159] 410 Microscope

[0160] 420 Computer

Claims

1. A computer-based method for setting optical focus comprising the steps of: - obtaining a first threshold value for the drift of the optical focus (104); - obtaining a first drift value (106), wherein the first drift value is based on measurements of at least one focus parameter at different times; - comparing the first drift value with a first threshold value; - outputting the first drift value if the first drift value inadmissibly exceeds the first threshold value.

2. The method according to claim 1, in, The first threshold (104) is predetermined based on at least one or more of the following parameters: - contains information about the manufacturer of the device being focused; - Information from the focused environment; - Sample specific information.

3. The method according to any one of the preceding claims, The first drift value (106) is based on image-based measurements of the focus that are performed successively in time.

4. The method according to any one of the preceding claims, in, The first drift value (106) is based on a laser-based measurement of the focus.

5. The method according to any one of the preceding claims, in, Measurements of the focus are recorded at different positions.

6. The method according to any one of the preceding claims, comprising the steps of: - When the first drift value is equal to or exceeds the first threshold value (104), discarding the first drift value (106).

7. The method according to any one of the preceding claims, comprising the steps of: - obtaining a second drift value; - comparing the second drift value with a first threshold value (104); - When the second drift value does not exceed the first threshold, output the second drift value.

8. The method according to any one of the preceding claims, comprising the steps of: - obtaining a second threshold value; - obtaining a second drift value; - comparing the second drift value with a second threshold value; - When the second drift value does not exceed the second threshold, outputting the second drift value.

9. The method according to the preceding claim, in, The second threshold is based on the first drift value (106), in particular when the first drift value is smaller than the first threshold.

10. The method according to claim 8, wherein the second threshold is based on a laser-based measurement of the focus; and the determination of the second drift value is based on an image-based measurement of the focus, or vice versa.

11. The method according to the preceding claim, in, The laser-based measurement of the focus is performed at a higher repetition rate than the image-based measurement of the focus.

12. The method according to any one of the preceding claims, in, The first threshold (104) and / or the second threshold are determined by machine learning and based on information in a database.

13. A computer-based method for setting optical focus comprising the steps of: - obtaining a first search range for focus measurement (204a); - obtaining a first drift value (208a) and / or a first focus value; - determining a second search range (204b) based on at least one of the following information: --First drift value; --First focus value.

14. The method according to the preceding claim, comprising the steps of: - determining a threshold value for a drift of the optical focus, wherein the threshold value is based on a second search range (204b); - obtaining drift values, wherein the drift values ​​are based on focus measurements at different times; - compare the drift value with the threshold; - When the drift value does not exceed the threshold value, output the drift value.

15. A medical device, in particular a microscope, with automatic focusing, which is designed to: - carrying out a method according to any one of the preceding claims; or - communicating with a device, in particular via an application user interface, which device implements a method according to any of the preceding claims.

Images