IMAGING SYSTEM AND METHOD FOR PROVIDING AN OPERATIONAL SETTING OF AN IMAGING DEVICE

Description

Technical field

[0001] The present invention relates to a method for providing an operational setting of an imaging device. Further, the present invention relates to an imaging system comprising an imaging device and a controller for providing an operational setting thereof.Background

[0002] In the field of microscopy, it is a crucial demand to find an operational setting of the microscope in order to achieve an optimal image quality. Usually, the operational setting comprises a plurality of setting parameters which have to be optimized simultaneously before the image acquisition can be started. Optimizing the setting parameters is a cumbersome and time-consuming task for a user, which has to be repeated for each image acquisition. Therefore, in order to achieve an optimal image quality in an easy and reproducible manner, it is highly desirable to provide the user with a tool to make it easier for him or her to select an optimal operational setting.

[0003] Recently, machine learning algorithms have been developed assisting the user to find a proper operational setting without having to adjust a large number of setting parameters each time the microscope is used. An example of a microscope system using a machine learning algorithm for providing an operational setting is disclosed in the document DE 10 2014 102 080 A1. The machine learning algorithm disclosed therein allows the microscope system to learn rules according to which setting parameters are adjusted. This learning of the system happens during the manufacturing or development cycle of the microscopes.

[0004] An imaging system according to the preamble of claim 1 is disclosed in EP 3 340 251 A1.Summary

[0005] It is an object of the present invention to provide an imaging system comprising an imaging device and a controller as well as a method for enabling a user to find a proper operational setting of the imaging device fast and with minimum effort.

[0006] In order to achieve the afore-mentioned object, an imaging system according to claim 1 is proposed, comprising an imaging device and a controller for providing an operational setting of the imaging device.

[0007] Before an image is acquired, the controller provides the user with a preferred operational setting. This preferred operational setting may directly be used for the subsequent image acquisition, i.e. without being changed by the user prior to the image acquisition. Alternatively, the preferred operational setting may be changed by the user before the image is acquired. In such a case, the preferred operational setting is overwritten with the changed operational setting, this changed operational setting subsequently being used for the image acquisition.

[0008] After the image has been acquired based on the operational setting, which may be the afore-mentioned preferred setting provided by the controller in the first place or the setting changed by the user and overwriting the preferred setting, the controller receives a user input based on which the controller can decide whether or not the image generated by the imaging device according to the selected operational setting is discarded by the user. Subsequently, based on this discard information derived from the user input and based on the operational setting used for acquiring the image, the controller updates the preferred operational setting applying a machine learning algorithm.

[0009] The discard information represents a simple decision criterion which can be used by the machine learning algorithm for updating the preferred operational setting of the imaging device efficiently. In particular, as the discard information is generated based on the user input during the imaging operation performed by the imaging device, the process for updating the operational setting using the machine learning algorithm can be performed while the user is actually working with the imaging device. Thus, it is not necessary to implement a separate learning process preceding the actual image acquisition during which the user is working with the imaging device. In other words, the user does not have to manually vote if the acquired image is a good one or not, as it is e.g. taught by the aforementioned document DE 10 2014 102 080 A1. In contrast the controller of the imaging system according to embodiments of the present invention concludes based on the behavior of the user if the acquired image is a good one or not. Based on the conclusion the controller can feed the machine learning algorithm.

[0010] The proposed controller enables a reliable prediction of a preferred operational setting and an immediate application of the predicted setting. Thus, it is possible to improve the work flow, and the user is relieved of the need to adjust a plurality of setting parameters before the image acquisition can even be started. The prediction is based on preceding user inputs which indicate whether the current image acquired in the on-going imaging process is accepted by the user or discarded.

[0011] Preferably, the controller is further configured to conclude that the image is a good image when the discard information indicates that the image is not discarded by the user, to conclude that the image is not a good image when the discard information indicates that the image is discarded by the user, and to correspondingly feed the machine learning algorithm with an information indicating whether or not the image was concluded to be a good image. In the following, a good image may be understood as an image meeting the user's expectations in particular in terms of image quality so that the resulting user input subsequent to the image acquisition indicates that the image is accepted, i.e. not discarded by the user. Thus, a good image may be an image with which the user continues to work during the imaging process in one way or another. An image not accepted by the user to be a good image as defined above will be referred to as a bad image hereinafter.

[0012] The controller is configured to update the preferred operational setting further based on the operational setting used for acquiring the image. The latter setting, which is used for image acquisition, may be either the afore-mentioned preferred operational setting, which is provided by the controller in the first place, or a setting which is deliberately adjusted by the user to be used for the image acquisition. This deliberately adjusted setting may be derived from the preferred operational setting.

[0013] The update of the preferred operational setting, i.e. the prediction of a next setting to be used for image acquisition, is made on the basis of the previous setting. Thus, the controller feeds the machine learning algorithm with the current operational setting in order to make a decision on an operational setting which is likely to be required by the user at a later stage.

[0014] Preferably, the controller is configured to update the preferred operational setting so as to reinforce the operational setting used for acquiring the image when the image was concluded to be a good image, and / or to update the preferred operational setting so as to attenuate the operational setting used for acquiring the image when the image was concluded not to be a good image. In such an embodiment, a reinforcement machine learning may be performed in such a way that a good image causes a reinforcement of the selected operational setting, so to speak, the update of the preferred operational setting integrates the setting used for acquiring the image. In contrast, a bad image causes an attenuation of the current operational setting, i.e. the update of the preferred operational setting departs from the setting used for acquiring the image. For instance, it is assumed that a preferred setting parameter has value A. Then, the user overwrites A with a changed parameter value B, and the image is acquired based on value B. In case that the acquired image is a good image, then the subsequent preferred parameter may be B. On the other hand, in case that the acquired image is a bad image, the next preferred setting parameter may be again A.

[0015] The controller is configured to generate the discard information indicating that the image is discarded if after acquiring the image a setting parameter of a first group of parameters of the operational setting is changed by the user compared to the previous operational setting with which the image was acquired.

[0016] The first group of parameters includes at least one of exposure time setting, gain setting, contrast setting and illumination light setting.

[0017] The illumination light setting may comprise a setting for each light type including an ON / OFF setting for a ring light in combination with light segments, an ON / OFF setting for coaxial light, an ON / OFF setting for transmitted light.

[0018] The controller is configured to generate the discard information indicating that the image is not discarded if after acquiring the image only a setting parameter of a second group of parameters of the operational setting is changed by the user compared to the previous operational setting.

[0019] The second group of parameters includes at least one of a zoom setting and a stage setting.

[0020] The controller may be configured to control the imaging device automatically based on the updated operational setting.

[0021] The imaging system is a microscope system, for instance a confocal microscope system.

[0022] According to another aspect, a method according to claim 5 is provided.

[0023] According to another aspect, a computer program according to claim 6 is provided.Short Description of the Figures

[0024] Hereinafter, preferred embodiments are described with reference to the drawings in which: Figure 1 is a block diagram showing an imaging system including a controller according to an embodiment; and Figure 2 is a flow chart illustrating a work flow for providing an operational setting of the imaging device. Detailed Description

[0025] An imaging system 100 according to an embodiment is shown in the block diagram of Figure 1. According to the invention, the imaging system 100 is formed by a microscope system.

[0026] The imaging system 100 comprises an imaging device 102, which is a microscope, and a controller 104. In the present invention, the controller 104 may include an input module 106 and a learning module 108. Both modules 106, 108 may be implemented in hardware and / or software. Further, in the present embodiment, the controller 104 is configured to control the overall operation of the imaging system 100. In particular, the controller 104 provides a user with a preferred operational setting when starting the imaging device 102. This preferred operational setting comprises a plurality of setting parameters which serve to control the imaging device 102 in order to acquire an image. The preferred operational setting may have been stored in advance in memory means included in the controller 104. In order to inform the user on the preferred operational setting, the imaging system 100 may comprise e.g. a monitor not shown in Figure 1, on which the preferred operational setting is displayed.

[0027] After the preferred operational setting has been adjusted, the imaging device 102 is operated in order to generate the image. For this, the imaging device 102 is either controlled based on the preferred operational setting or based on an operational setting which has been changed by the user before the image is acquired. In the latter case, the preferred operational setting may be overwritten in the memory means of the controller 104 in response to a corresponding user input.

[0028] After the image has been captured by the imaging device, the input module 106 of the controller 104 receives a user input made by the user by means of a suitable input device included in the imaging system 100. Such an input device may be a control panel, a keyboard etc. configured to transmit the user input to the input module 106. The controller 104 is configured to generate a discard information based on the user input, i.e. to derive said discard information from the user input such that the discard information indicates whether or not the image generated by the imaging device 102 is discarded by the user.

[0029] Subsequently, based on the discard information derived from the user input, the controller 104 updates the preferred operational setting using the learning module 108. For this, the learning module 108 is configured to apply a machine learning algorithm on input data including the discard information. This algorithm enables a prediction of an operational setting which is assumed to be preferred by the user when operating the imaging device for the next image acquisition. In this respect, it is worth noting that the controller 104 is configured to update the operational setting by means of the learning module 108 while the user is actually working with the imaging device 102. In other words, the image acquisition and the process for updating the operational setting are performed simultaneously. Thus, the controller 104 is enabled to control the imaging device 102 automatically based on the updated operational setting.

[0030] In addition to the discard information indicating whether or not the image generated by the imaging device 102 has been discarded by the user, the learning module 108 of the controller 104 is further fed with the operational setting which is currently used for acquiring the image in order to predict the next update of the setting.

[0031] Figure 2 is a flow chart illustrating an exemplary workflow for updating the operational setting of the imaging device 102 included in the imaging system 100 of Figure 1. For the sake of simplicity, Figure 2 is a kind of hybrid diagram combining elements of a block diagram and a flow diagram. In particular, Figure 2 illustrates the interaction between the learning module 108 and the remaining parts of the imaging system 100.

[0032] In step S1 of the workflow shown in Figure 2, the controller 104 determines whether or not the image generated by the imaging device 102 is discarded by the user. For this, the controller 104 analyses the discard information derived from the user input which is entered by the user in response to the image presented to him or her e.g. by being displayed on a monitor. If the controller 104 determines in S1 that the image is discarded, the controller 104 concludes in step S2 that the image is a good image. On the other hand, if the controller determines in S1 that the image is discarded by the user, the controller concludes the image in step S3 to be a bad image.

[0033] The conclusion whether or not the image is discarded, i.e. whether the image is a good image or a bad image, is drawn depending on whether one or more specific setting parameters of the operational setting are changed by the user after the image has been acquired. The image is concluded to be a bad image, i.e. discarded (step S3) if after acquiring the image a setting parameter of a first group of parameters of the operational setting is changed by the user compared to the previous operational setting with which the image was acquired. The afore-mentioned first group of parameters includes at least one of exposure time setting, gain setting, contrast setting and illumination light setting.

[0034] On the other hand, the image is concluded to be a good image, i.e. not discarded (step S2), if after acquiring the image only a setting parameter of a second group of parameters of the operational setting is changed by the user compared to the previous operational setting with which the image was acquired. The afore-mentioned second group of parameters includes at least one of a zoom setting and a stage setting. In other words, the image is not discarded if only the zoom and / or stage setting has been changed.

[0035] In step S4 of the work flow shown in Figure 2, the learning module 108 of the controller 102 is fed with a discard information which indicates whether the current image is concluded to be a good image or a bad image. Further, in the present embodiment, the learning module 108 is further fed with the operational setting that was used for acquiring the current image. Applying the learning algorithm taking into account the afore-mentioned information, the learning module 108 makes a prediction based on which the preferred operational setting to be used in the next image acquisition is updated. As a result, the learning module 108 provides the imaging system 100 with the updated preferred operational setting to be used next.

[0036] When using the operational setting which was used for acquiring the current image, the learning module may update the preferred operational setting either in the affirmative or in the negative of the operational setting. Thus, the learning module 108 may be configured to apply a reinforcement machine learning in such a way that a good image causes a reinforcement of the selected operational setting whereas a bad image causes an attenuation of the current operational setting.

[0037] Although some aspects have been described in the context of an apparatus, it is clear that these aspects also represent a description of the corresponding method, where a block or device corresponds to a method step or a feature of a method step. Analogously, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus. Some or all of the method steps may be executed by (or using) a hardware apparatus, like for example, a processor, a microprocessor, a programmable computer or an electronic circuit. In some embodiments, some one or more of the most important method steps may be executed by such an apparatus.

[0038] Depending on certain implementation requirements, embodiments of the invention can be implemented in hardware or in software. The implementation can be performed using a non-transitory storage medium such as a digital storage medium, for example a floppy disc, a DVD, a Blu-Ray, a CD, a ROM, a PROM, and EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed. Therefore, the digital storage medium may be computer readable.

[0039] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

[0040] In other words, an embodiment of the present invention is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on the controller of the imaging system.

[0041] A further embodiment of the present invention is, therefore, a storage medium (or a data carrier, or a computer-readable medium) comprising, stored thereon, the computer program for performing one of the methods described herein when it is performed by the controller of the imaging system.

[0042] The data carrier, the digital storage medium or the recorded medium are typically tangible and / or non-transitionary. A further embodiment of the present invention is an apparatus as described herein comprising a processor and the storage medium.

[0043] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0044] In some embodiments, a programmable logic device (for example, a field programmable gate array) may be used to perform some or all of the functionalities of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor in order to perform one of the methods described herein. Generally, the methods are preferably performed by any hardware apparatus.List of Reference Signs

[0045] 100imaging system 102imaging device 104controller 106input module 108learning module S1 to S4method steps

Claims

1. An imaging system (100), comprising an imaging device (102) and a controller (104), wherein the controller is configured to: provide a user with a preferred operational setting of the imaging device (102) for acquiring an image; control the imaging device (102) to acquire the image based on an operational setting, wherein the preferred operational setting is used for the image acquisition with or without being changed by the user prior to the image acquisition, receive a user input and to generate a discard information based on the user input, said discard information indicating whether or not the image generated by the imaging device (102) is discarded by the user; and update the preferred operational setting using a machine learning algorithm based on the discard information, characterized in that the controller (104) is configured to update the preferred operational setting further based on the operational setting used for acquiring the image, wherein the controller (104) is configured to generate the discard information indicating that the image is discarded if after acquiring the image a setting parameter of a first group of parameters of the operational setting for a next image acquisition is changed by the user compared to the operational setting with which the image was acquired, wherein the controller (104) is configured to generate the discard information indicating that the image is not discarded if after acquiring the image only a setting parameter of a second group of parameters of the operational setting is changed by the user compared to the operational setting with which the image was acquired, wherein the first group of parameters includes at least of one of exposure time setting, gain setting, contrast setting, and illumination light setting, wherein the second group of parameters includes at least one of a zoom setting and a stage setting, and wherein the imaging system (100) is a microscope system.

2. The imaging system (100) according to claim 1, wherein the controller (104) is configured to conclude that the image is a good image when the discard information indicates that the image is not discarded by the user, to conclude that the image is not a good image when the discard information indicates that the image is discarded by the user, and to correspondingly feed the machine learning algorithm with an information indicating whether or not the image was concluded to be a good image.

3. The imaging system (100) according to claim 2, wherein the controller (104) is configured to update the preferred operational setting so as to reinforce the operational setting used for acquiring the image when the image was concluded to be a good image, and / or to update the preferred operational setting so as to attenuate the operational setting used for acquiring the image when the image was concluded not to be a good image.

4. Imaging system (100) according to one of the preceding claims, wherein the controller (104) is configured to control the imaging device (102) automatically based on the updated operational setting.

5. A method for controlling an imaging device (102), comprising the following steps: providing a user with a preferred operational setting of the imaging device (102) for acquiring an image; controlling the imaging device (102) to acquire the image based on an operational setting, wherein the preferred operational setting is used for the image acquisition with or without being changed by the user prior to the image acquisition, receiving a user input and generating a discard information based on the user input, said discard information indicating whether or not the image generated by the imaging device (102) is discarded by the user; and updating the preferred operational setting using a machine learning algorithm based on the discard information, characterized by updating the preferred operational setting further based on the operational setting used for acquiring the image, generating the discard information indicating that the image is discarded if after acquiring the image a setting parameter of a first group of parameters of the operational setting for a next image acquisition is changed by the user compared to the operational setting with which the image was acquired, by generating the discard information indicating that the image is not discarded if after acquiring the image only a setting parameter of a second group of parameters of the operational setting is changed by the user compared to the operational setting with which the image was acquired, wherein the first group of parameters includes at least of one of exposure time setting, gain setting, contrast setting, and illumination light setting, wherein the second group of parameters includes at least one of a zoom setting and a stage setting, and wherein the imaging device (102) is a microscope.

6. Computer program with a program code for performing the method according to claim 5, when the computer program is run on the controller of the imaging system (100) according to any one of the claims 1 to 4.