Method for processing digital image of microscope sample and microscope system

Abstract

The concept of the present invention relates to a microscope system (10) and a method (40) for processing a plurality of digital images of a sample (182). The method (40) includes illuminating the sample (182) with a first subset of a plurality of illumination patterns by an illumination system (100), the illumination system comprising a plurality of light sources, each light source of the plurality of light sources being configured to illuminate the sample from one of a plurality of directions, each illumination pattern of the plurality of illumination patterns being formed by one or more of the plurality of light sources, obtaining a first input set of digital images by capturing a digital image of the sample (182) for each illumination pattern of the first subset of the plurality of illumination patterns, inputting the first input set of digital images into a first set of machine learning models configured to output a first inference output, illuminating the sample with a second subset of the plurality of illumination patterns by the illumination system (100), obtaining a second input set of digital images by capturing a digital image of the sample for each illumination pattern of the second subset of the plurality of illumination patterns, inputting the second input set of digital images into a second set of machine learning models configured to output a second inference output, wherein the second set of machine learning models is different from the first set of machine learning models, and inputting the first inference output and the second inference output into an image processing machine learning model trained to process a plurality of digital images of the sample using the first inference output and the second inference output (S416).

Description

Technical Field

[0001] The concept of the present invention relates to the processing of a plurality of digital images of a sample.

Background Art

[0002] Microscopy is widely used for the analysis of various samples. Examples of such samples include, but are not limited to, blood and bone marrow. As digitization has progressed, new and exciting technologies have been developed to image small features of imaged samples in great detail. Some of these technologies are typically referred to as computational imaging. In computational imaging, an image of a sample is processed in some way to generate information that is more detailed than what is normally possible. An example of such a technology is Fourier ptychographic microscopy (FPM). FPM is a technology that enables the construction of an image of a sample that has a higher resolution than what is normally possible with the optical components used to image the sample. This is done by combining image data in the Fourier space in an iterative and computationally intensive process after illuminating the sample from a plurality of different directions that are different from each other.

Summary of the Invention

Problems to be Solved by the Invention

[0003] Problems associated with FPM are the large number of images (usually several hundred) taken using illumination from different directions necessary to construct a high-resolution image of a sample and the time required for the actual image construction process. Due to the iterative nature of the image construction process, image reconstruction typically requires a large amount of time. In other words, the disadvantage of FPM is a slow process for creating a high-resolution image of a sample. Therefore, there is a need for improvement in the art.

[0004] The object of the present invention is, at least in part, to alleviate, reduce or eliminate one or more of the above-identified drawbacks and disadvantages in the art, either alone or in any combination, and to solve at least the above problems.

[0005] The object of the concept of the present invention is to provide a more time-efficient method for processing digital images of samples.

[0006] The object of the present invention is to provide a microscope system capable of processing digital images of samples with higher time efficiency.

[0007] The object of the present invention is to provide a method and / or a microscope system capable of analyzing samples more efficiently (e.g., with higher time efficiency).

Means for Solving the Problems

[0008] According to a first aspect, a method for processing a plurality of digital images of a sample is provided. The method includes: a. illuminating the sample with a first subset of a plurality of illumination patterns by an illumination system, the illumination system comprising a plurality of light sources, each light source of the plurality of light sources being configured to illuminate the sample from one of a plurality of directions, each illumination pattern of the plurality of illumination patterns being formed by one or more of the plurality of light sources, and obtaining a first input set of digital images by capturing a digital image of the sample for each illumination pattern of the first subset of the plurality of illumination patterns; b. inputting the first input set of digital images into a first set of machine learning models configured to output a first inference output; c. illuminating the sample with a second subset of the plurality of illumination patterns by the illumination system, and obtaining a second input set of digital images by capturing a digital image of the sample for each illumination pattern of the second subset of the plurality of illumination patterns and forming the second input set of digital images, and inputting the second input set of digital images into a second set of machine learning models configured to output a second inference output, wherein the second set of machine learning models is different from the first set of machine learning models; and inputting the first inference output and the second inference output into an image processing machine learning model trained to process the plurality of digital images of the sample using the first inference output and the second inference output.

[0009] In the context of the present disclosure, the term "machine learning model" should be construed as a machine learning model suitable for image processing. The first set of machine learning models and the second set of machine learning models may have the same structure (e.g., the number of machine learning models, the number of layers of each machine learning model, activation functions, etc.) or different structures. Each set of machine learning models may comprise one or more machine learning models. For example, the first set of machine learning models may comprise a single machine learning model. Similarly, the second set of machine learning models may comprise a single machine learning model. However, it should be understood that the number of machine learning models in each of the first and second sets of machine learning models may be different. Thus, each of the first and second sets of machine learning models may comprise a plurality of machine learning models.

[0010] According to the concept of the present invention, the period required for processing a plurality of digital images of a sample can be shortened. In particular, since the process of acquiring digital images and the inference of the acquired digital images are time-consuming, it is advantageous to divide the step of acquiring digital images of the sample into at least two steps. Thereby, the inference of the first input set of digital images can be started before / during the acquisition process of the second input set of digital images, and the time taken for processing a plurality of digital images can be shortened. By shortening the period related to image processing, the time taken for analyzing the sample can be shortened.

[0011] Another related advantage is that the memory requirements related to the processing of a plurality of digital images can be reduced. For example, the first input set of digital images and the second input set of digital images may be processed at least partially sequentially (i.e., input to each set of machine learning models), and the two input sets of digital images may thereby be stored in memory (e.g., working memory) at least partially sequentially. In other words, the first input set and the second input set of digital images may be processed without storing both the first input set and the second input set of digital images in memory simultaneously.

[0012] Steps b and c may be executed at least partially in parallel. In other words, the processing of the first set of input digital images by the first set of machine learning models may be executed at least partially in parallel with the acquisition of the second set of digital images.

[0013] The associated advantage is that the period required to process multiple images of the sample can be further shortened. This is because the inference of the first set of input digital images can be executed at least partially in parallel with the acquisition process of the second set of input digital images. Preferably, step c starts as soon as step a ends. Thus, step c may start before step b.

[0014] The lighting system may comprise a plurality of light sources, and each lighting pattern of the plurality of lighting patterns may be formed by one or more of the plurality of light sources.

[0015] The associated advantage is that digital images of the sample can be captured under various lighting conditions, thereby enabling more information about the sample (e.g., information related to the refractive index) to be collected in the captured digital images.

[0016] Each light source of the plurality of light sources may be configured to irradiate the sample from one of the plurality of directions.

[0017] By illuminating a sample from a plurality of different directions and capturing digital images for each of the plurality of directions, information about the fine details of the sample can be captured that is more than what can typically be resolved by a conventional microscope used to image the sample (i.e., by using conventional microscope illumination). This can be understood as information from different parts of the Fourier space (i.e., the spatial frequency domain) associated with the sample being captured for different illumination patterns (e.g., different illumination directions). This technique is known in the art as Fourier ptychography. Further, by illuminating the sample with a plurality of different illumination patterns and capturing digital images for each of the plurality of illumination patterns, information regarding the refractive index associated with the sample can be captured. This can be understood as an effect where the refraction of light depends on the angle of incidence of the light illuminating the sample and the refractive index of the sample. With information regarding the refractive index of the sample, phase information associated with the sample (which is generally referred to as quantitative phase in the art) can be determined. Since the plurality of digital images contain information related to one or more of the fine details of the sample, the refractive index associated with the sample, and the phase information associated with the sample, this information may be used by one or more of a first set of machine learning models, a second set of machine learning models, and an image processing machine learning model. Thereby, more information related to the sample can be extracted from the plurality of digital images than when the plurality of digital images are captured from only one direction or by using conventional microscopy. By using conventional microscopy (e.g., by illuminating the sample from most of a plurality of directions up to the numerical aperture of the microscope objective lens used to image the sample), it may be difficult or impossible to capture information related to the refractive index associated with the sample and / or phase information associated with the sample. By illuminating the sample with a plurality of different illumination patterns, it is also possible to capture information related to finer details of the sample than what is normally allowed by the microscope objective lens used to image the sample.Therefore, it is possible to capture information related to the fine details of the sample while using a relatively low magnification microscope objective lens. By using a relatively low magnification microscope objective lens, a relatively large portion of the sample can be imaged at each imaging position. Therefore, by imaging at relatively few positions, the entire sample can be scanned, and the entire sample or at least a major portion of the sample can be imaged more quickly.

[0018] At least one of the plurality of directions may correspond to an angle greater than the numerical aperture of the microscope objective lens used to image the sample.

[0019] The numerical aperture of the microscope objective lens may be a dimensionless number related to the range of angles at which the microscope objective lens accepts light. Therefore, a direction greater than the numerical aperture can be understood as a direction corresponding to an angle greater than the range of angles at which the microscope objective lens accepts light.

[0020] By illuminating the sample from a direction corresponding to an angle greater than the numerical aperture of the microscope objective lens, the digital image captured for that illumination angle can contain information regarding higher spatial frequencies of the sample than what is normally possible with the microscope objective lens (e.g., using conventional microscope illumination), and thus, information regarding fine details. Thereby, the microscope objective lens can capture phase information related to the sample and / or information related to details that are not normally resolvable with the microscope objective lens. Such information may be extracted and / or used by one or more of a first set of machine learning models, a second set of machine learning models, and an image processing machine learning model during the processing of the digital image. In other words, by illuminating the sample from a direction corresponding to an angle greater than the numerical aperture of the microscope objective lens, a digital image can be obtained from which more information related to the sample can be extracted than what is normally allowed by a machine learning model (e.g., when using conventional microscope illumination).

[0021] The first subset of illumination patterns may be different from the second subset of illumination patterns.

[0022] A related advantage is that the period required for processing and / or capturing a plurality of images of the sample can be further shortened. This is because the first input set of digital images and the second input set of digital images contain less redundant information. In other words, the first input set of digital images and the second input set of digital images may not include digital images captured under the same illumination conditions (i.e., illumination patterns). Since the first inference output and the second inference output are used by an image processing machine learning model, it may not be necessary to have redundant information in the first input set and the second input set of digital images. Redundant information in the first inference output and the second inference output may lead to unnecessary calculations by the image processing machine learning model. Therefore, by reducing redundant information, the computational resources associated with processing the first inference output and the second inference output by the image processing machine learning model can be reduced.

[0023] The image processing machine learning model may be trained to process a plurality of digital images using the first inference output and the second inference output by training one or more of constructing a digital image of the sample, classifying the sample into one or more classes, and detecting one or more objects within the sample. The image processing machine learning model may be configured to construct a digital image depicting the sample.

[0024] When the image processing machine learning model is trained to construct a digital image of the sample, that digital image of the sample can have a relatively higher resolution than a digital image captured by a microscope system using conventional microscope illumination (e.g., brightfield conditions). In other words, the image processing machine learning model may be trained to construct a digital image of the sample that has a relatively higher resolution than the digital images of the first input set of digital images and / or the digital images of the second input set of digital images.

[0025] When training an image processing machine learning model to classify samples into one or more classes, those classes may differ from each other depending on the type of sample. For example, when the sample is a biological sample (e.g., blood, bone marrow, body fluid, etc.), the one or more classes include, but are not limited to, whether the sample is cancerous, infected with a disease, abnormal, belongs to a specific classification, etc. In other words, what information the one or more classes contain may depend on the type of sample. For example, when the sample contains white blood cells, the one or more classes may include information regarding different white blood cell types. When the sample contains red blood cells, the one or more classes may include information regarding the morphology of the red blood cells. When the sample contains cervical fluid, the one or more classes may include information regarding pap smear abnormalities. The sample may be a non-biological sample. For example, the sample may be at least a part of a printed circuit board (PCB). In such a case, the one or more classes may be whether a part of the PCB is as per the specifications (e.g., correctly manufactured, damaged, etc.).

[0026] When training an image processing machine learning model to detect one or more objects within a sample, such objects may include, but are not limited to, malaria, white blood cells, red blood cells, platelets, platelet clumps, etc. The one or more objects may be, for example, a part of the sample. For example, the object may be a part of the sample suitable for collection and classification of a specific cell type. This is useful when the sample is bone marrow. The image processing machine learning model may be further trained to determine the type of the detected object. This may be a classification of the sample into one or more classes. The image processing machine learning model may be further configured to determine the number of the detected objects. Thus, the image processing machine learning model may be trained to count the objects within the sample.

[0027] According to a second aspect, a microscope system is provided. The microscope system is an illumination system configured to illuminate a sample with a plurality of illumination patterns, comprising a plurality of light sources, each light source of the plurality of light sources being configured to illuminate the sample from one of a plurality of directions, each illumination pattern of the plurality of illumination patterns being formed by one or more of the plurality of light sources, an illumination system, an image sensor configured to capture a digital image of the sample, a microscope objective lens configured to image the sample onto the image sensor, a first acquisition function configured to control the illumination system to illuminate the sample with a first subset of the plurality of illumination patterns and to control the image sensor to capture a digital image of the sample for each illumination pattern of the first subset of the plurality of illumination patterns to form a first input set of digital images, a first inference function configured to input the first input set of digital images into a first set of machine learning models configured to output a first inference output, a second acquisition function configured to control the illumination system to illuminate the sample with a second subset of the plurality of illumination patterns and to control the image sensor to capture a digital image of the sample for each illumination pattern of the second subset of the plurality of illumination patterns to form a second input set of digital images, a second inference function configured to input the second input set of digital images into a second set of machine learning models configured to output a second inference output, the second set of machine learning models being different from the first set of machine learning models, a second inference function, and a processing function configured to input the first inference output and the second inference output into an image processing machine learning model trained to process a plurality of digital images of the sample using the first inference output and the second inference output, and a circuit configured to execute the functions.

[0028] The circuit may be configured to execute the first inference function and the second acquisition function at least partially in parallel.

[0029] The illumination system may include a plurality of light sources, and each illumination pattern of the plurality of illumination patterns may be formed by one or more of the plurality of light sources.

[0030] Each of the plurality of light sources may be configured to irradiate a sample from one of a plurality of directions.

[0031] At least one of the plurality of directions may correspond to an angle greater than the numerical aperture of a microscope objective lens configured to image the sample on the image sensor.

[0032] The plurality of light sources may be arranged on a curved surface that is recessed along at least one direction.

[0033] Arranging the plurality of light sources on the curved surface can be advantageous in that the distances from each light source to the current imaging position of the microscope system (i.e., the position or portion of the sample being currently imaged) can be made the same. Since this distance is the same, the intensity of the light emitted from each light source can be made the same at the current imaging position (considering that each light source is configured to emit light of the same intensity). This can be understood as the effect of the inverse square law. Thus, the sample can be illuminated with light having a similar intensity in each of a plurality of directions, which can provide a more uniform illumination of the sample that is independent of the illumination direction and / or illumination pattern (when each illumination pattern is formed by the same number of light sources). It may be advantageous to configure the illumination system such that the distance from each light source to the current imaging position is large enough so that each light source can be regarded as a point source. Thereby, the light can be made quasi-coherent at the current imaging position. Thus, the distance from each light source to the current imaging position can be selected such that the intensity of the light from each light source at the current imaging position is high enough to generate an input set of digital images.

[0034] The curved surface may be formed by facets. In other words, the curved surface may be composed of a plurality of facets. In other words, the curved surface may be piecewise flat.

[0035] The related advantages are that the manufacturing of the lighting system is facilitated and the related economic costs are reduced. Another related advantage is that the lighting system is modular. Thereby, it becomes even easier to replace one or more light sources (for example, when the light source is broken and / or has a defect).

[0036] The numerical aperture of the microscope objective lens is 0.4 or less. In other words, the magnification of at least one microscope objective lens is 20 times or less.

[0037] The related advantage is that a relatively large portion of the sample can be imaged at once compared to a microscope objective lens having a relatively high numerical aperture. Thereby, the number of individual imaging positions required to image a large portion of the sample can be reduced. Therefore, the time required to image a large portion (for example, the whole) of the sample can be shortened.

[0038] The image processing machine learning model may be trained to process a plurality of digital images using a first inference output and a second inference output by training one or more of the construction of a digital image of the sample, and / or the classification of the sample into one or more classes, and / or the detection of one or more objects within the sample.

[0039] The above-described features of the first aspect are also applicable to this second aspect, where applicable. See the above for avoiding excessive repetition.

[0040] The further scope of the present disclosure will become apparent from the detailed description shown below. However, it should be understood that the detailed description and specific examples, while indicating preferred variations of the concept of the invention, are given by way of illustration only, since various changes and modifications within the scope of the concept of the invention will be apparent to those skilled in the art from this detailed description.

[0041] Accordingly, since the described methods and the described systems may be modified, it should be understood that the concepts of the present invention are not limited to specific steps of such methods or components of such systems. Also, it should be understood that the terms used herein are for the purpose of describing particular embodiments only and are not intended to be limiting. As used in this specification and the appended claims, the articles "a," "an," "the," and "said" are intended to mean that there are one or more elements unless the context clearly dictates otherwise. Thus, for example, references to "a unit" or "the unit" may include, among other things, a plurality of devices. Further, the terms "comprising," "including," "containing," and the like do not exclude the presence of other elements or steps.

Brief Description of the Drawings

[0042] Next, the above and other aspects of the concepts of the present invention will be described in more detail with reference to the accompanying drawings showing variations of the concepts of the present invention. The figures are not to be considered as limiting the concepts of the present invention to specific variations, but are used for the purpose of explaining and understanding the concepts of the present invention. As shown, the sizes of the layers and regions are exaggerated for purposes of explanation and are thus provided to illustrate the general structure of variations of the concepts of the present invention. Like reference numerals refer to like elements throughout.

[0043]

Figure 1

Figure 2

Figure 3

Figure 4

Best Mode for Carrying Out the Invention

[0044] The concept of the present invention will now be explained more fully with reference to the accompanying drawings, which show presently preferred embodiments of the concept of the invention. However, the concept of the invention may be implemented in many different forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided for thoroughness and completeness and to fully convey the scope of the concept of the invention to those skilled in the art.

[0045] FIG. 1 shows a microscope system 10. The microscope system 10 includes an illumination system 100, an image sensor 110, a microscope objective lens 120, and a circuit 130. As shown in the example of FIG. 1, the microscope system 10 may further include one or more of a sample holder 180 and a relay lens 190. The circuit 130 is configured to execute a first acquisition function 1402, a first inference function 1404, a second acquisition function 1406, a second inference function 1408, and a processing function 1410. As shown in the example of FIG. 1, the circuit 130 may be further configured to execute a focusing function 1412.

[0046] In the example of FIG. 1, the sample holder 180 includes a microscope slide on which a sample 182 is applied. It should be understood that the sample 182 may be covered with a coverslip (not shown in FIG. 1). The sample holder 180 may be configured to hold the sample 182. The sample holder 180 may be movable (e.g., by being coupled to a manual and / or motorized stage), thereby allowing the sample 182 to be moved so that various portions of the sample 182 can be imaged by the microscope objective lens 120. The sample holder 180 may be movable within a plane parallel to the major surface of the sample holder 180. In the example of FIG. 1, the sample holder 180 is movable within a plane parallel to the first axis X and the second axis Y. The sample holder 180 may be movable along a horizontal plane. Further, it should be understood that the sample holder 180 may be movable along a third axis Z. Thereby, the sample 182 can be moved relative to the microscope objective lens 120 so as to be in focus on the sample 182.

[0047] In FIG. 1, circuit 130 is shown as a separate entity, but it should be understood that circuit 130 may form part of a computer device. For example, the computer device may be a computer, a server (e.g., a local server and / or a cloud server), a smartphone, etc. It should be further understood that the functions of circuit 130 may be distributed among multiple computer devices. Although not explicitly shown in FIG. 1, the computer device may also include other components, such as an input device (e.g., a mouse, a keyboard, a touch screen, etc.) and / or a display. As shown in the example of FIG. 1, circuit 100 may include one or more of memory 140, processing device 150, transceiver 160, and data bus 170. Memory 140, processing device 150, and transceiver 160 may communicate (e.g., exchange data) via data bus 170. Processing device 150 may include a central processing unit (CPU) and / or a graphics processing unit (GPU). Transceiver 160 may be configured to communicate with external devices. For example, transceiver 160 may be configured to communicate with a server, a computer, an external peripheral device (e.g., an external storage device), etc. The external device may be a local device or a remote device (e.g., a cloud server). Transceiver 160 may be configured to communicate with external devices via an external network (e.g., a local area network, the Internet, etc.). Transceiver 160 may be configured to perform wireless communication and / or wired communication. Techniques suitable for wireless communication are known to those skilled in the art. Some non-limiting examples include Wi-Fi (registered trademark) and near field communication (NFC). Techniques suitable for wired communication are known to those skilled in the art. Non-limiting examples include USB, Ethernet (registered trademark), and Firewire (registered trademark).

[0048] Memory 140 may be a non - transitory computer - readable storage medium. Memory 140 may be a random - access memory. Memory 140 may be a non - volatile memory. As shown in the example of FIG. 1, memory 140 may store program code portions 1402, 1404, 1406, 1408, 1410, 1412 corresponding to one or more functions. The program code portions 1402, 1404, 1406, 1408, 1410, 1412 may be executable by a processing device 150 that performs the functions. Thus, when it is mentioned that circuit 130 is configured to perform a particular function, the processing device 150 may execute the program code portions 1402, 1404, 1406, 1408, 1410, 1412 corresponding to the particular function stored in memory 140. However, it should be understood that one or more functions of circuit 130 may be implemented hardware - wise and / or implemented in a specific integrated circuit. For example, one or more functions may be implemented using a field - programmable gate array (FPGA). Thus, one or more functions of circuit 130 may be implemented in hardware, software, or as a combination of the two.

[0049] Image sensor 110 is configured to capture a digital image of sample 182. Although image sensor 110 is shown as separate in FIG. 1, it should be understood that image sensor 110 may form part of a camera. Image sensor 110 may be a charge - coupled device (CCD) sensor or a complementary metal - oxide - semiconductor (CMOS) sensor. As shown in the example of FIG. 1, image sensor 110 may communicate with circuit 130 via transceiver 160. However, it should be understood that image sensor 110 may communicate with circuit 130 via data bus 170.

[0050] The microscope objective lens 120 is configured to form an image of the sample 182 on the image sensor 110. In other words, the microscope objective lens 120 may be arranged such that the object plane of the microscope objective lens 120 coincides with the sample 182 and the image plane of the microscope objective lens 120 coincides with the image sensor 110. It should be understood that whether an image depicting the sample is formed on the image sensor 110 may depend on the illumination pattern used. For example, when the illumination system 100 is configured to illuminate the sample 182 using an illumination pattern similar to conventional microscope illumination (e.g., bright-field illumination), an image similar to the sample 182 can be formed on the image sensor 110. An illumination pattern similar to conventional microscope illumination may be formed by using most (or all) of the light sources of the illumination system 100. For example, the light source of the illumination device 100 that emits light in a direction corresponding to an angle smaller than (and in some cases equal to) the numerical aperture 122 of the microscope objective lens 120 may be used to form an illumination pattern similar to conventional microscope illumination. However, when forming the illumination pattern by using only the light sources that emit light in a direction corresponding to an angle larger than the numerical aperture 122 of the microscope objective lens 120, the image formed on the image sensor 110 may not depict the sample 182. However, those images may contain information related to the sample 182, and that information may be used by one or more of the first inference function 1404, the second inference function 1408, and the image processing function 1410.

[0051] The numerical aperture of the microscope objective lens 120 may be 0.4 or less. In other words, the magnification of the microscope objective lens 120 may be 20× or less. Therefore, the digital image captured using the microscope objective lens 120 can include information related to a larger portion of the sample 182 as compared to a digital image captured using a microscope objective lens having a relatively high numerical aperture. Thereby, the number of individual imaging positions necessary to collect information related to most (or all) of the sample 182 can be reduced. Therefore, the time necessary to collect information on most of the sample 182 can be shortened.

[0052] It should be understood that the microscope system 10 may be provided with an optical system that may be used with the microscope objective lens 120 and the image sensor 110 when capturing a digital image of the sample 180. For example, as shown in the example of FIG. 1, the microscope system 10 may include a relay lens 190 arranged such that the sample 182 is imaged on the image sensor 110 by the microscope objective lens 120 and the relay lens 190. In other words, the microscope objective lens 120 and the relay lens 190 may form an optical system, and the object plane of the optical system may coincide with the sample 182 and the image plane of the optical system may coincide with the image sensor 110. It should be further understood that the relay lens 190 may be selected according to the magnification and / or numerical aperture 122 of the microscope objective lens 120 (e.g., focal length, material, size, etc.). The microscope objective lens 120 may be movable in a third direction Z by being coupled to a manual stage and / or a motorized stage. The third direction Z may be parallel to the optical axis of the microscope system 10. In other words, the microscope objective lens 120 may be movable in the focusing direction of the microscope system 10. The microscope objective lens 120 and / or the sample holder 180 may be movable such that a focused image of the sample 182 can be captured by the image sensor 110 (assuming that the illumination system 260 is configured for bright-field illumination). The position of the microscope objective lens 120 along the third direction Z may be controlled by the circuit 130. For example, the circuit 200 may be configured to execute a focusing function 1412 configured to adjust the position of the microscope objective lens 120 along the third direction Z. The focusing function 1412 may be configured to automatically adjust the position of the microscope objective lens 120 along the third direction Z. In other words, the focusing function 1412 may be an autofocus function. The focusing function 1412 may control the position of the microscope objective lens 120 along the third direction Z by communicating through the transceiver 160 as in the example of FIG. 1. However, it should be understood that the focusing function 1412 may control its position by communicating through the data bus 170.

[0053] The illumination system 100 is configured to illuminate the sample 182 with a plurality of illumination patterns. As shown in FIG. 1, the illumination system 100 may include a plurality of light sources 102. The light source may be a light emitting diode (LED). The light source may be a laser. The light source may emit incoherent light, quasi-coherent light or coherent light. The light source may be configured to emit monochromatic light or polychromatic light. The light source may be configured to emit light of a single color (e.g., red, green or blue). The light source may be configured to emit light of various colors (e.g., red, green or blue). The light source may be configured to emit white (or pseudo-white) light. Each illumination pattern of the plurality of illumination patterns may be formed by one or more of the plurality of light sources. In other words, each illumination pattern of the plurality of illumination patterns may be formed by emitting light from one or more of the plurality of light sources 102. Thus, the sample 182 may be illuminated under various illumination conditions (i.e., by using various illumination patterns), and more information about the sample 182 (e.g., information related to the refractive index) may be collected by capturing a digital image of the sample 182 when the sample 182 is illuminated under various illumination conditions. The illumination pattern may be formed by 10 or fewer light sources. Each illumination pattern of the plurality of illumination patterns may be formed by 10 or fewer light sources. The illumination pattern may be formed by 5 or fewer light sources. Each illumination pattern of the plurality of illumination patterns may be formed by 5 or fewer light sources. The illumination pattern may be formed by one light source. Each illumination pattern of the plurality of illumination patterns may be formed by one light source. Each of the plurality of light sources 102 may be configured to illuminate the sample 182 from one of the plurality of directions 104. Thus, the illumination pattern may be formed by emitting light from one or more of the plurality of directions 104. In other words, the illumination device 100 may be configured to simultaneously illuminate the sample 182 from one or more of the plurality of directions 104. When the illumination pattern is formed by a plurality of light sources, the light sources may correspond to non-overlapping portions of the Fourier space related to the sample 182.

[0054] At least one direction 104A of the plurality of directions 104 may correspond to an angle greater than the numerical aperture 122 of the microscope objective lens 120 configured to image the sample onto the image sensor 110. The numerical aperture 122 of the microscope objective lens 120 may be a dimensionless number related to the range of angles at which the microscope objective lens 120 accepts light. Thus, a direction greater than the numerical aperture 122 can be understood as a direction corresponding to an angle greater than the range of angles at which the microscope objective lens accepts light. By illuminating the sample 182 from a direction corresponding to an angle greater than the numerical aperture 122 of the microscope objective lens 120, the digital image captured for that angle may contain information about higher spatial frequencies of the sample 182 than what the microscope objective lens 120 normally allows (e.g., using conventional microscope illumination), and thus, information about even finer details. Thereby, the microscope objective lens 120 can capture phase information related to the sample 182 and / or information related to details of the sample 182 that are not normally resolvable by the microscope objective lens 120. Such information may be used by one or more of a first set of machine learning models, a second set of machine learning models, and an image processing machine learning model during the processing of the digital image. In other words, by illuminating the sample 182 from the direction 104A corresponding to an angle greater than the numerical aperture 122 of the microscope objective lens 120, a digital image can be obtained from which the machine learning model can extract more information related to the sample 182 than what is normally allowed (e.g., when using conventional microscope illumination). Thereby, for example, the image processing machine learning model can construct a digital image of the sample 182 having a relatively high resolution (or magnification) compared to what is normally allowed by the microscope objective lens 120 using conventional microscope illumination (e.g., brightfield illumination).

[0055] As further shown in FIG. 1, a plurality of light sources 102 may be arranged on a curved surface 106 that is concave along at least one direction. As shown in the example of FIG. 1, the curved surface 106 may be concave along at least one direction (e.g., the second direction Y). For example, the curved surface 106 may be a cylindrical surface. The curved surface 106 may be concave along two perpendicular directions (e.g., the first direction X and the second direction Y). For example, the curved surface 106 may have a shape similar to a spherical segment. The segment of the sphere may be a spherical cap or a spherical dome. Arranging a plurality of light sources 102 on the curved surface 106 is advantageous in that the distance R from each light source to the current imaging position P of the microscope system 10 is the same. Since this distance is the same, the intensity of the light emitted from each light source and reaching the current imaging position P can be made the same. This can be understood as the effect of the inverse square law. Therefore, the sample 182 can be illuminated with light having a similar intensity for each of the plurality of directions 104. The distance R from each light source to the current imaging position P can be in the range of 4 cm to 15 cm. It may be advantageous to configure the illumination system 100 such that the distance R from each light source to the current imaging position P is sufficiently large so that each light source can be regarded as a point source. This may enable the light to be quasi-coherent at the current imaging position P. Considering that the intensity of the light from at least most of the plurality of light sources 102 at the current imaging position P is high enough to generate a set of digital images, the distance R from each light source to the current imaging position P may be greater than 15 cm. In particular, when one or more of the plurality of light sources 102 are lasers, the distance R between each light source may be greater than 15 cm. However, it should be understood that the plurality of light sources 102 may be arranged on a flat surface or a surface having an irregular shape. It should be further understood that FIG. 1 shows a cross-section of the microscope system 10, particularly the illumination system 100. Therefore, the curved surface 106 of the illumination device 100 shown in FIG. 1 may be a part of a cylindrical surface or a spherical surface (or a quasi-spherical surface). The curved surface 106 of the illumination device 100 may be bowl-shaped.

[0056] The curved surface 106 may be formed by the facets 106A shown in the example of FIG. 2. In other words, the curved surface 106 may be composed of a plurality of flat surfaces. Therefore, the curved surface 106 may be partially flat. The curved surface 106 may be part of a quasi-spherical surface having a plurality of facets or segments. Therefore, the curved surface 106 may be part of the surface of a polyhedron. An example of such a polyhedron may be a truncated regular icosahedron. The plurality of light sources 102 may be arranged on the facets 106A. Each light source may be arranged so as to emit light in a direction substantially parallel to the normal of the facet to which the light source is associated. It should be understood that FIG. 2 shows a cross-section of the illumination system 100 as in the example shown in FIG. 1. Therefore, the curved surface 106 of the illumination system 100 shown in FIG. 2 may be part of a quasi-cylindrical surface or a quasi-spherical surface. The curved surface 106 of the illumination system 100 in FIG. 2 may have a shape similar to a bowl. Therefore, although the facets 106A in FIG. 2 are shown as lines, it should be understood that each facet 106A may be a flat surface having at least three sides. For example, the curved surface 106 may be formed by facets having five sides (e.g., similar to the inner surface of a soccer ball) and facets having six sides. Even if the curved surface 106 in FIG. 2 is shown as a continuous surface, it should be understood that each facet 106A may be separated from other facets. Therefore, the curved surface 106 may be formed by a plurality of parts, and each facet 106A may be formed by at least one of those parts. Furthermore, such parts may be arranged in contact with adjacent parts or at a distance from adjacent parts. It should be further understood that the number of facets 106A of the illumination system 100 shown in FIG. 2 is an example and that other numbers of facets 106A may be used to form the curved surface 106 of the illumination system 100. It should be further understood that the number of light sources on each facet 106A is merely illustrative and that the number may be changed.

[0057] The first acquisition function 1402 is configured to control the illumination system 100 to illuminate the sample 182 with a first subset of a plurality of illumination patterns and to control the image sensor 110 to form a first input set of digital images by capturing a digital image of the sample 182 for each illumination pattern of the first subset of the plurality of illumination patterns.

[0058] The first inference function 1404 is configured to input the first input set of digital images into a first set of machine learning models configured to output a first inference output. The first set of machine learning models may be trained to output a first inference output.

[0059] The second acquisition function 1406 is configured to control the illumination system 100 to illuminate the sample 182 with a second subset of a plurality of illumination patterns and to control the image sensor 110 to form a second input set of digital images by capturing a digital image of the sample 182 for each illumination pattern of the second subset of the plurality of illumination patterns. The second subset of illumination patterns may be different from the first subset of illumination patterns. In this way, the first input set of digital images and the second input set of digital images may include even less redundant information. In other words, the first input set of digital images and the second input set of digital images may not include digital images captured under the same illumination conditions (i.e., using the same illumination pattern). The first acquisition function 1402 and the second acquisition function 1406 may be implemented as a single function. For example, such a function may be referred to as an acquisition function. The number of digital images in each of the first input set of digital images and the second input set of digital images may be 30 or more. The number of digital images in each of the first input set of digital images and the second input set of digital images may be 50 or less. It should be understood that these numbers are illustrative and the numbers may be even more and / or even less. For example, when the illumination pattern is formed by one or more of a plurality of light sources, the number of digital images required may be further reduced. As another example, when training an image processing machine learning model to construct a digital image having a relatively higher resolution than the images of the first input set of digital images and the second input set of digital images, the number of digital images may depend on the difference in resolution. In other words, the number of digital images in the first input set of digital images and the second input set of digital images may depend on the increase in the resolution of the constructed digital image compared to the resolution of the digital images in the first input set and the second input set.

[0060] Circuit 130 may be configured to execute at least partially in parallel the first inference function 1404 and the second acquisition function 1406. In other words, the inference of the first input set of the digital image may be executed at least partially in parallel with the acquisition of the second input set of the digital image. Preferably, the second acquisition function 1406 is executed as soon as the first acquisition function 1402 ends (i.e., as soon as the first input set of the digital image is acquired). Thus, the second acquisition function 1406 may be executed before the first inference function 1404.

[0061] The second inference function 1408 is configured to input the second input set of the digital image into a second set of machine learning models configured to output a second inference output. The second set of machine learning models may be trained to output a second inference output. The second set of machine learning models is different from the first set of machine learning models.

[0062] The processing function 1410 is configured to input the first inference output and the second inference output into an image processing machine learning model trained to process a plurality of digital images of the sample 182 using the first inference output and the second inference output. The first inference output and the second inference output can be regarded as intermediate outputs of the image processing executed by the circuit 130. The intermediate output may include compressed data. The structure of the compressed data may vary. For example, the structure of the compressed data may depend on the design of the machine learning models in the set of machine learning models (e.g., memory constraints, processing power constraints, layout, design, etc.). Further, the structure of the compressed data may depend on the output type of the image processing machine learning model. The intermediate output may include information used by the image processing machine learning model to output the image processing output. In other words, the first set and the second set of machine learning models may output intermediate outputs that may be used by the image processing machine learning model to form the image processing output. The first set and the second set of machine learning models may output information related to the characteristics of the sample 182. These characteristics are relayed to an image processing machine learning model that may combine the characteristics into the output. The image processing machine learning model may be trained to process a plurality of digital images using the first inference output and the second inference output by training one or more of the construction of the digital image of the sample 182, and / or the classification of the sample 182 into one or more classes, and / or the detection of one or more objects within the sample 182. When training the image processing machine learning model to construct a digital image of the sample, the constructed digital image of the sample can have a relatively higher resolution than the captured digital image by the microscope system using conventional microscope illumination (e.g., bright field conditions). In other words, the image processing machine learning model may be trained to construct a digital image of the sample that has a relatively higher resolution than the digital images of the first input set of digital images and / or the digital images of the second input set of digital images.

[0063] By illuminating the sample 182 with a plurality of different illumination patterns and capturing a digital image for each illumination pattern of the plurality of illumination patterns, more information related to the sample 182 can be captured than is normally possible with a conventional microscope used to image the sample (e.g., by using conventional microscope illumination). This can be understood as capturing information from different parts of the Fourier space (i.e., the spatial frequency domain) related to the sample for different illumination patterns (e.g., different illumination directions). This technique is known in the art as Fourier ptychography. Further, by illuminating the sample 182 with a plurality of different illumination patterns and capturing a digital image for each of the plurality of illumination patterns, information regarding the refractive index related to the sample 182 may be captured. This can be understood as an effect where the refraction of light depends on the angle of incidence of the light illuminating the sample 182 and the refractive index of the sample 182. With the information regarding the refractive index of the sample 182, it may be possible to determine phase information related to the sample 182 (which is typically referred to as quantitative phase in the art). Since the plurality of digital images contain information related to one or more of the fine details of the sample 182, the refractive index related to the sample 182, and the phase information related to the sample 182, this information may be used by one or more of a first set of machine learning models, a second set of machine learning models, and an image processing machine learning model. Thereby, more information related to the sample 182 can be extracted from the plurality of digital images of the sample 182 than is allowed when capturing the plurality of digital images from only one direction or when using conventional microscopy. It may be difficult or impossible to capture information related to the refractive index related to the sample 182 and / or phase information related to the sample 182 by using conventional microscopy (e.g., by simultaneously illuminating the sample 182 from most of a plurality of directions up to the numerical aperture 122 of the microscope objective lens 120 used to image the sample 182 in FIG. 1).In particular, it may not be possible to depict the sample 182 and construct a digital image having a resolution (or magnification) that is relatively high compared to the resolution (or magnification) normally allowed by the microscope objective lens 120.

[0064] The microscope system 10 of FIG. 1 optionally includes the illumination system 100 shown in FIG. 2, which can reduce the period required for processing a plurality of digital images of the sample 182. In particular, since the process of acquiring digital images and the inference of the acquired digital images are time-consuming, it can be advantageous to divide the acquisition of digital images of the sample 182 (e.g., by the first acquisition function 1402, the second acquisition function 1406, the first inference function 1404, and the second inference function 408) into at least two instances. This concept is schematically shown in FIG. 3. Here, a first input set of digital images 300A is input to a first set of machine learning models 302A, 304A that form a first inference output 306A. The first set of machine learning models 302A, 304A is represented in FIG. 3 by the first machine learning model 302A and the second machine learning model 304A. However, it should be understood that this is merely an example, and the number of machine learning models in the first set of machine learning models 302A, 304A may be changed. After the acquisition of the first input set of digital images 300A, a second input set of digital images 300B is acquired. As described above, the acquisition of the second input set 300B of digital images may be performed at least partially in parallel with the processing of the first input set 300A of digital images by the first set of machine learning models 302A, 304A. After the acquisition of the second input set 300B of digital images, the second input set 300B of digital images is input to a second set of machine learning models 302B, 304B that form a second inference output 306B. The second set of machine learning models 302B, 304B is represented in FIG. 3 by the third machine learning model 302B and the fourth machine learning model 304B. However, it should be understood that this is merely an example, and the number of machine learning models in the second set of machine learning models 302B, 304B may be changed. After forming the first inference output 306A and the second inference output 306B, they are input to the image processing machine learning model 308. The first inference output 306A and the second inference output 306B may be input to the image processing machine learning model 308 simultaneously or sequentially.When sequentially inputting the first inference output 306A and the second inference output 306B, the image processing machine learning model 308 may be configured to start image processing as soon as it receives the first inference output 306A. In other words, the image processing machine learning model 308 may be configured to start forming the image processing output 310 as soon as it receives the second inference output 306B. This may enable further shortening of the time required for processing the digital image of the sample. As described above, the image processing output 310 from the image processing machine learning model may include one or more of the constructed digital image of the sample (i.e., the digital image depicting the sample), the classification of the sample into one or more classes, and the detection of one or more objects within the sample. The image processing output 310 may be used for analyzing the sample. Therefore, by using the microscope system 10 of FIG. 1, the time required for analyzing the sample can be shortened.

[0065] FIG. 4 is a block scheme of a method 40 for processing a plurality of digital images of a sample. The method 40 may be a computer-executed method. The method 40 includes: a. illuminating the sample with a first subset of a plurality of illumination patterns by an illumination system (S402), the illumination system comprising a plurality of light sources, each light source of the plurality of light sources being configured to illuminate the sample from one of a plurality of directions, each illumination pattern of the plurality of illumination patterns being formed by one or more of the plurality of light sources, and obtaining a first input set of digital images by capturing a digital image of the sample for each illumination pattern of the first subset of the plurality of illumination patterns (S404). The illumination system may comprise a plurality of light sources, and each illumination pattern of the plurality of illumination patterns may be formed by one or more of the plurality of light sources. Each light source of the plurality of light sources may be configured to illuminate the sample from one of a plurality of directions. At least one of the plurality of directions may correspond to an angle greater than the numerical aperture of a microscope objective lens used to image the sample. The method 40 may further comprise: b. inputting the first input set of digital images into a first set of machine learning models configured to output a first inference output (S406). The method 40 may further comprise: c. illuminating the sample with a second subset of a plurality of illumination patterns by the illumination system (S410), and obtaining a second input set of digital images by forming a second input set of digital images by capturing a digital image of the sample for each illumination pattern of the second subset of the plurality of illumination patterns (S412). Steps b and c may be performed at least partially in parallel. In other words, the operation of inputting the first input set of digital images into the first set of machine learning models S406 and the operation of obtaining the second input set of digital images S408 may be performed at least partially in parallel. The second subset of illumination patterns may be different from the first subset of illumination patterns.Put another way, one or more lighting patterns of the first subset of lighting patterns may be different from one or more lighting patterns of the second subset of lighting patterns. All lighting patterns of the first subset of lighting patterns may be different from all lighting patterns of the second subset of lighting patterns. Method 40 further comprises inputting (S414) a second input set of digital images to a second set of machine learning models configured to output a second inference output. The second set of machine learning models is different from the first set of machine learning models. Method 40 further comprises inputting (S416) the first inference output and the second inference output to an image processing machine learning model trained to process a plurality of digital images of the sample using the first inference output and the second inference output. The image processing machine learning model may be trained to process a plurality of digital images using the first inference output and the second inference output by training the image processing machine learning model for one or more of construction of digital images of the sample, classification of the sample into one or more classes, and detection of one or more objects within the sample.

[0066] A person skilled in the art is aware of how to train a machine learning model in particular and / or how to use a trained machine learning model. However, to briefly explain, a machine learning model may be a type of supervised machine learning model, for example, it may be a network such as U-net or Pix2pix. The machine learning model may be a transformer-based network such as SwinIR. The machine learning model may be a convolutional neural network. The machine learning model may be trained to predict a desired output using exemplary input training data and ground truth, that is, the "correct" or "true" output. In other words, the ground truth may be used as the label of the input training data. The input training data may include data related to various results, whereby each input training data may be associated with the ground truth related to that particular input training data. Therefore, each input training data may be labeled with the associated ground truth (i.e., the "correct" or "true" output). The machine learning model may comprise a plurality of neuron layers, and each neuron may represent a mathematical operation applied to the input training data. Typically, the machine learning model comprises an input layer, one or more hidden layers, and an output layer. The first layer may also be called the input layer. The output of each layer (excluding the output layer) of the machine learning model is fed to the subsequent layer, and the subsequent layer generates a new output. The new output can be fed to a further subsequent layer. The output of the machine learning model may be the output of the output layer. This process may be repeated for all layers of the machine learning model. Typically, each layer further includes an activation function. The activation function can further define the output of the neurons in the layer. For example, the activation function may prevent the output from the layer from being too large or too small (e.g., from tending towards positive or negative infinity). Furthermore, the activation function may introduce non-linearity into the machine learning model. During the training process, the weights and / or biases associated with the neurons in the layer may be adjusted until the machine learning model generates a prediction that reflects the ground truth for the input training data.Each neuron may be configured to multiply the inputs to the neuron by weights associated with that neuron. Each neuron may be further configured to add to the inputs a bias associated with that neuron. Stated another way, the output from a neuron is the sum of the bias associated with the neuron and the product of the weights associated with the neuron and the inputs. The weights and biases may be adjusted by a recursive process and / or an iterative process. This is known in the art as backpropagation. A convolutional neural network may be a type of neural network comprising one or more layers representing a convolutional operation. In this context, the input learning data includes digital images. The digital images may be represented as a matrix (or as an array), and each element of the matrix (or array) may represent a corresponding pixel of the digital image. Thereby, the value of the element may represent the pixel value of the corresponding pixel of the digital image. Thus, the inputs and outputs of the machine learning model may be numerical values (e.g., a matrix or an array) representing digital images. In this context, the input is a set of digital images (i.e., a training set or an input set). Thus, the input to the machine learning model may be a plurality of matrices or a three-dimensional matrix. It should be understood that the machine learning model may receive another input during training.

[0067] In this specific example, a first set of machine learning models, a second set of machine learning models, and an image processing machine learning model are trained using a first training set of digital images, a second training set of digital images, and a ground truth. The first training set of digital images is obtained in a similar manner as obtaining the first input set of digital images, and the second training set of digital images is obtained in a similar manner as obtaining the second input set of digital images. In other words, the first training set of digital images may comprise digital images captured when illuminating the training samples with a first subset of illumination patterns, and the second training set of digital images may comprise digital images captured when illuminating the training samples with a second subset of illumination patterns. The ground truth may be modified according to the desired image processing output from the (trained) image processing machine learning model. For example, if a digital image depicting the sample is part of the desired output, the ground truth may include the digital image of the training sample. In particular, when the desired output is a digital image having a relatively higher resolution (or magnification) than the digital images of the first input set and / or the second input set of digital images, the ground truth may include a digital image depicting the training sample and having a relatively higher resolution (or magnification of the training sample) than the digital images of the first training set and / or the second training set of digital images. Further, when the desired output is the classification of the sample and / or the detected objects within the sample, the ground truth may include information related to the classification of the training sample and / or the objects within the training sample. Furthermore, the types of data constituting the first inference output and the second inference output may be modified according to the types of processes for which the image processing machine learning model is trained to perform. For example, the first inference output and the second inference output may be digital images (or the form prior to the digital images) that the image processing machine learning model may be trained to construct a digital image depicting the sample.As another example, the first inference output and the second inference output may be one or more of an array, a vector, and a matrix containing data that may be used for one or more of constructing a digital image of a sample by an image processing machine learning model, classifying the sample into one or more classes, and detecting one or more objects within the sample. In the training process, the first set of machine learning models, the second set of machine learning models, and the image processing machine learning model may be determined as a common machine learning model. Thus, the first inference output and the second inference output may be determined during the training of the common machine learning model. Thus, the common machine learning model may be trained using the first training set and the second training set of digital images as inputs and using the ground truth as the desired output. In other words, the common machine learning model may be trained until the difference between the output of the common machine learning model and the ground truth is smaller than a threshold. This difference is sometimes referred to in the art as a loss function. In some cases, it may be preferable to train the common machine learning model until the loss function is minimized. In other words, the common machine learning model may be trained until the difference between the output of the common machine learning model and the ground truth is minimized. The training process may be repeated for a plurality of training samples that are different from each other (for example, different training samples of the same type and / or different types from each other), whereby the first set of machine learning models, the second set of machine learning models, and the image processing machine learning model may be able to process a wider range of sample types and / or digital images with higher accuracy. Thus, the first set of machine learning models, the second set of machine learning models, and the image processing machine learning model may be trained as a single machine learning model. However, when processing the digital image of the sample to be analyzed, the first set of machine learning models, the second set of machine learning models, and the image processing machine learning model may be treated as separate processes, whereby the image processing may be made parallelizable. The structure of the first set of machine learning models (for example, the number of layers, activation functions, etc.) may be the same as the structure of the second set of machine learning models.However, in the training process, the weights associated with the structure (e.g., layers) of the first set of machine learning models (i.e., the parameters determined during the training process) may be different from the weights associated with the structure of the second set of machine learning models. This is the case when the first set and the second set of machine learning models are trained using different digital image sets.

[0068] Those skilled in the art will understand that the concept of the present invention is by no means limited to the preferred variations described above. On the contrary, many changes and modifications are possible within the scope of the appended claims.

[0069] For example, the concept of the present invention has been described as using a first set of machine learning models and a second set of machine learning models. However, it should be understood that this is merely an illustration and additional sets of machine learning models may be used. For example, in addition to the first set and the second set of machine learning models, a third input set of digital images may be input into a third set of machine learning models. In such a case, the third input set of digital images may be obtained at least partially in parallel with the second set of machine learning models that form the second inference output.

[0070] As another example, although not explicitly shown in FIG. 1, it is assumed that at least a circuit and an image sensor are provided in the camera.

[0071] Furthermore, variations to the disclosed variations can be understood and effective by those skilled in the art who practice the invention described in the claims from the study of the drawings, the disclosure, and the appended claims.

Claims

**Claim 1** A method (40) for processing a plurality of digital images of a sample, comprising: a. illuminating the sample (S402) with a first subset of a plurality of illumination patterns by an illumination system, the illumination system comprising a plurality of light sources, each light source of the plurality of light sources being configured to illuminate the sample from one of a plurality of directions, and each illumination pattern of the plurality of illumination patterns being formed by one or more of the plurality of light sources; obtaining a first input set of digital images (S400) by forming a first input set of digital images by capturing (S404) a digital image of the sample for each illumination pattern of the first subset of the plurality of illumination patterns; b. inputting the first input set of digital images into a first set of machine learning models configured to output a first inference output (S406); c. illuminating the sample (S410) with a second subset of the plurality of illumination patterns by the illumination system; obtaining a second input set of digital images (S408) by forming a second input set of digital images by capturing (S412) the digital image of the sample for each illumination pattern of the second subset of the plurality of illumination patterns; d. inputting the second input set of digital images into a second set of machine learning models configured to output a second inference output (S414), the second set of machine learning models being different from the first set of machine learning models; e. inputting the first inference output and the second inference output into an image processing machine learning model trained to process a plurality of digital images of the sample using the first inference output and the second inference output (S416); and performing steps b (S406) and c (S408) at least partially in parallel. A method (40). **Claim 2** The method (40) according to claim 1, wherein at least one of the plurality of directions corresponds to an angle greater than the numerical aperture of a microscope objective lens used for imaging the sample. **Claim 3** The method (40) according to claim 1 or 2, wherein the first subset of the illumination patterns is different from the second subset of the illumination patterns. **Claim 4** The image processing machine learning model is Constructing a digital image of a sample, classifying the sample into one or more classes, and training one or more of detecting one or more objects within the sample, using the first inference output and the second inference output to process the plurality of digital images, the method (40) according to any one of claims 1 to 3.

5. An illumination system (100) configured to illuminate a sample (182) with a plurality of illumination patterns, comprising a plurality of light sources (102), each light source (102A) of the plurality of light sources (102) being configured to illuminate the sample from one direction (104A) of a plurality of directions (104), each illumination pattern of the plurality of illumination patterns being formed by one or more light sources (102A) of the plurality of light sources (102), the illumination system (100), An image sensor (110) configured to capture a digital image of the sample (182); A microscope objective lens (120) configured to image the sample (182) onto the image sensor (110); A first acquisition function (1402) configured to control the illumination system (100) to illuminate the sample (182) with a first subset of the plurality of illumination patterns and to control the image sensor (110) to capture a digital image of the sample (182) for each illumination pattern of the first subset of the plurality of illumination patterns, thereby forming a first input set of digital images; A first inference function (1404) configured to input the first input set of digital images into a first set of machine learning models configured to output a first inference output; A second acquisition function (1406) configured to control the illumination system (100) to illuminate the sample (182) with a second subset of the plurality of illumination patterns and to control the image sensor (110) to capture the digital image of the sample (182) for each illumination pattern of the second subset of the plurality of illumination patterns, thereby forming a second input set of the digital images; A second inference function (1408) configured to input the second input set of the digital images into a second set of machine learning models configured to output a second inference output, wherein the second set of the machine learning models is different from the first set of the machine learning models, the second inference function (1408), and A processing function (1410) configured to input the first inference output and the second inference output into an image processing machine learning model trained to process a plurality of digital images of a sample using the first inference output and the second inference output; and A circuit (130) configured to execute; Comprising The circuit (130) is a microscope system (10) configured to execute the first inference function (1404) and the second acquisition function (1406) at least partially in parallel.

6. The microscope system (10) according to claim 5, wherein at least one direction (104A) of the plurality of directions (104) corresponds to an angle larger than the numerical aperture (122) of a microscope objective lens (120) configured to form an image of a sample on the image sensor (110).

7. The microscope system (10) according to claim 5 or 6, wherein the plurality of light sources (102) are arranged on a curved surface (106) recessed along at least one direction.

8. The microscope system (10) according to claim 7, wherein the curved surface (106) is formed of facets (106A).

9. The numerical aperture (122) of the microscope objective lens (120) is 0.4 or less. The microscope system (10) according to any one of claims 5 to 8.

10. The image processing machine learning model is Construction of a digital image of a sample, and / or Classification of a sample into one or more classes, and / or The microscope system (10) according to any one of claims 5 to 9, trained to process the plurality of digital images using the first inference output and the second inference output by training one or more of detection of one or more objects within the sample.

Images