Improved bundle-based scanning

Abstract

The present invention provides, inter alia, a method of beam-based scanning of a sample (1) comprising a heterogeneity, comprising providing said sample (1) comprising said heterogeneity; irradiating the sample (1) by means of a beam (2); performing, based on an interaction of the beam (2) and the sample (1 ), a first spot measurement (5, 6); changing, by means of changing means (3, 4), a constellation of said sample (1 ) and said beam (2); performing a second spot measurement (5, 6), based on an interaction of the beam (2) and the sample (1 ) under the modified constellation; calculating, based on said first and second spot measurements (5, 6), a result of said scanning; wherein said calculating is at least partly done, based on a trained machine learning algorithm; wherein the training of said machine learning algorithm is related to distinguishing a foreground (8) and a background (9) which are characteristic of said heterogeneity.

Description

[0001] IMPROVED BUNDLE-BASED SCANNING

[0002] TECHNICAL FIELD

[0003] The invention is related to a method, device and system for scanning samples using an electromagnetic beam or ion beam originating from, for example, a light source or an ion source, more in particular beam-based scanning of samples comprising an heterogeneity.

[0004] STATE OF THE ART

[0005] The present invention is related to spectroscopic imaging of samples, wherein samples are scanned. Such scanning plays an important role in a wide range of applications, including Raman spectroscopy and mass spectrometry. The scanning process can be implemented in various ways, and is usually at least partly beambased. In this process, the sample is irradiated with an electromagnetic beam and / or an ion beam. For Raman spectroscopy, for example, this involves an electromagnetic light beam, with a laser as the light source. In mass spectroscopy, both laser-based and ion beam-based scanning are common. An important variant of the scanning process consists of irradiating the sample with a point source, in order to measure the spectra at each point of the irradiated area, based on spot measurements. This is to be distinguished from a scanning process according to the “ wide-field imaging “ principle, wherein spectra are collected over the entire area comprising the sample, e.g., the entire substrate on which the sample is present, and the measurement is done integrally, typically with tuneable filters, such as an acousto-optic tuneable filter (AOTF) or a liquid crystal tuneable filter (LCTF).

[0006] Scanning based on spot measurements presents several problems, wherein the characteristics of the sample can be of decisive importance. For a completely homogeneous sample it is clear that each spot measurement can provide a valid measurement for the entire sample. In many practical applications, however, samples exhibit a form of heterogeneity. In practice, a single spot measurement can therefore be little or not representative of the characteristics of the entire sample, which means that multiple spot measurements are necessary. US8988678, US8547540, US11397151 and W02020102442 disclose related concepts, but are not or insufficiently suitable for scanning samples containing an heterogeneity.

[0007] There is a need for methods, devices and systems for scanning samples that better deal with heterogeneity present in the sample. The invention aims to solve the above-mentioned problems.

[0008] SUMMARY OF THE INVENTION

[0009] In a first aspect, the invention provides a method for beam-based scanning of a sample comprising a heterogeneity, comprising providing said sample comprising said heterogeneity; irradiating the sample by means of a beam, preferably a laser beam, preferably using a lens; performing, based on an interaction of the beam and the sample, a first spot measurement (5, 6); changing, by means of changing means, a constellation of said sample and said beam; performing, based on an interaction of the beam and the sample under the modified constellation, a second spot measurement; calculating, based on said first and second spot measurements, a result of said scanning; wherein said calculating is at least partly done, based on a trained machine learning algorithm; wherein the training of said machine learning algorithm is related to distinguishing a foreground and a background which are characteristic for said heterogeneity.

[0010] Such a method allows to deal advantageously with the heterogeneity of a sample. Inherently, the heterogeneity is due to a suboptimal alignment of the beam and the sample, causing some spot measurements to end up next to the sample, and other spot measurements to only partially overlap with the sample. If the focal spot misses the target, only background is measured, and if it hits the target but the focal spot is too large, a spectrum is measured that contains a background signal. The method meets this requirement by distinguishing foreground and background using a trained machine learning algorithm, i.e. an artificial intelligence (Al) algorithm. The method thus allows to still perform an accurate scan of the sample using simple changing means, such as an actuation of a lens or a mirror or of the sample, and simple implementations of sensors. In exemplary embodiments, the actuation is based on the movement of a mirror, and / or actuation according to a curve. Such a curve can, for example, be compared with the orbit involved in the orbital raster scan, described in US8988678, although US8988678 does not concern spot measurements, but averaged measurements. An approach as described in US8988678, which results in an averaged spot measurement, is suitable for homogeneous samples, but not for samples that contain an heterogeneity. This heterogeneity can be related to, for example, the relatively small dimension of the sample, compared to the spot size, and / or the curve, which means that some spot measurements do not concern the sample (which is related to the foreground), but only the surrounding area (which is related to the background). Heterogeneity can also be related to the issue of whether the focal point is on or in front of or behind the sample. The invention addresses this issue by distinguishing between foreground and background using a trained machine learning algorithm.

[0011] Distinguishing foreground from background allows for advantageous forms of scanning, which are different from scanning based on averaging, as described, for example, in US8988678. In exemplary embodiments of the invention, a plurality of spot measurements is performed, and the method comprises a triage step, wherein at least one spot measurement is deleted and not used for calculating the result. Here, the deletion of respective spot measurements is preferably triggered by detecting, by means of said trained machine learning algorithm, a background and / or a lack of sufficient foreground for the respective spot measurement. In this way, a more accurate and / or more robust scanning is obtained. In related embodiments, at least one spot measurement is assigned a lower or higher weight according to an algorithmic weighting, wherein the weight is higher as the spot measurement is considered more dependable and / or more relevant.

[0012] Another advantage of the invention is that it enables high-quality classification of measured spectra (i.e., spot measurements) by means of the trained machine learning algorithm. As a result, the invention offers an alternative to conventional methods. These are often based on generic metrics such as signal-to-noise ratio (SNR), which are, however, not very applicable to samples with heterogeneity, since background spectra are often difficult to characterize by such a simple metric. The machine learning algorithm can be related to forms of template matching or forms of deep learning, such as those, based on neural networks.

[0013] In preferred embodiments, the first and second spot measurements are located in a limited zone, such as is the case with a curve according to said orbital raster scan, but then with spot measurements instead of continuous measurement as is the case with US8988678. In exemplary embodiments, more than two spot measurements are made, for example twenty, thirty or fifty spot measurements, and this preferably in a short period of time, for example less than five minutes or less than one minute. The method allows such a plurality of spot measurements to be processed accurately, by making a distinction between foreground and background for each spot measurement.

[0014] In a second aspect, the invention is related to a device for laser-based scanning of a sample comprising an heterogeneity, comprising a beam source, preferably a laser, for generating a beam, preferably a laser beam; preferably, a lens (7) for focusing said beam, preferably a laser beam; a sensor for measuring an interaction of the sample and the beam; changing means for changing a constellation of the sample and the beam; a memory comprising a trained machine learning algorithm; a processor, at least connected to said sensor, said means for actuation and said memory; wherein said processor is configured for: irradiating the sample comprising said heterogeneity with the beam; performing, by means of said sensor, a first spot measurement; changing, by means of the changing means, the constellation of said sample and said beam; performing a second spot measurement using said sensor; calculating, based on said first and second spot measurements, a result of said scanning; wherein said calculating is at least partly done, based on said trained machine learning algorithm; wherein the training of said machine learning algorithm is related to distinguishing a foreground and a background which are characteristic for said heterogeneity.

[0015] In a third aspect, the invention provides a kit, comprising said device and a plurality of substrates with known nanostructure, wherein said training of said machine learning algorithm has been done with said substrate with known nanostructure as background. This may have the advantage that the trained machine learning algorithm is optimally tuned to detect background which corresponds to the substrate.

[0016] Embodiments according to the dependent claims and their respective possible advantages are explained in the detailed description.

[0017] DESCRIPTION OF THE FIGURES

[0018] Figure 1 is a sketch of an exemplary embodiment with actuation according to a curve in the plane of the sample.

[0019] Figure 2 shows a first, second and third phase (Fig. 2A, 2B and 2C respectively) of an exemplary embodiment with actuation wherein the distance between lens and sample is changed.

[0020] Figure 3 shows a first, second and third aspect (Fig. 3A, 3B and 3C respectively) of an exemplary embodiment with actuation with intermediate result.

[0021] DETAILED DESCRIPTION

[0022] Unless otherwise defined, all terms used in the description of the invention, including technical and scientific terms, have the meaning as commonly understood by one skilled in the technical field of the invention. For a better appreciation of the description of the invention, the following terms are explicitly explained. “ A, “ “ an, “ and “ the “ refer in this document to both the singular and the plural unless the context clearly suggests otherwise. For example, “ a lens “ means one or more than one lens. Quoting numerical intervals through the endpoints includes all integers, fractions, and / or real numbers between the endpoints, including these endpoints. In this document, the term “ heterogeneity “ refers to a characteristic of a sample and the zone surrounding the sample, which is related to a foreground and a background, wherein the foreground is related to a region of interest. The background, on the other hand, can be related to very diverse aspects, such as the substrate, the subsurface (zone surrounding the substrate), and possible nanoparticles. In contrast to a completely homogeneous sample (wherein each portion of the sample is a potential region of interest), a sample with heterogeneity exhibits at least a first zone and an adjacent second zone, wherein the visual characteristics of the first zone are measurably different from the first zone. The terms “ first zone “ and “ second zone “ may refer to non-overlapping sections of the (two-dimensionally projected) upper surface of the sample, but they may equally well refer to depth-related zones, such as superimposed layers at different depths and both externally observable, for example, as with samples that are porous and / or transparent and / or semi-transparent and / or liquid and / or partially solid. In particular, the partially solid nature of the sample may lead to an increased degree of heterogeneity. Likewise, heterogeneity may involve only the first zone belonging to the sample, and the second zone being adjacent to the sample, for example wherein the second zone extends over the substrate. Heterogeneity is therefore also decisive for the issue of whether the focal point is on, in front of or behind the sample. In embodiments, the heterogeneity is at least partly or even exclusively related to the relatively small dimension of the sample relative to the spot size, and / or the relatively small dimension of the sample relative to the curve or circle described by redirection of the laser beam. In exemplary embodiments with such a curve and with fixed focal point size (as in Example 1), the ratio of the maximum dimension of the sample and the maximum diameter of the curve is less than 3:1 and / or less than 2:1 and / or less than 3:2 and / or less than 4:3 and / or less than 1 and / or less than 3:4 and / or less than 2:3 and / or less than 1 :2. In exemplary embodiments with variable focal point size (as in Example 2), the ratio of the maximum sample dimension to the maximum focal point diameter is less than 3:1 and / or less than 2:1 and / or less than 3:2 and / or less than 4:3 and / or less than 1 and / or less than 3:4 and / or less than 2:3 and / or less than 1 :2.

[0023] In this document, the terms “ beam “ and “ ray “ are interchangeable. Terms such as “ laser beam “ and “ ion beam “ are therefore interchangeable with “ laser ray “ and “ ion ray “, respectively. The term “ beam “ is a collective term for any form of beam or ray, regardless of whether it is referred to as “ ray “ or “ beam “.

[0024] In this document, beam-based scanning refers to scanning a sample using a beam of electromagnetic waves and / or electrons and / or ions and / or a mass-containing particle different from an electron or ion. In embodiments, it involves irradiation by means of a beam of light, for example light comprising visible light and / or infrared and / or UV. In embodiments, the beam is a light beam comprising coherent light, such as light from a laser beam. In embodiments, the beam is a light beam comprising incoherent light, such as light from certain types of LEDs. In embodiments, the beam is an ion beam; this is related, for example, to matrix-assisted laser desorption I ionisation (MALDI). In embodiments involving Raman spectroscopy, it is an electromagnetic beam and / or electron beam and / or ion beam, preferably an electromagnetic light beam, more preferably a laser beam (or, equivalently, a laser ray). In embodiments involving mass spectroscopy, the scanning preferably relates to laser-based and / or electron-based scanning and / or ion beam-based scanning.

[0025] In this document, the term “ interaction “ refers to any propagation of the beam when incident on the sample. In exemplary embodiments, the sample is irradiated with a laser beam, and the interaction is associated with a diffraction of the laser beam onto and / or through the sample. In exemplary embodiments, the sample is irradiated with a laser beam, and the interaction is associated with a reflection onto and / or through the sample. In exemplary embodiments, the sample is irradiated with an ion beam, and the interaction is associated with diffraction. In exemplary embodiments, the sample is irradiated with an ion beam, and the interaction is associated with a reflection.

[0026] As used herein, the term “ constellation “ refers to the relationship between the sample and the beam. In exemplary embodiments, the constellation of sample and beam is related to the relative orientation of the beam and the sample, and a change in the relative orientation is related to a change in distance and / or angle and / or spot size relative to the beam and sample, for example, by actuation of the beam and / or the beam source and / or the sample and / or a lens and / or a mirror. In exemplary embodiments, the constellation of sample and beam is related to the guidance of the beam towards the sample, and a change in such guidance is related to a change in spot size and / or intensity and / or spectral characteristics of the beam as it impinges on the sample, for example, by filtering the beam and / or diffraction of the beam by a filter or grid and / or by actuation of a lens or mirror that guides the beam towards the sample. Related to this, the term “ actuation “ is a collective term for changing means which may be related to any and every combination of an electronic, magnetic, mechanical, optical or other signal which, within the framework of the present invention, can bring about a changing of the constellation of the laser beam. The term “ actuation “ may be related to a mechanical actuation, such as in manipulating a position and / or angle of the laser and / or the lens and / or an intermediate optical element such as a mirror, but is by no means limited to this.

[0027] In preferred embodiments, the constellation of sample and beam is related to mechanical actuation of the sample, wherein the relative position of the beam and the sample changes. This can be related, for example, to the movement of the sample in a plane. In exemplary embodiments, this is related to mechanical actuation of the sample by means of an x-y table, wherein the sample moves relative to the beam.

[0028] In exemplary embodiments, the beam is perpendicular to the plane (for example, the x-y table). In other exemplary embodiments, the beam is at an angle to the plane (for example, the x-y table). In further embodiments, the constellation of sample and beam is related to mechanical actuation of the beam, or to mechanical actuation of both the beam and the sample.

[0029] In embodiments, the beam source is a light source, and the change of constellation is related to the actuation of a lens or mirror that is intermediate between the beam source and the sample. The mirror may be related to, for example, a MEMS (Microelectromechanical systems) mirror and / or a liquid lens.

[0030] In embodiments, the beam source is a light source, and the change in constellation is related to actuation of the sample relative to the beam, by moving and / or tilting the sample.

[0031] Preferably, the first aspect of the invention is related to a method for adaptive beam-based scanning of the sample and / or the second aspect of the invention is related to a device for adaptive laser-based scanning of the sample. Adaptive scanning refers to changing a scan position, preferably in real time, based on a previous measurement. An adaptive scanning process allows to change a scan path based on a previous measurement. It is advantageous that the scanning is performed fully adaptively. Adaptive scanning according to the present invention contributes to improving a time-efficient scanning process.

[0032] In this document, the terms “ point measurement “ and “ spot measurement “ are interchangeable. The term “ spot measurement “ refers to irradiation of the sample by means of a beam, resulting in a measurement with respect to a spot. In embodiments, this may involve a laser beam originating from a (laser) source, for example a source with a variable spectrum, or a source with a fixed spectrum. In embodiments, this may involve an ion beam originating from an ion source. Preferably, at least one spot measurement, more preferably at least the first and second spot measurements, is related to a measurement with respect to a measurement spectrum, which is characterised by one or more contiguous wavelength intervals, or alternatively by two or more non-contiguous wavelength intervals. These wavelength intervals may extend completely or partially into one or more of: the spectral band visible to the human eye, infrared, UV, or a band with a wavelength shorter than UV, or a band with a wavelength longer than infrared. The use of multiple wavelength intervals may be related to restrictions on the possible operating ranges of the sensors and / or the tuneable filters. In examples, the wavelength ranges are Ultraviolet (“ UV “), visible (“ VIS “), near infrared (“ NIR “), shortwave infrared (“ SWIR “), mid infrared (“ MIR “), for example corresponding to 180-380 nm (UV), 380-700 nm (VIS), 700-2500 nm (NIR), 900-1700 nm (SWIR) and 2500-25000 nm (MIR). In embodiments with an auxiliary optical element, such as a lens or mirror, or with one or more lens parts belonging to a lens, this auxiliary optical element is adapted to said measurement spectrum, which is characterised by either one wavelength interval or by two or more non-contiguous wavelength intervals.

[0033] In this document, the term “ focal point “ refers to the point at which the beam is incident on or near the sample. In the context of the invention, the focal point is related to the projection of the beam onto and / or near the sample, as well as to the contour of that projection. The terms “ focal point “ and “ spot “ are therefore interchangeable, as are the terms spot size and focal point size. The term “ focal point “ is thus not limited to the exact position (often denoted as “ f “) at which the light beams from an infinitely distant object are focused after refraction by a lens, although the focal point may, in embodiments with a light beam and a lens, be located in the vicinity of that exact position. Likewise, irradiating the sample by means of a beam is not limited to embodiments using a lens, and irradiation of the sample may be accomplished in embodiments without the use of any lens.

[0034] In this document, “ substrate “ refers to any object suitable for providing the material to be scanned. In embodiments, the substrate comprises a nanostructure. In embodiments, the substrate is filter-based. In exemplary embodiments, the substrate is a solid substrate. In exemplary embodiments, the substrate is related to a glass fibre filter. In exemplary embodiments, the substrate is related to a stained filter. Here, a stained filter is preferably a filter pretreated with nanoparticles. In embodiments, the substrate comprises a metal, preferably gold and / or silver and / or aluminium. In embodiments, the substrate is part of the sample, and contributes to the heterogeneity of the sample. In embodiments, the substrate is related to the background, and the material to be scanned is related to the foreground. For example, this can be related to samples / substrates wherein the beam does not penetrate the material to be scanned, and the substrate is only characteristic of the background but not of the foreground. In other embodiments, the substrate is related to both the foreground and the background. This may be related to samples / substrates wherein the material penetrates into the substrate (causing the foreground to exhibit substrate-related features) and / or wherein the beam at least partially penetrates through the material to be scanned (causing the foreground to exhibit substrate-related features, e.g. diffraction of the beam through the sample). In embodiments, the substrate comprises a variation source, which leads to a variation in the measurement. This may be related to substrates that exhibit a large variation or irregularity in the surface, such as roughened aluminium, wherein the roughening is the variation source. In embodiments, the substrate does not comprise a variation source, such as, for example, in the case of a completely smooth and homogeneous substrate surface.

[0035] In this document, the terms “ Surface Enhanced Raman Scattering “ and “ Surface Enhanced Raman Spectrometry “ are used interchangeably. Both are referred to by the abbreviation SERS. In embodiments, said first and second spot measurement are related to Raman spectroscopy. In preferred embodiments, this involves surface-enhanced Raman spectroscopy (SERS).

[0036] In embodiments, the device comprises or is a Raman spectrometer. In embodiments, the device is part of a Raman spectrometer.

[0037] In embodiments, said first and second spot measurement are related to mass spectrometry. In some embodiments with mass spectroscopy, the interaction of the beam and the sample may be destructive, wherein portions of the sample are separated during the first and / or second spot measurement. Such embodiments can advantageously be combined with embodiments in which the changing and changing means are related to changing the constellation of the sample and an ion source.

[0038] In embodiments, the machine learning algorithm is related to template matching.

[0039] In embodiments, the trained machine learning algorithm is related to deep learning, preferably with a neural network trained with respect to measurement spectra, for example a convolutional neural network (CNN) or a multi-layer perceptron (MLP).

[0040] In embodiments, the machine learning algorithm involves XGBoost and / or a transformer model.

[0041] In embodiments, the sensor is associated with a plurality of sensor elements. For example, it may be two-dimensional imaging focal plane array (FPA) detectors and / or silicon (Si) charge-coupled device (CCD) detectors and / or CMOS detectors and / or indium gallium arsenide (InGaAs) detectors.

[0042] In embodiments, said changing by said changing means is related to describing a curve in a plane of the sample, wherein said first and second spot measurement are related to non-overlapping zones of said sample. In embodiments, the curve is related to a grid. In still other embodiments, the curve is related to a circle, whether or not in combination with a grid. In embodiments, said distinguishing the foreground and background is related to classifying each of the first and second spot measurement as either foreground or background.

[0043] In embodiments, said redirecting by said actuation is related to adjusting the shape and / or size of a focal point of the laser beam.

[0044] In embodiments, said actuation is related to changing a distance between said lens and said sample and / or to changing an angle between said laser beam and said sample.

[0045] In embodiments, said distinguishing of the foreground and the background is related to detecting a background with respect to the first and / or the second spot measurement.

[0046] In embodiments, said calculating said result with said machine learning algorithm comprises calculating an intermediate result based on said first spot measurement, wherein said redirecting is done, based on said intermediate result.

[0047] In embodiments, said changing is related to the stepwise changing of a constellation according to a sampling pattern, preferably a random sampling pattern and / or a predetermined sampling pattern. It is conceivable that the sampling pattern changes between the first spot measurement and the second spot measurement.

[0048] In embodiments, the sampling pattern is related to a sampling grid, preferably a sparse sampling grid. The use of a sparse sampling grid contributes to a time-efficient scanning process.

[0049] In embodiments, a constellation is modified such that the first spot measurement and the second spot measurement, and optionally further spot measurements, are performed at different locations. Preferably, none of the spot measurements are performed at the same location. If the sampling pattern is a sampling grid, such as a sparse sampling grid, preferably one, more preferably a maximum one, spot measurement is performed per grid cell. In embodiments, the method is performed for two or more than two spot measurements. Here, preferably for each spot measurement, an intermediate result is calculated to determine whether the foreground is present, wherein performing or not performing a next spot measurement is determined based on the intermediate results already determined.

[0050] In embodiments, calculating the result with the machine learning algorithm comprises calculating an intermediate result of at least one spot measurement and / or a series of intermediate results of multiple spot measurements. Based on said intermediate result and / or at least a part of the series of intermediate results, a location of at least one subsequent spot measurement, in particular at least the second spot measurement, can be determined. Preferably, based on said intermediate result and / or at least a part of the series of intermediate results, a constellation is changed such that at least two, preferably all, spot measurements are performed at mutually different locations. Based on said intermediate result and / or based on at least a part of the series of intermediate results, a distance between successive spot measurements is preferably determined. Optionally, based on said intermediate result and / or the series of intermediate results, a constellation is changed such that a distance between spot measurements mutually varies at least partially. It is possible to change a constellation such that a distance between spot measurements belonging to the foreground and a distance between spot measurements belonging to the background differ, wherein preferably the distance between spot measurements belonging to the foreground is smaller than the distance between spot measurements belonging to the background.

[0051] In embodiments, said actuation relates both to describing a curve in the plane of the sample and to adjusting the shape and / or size of the focal point of the laser beam.

[0052] In embodiments, providing said sample comprises applying said sample to a substrate of known nanostructure, wherein said training of said machine learning algorithm is done with said substrate of known nanostructure as background.

[0053] In preferred embodiments, spot measurements, preferably spot measurements belonging to the foreground (such as those of Example 1 ), or parts of spot measurements, preferably parts of spot measurements belonging to the foreground (such as those of Example 2), are compared with reference spectra belonging to a pattern which is (or is not) present in the sample. In exemplary embodiments, the memory comprises a first and second reference spectrum, wherein the first reference spectrum belongs to, for example, a first bacterial pattern and the second reference spectrum belongs to, for example, a second bacterial pattern, wherein the machine learning algorithm is trained to, for example, based on a certain distance measure and / or based on supervised learning, to determine which bacterial pattern is present in the sample, i.e. the first and / or the second. In further exemplary embodiments, the number of reference spectra is greater than two, three, greater than three, four, greater than four, five, greater than five.

[0054] In embodiments, the substrate exhibits a known nanostructure. In exemplary embodiments, the substrate comprises a metal nanolayer with a high degree of homogeneity. Such a substrate may have the advantage that the sample, which typically extends over the substrate according to an irregular geometry, is easier to distinguish from the other parts of the substrate. In embodiments, the machine learning algorithm is primarily trained to recognize the background (which the substrate exhibits “ unchanged “), and only secondarily to recognize the foreground. In other embodiments, the machine learning algorithm is primarily trained to recognize the foreground, or on the contrary, trained to jointly recognize both foreground and background. This may also have the advantage that more stable and regular spectra are obtained from the spot measurements. This may be related to the training of said machine learning algorithm with said substrate. Because the nanostructure is known, the algorithm can better recognize the substrate as being background, to be distinguished from the sample that forms the foreground, for which the known nanostructure is not or only to a lesser extent present. In exemplary embodiments with a metal nanolayer, this may involve a background with a high degree of homogeneity and / or symmetry, thereby simplifying recognition by a machine learning algorithm. Spot measurements (such as those of Example 1 ), or portions of spot measurements (such as those of Example 2), that are related to the background (reference numeral 9 in both Examples 1 and 2), will indeed yield spectra that closely resemble a particular reference spectrum associated with the substrate. On the other hand, spot measurements (such as those of Example 1), or portions of spot measurements (such as those of Example 2), that are related to the foreground (reference numeral 8 in both Examples 1 and 2), will yield spectra that significantly differ from the reference spectrum associated with the substrate. In preferred embodiments, such measurements associated with the foreground may additionally or alternatively be compared with reference spectra, associated with a pattern that is (or is not) present in the sample. For example, a first reference spectrum may be available corresponding to a first bacterial pattern, and a second reference spectrum corresponding to a second bacterial pattern, wherein the machine learning algorithm is able to determine, for example based on a certain distance measure, which bacterial pattern is present in the sample, i.e. the first and / or the second.

[0055] In embodiments, the invention is related to more than one beam. In exemplary embodiments, this involves a multi-channel approach, wherein a first channel is used to drive the scanning, and a second channel is used to perform the measurement. For example, the first channel is related to light microscopy, and the second channel is related to Raman spectroscopy.

[0056] In the following, the invention is described by means of non-limiting examples which illustrate the invention, and which are not intended or should be construed to limit the scope of the invention.

[0057] EXAMPLES

[0058] EXAMPLE 1 : EXEMPLARY EMBODIMENT WITH ACTUATION IN X-Y PLANE Figure 1 is a sketch of an exemplary embodiment with actuation according to a curve in the plane of the sample. The change in constellation thus corresponds to the movement according to the curve. This corresponds to actuation 3 according to the x-y plane, i.e., the plane of the substrate 10, over which the sample 1 extends.

[0059] The purpose of the scanning in this example is to detect (or not detect) a certain bacterial pattern in sample 1 .

[0060] The beam source is a laser, and the beam is a laser beam. The laser beam 2 is focused on the sample 1 using a lens 7. The change in constellation is achieved by deflecting the laser beam. The laser beam is deflected with respect to the sample by means of an actuation 3. This actuation can, for example, be related to the movement of a mirror, wherein the mirror belongs to the changing means. The actuation 3 is such that a circular curve is described over the x-y plane, with shortterm spot measurements 5 on this circle. The circle is traversed with an angular velocity such that successive spot measurements are non-overlapping (not too low angular velocity) and that multiple spot measurements are spread over the circle (not too high angular velocity).

[0061] The sample 1 contains a heterogeneity. On the one hand, the heterogeneity is related to an irregular contour, as illustrated in Fig. 1 . On the other hand, the heterogeneity is related to the relatively limited dimensions of the sample with respect to the circle of the spot measurements 5. As a result, regardless of the precise alignment of the circle with respect to the sample, it is only possible to perform part of the spot measurements 5 on the sample itself, leading to a first group, and some spot measurements inevitably fall outside the contour, leading to a second group.

[0062] The method according to the invention allows, with the aid of a trained machine learning algorithm, to distinguish between the two groups. Spot measurements of the first group are automatically classified by the algorithm as belonging to the foreground 8. Spot measurements of the second group, on the other hand, are automatically classified by the algorithm as belonging to the background 9.

[0063] The classification is hereby extremely dependable, because the substrate 10 exhibits a previously known nanostructure, which is the same for all substrates used in the method. In addition, the machine learning algorithm is trained on exactly these substrates, which makes the classification of spot measurements of the second group very dependable. These spot measurements correspond to known spectra that are related to the substrate, which makes identification by a machine learning algorithm easier.

[0064] All this allows to calculate a reliable result of the scanning, wherein only spot measurements of the first group are taken into account. These spot measurements are preferably classified by a further machine learning algorithm as corresponding to a reference spectrum belonging to the bacterial pattern for which scanning was done. Such a machine learning algorithm is trained to determine, based on an extensive training set with a large number of samples without this bacterial pattern and a large number of samples with this bacterial pattern, whether the bacterial pattern is present in the sample.

[0065] In conclusion, this example illustrates that a triage (first group versus second group) followed by a comparison with a reference spectrum leads to an accurate scanning. It is clear that a scanning process, based on averaging all spot measurements (both first group and second group), would not be successful and / or not accurate, because the averaged spectrum would not be sufficiently representative for the bacterial pattern sought.

[0066] EXAMPLE 2: EXEMPLARY EMBODIMENT WITH ACTUATION ALONG Z-AXIS Figure 2 shows a first, second and third phase (Fig. 2A, 2B and 2C respectively) of an exemplary embodiment with actuation wherein the distance between lens 7 and sample 1 is changed. This corresponds to actuation 4 along the z-direction, i.e. , the direction of the laser beam 2, which is directed substantially perpendicularly to the x-y plane of the substrate 10, over which the sample 1 extends.

[0067] As in Example 1 , the purpose of the scanning in this example is to detect (or not detect) a certain bacterial pattern in the sample 1. However, the shape of actuation 4 is different, and no circular curve is described over the x-y plane. On the contrary, it concerns short-term spot measurements 6. Due to the movement of the lens 7, the focal point 6 that falls on the sample 1 gradually becomes larger, going from Fig. 2A via Fig. 2B to Fig. 2C. In Fig. 2A and 2B, the entire focal point still extends over the sample 1 , i.e., all parts of the focal point belong to the sample 1 . In Fig. 2C, only the medial part of the focal point still overlaps with the sample, and there are two distal parts that extend beyond the contour of the sample 1 .

[0068] Also in this example, sample 1 contains a heterogeneity. The heterogeneity is again related to an irregular contour, as illustrated in Fig. 2. On the other hand, the heterogeneity is related to the relatively limited dimensions of the sample with respect to the focal point 6, which causes the distal parts of the focal point, in the third phase (Fig. 2C), to fall outside sample 1. As a result, spot measurements associated with the third phase (Fig. 2C) will be different from those associated with the first and second phases (Fig. 2A and Fig. 2B). In particular, it is expected that the spot measurement of the third phase will not measure accurately enough, and therefore may not be sufficiently representative of the bacterial pattern sought.

[0069] The invention meets this requirement by performing a detection of background 9 present in the spot measurement of the third phase, using a trained machine learning algorithm. This background corresponds to the distal parts depicted in Fig. 2C. On the other hand, for the spot measurements of the first and second phases, it is detected that it concerns foreground 8. By extension, the presence of foreground 8 is also detected for the spot measurement of the third phase. This foreground 8 corresponds to the medial part of the focal point 6 depicted in Fig. 2C.

[0070] Also in this example, the classification is extremely dependable, because the substrate 10 exhibits a previously known nanostructure, and the machine learning algorithm is trained on exactly these substrates, such that the classification of spot measurements according to foreground and background is dependable.

[0071] In variations of this example (not illustrated), an intermediate result is calculated for each spot measurement, and the actuation 4 along the z -direction is continued or not, based on whether or not background 9 is detected in the spot measurement. This means that the laser beam is redirected each time as a function of the intermediate result. This allows for a far-reaching form of automation, in which the alignment of laser beam and sample is automatically adjusted. Furthermore, this also allows for a smoother cycle for scanning, in which the execution or discontinuation of a next spot measurement is determined based on the intermediate results already determined. Finally, this example also illustrates the various advantages of distinguishing between foreground and background. Here too, it is clear that a scanning process, based on averaging all spot measurements (with both medial and distal parts, in the case of Fig. 2C), would not be successful and / or not accurate, because the averaged spectrum would not be sufficiently representative of the bacterial pattern sought.

[0072] EXAMPLE 3: EXEMPLARY EMBODIMENT WITH ACTUATION ACCORDING TO INTERMEDIATE RESULT

[0073] Figure 3 shows a first, second and third aspect (resp. Fig. 3A, 3B and 3C) of an exemplary embodiment with actuation with intermediate result. In this example, calculating the result with the machine learning algorithm comprises calculating an intermediate result. The intermediate result is calculated based on a set of first spot measurements, of which the first five are provided with reference numerals 5a-5e. Changing 10 the constellation, in this example an actuation, is then done, based on the intermediate result.

[0074] According to a first aspect, illustrated by Fig. 3A, the constellation is changed step by step according to a coarse sampling pattern 10 that has been determined in advance, with a spot measurement as a step each time (the first five provided with reference numerals 5a-5e). The spot measurements are located in a square zone. As in Examples 1 and 2, sample 1 is surrounded by a background. The technical challenge is to detect the environment of the actual sample 1 , which is made possible by the sampling pattern 10. In spot measurement 5a, no foreground is detected. In this example, work is then carried out step by step according to a fixed sampling pattern 10 and corresponding sequence, which leads to spot measurements 5b-5e and subsequent. A total of sixteen spot measurements are carried out in this way, each corresponding to a spectrum. As is clear from Fig. 3A, the seventh and eleventh spot measurements of the sequence are particularly relevant, because they only measure the foreground. In this example, the entire sampling pattern is traversed, regardless of the results of the spot measurements. In other variations of this example (not shown), the spot measurement is analysed in real-time using the trained machine learning algorithm. The intermediate result is then always the total of the already measured spot measurements, and detecting only foreground on e.g. the seventh spot measurement can lead to the actuation being stopped, based on this intermediate result. In yet other variations of this example (not shown), the sampling pattern is not fixed but, on the contrary, random, and the successive spot measurements are performed at random locations on or near sample 1 , without a predetermined pattern.

[0075] According to a second aspect of this example, illustrated by Fig. 3B, the spot measurements 5a-5e of Fig. 3A are used as the first iteration of a feedback loop, to create a heat map of spot measurements 5, as shown in Fig. 3B. After classification of each spot measurement by the machine learning algorithm, more intensity is assigned to spot measurements in which more foreground is detected, and less to spot measurements with less foreground. This heat map is the result of the first iteration, and forms the intermediate result based on which the actuation is adjusted. Adjustment is done by applying a new, more finely-grained sample pattern to a new region of interest, as the second iteration of the feedback loop. The region of interest of the second iteration is defined based on the heat map, e.g. by selecting the rectangular or square zone of the heat map with the highest intensity.

[0076] The third aspect of this example concerns this second iteration, illustrated by Fig. 3C. The spot measurements 5, 5', 5", 5"' are again located in a square zone, as in Fig. 3A. This zone is however smaller, with smaller mutual distance between the spot measurements. In examples wherein the beam is a laser beam, this is related to a stronger focus, for example by actuation of a lens. In other examples, the second iteration is related to a movement along the z-direction, which causes the spot measurements to be closer together. As illustrated in Fig. 3C, the zone of spot measurements is then better adapted to the location of sample 1 , with much more overlap with sample 1 , and thus more foreground present in the spot measurements. For the second iteration, a fixed sampling pattern can again be used (such as pattern 10 of the first iteration, illustrated in Fig. 3A). Alternatively, a random sampling pattern can be used here, or any other pattern, which does not have to be the same as that of the first iteration. From this second iteration, a final result can then be calculated, but the result can just as well be an intermediate result, with a new heat map, which can be used to calculate a new region of interest, for a third iteration.

[0077] Finally (after two, three, or more than three iterations), the final result of the measurement is obtained. In doing so, the heterogeneity of the sample is adequately accommodated by ensuring that the foreground share in the spot measurements is as large as possible.

[0078] (END EXAMPLE 3)

[0079] It is understood that the present invention is not limited to the embodiments described above and that some modifications or changes may be added to the described examples without reevaluating the appended claims. For example, the present invention has been described with Raman spectroscopy but it may also concern other forms of spectroscopy or related spectrum-based measurement, such as mass spectrometry.

Claims

CLAIMS1. Method for beam-based scanning of a sample (1 ) comprising a heterogeneity, comprising providing said sample (1 ) comprising said heterogeneity; irradiating the sample (1) by means of a beam (2), preferably a laser beam, preferably using a lens (7); performing, based on an interaction of the beam (2) and the sample (1), a first spot measurement (5, 6); changing, by means of changing means (3, 4), a constellation of said sample (1 ) and said beam (2); performing, based on an interaction of the beam (2) and the sample (1) under the modified constellation, a second spot measurement (5, 6); calculating, based on said first and second spot measurements (5, 6), a result of said scanning; wherein said calculating is at least partly done, based on a trained machine learning algorithm; wherein the training of said machine learning algorithm is related to distinguishing a foreground (8) and a background (9) which are characteristic of said heterogeneity.

2. Method according to claim 1 , wherein said calculating of said result with said machine learning algorithm comprises calculating an intermediate result, based on said first spot measurement (5a-5e), and wherein said changing (10) is done, based on said intermediate result, wherein said changing is preferably related to stepwise changing of a constellation according to a sampling pattern, preferably a random sampling pattern and / or a predetermined sampling pattern.

3. Method according to claims 1 or 2, wherein at least one spot measurement is related to a measurement with respect to a measurement spectrum, preferably at least said first and second spot measurements are related to a measurement with respect to a measurement spectrum.

4. Method according to claims 1 -3, wherein said first and second spot measurements are related to Raman spectroscopy, preferably surface-enhanced Raman spectroscopy, SERS.

5. Method according to claims 1 -4, wherein said machine learning algorithm is related to template matching.

6. Method according to claims 1 -5, wherein said machine learning algorithm is related to deep learning, preferably deep learning based on neural networks.

7. Method according to claims 1 -6, wherein said changing by means of said changing means (3) are related to describing a curve, preferably a circle, in a plane of the sample, and wherein said first and second spot measurements (5) are related to non-overlapping zones of said sample.

8. Method according to claim 7, wherein said distinguishing of the foreground (8) and the background (9) is related to classifying each of the first and second spot measurements (5) as either foreground (8) or background (9).

9. Method according to claims 1 -8, wherein said changing by said changing means (4) is related to adjusting the shape and / or size of a focal point of the beam (2), wherein said changing means (4) are preferably related to actuation of a beam source and / or the sample and / or a lens (7).

10. Method according to claim 9, wherein said changing means (4) are related to actuation of a lens (7), and wherein said actuation (4) is related to changing a distance between said lens (7) and said sample (1) and / or to changing an angle between said beam (2) and said sample (1 ).11 . Method according to claims 9-10, wherein said distinguishing of the foreground (8) and the background (9) is related to detecting a background (9) with respect to the first and / or the second spot measurement (5).

12. Method according to claims 2-11 , wherein the method is performed for two or more spot measurements (5, 6); wherein an intermediate result is calculatedfor each spot measurement to determine whether the foreground (8) is present; and wherein performing or no longer performing a subsequent spot measurement is determined based on the intermediate results already determined.

13. Method according to claims 6-12, wherein said changing means (3, 4) relate both to describing a curve in the plane of the sample (1 ) and to changing the shape and / or size of the focal point of the beam (2).

14. Method according to claims 6-13, wherein providing said sample comprises applying said sample to a substrate with known nanostructure, and wherein said training of said machine learning algorithm is done with said substrate with known nanostructure as background (9).

15. Device for laser-based scanning of a sample (1 ) comprising a heterogeneity, comprising a beam source, preferably a laser, for generating a beam (2), preferably a laser beam (2); preferably, a lens (7) for focusing said beam, preferably a laser beam; a sensor for measuring an interaction of the sample (1) and the beam (2); changing means (3, 4) for changing a constellation of the sample (1) and the beam (2); a memory, comprising a trained machine learning algorithm; a processor, at least connected to said sensor, said means for actuation and said memory; wherein said processor is configured for: irradiating the sample (1) comprising said heterogeneity with the beam (2); performing, by means of said sensor, a first spot measurement (5, 6); changing, by means of the changing means (3, 4), the constellation of said sample (1) and said beam (2); performing a second spot measurement (5, 6) using said sensor; calculating, based on said first and second spot measurements (5, 6), a result of said scanning; wherein said calculating is at least partly done, based on said trained machine learning algorithm;wherein the training of said machine learning algorithm is related to distinguishing a foreground (8) and a background (9) which are characteristic of said heterogeneity.

16. Kit, comprising said device according to claim 15 and a plurality of substrates with known nanostructure, wherein said training of said machine learning algorithm has been done with said substrate with known nanostructure as background (9).

Images