Method for classifying an input image containing particles in a sample

Abstract

The present invention relates to a method for classifying at least one input image containing target particles (11a-11f) in a sample (12), the method being characterized in that it includes performing the following steps via a data processing device (20) of a client (2): (b) extracting a vector of characteristics of the target particles (11a-11f), the characteristics being digital coefficients, each digital coefficient being associated with a base image in a set of base images, each base image representing a reference particle, such that a linear combination of the base images weighted by the coefficients approximates a representation of the target particles (11a-11f) in the input image; and (c) classifying the input image depending on the extracted vector of characteristics.

Description

Technical Field

[0001] This invention relates to the field of optical acquisition of biological particles. These biological particles can be microorganisms, such as bacteria, fungi, or yeast. It could also be cells, multicellular organisms, or any other type of particle such as contaminants or dust.

[0002] This invention is particularly advantageous for analyzing the state of biological particles, for example, focusing on determining the metabolic state of bacteria after antibiotic application. This invention makes it possible, for example, to determine the antibacterial spectrum of bacteria. Background Technology

[0003] Antibiogram is a laboratory technique designed to test the phenotype of bacterial strains against one or more antibiotics. Conventionally, antibiograms are determined by culturing a sample containing both bacteria and the antibiotic.

[0004] European Patent Application No. 2,603,601 describes a method for determining an antibacterial spectrum, which involves visualizing the state of bacteria after an incubation period in the presence of antibiotics. To visualize the bacteria, they are labeled with fluorescent markers, revealing their structure. Measuring the fluorescence of the markers then allows determination of whether the antibiotics have effectively acted on the bacteria.

[0005] The routine procedure for determining the effectiveness of antibiotics against a given bacterial strain involves obtaining a sample containing the strain (e.g., from a patient, animal, food batch, etc.) and sending the sample to an analytical center. Upon receiving the sample, the analytical center first cultures the bacterial strain to obtain at least one colony, which takes 24 to 72 hours. Several samples containing different antibiotics and / or different concentrations of antibiotics are then prepared from this colony and incubated again. After a new incubation period (also requiring 24 to 72 hours), each sample is manually analyzed to determine antibiotic effectiveness. The results are then sent back to the physician so that they can apply the most effective antibiotic and / or antibiotic concentration.

[0006] However, the labeling process is particularly lengthy and complex, and these chemical markers are cytotoxic to bacteria. Therefore, this visualization method does not allow for multiple observations of bacteria during culture; as a result, bacteria must be cultured for a sufficiently long time, approximately 24 to 72 hours, to ensure the reliability of the measurements. Other methods for visualizing biological particles use microscopy, allowing for non-destructive measurements of the sample.

[0007] Digital holographic microscopy, or DHM, is an imaging technique that overcomes the depth-of-field limitations of conventional optical microscopy. Illustratively, it involves recording a hologram formed by the interference between light waves diffracted from the observed object and a spatially coherent reference wave. This technique is described in Myung K. Kim's commentary entitled "Principles and techniques of digital holography microscopy," published in the January 2010 issue of SPIE Review, Volume 1, No. 1.

[0008] Recently, the use of digital holographic microscopy for automated identification of microorganisms has been proposed. Therefore, international application WO2017 / 207184 describes a method for acquiring particles that combines simple defocus acquisition with digital focus reconstruction, thereby enabling the observation of biological particles while limiting acquisition time.

[0009] Typically, this solution allows for the detection of bacterial structural modifications after only about 10 minutes of incubation in the presence of antibiotics, and the determination of sensitivity (detecting the presence or absence of division or indicating the pattern of division) after two hours, unlike conventional methods described above which may take several days. Specifically, because the measurements are non-destructive, analysis can be performed very early in the culture process without the risk of damaging the sample and thus prolonging the analysis time.

[0010] It is even possible to track particles across multiple consecutive images, thus creating a film representing the progress of the particles over time (because the particles are not destroyed after the first analysis), in order to visualize their behavior, such as their speed of movement or their cell division process.

[0011] Therefore, it can be understood that this visualization method yields excellent results. The difficulty lies in interpreting these images or the film itself, for example, whether to draw conclusions about the susceptibility of bacteria to antibiotics present in the sample.

[0012] Various techniques have been proposed, ranging from simply counting bacteria over time to so-called morphological analysis, which aims to detect specific “configurations” via image analysis. For example, when bacteria are preparing to divide, bipolar distributions appear in the distribution long before the division itself, causing the distribution to split into two distinct segments.

[0013] The article [Choi et al., 2014] has proposed combining these two techniques to assess antibiotic efficacy. However, as the authors emphasize, their method requires very fine calibration of a number of thresholds that are highly dependent on the nature of the morphological changes induced by the antibiotic.

[0014] Recently, a deep learning-based approach was described in the paper [Yu et al., 2018]. The authors proposed using a convolutional neural network (CNN) to extract morphological features and features related to bacterial movement. However, this solution proved to be computationally intensive, requiring a large database of training images to train the CNN.

[0015] Therefore, the objective technical problem of the present invention is to provide a more efficient and less resource-intensive solution for classifying images of biological particles. Summary of the Invention

[0016] According to a first aspect, the present invention relates to a method for classifying at least one input image representing target particles in a sample, the method being characterized in that it includes the following steps performed by a data processing device of a client:

[0017] (b) Extracting feature vectors of the target particle’s features, the features being digital coefficients, each of which is associated with a base image in a set of base images, each base image representing a reference particle, such that a linear combination of the base images weighted by the coefficients approximates the representation of the target particle in the input image;

[0018] (c) The input image is classified based on the extracted feature vector.

[0019] Based on the advantageous but non-restrictive features:

[0020] The particle is represented in a uniform manner in the input image and in each base image, specifically centered and aligned along a predetermined direction.

[0021] The method includes step (a): extracting the input image from the overall image of the sample in order to represent the target particle in the uniform manner.

[0022] Step (a) includes: segmenting the overall image to detect the target particles in the samples, and then re-cropping the input image according to the detected target particles.

[0023] Step (a) includes obtaining the overall image from an intensity image of the sample acquired by an observation device.

[0024] The method includes step (b0): unsupervised training of the base image using a database of training images of particles in the sample.

[0025] The trained reference images are those that allow for the best approximation of the representation of particles in the training images through a linear combination of the base images.

[0026] Step (c) is implemented using a classifier, which includes step (a0): the data processing device of the server trains the parameters of the classifier using a training database of classified feature vectors / matrices of particles in the sample.

[0027] The classifier is selected from support vector machines, k-nearest neighbor algorithm, or convolutional neural networks.

[0028] Step (c) includes reducing the number of variables in the feature vector by using the t-SNE algorithm.

[0029] This method is used to classify an input image sequence representing the target particle in a sample over time, wherein step (b) includes obtaining a feature matrix of the target particle by concatenating the extracted feature vectors of each input image of the sequence.

[0030] According to a second aspect, a system is provided for classifying at least one input image representing target particles in a sample, the system comprising at least one client, the client including a data processing device, characterized in that the data processing device is configured to implement:

[0031] - Extract feature vectors of the target particle's features, where the features are digital coefficients, each of which is associated with a base image in a set of base images, each base image representing a reference particle, such that a linear combination of the base images weighted by the coefficients approximates the representation of the target particle in the input image;

[0032] The input image is classified based on the extracted feature vector.

[0033] According to an advantageous but not limiting feature, the system also includes: a device for observing the target particles in a sample.

[0034] According to the third and fourth aspects, the following are provided: a computer program product including code instructions for performing a method according to the first aspect for classifying at least one input image representing target particles in a sample; and a storage medium readable by a computer device, wherein the computer program product includes code instructions for performing a method according to the first aspect for classifying at least one input image representing target particles in a sample. Attached Figure Description

[0035] Other features and advantages of the invention will become apparent upon reading the following description of preferred embodiments. This description will be given with reference to the accompanying drawings, in which:

[0036] - Figure 1 This is a schematic diagram of an architecture for implementing the method according to the present invention;

[0037] - Figure 2 An example of a device for observing particles in a sample is shown, which is used in a preferred embodiment of the method according to the invention;

[0038] - Figure 3a The acquisition of an input image is illustrated in one embodiment of the method according to the present invention;

[0039] - Figure 3b The acquisition of an input image is illustrated in a preferred embodiment of the method according to the present invention;

[0040] - Figure 4 The steps of a preferred embodiment of the method according to the present invention are shown;

[0041] - Figure 5a An example of a dictionary of basic images used in a preferred embodiment of the method according to the invention is shown;

[0042] - Figure 5b An example of feature vector and matrix extraction according to a preferred embodiment of the method according to the present invention is shown;

[0043] - Figure 6 An example of t-SNE projection used in a preferred embodiment of the method according to the invention is shown. Detailed Implementation

[0044] Architecture

[0045] The present invention relates to a method for classifying at least one image representing particles 11a-11f (referred to as target particles) present in sample 12. It should be noted that the method can be implemented in parallel for all or some of the particles 11a-11f present in sample 12, each particle being sequentially considered a target particle.

[0046] As will be seen, this approach may include one or more machine learning components, specifically one or more classifiers, including convolutional neural networks (CNNs).

[0047] The input or training data is of image type and represents target particles 11a-11f in sample 12 (in other words, these are images of the samples in which the target particles are visible). As will be seen, a sequence of images of the same target particles 11a-11f (or, where appropriate, multiple sequence of images of particles 11a-11f in sample 12 if multiple particles are considered) can be provided as input.

[0048] Sample 12 consists of a liquid, such as water, buffer, culture medium or reaction medium (with or without antibiotics), in which the particles to be observed 11a-11f are located.

[0049] As a variation, sample 12 can take the form of a semi-transparent solid medium such as agar, in which particles 11a-11f are located. Sample 12 can also be a gaseous medium. Particles 11a-11f can be located inside the medium or on the surface of sample 12.

[0050] Particles 11a-11f can be microorganisms such as bacteria, fungi, or yeast. It could also be a problem with cells, multicellular organisms, or any other type of particle such as contaminants or dust. In the remainder of this description, a preferred example of particles being bacteria (and, as will be seen, sample 12 contains antibiotics) will be considered. The observed particle sizes 11a-11f vary between 500 nanometers and several hundred micrometers, or even a few millimeters.

[0051] The “classification” of the input image (or sequence of input images) includes determining at least one category from a set of possible categories describing the image. For example, in the case of bacterial type particles, a binary classification can be used, that is, two possible categories can be used, indicating “division” or “no division”, respectively, demonstrating the presence or absence of resistance to the antibiotic. The invention is not limited to any particular classification, although examples of binary classification of the effects of antibiotics on the target particles 11a-11f will be primarily described.

[0052] This method utilizes server 1 and client 2 in, for example... Figure 1 The architecture shown is implemented as follows. Server 1 is the device being trained (implementing the training method), and client 2 is the user device (implementing the classification method), such as a doctor's or hospital's terminal.

[0053] These two devices 1 and 2 can be combined, but preferably, server 1 is a remote device, while client 2 is a mass-market device, particularly a desktop computer, laptop computer, etc. Client device 2 is advantageously connected to observation device 10 so as to be able to directly acquire the input image (or, as will be seen below, “raw” acquisition data such as the overall image of sample 12 or even the electromagnetic matrix), typically with an eye toward direct processing. Alternatively, the input image will be loaded onto client device 2.

[0054] In all cases, focusing on data exchange, each of devices 1 and 2 is typically a remote computer device connected to a local area network or wide area network such as the Internet. Each device includes a processor-type data processing device 3, 20, and a data storage device 4, 21 such as computer memory, for example, flash memory or a hard disk. Client 2 typically includes a user interface 22, such as a screen that allows interaction.

[0055] Server 1 advantageously stores a training database, namely, a set of images of particles 11a-11f under various conditions (see below) and / or a set of classified feature vectors / matrices (e.g., associated with the label “split” or “non-split” indicating sensitivity or tolerance to antibiotics). It should be noted that the training data will likely be associated with labels defining test conditions, such as “strain,” “antibiotic condition,” “time,” etc., indicating information about bacterial culture.

[0056] collection

[0057] As explained above, this method can directly use any image of the target particles 11a-11f obtained in any manner as input. However, this method preferably begins at step (a), i.e., obtaining the input image from the data delivered by the observation device 10.

[0058] In a known manner, those skilled in the art will be able to use DHM technology (DHM stands for Digital Holographic Microscopy), particularly as described in International Application WO2017 / 207184. Specifically, an intensity image of sample 12 can be acquired that is not focused on the target particle (this image is referred to as "out of focus") but can be processed by a data processing device (e.g., those devices 20 integrated into device 10 or client 2, see below), such an image is referred to as a hologram. It will be understood that the hologram "represents" all particles 11a-11f of the sample in some way.

[0059] Figure 2An example of a device 10 for observing particles 11a-11f present in a sample 12 is shown. The sample 12 is arranged between a spatially and temporally coherent (e.g., laser) or pseudo-coherent (e.g., light-emitting diode, laser diode) light source 15 and a digital sensor 16 sensitive to the spectral range of the light source. Preferably, the light source 15 has a narrow spectral width, for example, narrower than 200 nm, narrower than 100 nm, or even narrower than 25 nm. Hereinafter, reference is made to the center emission wavelength of the light source, for example, located in the visible region. The light source 15 emits a coherent signal Sn toward a first surface 13 of the sample, which is transmitted, for example, by a waveguide such as an optical fiber.

[0060] Sample 12 (such as a culture medium as typically interpreted) is contained in an analytical chamber vertically defined by a download slide and a top slide (such as a conventional microscope slide). The analytical chamber is laterally defined by adhesive or any other sealing material. The download slide and top slide are transparent to the wavelength of light source 15, and the sample and analytical chamber allow, for example, more than 50% of the light source wavelength to pass through on the download slide under normal incidence.

[0061] Preferably, particles 11a-11f are located in sample 12 near the upper slide. For this purpose, the bottom surface of the upper slide includes ligands that allow particle attachment, such as polycations (e.g., poly-L-lysine) in the context of microorganisms. This allows for the inclusion of particles with a thickness equal to or close to the depth of field of the optical system (i.e., less than 1 mm (e.g., a tube lens), preferably less than 100 μm (e.g., a microscope objective)). Particles 11a-11f, however, can move within sample 12.

[0062] Preferably, the device includes an optical system 23, which, for example, consists of a microscope objective lens and a tube lens, positioned in the air and maintained at a fixed distance from the sample. The optical system 23 may optionally be equipped with a filter, which may be located in front of the objective lens or between the objective lens and the tube lens. The optical system 23 is characterized by an optical axis; an object plane (also referred to as the focal plane) located at a distance from the objective lens; and an image plane, which is conjugate to the object plane by the optical system. In other words, for an object located in the object plane, there is a corresponding sharp image of that object in the image plane (also referred to as the focal plane). The optical properties of the system 23 are fixed (e.g., a fixed-focal-length optics). The object plane and the image plane are orthogonal to the optical axis.

[0063] The second surface 14 of the image sensor 16, facing the sample, is located in or near the focal plane. The sensor, such as a CCD or CMOS sensor, comprises a periodic two-dimensional array of fundamental sensitive sites, and associated electronics that adjust the exposure time and zero the sites in a manner known per se. The signal output from the fundamental sites depends on the amount of radiation in the spectral range incident on said sites during the exposure time. This signal is then converted, for example, by the associated electronics into image points or “pixels” of a digital image. Thus, the sensor produces a digital image in the form of a C-column, L-row matrix. The coordinates (c, l) in the matrix, each pixel of which corresponds in a manner known per se to the position of Cartesian coordinates (x(c, l), y(c, l)) in the focal plane of the optical system 23, for example, the position of the center of the rectangular fundamental sensitive sites.

[0064] The pitch and fill factor of the periodic array are selected to conform to the Nyquist criterion regarding the observed particle size, so as to define at least two pixels for each particle. Therefore, image sensor 16 acquires a transmission image of the sample across the spectral range of the light source.

[0065] The image acquired by image sensor 16 includes holographic information, in terms of the interference between the waves diffracted by particles 11a-11f and the reference waves that have passed through the sample without interacting with it. As stated above, it should be apparent that, in the context of a CMOS or CCD sensor, the acquired digital image is an intensity image, and phase information is thus encoded in this intensity image.

[0066] Alternatively, the coherent signal Sn generated by the light source 15 can be divided into two components, for example, by means of a translucent plate. The first component then acts as a reference wave, and the second component is diffracted by the sample 12. The image in the image plane of the optical system 23 is generated by the interference between the diffracted wave and the reference wave.

[0067] Reference Figure 3a In step (a), at least one overall image of sample 12 can be reconstructed from the hologram, and then the input image can be extracted from the overall image of the sample.

[0068] Specifically, it will be understood that the target particles 11a-11f must be represented in a uniform manner in the input image, and specifically centered and aligned along a predetermined direction (e.g., horizontally). The input image must also have a standardized size (and it is expected that only target particles 11a-11f will be visible in the input image). Therefore, this input image is referred to as a "thumbnail," and its size can be defined, for example, as 250×250 pixels. In the case of a sequence of input images, for example, one image is taken every minute at a time interval of 120 minutes, and this sequence thus forms a 3D "stack" of size 250×250×120.

[0069] As explained, the overall image is reconstructed by the data processing device of device 10 or by the devices 20 of client 2.

[0070] Typically, (for each given acquisition time) a series of complex matrices, called “electromagnetic matrices”, are constructed. These matrices are modeled based on the intensity image (hologram) of sample 12, for the wavefront of light waves propagating along the optical axis with respect to multiple deviations (particularly deviations located in the sample) relative to the focal plane of optical system 23.

[0071] These matrices can be projected into real space (e.g., via the Hermitian norm) to form a stack of the overall image at various focal lengths.

[0072] This allows us to determine the average focal length (and select the appropriate overall image, or recalculate it from the hologram), or even determine the optimal focal length for the target particle (and again select the appropriate overall image, or recalculate it from the hologram).

[0073] In any case, refer to Figure 3b Step (a) advantageously includes segmenting the one or more overall images to detect the target particles in the samples, followed by cropping. Specifically, the input image can be extracted from the overall image of the samples to represent the target particles in the uniform manner.

[0074] Generally, segmentation allows for the detection of all particles of interest while removing artifacts such as filaments or microcolonies to improve one or more overall images. Then, one of the detected particles is selected as the target particle, and its corresponding thumbnail is extracted. As explained, this can be done for all detected particles.

[0075] The partitioning can be implemented in any known manner. Figure 3bIn the example, fine segmentation is first performed to eliminate artifacts, followed by coarse segmentation to detect particles 11a-11f. Any segmentation technique known to those skilled in the art can be used.

[0076] If you want to obtain the input image sequence for the target particles 11a-11f, you can use tracking techniques to track any movement of the particles from one overall image to the next.

[0077] It should be noted that for a given sample (for multiple particles or even all particles in sample 12), all input images obtained over time can be pooled to form a descriptive corpus of sample 12 (in other words, the descriptive corpus of the experiment), such as Figure 3a As shown on the right, this corpus is specifically copied to storage device 21 of client 2. This is the "field" level, as opposed to the "particle" level. For example, if particles 11a-11f are bacteria, and sample 12 contains (or does not contain) antibiotics, then this descriptive corpus contains all the information about the growth, morphology, internal structure, and optical properties of these bacteria throughout the collection field. As will be seen, this descriptive corpus can be transferred to server 1 for integration into the training database.

[0078] Feature extraction

[0079] Reference Figure 4 Of particular note in this method is that the step (b) of extracting feature vectors from the input image is performed separately from the step (c) of classifying the input image based on the feature vectors, rather than attempting to classify the input image directly. As will be seen, each step may involve independent machine learning mechanisms, so the training database of server 1 may include particle images and feature vectors that have not necessarily been classified.

[0080] Therefore, the main step (b) is for the data processing device 20 of the client 2 to extract the feature vector of the target particle, that is, the "coding" of the target particle.

[0081] In the remainder of this description, a distinction will be made between the number of “dimensions” of the eigenvectors / matrices in a geometric sense, i.e., the number of independent directions in which these graphs extend (e.g., vectors are 1-dimensional objects, while matrices are 2-dimensional, advantageously 3-dimensional objects) and the number of “variables” of these eigenvectors / matrices, i.e., the size in each dimension, i.e., the number of independent degrees of freedom (this actually corresponds to the concept of dimension in vector space—more precisely, a set of eigenvectors / matrices with a given number of variables forms a vector space with dimensions equal to the number of those variables).

[0082] Therefore, the following will describe an example in which the feature matrix extracted at the end of step (b) is a two-dimensional object (i.e., an object with dimension 2) with a size of 60×25 and thus 1500 variables.

[0083] In this case, the special feature of this encoding is that the feature is a set of digital coefficients, each of which is associated with a base image in a set of base images, each base image representing a reference particle, such that a linear combination of the base images weighted by the coefficients approximates the representation of the particle in the input image.

[0084] This is called "sparse coding". The basic image is called an "atom", and the set of atoms is called a "dictionary". The idea behind sparse coding is to express any input image as a linear combination of these atoms by analogy with dictionary words. More precisely, for a dictionary D of size p, and denoted by α as a feature vector of size p, we find the best approximation Dα of the input image x. In other words, denoteing the best vector (the sparse code of the input image x) as α*, step (b) involves solving a minimization problem of a functional with a regularization parameter λ (which allows for a trade-off between the quality of the approximation and the sparsity of the vector, i.e., involving the fewest possible atoms). For example, the constraint minimization problem can be formulated as follows:

[0085]

[0086] It can also be expressed as a variational-formulaic problem:

[0087]

[0088] The coefficients advantageously have values ​​in the interval [0, 1] (which is simpler than in R), and it will be understood that due to the “sparse” nature of the encoding, most coefficients typically have a value of 0. The atom associated with a non-zero coefficient is called an activation atom.

[0089] Naturally, the base image is a thumbnail comparable to the input image, i.e., the reference particles are represented therein in the same uniform manner as in the input image, particularly centered and aligned along the predetermined direction, and the base image advantageously has the same size as the input image (e.g., 250×250).

[0090] therefore, Figure 5a An example dictionary of 36 basic images is shown (in the case of E. coli with the antibiotic cefpodoxime).

[0091] Given a sequence of input images, step (b) advantageously includes extracting a feature vector from each input image, which can be combined into a feature matrix called the “profile” of the target particle. More precisely, these vectors all have the same size (number of atoms) and form a vector sequence, so juxtaposing them in the order of the input images is sufficient to obtain a sparse two-dimensional code (encoding spatiotemporal information, and thus two-dimensional).

[0092] Alternatively or additionally, the feature vectors / matrices corresponding to the multiple input images associated with the multiple particles 11a-11f of sample 12 can be summed.

[0093] Therefore, this technique allows for the acquisition of high-semantic-level feature vectors without requiring a large amount of computing power or annotated databases.

[0094] Figure 5b Another example of extracting feature vectors is shown, this time using a dictionary of 25 atoms. The entire overall image obtained at a given time T1, along with the various extracted input images (corresponding to the detected particles), has been shown. Therefore, the image representing the second target particle can be approximated as 0.33 times that of atom 13 plus 0.21 times that of atom 2 plus 0.16 times that of atom 9 (i.e., vector (0; 0.21; 0; 0; 0; 0; 0; 0; 0.16 0; 0; 0; 0.33; 0; 0; 0; 0; 0; 0; 0; 0; 0; 0; 0; 0).

[0095] The middle section shows a summation vector known as a "cumulative histogram." Advantageously, the coefficients are normalized so that their sum equals 1. On the right is a summation matrix (summary over 60 minutes) known as an "activation profile"—it can be seen that its size is therefore 60×25.

[0096] It will be understood that the activation distribution map is a high-level feature map representing sample 12 (over time).

[0097] Learning about atoms

[0098] The reference image (atom) can be predefined. However, preferably, the method includes a step (b0) of learning from a training database, in which the reference image (i.e., the image of the dictionary) is learned, in particular, by the data processing device 3 of server 1, so that the method does not require any human intervention at any time.

[0099] This learning method is called "dictionary learning" because it involves learning a dictionary. It is unsupervised in that it does not require annotating the images in the training database, and is therefore extremely simple to implement. Specifically, it will be understood that manually annotating thousands of images would be very time-consuming and very expensive.

[0100] The idea is to simply provide a training database with thumbnails representing particles 11a-11f under various conditions, and based on this, find atoms that allow any thumbnail to be represented as easily as possible.

[0101] Preferably, different dictionaries may exist for each type of particle 11a-11f and / or each type of sample 12. Specifically, in embodiments where particles 11a-11f are bacteria, a dictionary exists for each type of bacteria and each antibiotic. Various conditions are obtained, in particular, using various concentrations of antibiotics. However, it is conceivable to use the same training database for multiple antibiotics, etc.

[0102] It will be noted that step (b0) can be performed very far upstream, or the results of step (a) can be waited for (representing the database of ongoing experiments) to improve the results.

[0103] In any case, learning can be performed in any manner known to those skilled in the art, especially when it corresponds to an optimization problem. If the image representation of the training database is x i If i ≤ N, then the problem will be as follows, for example:

[0104]

[0105] Specifically, the aim is to find a dictionary D that allows each training image x to... i The best approximation of Dα i .

[0106] The SPAMS toolbox may be used, for example, to perform learning (SPAMS stands for Sparse Modeling Software).

[0107] therefore, Figure 5aThe 36 atoms were learned using a database of tens of thousands of input images obtained over 61 minutes from cultures of six strains of *E. coli* (two non-resistant and four resistant strains) containing up to four different concentrations of cefpodoxime (plus the absence of antibiotics). The 36 atoms were obtained with the regularization parameter λ set to 0.2. Atoms 5, 16, 19, and 32 correspond to bacteria in the process of (normal) division, while atoms 9, 11, 12, 26, 27, and 33 show morphological changes induced by cefpodoxime.

[0108] I have successively studied other dictionaries for other bacteria such as Staphylococcus aureus and / or other antibiotics such as cefoxitin, gentamicin, etc.

[0109] Classification

[0110] In step (c), the input image is classified based on the extracted feature vector.

[0111] It will be understood that it is possible to use any technique that allows for descriptive analysis of one or more feature vectors / matrices, particularly a classifier trained on the training database (several examples will be given below). In this regard, as in step (b0), the method may include step (a0): the data processing device 3 of server 1 trains the classifier using the training database. Specifically, this step is typically performed very far upstream, particularly by remote server 1. As explained, the training database may contain a certain number of feature vectors / matrices of the training images, i.e., their sparse codes, which occupy a small space.

[0112] The sparse code obtained in step (b) (especially in the case of matrices) may have a large number of variables, and therefore the visualization and interpretation of the analysis results are complex. It is best to use reduction techniques.

[0113] Therefore, the t-SNE algorithm (t-SNE stands for t-distributed stochastic neighbor embedding) can be used. This is a non-linear method to reduce the number of variables used for data visualization, allowing the representation of a set of points in a high-dimensional space (a value space of sparse codes / activation distributions) in two or three-dimensional space—the data can then be visualized using scatter plots. The t-SNE algorithm attempts to find a configuration (called the t-SNE projection) that is optimal in terms of point proximity according to information theory criteria: two points that are close (or far apart) in the original space must be close (or far apart) in the lower-dimensional space.

[0114] The t-SNE algorithm can be implemented at both the particle level (for target particles 11a-11f with vectors available in the training database) and the field level (for the entire sample 12 representing multiple input images of multiple particles 11a-11f), especially in the case of a single vector rather than a feature matrix.

[0115] It should be noted that t-SNE projection can be implemented efficiently, particularly with implementations in Python, and therefore can be performed in real time. To accelerate computation and reduce memory footprint, the first step of linear dimensionality reduction (e.g., PCA – Principal Component Analysis) can be performed before computing the t-SNE projections of the training database and the input image in question. In this case, the PCA projection of the training database can be stored in memory, and then the remaining task is to perform the projection using the sparse code of the input image in question.

[0116] For practical classifiers, the k-NN method (k-NN stands for k-nearest neighbors) can be used, especially for the results of the t-SNE algorithm (the obtained projection, or "embedding").

[0117] The idea is to examine the neighboring points of a point corresponding to a feature vector of one or more input images in question and check their classification. For example, if a neighboring point is classified as "non-split," then it can be assumed that the input image in question must be classified as "non-split." It should be noted that the range of nearest neighbors considered may be limited, for example, depending on the strain, antibiotic, etc. Figure 6 Two examples of t-SNE embeddings obtained by *E. coli* strains for various concentrations of cefpodoxime are shown. In the top example, two blocks are clearly visible, visually demonstrating the existence of a minimum inhibitory concentration (MIC) above which morphology and therefore cell division are affected. Vectors closer to the top are likely to be classified as "dividing," while vectors closer to the bottom are likely to be classified as "not dividing." In the bottom example, only the highest concentration stands out (and therefore appears to have antibiotic activity).

[0118] According to the second embodiment, a support vector machine (SVM) is used as the classifier to obtain a binary classification again (e.g., "split" or "not split" again). This simple method is particularly effective for a single input image (the SVM is applied to the feature vector). The hyperparameter C of the SVM can be optimized using grid search and so-called k-fold cross-validation (especially in the case of k=5, where the original database is divided into k samples, and then one of the k samples is selected as the validation set, while the k-1 other samples form the training set).

[0119] According to the third embodiment, in the case of an input image sequence (3D stack) and thus a feature matrix, a convolutional neural network (CNN) is used as a classifier.

[0120] This CNN can have a relatively simple architecture, such as an architecture consisting of a series of blocks of a convolutional layer, an activation layer (e.g., a ReLU function), and a pooling layer (e.g., a max pooling layer). Two such blocks are sufficient to achieve efficient binary classification. Furthermore, it is possible to downsample the input (especially in the "time" dimension) to further reduce its memory footprint.

[0121] CNNs can be trained in a conventional manner. The training cost function can consist of a conventional cost function (such as cross-entropy) and total variation regularization.

[0122] In all embodiments, where appropriate, the trained classifier can be stored on the data storage device 21 of client 2 for classification purposes. It will be noted that the same classifier can be installed on many clients 2 with only one training phase.

[0123] Computer program products

[0124] According to the second and third aspects, the present invention relates to a computer program product comprising code instructions for executing (particularly on data processing apparatus 3, 20 of server 1 and / or client 2) a method for classifying at least one input image representing target particles 11a-11f in sample 12, and a computer device readable storage device (memory 4, 21 of server 1 and / or client 2) on which the computer program product is stored.

Claims

1. A method for classifying at least one input image representing target particles (11a-11f) in a sample (12), the method being characterized in that it comprises the following steps performed by a data processing device (20) of a client (2): (b) Extract feature vectors of the features of the target particles (11a-11f), the features being digital coefficients, each of which is associated with a base image in a set of base images, each base image representing a reference particle, such that a linear combination of the base images weighted by the coefficients approximates the representation of the target particles (11a-11f) in the input image. (c) Classifying the input image based on the extracted feature vector, the method comprising the step (b0): performing unsupervised learning on the base image using a database of training images of particles in the sample, wherein, The learned base images are those base images that allow for the best approximation of the representation of particles in the training images through a linear combination of the base images.

2. The method according to claim 1, wherein, The particles (11a-11f) are represented in a uniform manner in the input image and in each basic image, specifically centered and aligned along a predetermined direction.

3. The method according to claim 2, comprising step (a): extracting the input image from the overall image of the sample to represent the target particles (11a-11f) in the uniform manner.

4. The method according to claim 3, wherein, Step (a) includes: segmenting the overall image to detect the target particles (11a-11f) in the sample (12), and then re-cropping the input image according to the detected target particles (11a-11f).

5. The method according to any one of claims 3 and 4, wherein, Step (a) includes obtaining the overall image from the intensity image of the sample (12) acquired by the observation device (10).

6. The method according to any one of claims 1 to 5, wherein, Step (c) is implemented using a classifier, the method comprising step (a0): the data processing device (3) of the server (1) trains the parameters of the classifier using a training database of classified feature vectors / matrices of particles (11a-11f) in the sample (12).

7. The method according to claim 6, wherein, The classifier is selected from support vector machines, k-nearest neighbor algorithm, or convolutional neural networks.

8. The method according to any one of claims 1 to 7, wherein, Step (c) includes: reducing the number of variables in the feature vector by means of the t-SNE algorithm.

9. The method according to any one of claims 1 to 8, used for classifying an input image sequence representing the target particles (11a-11f) in a sample (12) over time, wherein, Step (b) includes obtaining the feature vectors of the target particles (11a-11f) by concatenating the extracted feature vectors of each input image of the sequence.

10. A system for classifying at least one input image representing target particles (11a-11f) in a sample (12), the system comprising at least one client (2) including a data processing device (20), characterized in that, The data processing device (20) is configured to implement: - Extract feature vectors of the features of the target particles (11a-11f), the features being digital coefficients, each of which is associated with a base image in a set of base images, each base image representing a reference particle, such that a linear combination of the base images weighted by the coefficients approximates the representation of the target particles (11a-11f) in the input image. The input image is classified based on the extracted feature vectors. The data processing device is further configured to perform unsupervised learning on the base image using a database of training images of particles (11a-11f) in the sample (12), wherein the learned reference images are those that allow the best approximation of the representation of particles (11a-11f) in the training image through a linear combination of the base images.

11. The system of claim 10, further comprising: Device (10) for observing the target particles (11a-11f) in the sample (12).

12. A computer program product comprising code instructions for performing, when the program is executed on a computer, a method for classifying at least one input image representing target particles (11a-11f) in a sample (12) according to any one of claims 1 to 9.

13. A storage medium readable by a computer device, wherein a computer program product thereon comprises code instructions for performing a method according to any one of claims 1 to 9 for classifying at least one input image representing target particles (11a-11f) in a sample (12).

Images