METHOD FOR CLASSIFIING A SEQUENCE OF INPUT IMAGES REPRESENTING A PARTICLE IN A SAMPLE OVER TIME

Description

GENERAL TECHNICAL FIELD

[0001] The present invention relates to the field of optical acquisition of biological particles. The biological particles can be microorganisms such as bacteria, fungi or yeasts for example. They can also be cells, multicellular organisms, or any other particle such as pollutant particles or dust.

[0002] The invention finds a particularly advantageous application for analyzing the state of a biological particle, for example to know the metabolic state of a bacterium following the application of an antibiotic. The invention makes it possible, for example, to produce an antibiogram of a bacterium. STATE OF THE ART

[0003] An antibiogram is a laboratory technique used to test the phenotype of a bacterial strain against one or more antibiotics. An antibiogram is typically performed by culturing a sample containing bacteria and an antibiotic.

[0004] European Patent Application No. 2,603,601 describes a method for performing an antibiogram by visualizing the state of bacteria after a period of incubation in the presence of an antibiotic. To visualize the bacteria, the bacteria are labeled with fluorescent markers to reveal their structures. Measuring the fluorescence of the markers then makes it possible to determine whether the antibiotic has acted effectively on the bacteria.

[0005] The classic process for determining which antibiotics are effective on a bacterial strain involves taking a sample containing the strain (e.g., from a patient, an animal, a batch of food, etc.) and then sending the sample to an analysis center. When the analysis center receives the sample, it first cultivates the bacterial strain to obtain at least one colony of it, a culture lasting between 24 and 72 hours. It then prepares several samples from this colony containing different antibiotics and / or different antibiotic concentrations, and then incubates the samples again. After a further culture period of between 24 and 72 hours, each sample is manually analyzed to determine whether the antibiotic has worked effectively. The results are then sent back to the practitioner to apply the most effective antibiotic and / or antibiotic concentration.

[0006] However, the labeling process is particularly long and complex to carry out and these chemical markers have a cytotoxic effect on bacteria. It follows that this method of visualization does not allow the observation of bacteria at several times during the bacterial culture, hence the need to use a sufficiently long culture time, of the order of 24 to 72 hours, to guarantee the reliability of the measurement. Other methods of visualizing biological particles use a microscope, allowing non-destructive measurement of a sample.

[0007] Digital holographic microscopy (DHM) is an imaging technique that overcomes the depth-of-field constraints of conventional optical microscopy. It consists of recording a hologram formed by the interference between light waves diffracted by the object under observation and a spatially coherent reference wave. This technique is described in the journal article by Myung K. Kim entitled "Principles and techniques of digital holographic microscopy" published in SPIE Reviews Vol. 1, No. 1, January 2010.

[0008] Recently, it has been proposed to use digital holographic microscopy to identify microorganisms in an automated manner. Thus, international application WO2017 / 207184 describes a method for acquiring a particle integrating a simple acquisition without focusing associated with a digital reconstruction of the focusing, making it possible to observe a biological particle while limiting the acquisition time.

[0009] Typically, this solution makes it possible to detect structural changes in a bacterium in the presence of an antibiotic after an incubation of only ten minutes, and its sensitivity after two hours (detection of the presence or absence of a division or a motif coding for division) unlike the classic process previously described which can take several days. Indeed, since the measurements are non-destructive, it is possible to carry out analyses very early in the culture process without risking destroying the sample and therefore prolonging the analysis time.

[0010] It is even possible to follow a particle over several successive images so as to form a film representing the evolution of a particle over time (since the particles are not altered after the first analysis) in order to visualize its behavior, for example its speed of movement or its cell division process.

[0011] It is therefore clear that the visualization process produces excellent results. The difficulty lies in the interpretation of these images or this film if, for example, one wishes to conclude on the susceptibility of a bacterium to the antibiotic present in the sample, particularly automatically.

[0012] Various techniques have been proposed, ranging from simple counting of bacteria over time to so-called morphological analysis aimed at detecting particular "configurations" through image analysis. For example, when a bacterium prepares to divide, two poles appear in the distribution, well before the division itself, which results in two distinct portions of the distribution.

[0013] It was proposed in the article Choi, J., Yoo, J., Lee, M., et al. (2014). A rapid antimicrobial susceptibility test based on single-cell morphological analysis. Science Translational Medicine, 6(267). https: / / doi.org / 10.1126 / scitranslmed.3009650 to combine the two techniques to assess an antibiotic effect. However, as pointed out by the authors, their approach requires a very fine calibration of a number of thresholds that strongly depend on the nature of the morphological changes caused by antibiotics.

[0014] More recently, the article Yu, H., Jing, W., Iriya, R., et al. (2018). Phenotypic Antimicrobial Susceptibility Testing with Deep Learning Video Microscopy. Analytical Chemistry, 90(10), 6314-6322. https: / / doi.org / 10.1021 / acs.analchem.8b01128 describes an approach based on deep learning. The authors propose to extract morphological features as well as features related to the movement of bacteria using a convolutional neural network (CNN). However, this solution is very heavy in terms of computing resources, and requires a large database of training images to train the CNN. The documents “Embedded neural network system for microorganisms growth analysis” for D. Bliznuks et al., “Performance of convolutional neural networks for identification of bacteria in 3D microscopy datasets” for E. Hay et al.and "Phenotipic Antimicrobial Susceptibility Testing with Deep Learning Video Microscopy" by Hui Yu et al. would also be relevant.

[0015] The objective technical problem of the present invention is, therefore, to be able to have a solution that is both more efficient and lighter for classifying images of a biological particle. PRESENTATION OF THE INVENTION

[0016] According to a first aspect, the present invention relates to a method for classifying a sequence of input images representing a target particle in a sample, the method being characterized in that it comprises the implementation, by data processing means of a client, of steps of: (b) Concatenation of said input images of the sequence in the form of a three-dimensional stack; (c) Direct classification of said three-dimensional stack by means of a convolutional neural network, CNN, said CNN being composed of a succession of convolution blocks composed of a so-called 3D convolution layer, applying to a four-dimensional input feature map four-dimensional filters so as to generate a four-dimensional output feature map, an activation layer and a 3D pooling layer, then a flattening layer, and finally one or more fully connected layers.

[0017] According to advantageous and non-limiting characteristics: The particles are represented in a homogeneous manner in each input image, in particular centered and aligned in a predetermined direction.

[0018] The method comprises a step (a) of extracting from each input image a global image of the sample, so as to represent said target particle in said homogeneous manner.

[0019] Step (a) comprises, for each input image, segmenting said global image so as to detect said target particle in the sample, then cropping the input image onto said detected target particle.

[0020] Step (a) comprises obtaining said global image from an intensity image of the sample acquired by an observation device.

[0021] Said three-dimensional stack has two spatial dimensions and one temporal dimension, said filters and feature maps having as first three dimensions said spatial and temporal dimensions, and as fourth dimension a semantic depth.

[0022] The filters of said 3D convolution layer have a depth equal to the depth of the input feature map, and the output feature map has a depth equal to the number of filters of the 3D convolution layer.

[0023] The said CNN consists of only two convolution blocks.

[0024] The method comprises a step (a0) of learning, by data processing means of a server, the parameters of said classifier from a learning base of already classified sequences of particle images in said sample.

[0025] According to a second aspect, a system for classifying a sequence of input images representing a target particle in a sample over time is proposed, comprising at least one client comprising data processing means, characterized in that said data processing means are configured to implement: the concatenation of said input images of the sequence in the form of a three-dimensional stack; The direct classification of said three-dimensional stack by means of a convolutional neural network, CNN, said CNN being composed of a succession of convolution blocks composed of a so-called 3D convolution layer, applying to a four-dimensional input feature map four-dimensional filters so as to generate a four-dimensional output feature map, an activation layer and a 3D pooling layer, then a flattening layer, and finally one or more fully connected layers.

[0026] According to advantageous and non-limiting characteristics, the system further comprises a device for observing said target particle in the sample.

[0027] According to a third and a fourth aspect there is provided a computer program product comprising code instructions for executing a method according to the first aspect of classifying an input image sequence representing a target particle in a sample; and a storage means readable by a computer equipment on which a computer program product comprises code instructions for executing a method according to the first aspect of classifying an input image sequence representing a target particle in a sample. PRESENTATION OF FIGURES

[0028] Other characteristics and advantages of the present invention will appear on reading the following description of a preferred embodiment. This description will be given with reference to the appended drawings in which: there figure 1 is a diagram of an architecture for implementing the method according to the invention; the figure 2 represents an example of a device for observing particles in a sample used in a preferred embodiment of the method according to the invention; the figure 3a illustrates obtaining the input image in an embodiment of the method according to the invention; the figure 3b illustrates the obtaining of the input image in a preferred embodiment of the method according to the invention; the figure 4 represents the steps of a preferred embodiment of the method according to the invention; the figure 5 represents an example of a convolutional neural network architecture used in a preferred embodiment of the method according to the invention. DETAILED DESCRIPTION Architecture

[0029] The invention relates to a method for classifying a sequence of input images representative of a particle 11a-11f present in a sample 12, called a target particle. Note that the method can be implemented in parallel for all or part of the particles 11a-11f present in a sample 12, each being considered a target particle in turn.

[0030] As we will see, this process includes a machine learning component, and in particular a convolutional neural network, CNN.

[0031] The input or learning data are of image type, and represent the target particle 11a-11f in a sample 12 (in other words, these are images of the sample in which the target particle is visible). Said sequence consists of a plurality of input images of the same target particle 11a-11f over time. As will be seen, it is possible, if necessary, to have as input a plurality of sequences of images of particles 11a-11f of the sample 12 if several particles are considered.

[0032] Sample 12 consists of a liquid such as water, a buffer solution, a culture medium or a reactive medium (including or not including an antibiotic), in which the particles 11a-11f to be observed are located.

[0033] Alternatively, the sample 12 may be in the form of a solid, preferably translucent, medium, such as agar, in which the particles 11a-11f are located. The sample 12 may also be a gaseous medium. The particles 11a-11f may be located within the medium or on the surface of the sample 12.

[0034] The particles 11a-11f can be microorganisms such as bacteria, fungi or yeasts. They can also be cells, multicellular organisms, or any other particle of the polluting particle type, dust. In the remainder of the description, we will take the preferred example in which the particle is a bacterium (and as we will see, sample 12 incorporates an antibiotic). The size of the particles 11a-11f observed varies between 500nm and several hundred µm, or even a few millimeters.

[0035] The “classification” of an input image sequence consists of determining at least one class from a set of possible classes descriptive of the images. For example, in the case of bacteria-type particles, there may be a binary classification, i.e. two possible classes of “division” or “no division” effect, respectively indicating resistance or not to an antibiotic. The present invention will not be limited to any particular type of classification, even if the example of a binary classification of the effect of an antibiotic on said target particle 11a-11f will mainly be described.

[0036] The present methods are implemented within an architecture as represented by the Figure 1 ,using a server 1 and a client 2. Server 1 is the learning equipment (implementing the learning method) and client 2 is a user equipment (implementing the classification method), for example a terminal of a doctor or a hospital.

[0037] It is entirely possible that the two devices 1, 2 are merged, but preferably the server 1 is a remote device, and the client 2 a consumer device, in particular an office computer, a laptop, etc. The client device 2 is advantageously connected to an observation device 10, so as to be able to directly acquire said input image (or as we will see later “raw” acquisition data such as a global image of the sample 12, or even electromagnetic matrices), typically to process it directly, alternatively the input image will be loaded onto the client device 2.

[0038] In all cases, each device 1, 2 is typically a remote computer device connected to a local network or a wide area network such as the Internet network for the exchange of data. Each comprises data processing means 3, 20 of the processor type, and data storage means 4, 21 such as a computer memory, for example a flash memory or a hard disk. The client 2 typically comprises a user interface 22 such as a screen for interaction.

[0039] The server 1 advantageously stores a training database, i.e. a set of sequences of images of particles 11a-11f in various conditions (see below) already classified (for example associated with labels “with division” or “without division” indicating sensitivity or resistance to the antibiotic). Note that the training data may be associated with labels defining the test conditions, for example indicating for bacterial cultures “strains”, “antibiotic conditions”, “time”, etc. Acquisition

[0040] Even if as explained the present method can directly take as input any image of the target particle 11a-11f, obtained in any way. Preferably the present method begins with a step (a) of obtaining the input image from data provided by an observation device 10.

[0041] In a known manner, a person skilled in the art may use DHM digital holographic microscopy techniques, in particular as described in international application WO2017 / 207184. In particular, an intensity image of the sample 12 called a hologram may be acquired, which is not focused on the target particle (this is referred to as an “out-of-focus” image), and which may be processed by data processing means (integrated into the device 10 or those 20 of the client 2 for example, see below). It is understood that the hologram “represents” in a certain way all the particles 11a-11f in the sample.

[0042] There Figure 2 illustrates an example of an observation device 10 of a particle 11a-11f present in a sample 12. The sample 12 is arranged between a light source 15, spatially and temporally coherent (e.g. a laser) or pseudo-coherent (e.g. a light-emitting diode, a laser diode), and a digital sensor 16 sensitive in the spectral range of the light source. Preferably, the light source 15 has a small spectral width, for example less than 200nm, less than 100nm or even less than 25nm. In the following, reference is made to the central emission wavelength of the light source, for example in the visible range. The light source 15 emits a coherent signal Sn oriented on a first face 13 of the sample, for example conveyed by a waveguide such as an optical fiber.

[0043] The sample 12 (as typically explained a culture medium) is contained in an analysis chamber, delimited vertically by a lower slide and an upper slide, for example conventional microscope slides. The analysis chamber is delimited laterally by an adhesive or any other waterproof material. The lower and upper slides are transparent to the wavelength of the light source 15, the sample and the chamber allowing for example more than 50% of the wavelength of the light source to pass under normal incidence on the lower slide.

[0044] Preferably, the particles 11a-11f are arranged in the sample 12 at the level of the upper blade. The lower face of the upper blade comprises for this purpose ligands allowing the particles to be attached, for example polycations (e.g. poly-Llysine) in the context of microorganisms. This makes it possible to contain the particles in a thickness equal to, or close to, the depth of field of the optical system, namely in a thickness less than 1 mm (e.g. tube lens), and preferably less than 100 µm (e.g. microscope objective). The particles 11a-11f can nevertheless move in the sample 12.

[0045] Preferably, the device comprises an optical system 23 consisting, for example, of a microscope objective and a tube lens, arranged in the air and at a fixed distance from the sample. The optical system 23 is optionally equipped with a filter which can be located in front of the objective or between the objective and the tube lens. The optical system 23 is characterized by its optical axis, its object plane, also called the focusing plane, at a distance from the objective, and its image plane, conjugated to the object plane by the optical system. In other words, an object located in the object plane corresponds to a sharp image of this object in the image plane, also called the focal plane. The optical properties of the system 23 are fixed (e.g. fixed focal length optics). The object and image planes are orthogonal to the optical axis.

[0046] The image sensor 16 is located, opposite a second face 14 of the sample, in the focal plane or close to the latter. The sensor, for example a CCD or CMOS sensor, comprises a periodic two-dimensional array of sensitive elementary sites, and proximity electronics which regulate the exposure time and the resetting of the sites, in a manner known per se. The output signal of an elementary site is a function of the quantity of radiation of the spectral range incident on said site during the exposure time. This signal is then converted, for example by the proximity electronics, into an image point, or "pixel", of a digital image. The sensor thus produces a digital image in the form of a matrix with C columns and L rows.Each pixel of this matrix, with coordinates (c, l) in the matrix, corresponds in a manner known per se to a position with Cartesian coordinates (x(c, l), y(c, l)) in the focal plane of the optical system 23, for example the position of the center of the elementary sensitive site of rectangular shape.

[0047] The pitch and fill factor of the periodic grating are chosen to respect the Shannon-Nyquist criterion with respect to the size of the particles observed, so as to define at least two pixels per particle. Thus, the image sensor 16 acquires a transmission image of the sample in the spectral range of the light source.

[0048] The image acquired by the image sensor 16 includes holographic information insofar as it results from the interference between a wave diffracted by the particles 11a-11f and a reference wave having passed through the sample without having interacted with it. It is obviously understood, as described above, that in the context of a CMOS or CCD sensor, the digital image acquired is an intensity image, the phase information therefore being coded here in this intensity image.

[0049] Alternatively, it is possible to divide the coherent signal Sn from the light source 15 into two components, for example by means of a semi-transparent plate. The first component then serves as a reference wave and the second component is diffracted by the sample 12, the image in the image plane of the optical system 23 resulting from the interference between the diffracted wave and the reference wave.

[0050] In reference to the Figure 3a ,it is possible in step (a) to reconstruct from the hologram a plurality of global images of the sample 12, then to extract each input image from a global image of the sample.

[0051] It is understood that the target particle 11a-11f must be represented in a homogeneous manner in each input image, in particular centered and aligned in a predetermined direction (for example the horizontal direction). The input images must also have a standardized size (it is also desirable that only the target particle 11a-11f be seen in the input image). The input image is thus called a "thumbnail" (in English thumbnail); for example, a size of 250x250 pixels can be defined. To the extent that a sequence of input images is desired, for example, one image per minute is taken for a time interval of 120 minutes; thus, a sequence of 120 input images is obtained.

[0052] The reconstruction of each global image is implemented as explained by data processing means of the device 10 or those 20 of the client 2.

[0053] Typically, a series of complex matrices called “electromagnetic matrices” are constructed (for an acquisition instant), modeling from the intensity image of the sample 12 (the hologram) the light wavefront propagated along the optical axis for a plurality of deviations from the focusing plane of the optical system 23, and in particular deviations positioned in the sample.

[0054] These matrices can be projected into real space (e.g. via the Hermitian norm), so as to constitute a stack of global images at various focusing distances.

[0055] From there, an average focusing distance can be determined (and the corresponding global image selected, or recalculated from the hologram), or even an optimal focusing distance for the target particle can be determined (and the corresponding global image selected again, or recalculated from the hologram).

[0056] In any case, with reference to the Figure 3b , step (a) advantageously comprises segmenting said global images so as to detect said target particle in the sample, then cropping. In particular, each input image can be extracted from one of the global images of the sample, so as to represent said target particle in said homogeneous manner.

[0057] Typically, segmentation detects all particles of interest, removing artifacts such as filaments or microcolonies, so as to improve the overall image(s), then one of the detected particles is selected as the target particle, and the corresponding thumbnail is extracted. As explained, this work can be done for all detected particles.

[0058] Segmentation can be implemented in any known way. In the example of the figure 3b , we start with a fine segmentation to eliminate artifacts, then we implement a less fine segmentation to this time detect particles 11a-11f. Those skilled in the art can use any known segmentation technique.

[0059] To obtain the input image sequence for a target particle 11a-11f, tracking techniques can be implemented to follow any movements of the particle from one global image to the next.

[0060] Note that all the input images obtained for a sample (for several or even all the particles of sample 12, and this over time) can be pooled to form a descriptive base of sample 12 (in other words a descriptive base of the experiment), as seen on the right of the figure 3a , notably copied onto the storage means 21 of the client 2. We speak of the “field” level, as opposed to the “particle” level. For example, if the particles 11a-11f are bacteria and the sample 12 contains (or not an antibiotic), this descriptive base contains all the information on the growth, morphology, internal structure and optical properties of these bacteria over the entire acquisition field. As we will see, this descriptive base can be transmitted to the server 1 for integration into said learning base. Stacking

[0061] As we will see, the present method is distinguished by the fact that it can work directly on a sequence of input images, without the need to work image by image, nor to extract feature maps in an intermediate manner. In addition, we will see that a very simple and lightweight CNN is sufficient to achieve a reliable and efficient classification.

[0062] In reference to the Figure 4 , the present method comprises a step (b) of concatenating said input images of the sequence in the form of a three-dimensional stack, in other words a 3D “stack”. More precisely, the input images all have the same size and form a sequence of matrices, it is therefore sufficient to stack them according to the order of the input images so as to obtain the three-dimensional stack.

[0063] This three-dimensional stack can therefore be seen as an image presenting as many channels as there are acquisition time instants, even if, as we will see later, the present method will treat this stack in a very original way as a single three-dimensional object (for example, of size 250x250x120 if we have input images of size 250x250 and one image acquired per minute for 120 minutes) with a single channel, in the same way that an RGB image is a two-dimensional object with three channels. The first two dimensions are classically the spatial dimensions (i.e. the size of the input images) and the third dimension is the "temporal" dimension (acquisition time).

[0064] Preferably, step (b) comprises subsampling said three-dimensional stack, i.e. reducing the size of the input.

[0065] Said subsampling can be implemented on the temporal dimension of the stack and / or its spatial dimensions, preferably both.

[0066] Especially : As for the time dimension, it can be reduced by dividing the acquisition period into n intervals and selecting the n + 1images of the sequence corresponding to the ends of these intervals, for example by keeping only 5 images out of a sequence of 120 images, which amounts to taking an image every 120 / 4 = 30 minutes (in particular the images acquired after 1, 30, 60, 90 and 120 minutes). However, it remains possible to select the images in a non-regular manner (for example by selecting more images at the beginning of the acquisition period than at the end). As for the spatial dimensions, they can be reduced with any image subsampling technique with a given sampling factor (on each axis to keep the proportions), for example a factor of 2.

[0067] By implementing the two subsamplings above in combination, we go from a stack of size 250x250x120 to a stack of size 125x125x5 (we reduce the size of the stack by almost a factor of 100).

[0068] Note that this subsampling can be done upstream of the generation of the stack (by selecting and modifying the input images). Classification

[0069] In a step (c), said three-dimensional stack is classified directly by means of an adapted convolutional neural network, called "3D CNN" because of its ability to process three-dimensional objects that are stacks. It is indeed important to understand, as mentioned, that the three-dimensional stack is processed by the CNN as a single three-dimensional object (i.e. with a single channel), and not as a two-dimensional object with several channels (as is, for example, an RGB image).

[0070] By direct, or "end-to-end" classification, we mean without separate extraction of at least one feature map of said target particle 11a-11f: we understand that the CNN naturally has internal states in the form of feature maps, but the latter are never sent outside the CNN, the latter having as its only output the result of the classification.

[0071] It is recalled that in general, CNNs are particularly suitable for vision tasks and more particularly for image classification. Generally, a CNN uses a plurality of convolution layers, and the present 3D CNN uses at least one 3D convolution layer which models the spatio-temporal dependence between the different input images.

[0072] A 3D convolution layer is a convolution layer that applies four-dimensional filters, and is thus able to work on multiple channels of already three-dimensional stacks, i.e., a four-dimensional feature map. In other words, the 3D convolution layer applies four-dimensional filters to a four-dimensional input feature map so as to generate a four-dimensional output feature map. The fourth and final dimension is semantic depth, as in any feature map.

[0073] This is to be differentiated from classic convolution layers which are only able to work on three-dimensional feature maps representing several channels of two-dimensional objects (images).

[0074] This notion of 3D convolution may seem counter-intuitive, but it generalizes the notion of a convolution layer which only provides that we apply a plurality of "filters" of a depth equal to the number of channels of the input (i.e. the depth of the input feature map), by scanning them over all the dimensions of the input (in 2D for an image), the number of filters defining the output depth.

[0075] Our 3D convolution therefore applies four-dimensional filters of depth equal to the number of channels of three-dimensional stacks in input, and sweeps these filters over the entire volume of a three-dimensional stack, therefore the two spatial dimensions but also the temporal dimension, i.e. in 3D (hence the name 3D convolution). We thus obtain a three-dimensional stack per filter, i.e. a four-dimensional feature map. In a classic convolution layer, using a large number of filters certainly increases the semantic depth at the output (the number of channels), but we will always have a three-dimensional feature map.

[0076] We understand that a 3D convolution layer remains heavier and requires more computing power, but as explained, a very simple architecture (and much simpler than known CNNs such as VGG616) is sufficient. Figure 5 represents the architecture of an embodiment of the present 3D CNN.

[0077] Conventionally, this architecture advantageously comprises a succession of so-called convolution blocks composed of a 3D convolution layer, an activation layer (for example the ReLU function) to increase the depth of the feature maps, and a 3D pooling layer to reduce the size of the feature map (generally by a factor of 2). What is remarkable is that two convolution blocks may be sufficient, so that very preferably the present 3D CNN only comprises two convolution blocks.

[0078] By 3D pooling layer we mean, as for 3D convolution, a layer capable of working on four-dimensional feature maps, having one or more channels of already three-dimensional stacks. In other words, the size reduction is on all dimensions of a three-dimensional stack, i.e. the first three dimensions of a four-dimensional feature map.

[0079] In the remainder of this description, a distinction will be made between the number of "dimensions" of the feature maps, in the geometric sense, i.e. the number of independent directions in which these maps extend (for example, a vector is an object of dimension 1, an image is of dimension 2, and the present feature maps are of dimension 4), and the number of "variables" of these feature maps, i.e. the size according to each dimension, i.e. the number of independent degrees of freedom (which corresponds in practice to the notion of dimension in a vector space - more precisely, the set of feature maps having a given number of variables constitutes a vector space of dimension equal to this number of variables).

[0080] So in the example of the figure 5 , the 3D CNN starts as explained by 6 layers divided into 2 blocks. The first takes as input the three-dimensional stack with a single channel (thus forming an object of size 125x125x5x1 when it is advantageously downsampled as proposed above), and includes a convolution+ReLU sequence (a first 3D convolution layer, and an activation layer with ReLU function) raising the depth to 30 then a max pooling layer (we can also use global average pooling), with as output a feature map of size 62x62x2x30 (the 3D pooling layer works as explained on the three dimensions and not only the two spatial dimensions - so we divide by two including the temporal dimension).

[0081] In the example shown, the first 3D convolution layer uses 30 filters of dimension 3x3x3x1, and thus requires ((3*3*3*1)+1)*30=570 parameters.

[0082] The second block has an identical architecture to the first block and generates at the output of a new convolution+ReLU ensemble (a second 3D convolution layer, and an activation layer with ReLU function) a feature map of size 62x62x2x60 (doubled depth) and at the output of the max pooling layer a feature map of size 12x12x1x60 (we note that the spatial size reduction is this time by a factor of five, but the temporal size reduction is still by a factor of two).

[0083] This time, the second 3D convolution layer uses 60 filters of dimension 3x3x3x30, and thus requires ((3*3*3*30)+1)*60=32460 parameters.

[0084] At the output of the last convolution block (in this case the second), the 3D CNN advantageously includes a "flattening" layer transforming the "final" feature map (containing the "deepest" information) at the output of this block into a vector (object of dimension 1). Thus, for example, we go from the feature map of size 12x12x1x60 to a vector of size 12*12*1*60=8640. It will be understood that we are limited to no map / filter sizes at any level, and that the sizes mentioned above are only examples.

[0085] Finally, in a classic way, we finish with one or more fully connected layers (FC, or “dense” layers as indicated in the figure 5 ) and possibly a final activation layer, for example softmax. In the example shown, a first FC layer transforms the vector of size 8640 into a reduced vector of size 100 (which requires (8640+1)*100=864100 parameters), and a second FC layer transforms the vector of size 8640 into a final vector of size 2 (which requires (100+1)*2=202 parameters).

[0086] Preferably, the 3D CNN is composed of (i.e., includes exactly) a sequence of convolution blocks, then a flattening layer, and finally one or more fully connected layers.

[0087] This final part of the 3D CNN returns the expected result, in this case the class of the input image sequence (the vector of size 2 corresponds to a binary result).

[0088] We can therefore see that we have a total number of parameters less than 900,000, which is remarkably low for a CNN (we commonly have several tens of millions of parameters), particularly given that the input data are already very large image sequences. This 3D CNN can therefore be used by many clients 2, including those with modest computing resources.

[0089] Preferably, the method may comprise a step (a0) of learning, by the data processing means 3 of the server 1, from a learning base, the parameters of the 3D CNN. This step is in fact typically implemented very early on, in particular by the remote server 1. As explained, the learning base may comprise a certain number of learning data, in particular sequences of images associated with their class (for example “division” or “no division” if binary classification).

[0090] 3D CNN training can be performed in a classical way. The learning cost function can be composed of a classical data attachment - cross entropy - to be minimized via a gradient descent algorithm.

[0091] In all embodiments, the parameters of the learned CNN can be stored, if necessary, on data storage means 21 of the client 2 for use in classification. Note that the same CNN can be embedded on many clients 2, only one learning is necessary. Produit programme d'ordinateur

[0092] According to a second and a third aspect, the invention relates to a computer program product comprising code instructions for the execution (in particular on the data processing means 3, 20 of the server 1 and / or the client 2) of a method for classifying an input image sequence representing a target particle 11a-11f in a sample 12, as well as storage means readable by computer equipment (a memory 4, 21 of the server 1 and / or the client 2) on which this computer program product is found.

Claims

1. A method for classifying a sequence of input images representing a target particle (11a-11f) in a sample (12) over time, the method being characterised in that it comprises the implementation, by data processing means (20) of a client (2), the following steps: (b) concatenating said input images of the sequence in the form of a three-dimensional stack; (c) Direct classifying said three-dimensional stack by means of a convolution neural network, CNN, said CNN being composed of a succession of convolution blocks composed of a so-called 3D convolution layer, applying to an input feature map of dimension four filters of dimension four so as to generate an output feature map of dimension four, an activation layer and a 3D pooling layer, then a flattening layer, and finally one or more fully connected layers, the succession of convolution blocks comprising at least two convolution blocks, including a first block and a second block, the 3D convolution layer of the first block applying 30 filters of dimension 3x3x3x1 and the 3D convolution layer of the second block applying 60 filters of dimension 3x3x3x30.

2. The method according to claim 1, wherein the particles (11a-11f) are represented in a homogeneous manner in each input image, in particular centred and aligned in a predetermined direction.

3. The method according to claim 2, comprising a step (a) of extracting from each input image a global image of the sample, so as to represent said target particle (11a-11f) in said homogeneous manner4. The method according to claim 3, wherein said extraction step (a) comprises, for each input image, segmenting said overall image so as to detect said target particle (11a-11f) in the sample (12), and then cropping the input image onto said detected target particle (11a-11f).

5. The method according to one of claims 3 and 4, wherein step (a) comprises obtaining said global image from an intensity image of the sample (12) acquired by an observation device (10).

6. The method according to one of the preceding claims, wherein said three-dimensional stack has two spatial dimensions and one temporal dimension, said filters and feature maps having said spatial and temporal dimensions as the first three dimensions, and semantic depth as the fourth dimension.

7. The method according to claim 6, wherein the filters of said 3D convolution layer have a depth equal to the depth of the input feature map, and the output feature map has a depth equal to the number of filters of the 3D convolution layer.

8. The method according to one of the preceding claims, wherein said CNN comprises only two convolution blocks.

9. The method according to one of claims 1 to 8, comprising a step (a0) of learning, by data processing means (3) of a server (1), the parameters of said CNN from a learning base of already classified sequences of images of particles (11a-11f) in said sample (12).

10. The system for classifying a sequence of input images representing a target particle (11a-11f) in a sample (12) over time, comprising at least one client (2) comprising data processing means (20), characterised in that said data processing means (20) are configured to implement: - concatenating the said input images of the sequence in the form of a three-dimensional stack; - Direct classifying of the said three-dimensional stack by means of a convolution neural network, CNN, the said CNN being composed of a succession of convolution blocks made up of a so-called 3D convolution layer, applying to an input feature map of dimension four filters of dimension four so as to generate an output feature map of dimension four, an activation layer and a 3D pooling layer, then a flattening layer, and finally one or more fully connected layers, the succession of convolution blocks comprising at least two convolution blocks, including a first block and a second block, the 3D convolution layer of the first block applying 30 filters of dimension 3x3x3x1 and the 3D convolution layer of the second block applying 60 filters of dimension 3x3x3x30.

11. The system according to claim 10, further comprising an observation device (10) observing said target particle (11a-11f) in the sample (12).

12. Computer program product comprising code instructions for performing a method according to any one of claims 1 to 9 of classifying a sequence of input images representing a target particle (11a-11f) in a sample (12) over time, when said program is executed on a computer.

13. Computer-readable storage medium on which a computer program product comprises code instructions for executing a method according to any one of claims 1 to 9 of classifying a sequence of input images representing a target particle (11a-11f) in a sample (12) over time.