Sperm characterization process

Abstract

A method for characterizing a mobile particle in a sample (10), the method comprising: acquiring at least one image () of the sample during an acquisition period, using an image sensor (20) and forming a series of images, the series of images comprising at least one image; using each image of the series of images resulting from a) as an input image () of a convolutional neural network for detection (CNNd), the convolutional neural network for detection being configured to detect the particles and to produce, from each image, an output image () on which each detected particle is assigned an intensity distribution, centered on the particle and extending around the particle; for each detected particle, from each output image () resulting from b), estimating a position of each detected particle in each image of the series of images;characterization of each detected particle from the estimate of the position resulting from c), established from each image of the series of images.;

Description

Title of the invention: Method for characterizing spermatozoa Technical field

[0001] The technical field of the invention is the observation of mobile or motile microscopic particles in a sample, with a view to their characterization. One intended application is the characterization of spermatozoa. PREVIOUS ART

[0002] The observation of motile cellular particles, such as spermatozoa, in a sample is usually carried out using a microscope. The microscope comprises an objective defining an object plane, extending into the sample, as well as an image plane, merged with a detection plane of an image sensor. The microscope produces images of the spermatozoa according to a focused configuration. The choice of such a modality involves a compromise between spatial resolution, the observed field and the depth of field. The higher the numerical aperture of the objective, the better the spatial resolution, to the detriment of the size of the observed field. Similarly, a high numerical aperture decreases the depth of field.

[0003] It is understood that the observation of motile particles requires optimization of the objective, knowing that, in such a configuration, it is not possible to obtain good spatial resolution, a wide observed field and a large depth of field at the same time. However, these properties are particularly important when observing motile microscopic particles, especially when there are many of them: - the size of the particles requires good spatial resolution; - their number justifies an extended observed field, so as to be able to maximize the number of particles observed on the same image; - their movement requires a significant depth of field, so that the particles appear clearly in the image.

[0004] Given these requirements, it is usual to use a high magnification microscope. The small size of the observed field is compensated for by using a translation stage. The latter allows image acquisition by moving the objective relative to the sample, parallel to the latter. The small depth of field is compensated for by limiting the thickness of the sample: the latter is for example placed in a fluidic chamber of small thickness, typically less than 20 μm, so as to limit the movement of the particles in a direction perpendicular to the object plane. Furthermore, the objective can be moved closer to or further from the sample, so as to move the object plane in the sample, depending on its thickness. The result is a complex and expensive device, requiring precise movement of the lens.

[0005] An alternative to conventional microscopy has been proposed by lensless imaging. It is known that lensless imaging, coupled with holographic reconstruction algorithms, allows observation of cells while maintaining a high field of observation, as well as a large depth of field. Patents US9588037 or US8842901 describe, for example, the use of lensless imaging for the observation of spermatozoa. Patents US10481076 or US10379027 also describe the use of lensless imaging, coupled with reconstruction algorithms, to characterize cells.

[0006] It is known that the use of digital reconstruction algorithms makes it possible to obtain clear images of particles. Such algorithms are for example described in US10564602, US20190101484 or US20200124586. In this type of algorithm, from a hologram acquired in a detection plane, an image of the sample is reconstructed in a reconstruction plane, distant from the detection plane. It is usual for the reconstruction plane to extend across the sample. However, this type of algorithm can require a relatively long calculation time.

[0007] Such a constraint is acceptable when the particles are considered to be immobile in the sample. However, when one wishes to characterize mobile particles, and in particular motile particles, the computation time can become too long. Indeed, the characterization of mobile particles requires the acquisition of several images, at high frequencies, in order to be able to characterize the movement of the particles in the sample.

[0008] The inventors propose an alternative to the previously cited patents, making it possible to characterize motile particles, and in particular spermatozoa, using a simple observation method. The method designed by the inventors allows the characterization of a large number of particles without requiring movement of an objective relative to the sample. Statement of the invention

[0009] A first object of the invention is a method for characterizing at least one mobile particle in a sample, the method comprising: a. acquiring at least one image of the sample during an acquisition period, using an image sensor and forming a series of images from the acquired images; b. using each image of the series of images resulting from a) as an input image to a convolutional detection neural network, the convolutional detection neural network being configured to detect the particles and to produce, from each image, an output image on which each detected particle is assigned an intensity distribution centered on the particle and extending around the particle; c. for each detected particle, from each output image resulting from b), estimating a position of each detected particle in each image of the series of images; d. characterization of each detected particle from the position estimate resulting from c), established from each image of the series of images.

[0010] According to one possibility, steps a) to d) can be carried out with a single image. In this case, the series of images comprises a single image.

[0011] According to one possibility, on the output image, each particle can be represented in the form of a point.

[0012] The particle (or each particle) may in particular be a spermatozoon.

[0013] Step d) may include a characterization, in particular morphological, of each spermatozoon detected. Step d) then includes: - for each of the detected spermatozoon, from each image resulting from a), and the positions resulting from c), extraction of a thumbnail comprising the detected spermatozoon, the position of the detected spermatozoon in the thumbnail being predetermined, so as to obtain, for each detected spermatozoon, a series of thumbnails, the size of each thumbnail being less than the size of each image acquired during step a); - for each detected spermatozoon, using the series of thumbnails as input data to a classification neural network, the classification neural network being configured to classify the spermatozoon among predetermined classes. These may in particular be morphological classes.

[0014] The method may be such that each detected spermatozoon is centered relative to each thumbnail.

[0015] Step d) may include a characterization of the motility of the spermatozoon. Step d) may then include, from the positions of the spermatozoon resulting from step c): - a determination of a sperm trajectory during the acquisition period; - a calculation of the sperm speed from the trajectory; - a classification of sperm according to speed.

[0016] The method may comprise: - a determination of a distance, in a straight line, between a first point and a last point of the trajectory, and a calculation of a linear trajectory speed from said distance; - and / or a determination of distances traveled between each acquired image, and a calculation of a speed of the curvilinear trajectory from said distances; - and / or a determination of a smoothed trajectory, and a calculation of an average trajectory speed from the smoothed trajectory.

[0017] According to one embodiment, - the sample extends along a sample plane; - the image sensor extends along a detection plane; - an optical system extends between the sample and the image sensor, the optical system defining an object plane and an image plane; - the object plane is offset from the sample plane by an object defocus distance and / or the image plane is offset from the sample plane by an image defocus distance.

[0018] In such an embodiment, the method may be such that: - the sample is arranged on a sample holder, resting on at least one spring, the spring being configured to push the sample holder towards the optical system; - the optical system is connected to at least one stop, extending from the optical system towards the sample holder; - such that during step a), under the effect of the spring, the sample support rests on the stop.

[0019] According to one embodiment, no imaging optics extend between the sample and the image sensor.

[0020] According to one embodiment, step a) comprises a normalization of each acquired image by an average of said image or by an average of images from the series of images. In this case, the series of images is produced from each acquired image, after normalization.

[0021] According to one embodiment, step a) comprises applying a high-pass filter to each acquired image. In this case, the series of images is produced from each acquired image, after applying the high-pass filter.

[0022] Advantageously, during step b), the intensity distribution assigned to each particle is decreasing, such that the intensity decreases as a function of the distance from the particle.

[0023] Advantageously, during step b), the intensity distribution assigned to each particle can be a two-dimensional parametric statistical distribution.

[0024] A second object of the invention is a device for observing a sample, the sample comprising mobile particles, the device comprising: - a light source, configured to illuminate the sample; - an image sensor, configured to form an image of the sample; - a holding structure, configured to hold the sample between the light source and the image sensor; - a processing unit, connected to the image sensor, and configured to implement steps a) to d) of a method according to the first subject of the invention from at least one image acquired by the image sensor.

[0025] According to one embodiment, no imaging optics extend between the image sensor and the sample. The holding structure may be configured to maintain a fixed distance between the image sensor and the sample.

[0026] According to one embodiment, - the image sensor extends along a detection plane; - the device comprises an optical system, extending between the image sensor and the support plane, the optical system defining an image plane and an object plane, the device being such that: • the image plane is offset from the detection plane by an image defocusing distance; • and / or the support plane and offset from the object plane by an object defocusing distance.

[0027] The invention will be better understood upon reading the description of the exemplary embodiments presented in the remainder of the description, in conjunction with the figures listed below. FIGURES

[0028] [Fig. 1 A] [Fig. 1 A] represents a first embodiment of the device allowing implementation of the invention.

[0029] [Fig.lB] [Fig.lB] is a three-dimensional view of the device shown schematically in the [Fig.lA],

[0030] [Fig. IC] [Fig. IC] shows a setup for holding the sample at a fixed distance from an optical system.

[0031] [Fig.lD] [Fig.lD] shows the assembly shown in [Fig.lC] in an observation position.

[0032] [Fig.2] [Fig.2] represents a second embodiment of a device allowing implementation of the invention.

[0033] [Fig.3] [Fig.3] shows the main steps of a method for characterizing mobile particles in the sample.

[0034] [Fig.4] Figures 4A, 4B and 4C respectively represent an acquired image, a reference image and an image resulting from the detection neural network.

[0035] [Fig.5] Figure 5A is an example of an image of a sample obtained by putting implements a device according to the first embodiment of the invention. Figure 5B shows the image represented in Figure 5A after processing by a detection neural network. Figure 5C shows an example of obtaining trajectories of particles detected in image 5B.

[0036] [Fig.6] [Fig.6] shows thumbnails centered on the same spermatozoon, the thumbnails being extracted from a series of nine images acquired using a defocused imaging modality.

[0037] [Fig.7] Figures 7A, 7B and 7C respectively represent an acquired image, a reference image and an image resulting from the detection neural network.

[0038] [Fig.8] Figures 8A, 8B and 8C respectively represent an acquired image, a reference image and an image resulting from the detection neural network. In this series of images, Figure 8A has been processed by a high-pass filter.

[0039] [Fig.9A]

[0040] [Fig.9B] Figures 9A and 9B represent particle detection performances by the detection neural network, under different conditions. The performances are sensitivity ([Fig.9A]) and specificity ([Fig.9B]).

[0041] [Fig.10A]

[0042] [Fig.lOB]

[0043] [Fig.10C] Figures 10A to 10C represent confusion matrices, expressing the sperm classification performance, using images acquired according to the defocused modality, the classification being carried out respectively: - by implementing a holographic reconstruction algorithm from an image obtained by holographic reconstruction and processed by a neural network; - by implementing a classification neural network whose input layer comprises a single image acquired according to a defocused configuration, without holographic reconstruction; - by implementing a classification neural network, whose input layer comprises a series of images of a particle, each image being acquired in a defocused configuration, without holographic reconstruction. PRESENTATION OF SPECIAL EMBODIMENTS

[0044] [Fig. 1A] shows a first embodiment of a device 1 allowing an implementation of the invention. According to this first embodiment, the device allows the observation of a sample 10 interposed between a source light source 11 and an image sensor 20. The light source 11 is configured to emit an incident light wave 12 propagating to the sample parallel to a propagation axis Z.

[0045] The device comprises a sample holder 10s configured to receive the sample 10, such that the sample is held on the holder 10s. The sample thus held extends along a plane, called the sample plane P10. The plane of the sample corresponds for example to a mean plane around which the sample 10 extends. The sample holder may be a glass slide, for example 1 mm thick.

[0046] The sample comprises in particular a liquid medium 10m in which mobile and possibly motile particles 10; are bathed. The medium 10m may be a biological liquid or a buffer liquid. It may for example comprise a bodily fluid, in the pure or diluted state. By bodily fluid is meant a liquid generated by a living body. It may in particular be, without limitation, blood, urine, cerebrospinal fluid, sperm, lymph.

[0047] The sample 10 is preferably contained in a fluidic chamber 10c. The fluidic chamber is for example a fluidic chamber with a thickness of between 20 μm and 100 μm. The thickness of the fluidic chamber, and therefore of the sample 10, along the propagation axis Z, typically varies between 10 μm and 200 μm, and is preferably between 20 μm and 50 μm.

[0048] One of the objectives of the invention is the characterization of moving particles in the sample. In the embodiment described, the moving particles are spermatozoa. In this case, the sample comprises sperm, possibly diluted. In this case, the fluidic chamber 10c may be a counting chamber dedicated to the analysis of the mobility or concentration of cells. It may, for example, be a counting chamber marketed by Leja, with a thickness of between 20 μm and 100 μm.

[0049] According to other applications, the sample comprises mobile particles, for example microorganisms, for example microalgae or plankton, or cells, for example cells undergoing sedimentation.

[0050] The distance D between the light source 11 and the sample 10 is preferably greater than 1 cm. It is preferably between 2 and 30 cm. Advantageously, the light source 11, seen by the sample, is considered to be point-like. This means that its diameter (or its diagonal) is preferably less than one tenth, better still one hundredth of the distance between the sample and the light source.

[0051] The light source 11 is for example a light-emitting diode. It is preferably associated with a diaphragm 14, or spatial filter. The opening of the diaphragm is typically between 5 pm and 1 mm, preferably between 50 pm and 1 mm. In this example, the diaphragm has a diameter of 400 pm. According to another configuration, the diaphragm can be replaced by an optical fiber, a first end of which is placed facing the light source and a second end of which is placed opposite the sample 10. The device can also comprise a diffuser 13, arranged between the light source 13 and the diaphragm 14. The use of a diffuser / diaphragm assembly is for example described in US10418399

[0052] The image sensor 20 is configured to form an image of the sample according to a detection plane P20. In the example shown, the image sensor 20 comprises a pixel matrix, of the CCD or CMOS type. The detection plane P20 preferably extends perpendicular to the propagation axis Z. Preferably, the image sensor has a large sensitive surface, typically greater than 10 mm2. In this example, the image sensor is an IDS-UL3160CP-M-GL sensor comprising pixels of 4.8x4.8 pm2, the sensitive surface being 9.2 mm x 5.76 mm, or 53 mm2.

[0053] In the example shown in [Fig.lA], the image sensor 20 is optically coupled to the sample 10 by an optical system 15. In the example shown, the optical system comprises an objective 15i and a tube lens 152. The latter is intended to project a formed image onto the sensitive surface of the image sensor 20 (surface of 53 mm2).

[0054] In this example: - the 15i objective is a Motic CCIS EF-N Plan Achromat lOx objective, with a numerical aperture of 0.25; - lens 152 is a Thorlabs LBF254-075-A lens - focal length 75 mm.

[0055] Such an assembly provides a field of observation of 3 mm2, with a spatial resolution of 1 pm. The short focal length of the lens makes it possible to optimize the size of the device 1 and to adjust the magnification to the size of the image sensor.

[0056] The image sensor is configured to acquire images of the sample, according to an acquisition frequency of a few tens of images per second, for example 60 images per second. The sampling frequency is typically between 5 and 100 images per second.

[0057] The optical system 15 defines an object plane Po and an image plane P;. In the embodiment shown in [Fig.lA], the image sensor 20 is configured to acquire an image according to a defocused configuration. The image plane P; coincides with the detection plane P20, while the object plane Po is offset by an object focusing distance δ of between 10 pm and 500 pm, relative to the sample. The focusing distance is preferably between 50 pm and 100 pm, for example. example 70 pm. The object plane Po extends outside the sample 10. According to another possibility, the object plane extends into the sample, while the image plane is offset from the detection plane by an image defocusing distance. The image focusing distance is preferably between 50 pm and 100 pm, for example 70 pm. According to another possibility, the object plane Po and the image plane P; are both offset respectively from the sample plane and from the detection plane. Whatever the configuration chosen, the defocusing distance is preferably greater than 10 pm and less than 1 mm, or even 500 pm, and preferably between 50 pm and 150 pm. The observation of a cellular sample according to a defocused configuration has been described in US patent 10545329.

[0058] According to such a method, the image sensor 20 is exposed to a light wave, called the exposure light wave. The image acquired by the image sensor comprises interference patterns, which may also be designated by the term "diffraction patterns", formed by: - a part of the light wave 12 emitted by the light source 11, and having passed through the sample without interacting with the latter; - diffraction waves, formed by the diffraction of a part of the light wave 12 emitted by the light source in the sample.

[0059] A processing unit 30, comprising for example a microprocessor, is capable of processing each image acquired by the image sensor 20. In particular, the processing unit comprises a programmable memory 31 in which is stored a sequence of instructions for carrying out the image processing and calculation operations described in this description. The processing unit 30 can be coupled to a screen 32 allowing the display of images acquired by the image sensor 20 or resulting from the processing carried out by the processing unit 30.

[0060] The image acquired by the image sensor 20, according to a defocused imaging modality, is a diffraction pattern of the sample, sometimes called a hologram. It does not make it possible to obtain a precise representation of the observed sample. Usually in the field of holography, a holographic reconstruction operator can be applied to each image acquired by the image sensor so as to calculate a complex expression representative of the light wave to which the image sensor is exposed, and this at any point of coordinates (x, y, z) in space, and in particular in a reconstruction plane corresponding to the plane of the sample. The complex expression makes it possible to obtain the intensity or the phase of the exposure light wave. Such a holographic reconstruction is described in connection with the prior art, as well as in US 10545329.

[0061] However, for reasons of processing speed, the inventors followed a different approach from that suggested by the prior art, without recourse to a holographic reconstruction algorithm applied to the images acquired by the image sensor. The method implemented by the sensor is described subsequently, in connection with [Fig.3].

[0062] [Fig. 1B] is a representation of an example of a device as shown diagrammatically in [Fig. 1A]. Figures 1C and 1D represent a detail of the arrangement of the sample 10 facing the optical system 15, and more precisely facing the objective 15i. The sample support 10s is connected to elastic return means 16a, for example springs. The springs tend to bring the sample support 10s closer to the objective 15i. The device also comprises stops 16b, integral with the objective 15i. The objective 15i and the stops 16b are fixed relative to the image sensor 20. The stops 16b and the return means 16a form a holding structure 16, intended to hold the sample between the light source 11 and the image sensor 20.

[0063] Figures 1C and 1D show the holding of the sample on the sample support respectively during the placement of the sample and during its observation. A rigid connection 16c connects the objective 15i to the stops 16b. During the placement of the sample ([Fig.1C]), the springs 16a are compressed. During the observation of the sample ([Fig.1D]), under the effect of a relaxation of the springs 16a, the sample support 10s comes to bear against the stops 16b. This makes it possible to control the distance A between the objective 15i and the sample 10, independently of the thickness of the sample support 10s. This makes it possible to avoid variations in the thickness of the sample support. For example, when the latter is a 1mm thick glass slide, the thickness can fluctuate over a relatively large range, for example ±100 pm.The assembly described in connection with figures IC and 1D makes it possible to control the distance A between the objective 15i and the sample 10, to within ± 5 pm, independently of such fluctuations in the thickness of the slide 10s. The inventors believe that such an assembly makes it possible to avoid the use of an autofocus system to produce images.

[0064] In Figures 1C and 1D, a heating resistor 19 connected to a temperature controller 18 is also shown. The function of these elements is to maintain a temperature of the fluid chamber 10c at 37°C.

[0065] [Fig.2] represents a second embodiment of device 1' suitable for the implementation of the invention. The device 1' comprises a light source 11, a diffuser 13, a diaphragm 14, an image sensor 20, a holding structure 17 and a processing unit 30 as described in connection with the first embodiment. The holding structure 17 is configured to define a fixed distance between the sample and the image sensor. According to this embodiment, the device does not include an image-forming lens between the image sensor 20 and the sample 10. The image sensor 20 is preferably brought close to the sample, the distance between the image sensor 20 and the sample 10 typically being between 100 μm and 3 mm. According to this embodiment, the image sensor acquires images according to a lensless imaging modality. The sample is preferably contained in a fluidic chamber 10c, for example a “Leja” chamber as described in connection with the first embodiment. The advantage of such an embodiment is that it does not require precise positioning of an optical system 15 relative to the sample 10, and that it provides a high field of observation. The disadvantage is the obtaining of images of lower quality, but which remain usable.

[0066] Preferably, the holding structure is arranged so that the distance between the sample, when the sample 10 is arranged on the holding structure 17, and the image sensor 20, is constant. In the example shown in [Fig. 2], the holding structure 17 is connected to a base 20', on which the image sensor 20 extends. The base may for example be a PCB (Printed Circuit Board) type plate, on which the image sensor 20 is placed. The sample 10, contained in the fluidic chamber 10c, is held, by the sample support 10s, on the holding structure 17. The sample support is for example a transparent slide. The sample extends between the sample support 10s and the image sensor. Thus, the distance between the sample 10 and the image sensor 20 is not affected by a fluctuation in the thickness of the support 10s.

[0067] [Fig. 3] shows schematically the main steps of a method for processing several images acquired by an image sensor according to the defocused imaging modality (first embodiment) or lensless imaging (second embodiment). The method is described in connection with the observation of spermatozoa, it being understood that it can be applied to the observation of other types of motile particles.

[0068] Step 100: Acquisition of a series of images

[0069] During this step, a series of images Iq^ is acquired according to one of the previously described modalities. n is a natural integer designating the rank of each acquired image, with 1 n < AT, N being the total number of acquired images. The acquired images are images acquired either according to a defocused imaging modality or according to a lensless imaging modality. The number N of acquired images can be between 5 and 50. As previously indicated, the images can be acquired according to an acquisition frequency of 60 Hz.

[0070] In the results presented below, the images were acquired using a defocused imaging modality, as described in connection with Figures 1A to 1D. Step 110: Pre-treatment

[0071] The objective is to carry out a pre-processing of each acquired image, so as to limit the effects of a fluctuation in the intensity of the incident light wave or the sensitivity of the camera. The pre-processing consists of normalizing each acquired image by an average of the intensity of at least one acquired image, and preferably of all the acquired images.

[0072] Thus, from each image Iq^, normalization makes it possible to obtain a normalized image In, such as T _ kw or T _ 70ji where Jo is an average In — t' In — r' of one or more images of the series of images, and preferably of each image of the series of images.

[0073] According to one possibility, the preprocessing may include applying a high-pass filter to each image, possibly normalized. The high-pass filter makes it possible to eliminate low frequencies from the image.

[0074] According to one possibility, a Gaussian filter is applied, by performing a convolution product of the image In by a Gaussian kernel K. The width at half-height of the Gaussian kernel is for example 20 pixels. The preprocessing then consists of subtracting, from the image In, the image In*K resulting from the application of the Gaussian filter, so that the image resulting from the preprocessing is: I 'n = In- In*K. * is the convolution product operator.

[0075] The effect of such filtering is described below, in connection with Figures 7A to 7C and 8A to 8C. Step 120: Particle Detection

[0076] This step consists of detecting and precisely positioning the spermatozoa, and this on each image Iqd, resulting from step 100 or from each image I'n preprocessed during step 110. This step is carried out using a convolutional neural network for detection CNNd. The neural network CNNd comprises an input layer, in the form of an input image. The input image is either an acquired image Iq1v or a preprocessed image In, I'n. From the input image Iinrn, the detection neural network CNNd generates an output image. The output image IOUf n is such that each spermatozoon 10; detected on the input image IiBrB, in a position ( 1 appears in the form of a distribution i) intensity centered around said position. In other words, from an input image Iinn. the CNNd neural network allows: - detection of spermatozoa; - an estimate of the position / j of each sperm 10; detected; \XP Y j) - a generation of an output image Ioutn comprising a predetermined intensity distribution D. around each position (V yg

[0077] Thus, I^CNN^lJ

[0078] Each position / ] is a two-dimensional position, in the detection plane yg P20. The intensity distribution Di is such that the intensity is maximum at each position y} and that the intensity is considered negligible beyond a neighborhood Vj of each position. By neighborhood V j, we mean a region extending according to a predetermined number of pixels, for example between 5 and 20 pixels, around each position y ). The distribution Dj can be of the slot type, in which case in the neighborhood V each pixel is of constant intensity is high, and beyond the neighborhood V each pixel has zero intensity.

[0079] Preferably, the distribution Dj is centered on each position Xp y ) and is strictly decreasing around the latter. It may for example be a two-dimensional Gaussian intensity distribution, centered on each position ( Xj, y ). The width at half height is for example less than 20 pixels, and preferably less than 10 or 5 pixels. Any other form of parametric distribution can be envisaged, knowing that it is preferable for the distribution to be symmetrical around each position, and preferably strictly decreasing from the position Xj, y.}. Assigning an intensity distribution to each position Xp y J makes it possible to obtain an output image IOUfU in which each spermatozoon is simple to detect. In this, the output image is a detection image, on the basis of which each spermatozoon can be detected.

[0080] The output image of the neural network is formed by a resultant of each intensity distribution defined respectively around each position (Xp yJ.

[0081] In the example implemented by the inventors, the convolutional neural detection network comprised 20 layers comprising either 10 or 32 characteristics (more commonly referred to as "features") per layer. The number of characteristics was determined empirically. The transition from one layer to another is performed by applying a convolution kernel of size 3x3. The output image is obtained by combining the characteristics of the last convolution layer. The neural network was programmed in a Matlab environment (publisher: The Mathworks). The neural network was previously trained (step 80), in order to parameterize the convolution filters. During training, training sets were used, each set comprising: - an input image, obtained by the image sensor and preprocessed by normalization and possibly filtering, using animal seeds. - an output image, on which the position of each spermatozoon was manually annotated.

[0082] The training was performed using 10000 or 20000 annotated positions (i.e. between 1000 and 3000 annotated positions per image). The effect of the size of the training set (10000 or 20000 annotations) is discussed in connection with Figures 9A and 9B.

[0083] Figures 4A, 4B and 4C are a set of test images, comprising respectively: - an image of a sample of bovine semen acquired by the image sensor and having been the subject of normalization and high-pass filtering according to step 110; - a manually annotated image, which corresponds to a reference image, usually referred to as “ground truth” (reality); - an image resulting from the application of the convolutional neural detection network.

[0084] The image in Figure 4C is consistent with the reference image (Figure 4B), which attests to the detection performance of the CNNd neural network. The inventors noted that Figure 4C shows a position not represented in the reference image: a check showed that this was an omission of annotation in the reference image.

[0085] Thus, on the basis of a hologram, and with possible pre-processing of the normalization or high-pass filter type, to bring out the high frequencies, the application of the convolutional neural network for CNNd detection allows for precise detection and positioning of spermatozoa. And this without resorting to image reconstruction implementing a holographic propagation operator, as suggested in the prior art. The input image of the neural network is an image formed in the detection plane, and not in a reconstruction plane distant from the detection plane. In addition, the detection neural network generates low-noise output images: the signal-to-noise ratio associated with each spermatozoon detection is high, which facilitates subsequent operations.

[0086] Step 120 is repeated for different images of the same series of images, so as to obtain a series of output images Iout j.....^outN-

[0087] During step 120, from each output image IOUf i.....^out N' a local maximum detection algorithm is applied, so as to obtain, for each image, a list of 2D coordinates, each coordinate corresponding at a position of a spermatozoon. Thus, from each output image Iout J.....Iout,N' we establish a list Lout i.....Lout ^.Each list Lolltn contains corresponds to the 2D positions, in the detection plane, of spermatozoa detected in an image

[0088] Figure 5A represents an image acquired during step 100. More precisely, it is a detail of an image acquired according to a defocused modality. It is an image of a sample comprising bovine sperm. Such an image is difficult for a user to interpret. Figure 5B shows an image resulting from the application of the detection neural network to Figure 5A, after normalization and application of a high-pass filter. The comparison between images 5A and 5B shows the gain provided by the convolutional detection neural network in terms of signal-to-noise ratio. Step 130: Position tracking

[0089] During this step, from the lists LOUf j.....LOutN respectively established from the output images Iouf j.....loutN' resulting from the same series of images, a position tracking algorithm is applied, usually referred to as the "tracking algorithm". For each spermatozoon detected following step 120, an algorithm is implemented that allows tracking of the position of the spermatozoon, parallel to the detection plane, between the different images of the series of images. The implementation of the position tracking algorithm is efficient because it is carried out from the lists Lout .....Lout resulting from step 120. As previously indicated, these images have a high signal-to-noise ratio, which facilitates the implementation of the position tracking algorithm. The position tracking algorithm may be a "nearest neighbor" type algorithm.Step 130 allows determination of a trajectory of each spermatozoon parallel to the detection plane.

[0090] In Figure 5C, output images IOUf 1.....Iout N have been superimposed considering a stack of images of 30 images. It is observed that the trajectory, parallel to the detection plane, of each spermatozoon can be determined precisely. Step 140: Characterization of motility

[0091] During step 140, for each spermatozoon, each trajectory can be characterized on the basis of metrics applied to the trajectories resulting from step 130. Knowing the frequency of acquisition of the images, it is possible to quantify the movement speeds of each detected spermatozoon, and in particular: - a speed of the linear trajectory VSL, usually designated by the term “velocity straightline path”, which corresponds to the speed calculated on the based on a distance, in a straight line, between the first and last points of the trajectory (the first point is determined from the first image acquired in the series of images, and the last point is determined from the last image acquired in the series of images) - a speed of the curvilinear trajectory VCL, usually designated by the term “velocity curvilinear path”: this is a speed established by adding the distances traveled between each image, and dividing by the duration of the acquisition period; - a speed of the average trajectory VAP, usually designated by the term “velocity average path”: this is a speed established after smoothing the trajectory of a particle: the distance traveled according to the smoothed trajectory (or average trajectory) is divided by the duration of the acquisition period.

[0092] From the calculated speeds, it is possible to define indicators for characterizing the motility of spermatozoa, known to those skilled in the art. These are, for example, indicators of the type: - STR straightness indicator, obtained by a ratio between VSL and VAP, usually referred to as “straightness”. This indicator is higher when the sperm moves in a straight line; - linearity indicator LIN, obtained by a ratio between VSL and VCL, usually referred to as “linearity”. This indicator is also higher when the sperm moves in a straight line. - WOB oscillation indicator, obtained by a VAP / VCL ratio, usually referred to by the term wobble.

[0093] Quantifying the speeds or parameters listed above allows spermatozoa to be categorized according to their motility. For example, a spermatozoon is considered to be: - motile if the length of the trajectory is greater than a first threshold, for example 10 pixels, and its displacement along the average trajectory (VAPx At, At being the acquisition period) is greater than a predefined length, corresponding for example to the length of a sperm head; - progressive if the length of the trajectory is greater than the first threshold, and if the straightness STR and the speed of the average trajectory VAP are respectively greater than two threshold values ​​STRth and VAPth i; - slow if the length of the trajectory is greater than the first threshold and if the speed of the linear trajectory VSL and the speed of the average trajectory VAP are respectively less than two threshold values ​​VSLth 2 and VAPth 2; - static if the length of the trajectory is greater than the first threshold and if the speed of the linear trajectory VSL and the speed of the average trajectory VAP are respectively less than two threshold values ​​VSLth 3 and VAPth 3. - not categorized if the trajectory length is less than the first threshold.

[0094] The threshold values ​​STRth, VSLth 2 VSLth 3 are previously determined, with VSLth2>VSLth 3. The same applies to the values ​​VAPth b VAPth 2 and VAPth 3 with VAPth i>VAPth !>VAPth 3.

[0095] Step 150: Extraction of thumbnails for each spermatozoon.

[0096] During this image, a thumbnail V is extracted for each spermatozoon detected by the CNNd detection network, and this in each image acquired by the image sensor during step 100.

[0097] Each thumbnail V is a portion of an image Iq^ acquired by the image sensor. For each sperm detected 10; from each image a thumbnail Vautour of the position y. j assigned to the sperm is extracted. The position (Xj, y J of the sperm 10;, in each image Iqu, is obtained following the implementation of the position tracking algorithm (step 130).

[0098] The size of each thumbnail V is predetermined. Relative to each thumbnail Vjn, the position Xj, y J of the spermatozoon 10; considered is fixed: preferably, the position y J of the spermatozoon 10; is centered in each thumbnail V

[0099] A thumbnail VjD may for example comprise a few tens or even a few hundred pixels, typically between 50 and 500 pixels. In this example, a thumbnail comprises 64 x 64 pixels. Due to the size and the concentration of spermatozoa in the sample, a thumbnail may comprise several spermatozoa to be characterized. However, only the spermatozoon 10; occupying a predetermined position in the thumbnail Vy2, for example in the center of the thumbnail, is characterized using said thumbnail.

[0100] Figure 6 shows nine thumbnails V extracted from images Iout>n resulting from step 100, using the positions ( Xp y. j resulting from the trajectory tracking of a spermatozoon resulting from step 130.

[0101] Step 160 Classification of the morphology of each spermatozoon

[0102] During this step, a classification neural network CNNc is used, so as to classify each spermatozoon previously detected by the convolutional detection neural network CNNd. For each spermatozoon 10;, the neural network is fed by the thumbnails V2j... extracted during step 150.

[0103] The CNNc classification convolutional neural network can for example comprise 6 convolutional layers, comprising between 16 and 64 characteristics: 16 characteristics for the first four layers, 32 characteristics for the fifth layer, and 64 characteristics for the sixth layer. The transition from one layer to another is carried out by applying a convolution kernel of size 3 by 3. The output layer comprises nodes, each node corresponding to a probability of belonging to a morphological class. Each morphological class corresponds to a morphology of the spermatozoon analyzed. It can for example be a known classification, the classes being: - 0: undefined - 1: normal - 2: stump - 3: distal droplet - 4: beheaded - 5: microcephalic - 6: proximal droplet - 7: DMR (Distal Midpiece reflex, usually referred to as Curvature of the distal end of the intermediate piece) - 8: head anomaly - 9: flagellum anomaly: curved or coiled flagellum - 10: aggregates.

[0104] The output layer can thus have 11 classes.

[0105] The neural network is programmed in the Matlab environment (publisher: The Mathworks). The neural network has previously been trained (step 90), in order to configure the convolution filters. During training, training sets were used, each set comprising: - thumbnails, extracted from images acquired by the image sensor; - manual annotation of the morphology of each sperm analyzed. The annotation was carried out on the basis of thumbnails that had been subject to holographic reconstruction. Experimental tests

[0106] Various tests were carried out to examine the performance of the detection neural network CNNd. The sensitivity of the detection was tested with respect to the application of a high-pass filter during step 110. Figures 7A, 7B and 7C respectively represent an image acquired by the image sensor, a reference image, manually annotated, and an image resulting from the detection neural network. Image 7C was obtained without implementing the filter during step 110.

[0107] Figures 8A, 8B and 8C respectively represent an image acquired by the image sensor, a reference image, manually annotated, and an image resulting from the detection neural network. Image 8C was obtained by implementing a high-pass filter during step 110.

[0108] It is observed that image 8C includes detected spermatozoa, marked by arrows, which do not appear in image 7C. The application of the filter improves the detection performance of the classification neural network.

[0109] Figures 9A and 9B respectively represent the detection sensitivity and the detection specificity of the CNNd detection neural network (curves a, b and c), in comparison with a conventional algorithm (curve d). The conventional algorithm was based on erosion / dilation type morphological image processing operations.

[0110] In each of Figures 9A and 9B, curves a, b and c correspond respectively: - to the neural network trained with 10,000 annotations; - to the neural network trained with 20,000 annotations; - to the neural network trained with 20,000 annotations, each image acquired by the sensor having been subjected to high-pass filtering.

[0111] The sensitivity and specificity (y-axis) were determined on 14 different samples (x-axis) of diluted bovine semen (factor 10). It is observed that the performance of the neural network, whatever the configuration (curves a, b and c) is superior to that of the classical algorithm, which is remarkable. Furthermore, the best performance is obtained in configuration c, according to which the neural network is trained with 20,000 annotations and is fed with an image that has been preprocessed with a high-pass filter.

[0112] We recall that sensitivity corresponds to the rate of true positives over the sum of the rates of true positives and false negatives, and that specificity corresponds to the rate of true positives over the sum of the rates of true and false positives.

[0113] [Fig. 10A] is a confusion matrix representing the classification performance of a CNNc classification neural network configured to receive, as an input image, an image resulting from a holographic reconstruction. The neural network is a convolutional neural network, as previously described, trained using images acquired using a defocused device, as previously described. Using this neural network, each acquired image is subjected to a holographic reconstruction, in a sample plane extending across the sample, using a holographic reconstruction algorithm as described in US20190101484. From the reconstructed image, a thumbnail is extracted. centered on the sperm to be characterized. The thumbnail forms an input image of the neural network.

[0114] [Fig. 10B] is a confusion matrix representing the performance of classification of a classification neural network configured to receive, as input image, a single thumbnail, centered on the spermatozoon to be characterized, and extracted from an image acquired according to a defocused configuration. The neural network is a convolutional neural network, structured as described in step 160, except that the neural network is fed with a single thumbnail.

[0115] [Fig. 10C] is a confusion matrix representing the performance of classification of a classification neural network as described in step 160, fed by a series of 5 thumbnails extracted from images acquired in defocused configuration.

[0116] The axes of each confusion matrix correspond to classes 1 to 10 previously described.

[0117] It is noted that the classification performance of the algorithms described in connection with Figures 10B and 10C is superior to that described in [Fig. 10A]. For example, with regard to class 6 (“proximal droplet” anomaly), the classification performance increases from 31% ([Fig. 10A]), to 61% ([Fig. 10B]), and to 96.9% ([Fig. 10C]).

[0118] It is also noted that a single input image allows satisfactory classification performance to be obtained. Also, for morphological classification, it is not necessary to have a series of images comprising several images. A single image may be sufficient. However, the classification performance is better when several images are used.

[0119] The invention allows analysis of a sample containing spermatozoa without using a translation stage. Furthermore, it allows characterization directly from the image acquired by the image sensor, i.e. on the basis of diffraction patterns of the different spermatozoa, and this without requiring the use of digital reconstruction algorithms. Currently, the use of such an algorithm results in a processing time of 10 seconds per image. That is 300 seconds for a series of 30 images. Furthermore, as shown in Figures 10A to 10C, the invention allows more precise classification than a classification based on a neural network of identical structure, fed by reconstructed images.

[0120] Another advantage of the invention is the tolerance to defocusing. The inventors estimate that the invention tolerates offsets of ± 25 pm between the optical system and the sample.

Claims

Claims

1. Method for characterizing at least one mobile particle (10;) in a sample (10), the method comprising: a. acquiring at least one image (Zq„) of the sample during an acquisition period, using an image sensor (20) and forming a series of images, the series of images comprising at least one image, each image of the sample being acquired according to a defocused imaging modality or according to a lensless imaging modality, so that each particle forms, on each image, a diffraction pattern; b. using each image of the series of images resulting from a) as an input image (I,„ n) of a convolutional neural network for detection (CNNd), the convolutional neural network for detection being configured to detect the particles and to produce, from each image, an output image on which each detected particle is assigned an intensity distribution, centered on the particle and extending around the particle; c. for each detected particle, from each output image (Zoutn) resulting from b), estimate a position ( ) of each detected particle in each y dd image from the image series; d. characterization of each detected particle from the estimate of the position resulting from c), established from each image of the series of images; the method being characterized in that - each particle is a spermatozoon. - step d) comprises a morphological characterization of each detected spermatozoon, step d) comprising: • for each of the detected spermatozoon, from each image resulting from a), and the positions resulting from c), extraction of a thumbnail (V comprising the detected spermatozoon, the position of the detected spermatozoon in the thumbnail being predetermined, so as to obtain, for the detected spermatozoon, a series of thumbnails, the size of each thumbnail being smaller than the size of each image acquired during step a); • for each detected spermatozoon, use of the series of thumbnails as input data for a classification neural network (CNNc), the classification neural network being configured to classify the spermatozoon among predetermined morphological classes.

2. The method of claim 1, wherein each detected sperm is centered relative to each thumbnail.

3. Method according to claim 1 or claim 2, in which step d) comprises a characterization of the motility of the spermatozoon, step d) comprising, from the positions of the spermatozoon resulting from step c): - a determination of a trajectory of the spermatozoon during the acquisition period; - a calculation of a speed of the spermatozoon from the trajectory; - a classification of the spermatozoon according to the speed.

4. A method according to claim 3, comprising: - a determination of a distance, in a straight line, between a first point and a last point of the trajectory, and a calculation of a linear trajectory speed from said distance; - and / or a determination of distances traveled between each acquired image, and a calculation of a speed of the curvilinear trajectory from said distances; - and / or a determination of a smoothed trajectory, and a calculation of an average trajectory speed from the smoothed trajectory.

5. Method according to any one of the preceding claims, in which: - the sample extends along a sample plane (Pio); - the image sensor extends along a detection plane (P20); - an optical system (15) extends between the sample and the image sensor, the optical system defining an object plane (Po) and an image plane (P;); - the object plane is offset from the sample plane by an object defocusing distance and / or the image plane is offset from the sample plane by an image defocusing distance, so that during step a), each image of the sample is acquired according to a defocused imaging modality.

6. Method according to claim 5, wherein: - the sample is arranged on a sample holder (10s), resting on at least one spring (16a), the spring being configured to push the sample holder towards the optical system; - the optical system is connected to at least one stop (16b), extending, from the optical system, towards the sample holder; - such that during step a), under the effect of the spring, the sample holder rests on the stop.

7. A method according to any one of claims 1 to 4, wherein no imaging optics extend between the sample and the image sensor, so that during step a), each image of the sample is acquired using a lensless imaging modality.

8. Method according to any one of the preceding claims, in which step a) comprises a normalization of each acquired image by an average of said image or by an average of images of the series of images.

9. A method according to any preceding claim, wherein step a) comprises applying a high-pass filter to each acquired image.

10. A method according to any preceding claim, wherein in step b), the intensity distribution assigned to each particle is decreasing, such that the intensity decreases as a function of the distance from the particle.

11. A method according to any preceding claim, wherein in step b), the intensity distribution assigned to each particle is a two-dimensional parametric statistical distribution.

12. Device (1, 1') for observing a sample, the sample comprising mobile particles, the device comprising: - a light source (11), configured to illuminate the sample; - an image sensor (20), configured to form an image of the sample; - a holding structure (16, 17), configured to hold the sample between the light source and the image sensor; - a processing unit (30), connected to the image sensor, and configured to implement steps a) to d) of a method according to any one of the preceding claims from at least one image acquired by the image sensor.

13. The device of claim 12, wherein no imaging optics extend between the image sensor and the sample.

14. The device of claim 13, wherein the holding structure (17) is configured to maintain a fixed distance between the image sensor and the sample.

15. Device according to claim 12, wherein: - the image sensor extends along a detection plane; - the device comprises an optical system (15), extending between the image sensor and the support plane, the optical system defining an image plane and an object plane, the device being such that: • the image plane is offset from the detection plane by an image defocusing distance; • and / or the support plane and offset from the object plane by an object defocusing distance.

16. Device according to claim 15, wherein: - the holding structure (16) comprises a sample support (10s), mounted on a spring (16a); - the optical system (15) is mechanically connected to at least one stop (16b); - the spring is arranged so that the sample holder rests against the stop, so as to control a distance between the optical system and the sample.

Images