Method for observing a sample for identification purposes and associated devices

Abstract

The present invention relates to a method for observing a sample comprising a set of organisms, the method being implemented by an observation system (10) including a lensless imaging unit (34) and comprising: - a phase of obtaining images of the sample; and - an image analysis phase comprising: - calculating the probability that an organism belongs to a predefined species by applying a neural network in order to obtain a probability of belonging; - verifying that a condition of reliability of the probabilities is fulfilled by comparing a value arising from the probabilities with a predefined threshold; and - determining information relating to the identity of the organism, the information belonging to a species that is not among the predefined species or when the reliability condition is not fulfilled.

Description

[0001] Method of observing a sample for identification purposes and associated devices

[0002] The present invention relates to an observation method and associated devices.

[0003] The invention falls within the field of microbiology and, more specifically, that of the identification of biological organisms and the study of the interaction of organisms with an environment.

[0004] It is known to culture organisms on specific media. These media are agar nutrient media. These media allow the number of organisms sampled to be increased, either specifically or by obtaining spatial separation of the different types of organisms making up the sample. Specific amplification is achieved by the presence of inhibitors that promote the growth of certain organisms to the detriment of others. Spatial isolation of different types of organisms is only achieved in the case of a polymicrobial sample.

[0005] One of the disadvantages of culturing on such media is the time involved in obtaining from a single organism a set of organisms that can be observed and sampled by a human operator. This generally requires about twenty generations, a relatively long time of at least 24 hours.

[0006] There is therefore a need for a method of observing organisms that is rapid and robust to implement.

[0007] For this purpose, the description describes a method of observing a sample, the sample comprising:

[0008] - a set of organisms, and

[0009] - a solid substrate supporting the set of organisms, the method being implemented by an observation system comprising a lensless imaging unit and the method comprising:

[0010] - a phase of obtaining images of the sample, the obtaining phase comprising, for each image obtained, a step of:

[0011] - acquisition of a set of images of a part of the sample, each acquired image being called a thumbnail, each thumbnail having an overlapping area with at least one other thumbnail and each area of ​​the sample being visible in at least one thumbnail,

[0012] - a step of determining a transformation to be applied to each image to correct both translation and rotation offsets by using the overlap zones, to obtain a transformation specific to each image,

[0013] - a step of applying the transformation to each image to obtain corrected images, and

[0014] - a step of merging the thumbnails to obtain the image of the sample, and

[0015] - an image analysis phase, the analysis phase including:

[0016] - a step of determining at least one characteristic relating to the set of organisms by analyzing at least one of the images obtained.

[0017] According to particular embodiments, the observation method has one or more of the following characteristics, taken in isolation or in all technically possible combinations:

[0018] - the observation system comprises a sample storage space, the imaging unit comprising an optical sensor for which an acquisition zone is defined, the acquisition phase further comprising, for each image obtained, a step of transporting the sample from the storage space to the acquisition zone for the implementation of the acquisition step and a step of transporting the sample to the storage space after the implementation of the acquisition step.

[0019] - the two transport stages are implemented by a robotic arm equipped with gripping grippers.

[0020] - for each pair of neighboring images, the determination step includes determining the distortion between two neighboring images, determining the translational shift between the two overlapping zones of the neighboring images, determining an affine transformation making it possible to move from the overlapping zone of the first image to the second image of the two neighboring images and determining the transformation function to be applied to each image to minimize the distortions between each image.

[0021] - determining the transformation function to be applied to each image to minimize distortions between each image involves solving an overdetermined linear optimization problem.

[0022] - during the fusion step, each overlap area of ​​the image is an average of the overlap areas of the corresponding corrected thumbnails.

[0023] - during the acquisition stage, the images are acquired along successive lines to cover the entire sample.

[0024] - during the acquisition step, the number of images acquired is between 12 - the solid substrate is capable of growing at least part of the set of organisms and the sample comprises at least one zone in which the substrate comprises an antibiotic, the characteristic determined in the determination step being the sensitivity of an organism to the antibiotic.

[0025] - the sensitivity of the organism to the antibiotic is obtained by determining a characteristic dimension of the inhibition circle associated with the growth of at least part of the set of organisms on the area in which the substrate comprises an antibiotic, the determination comprising a step of extracting the contours of the inhibition circle and estimating the characteristic dimension from the extracted contours.

[0026] - a bacterial mat is defined for the sample, the analysis phase comprising a step of obtaining the bacterial mat, the determination step using the bacterial mat obtained to determine at least one characteristic relating to the set of organisms.

[0027] - the time interval between each image acquisition during the implementation of the acquisition phase is adjustable.

[0028] - the time interval depends on at least one determined characteristic.

[0029] - the value resulting from the probabilities is the maximum value of the calculated probabilities.

[0030] - the organisms are organized into colonies, the observation method further comprising a colony detection phase for detecting the colonies and locating them, the neural network of the calculation step also taking as input the location of at least one colony.

[0031] - the analysis phase includes a step of determining at least one property relating to the detected colonies and a step of selecting the colonies to obtain selected colonies, the obtaining step being implemented only for the selected colonies.

[0032] - a plurality of organisms correspond to the same species, the determination step comprising the implementation of an aggregation function of the different probabilities found for each of the organisms.

[0033] - the aggregation function gives as species for the plurality of organisms the maximum of the set of probabilities.

[0034] - the aggregation function gives as species for the plurality of organisms the species for which the sum on each organism of the probabilities that the organism belongs to the species is maximum

[0035] - in the calculation step, the neural network is a multi-class classification neural network. - a phylogenetic tree has been defined for the neural network, the phylogenetic tree comprising a set of nodes distributed according to at least two phylogenetic levels, the lowest phylogenetic level being the species, the nodes being linked together when there is a phylogenetic link between them, the verification step comprising the calculation of a probability for each node from the probabilities calculated in the calculation step and the verification of the conformity condition for each node.

[0036] - during the determination step, the information determined is the membership at the lowest phylogenetic level respecting a conformity condition.

[0037] - phylogenetic levels are chosen from the list consisting of phylum, class, order, family and genus.

[0038] - the number of phylogenetic levels is greater than or equal to 4.

[0039] - the predefined threshold has a value between 0.7 and 0.9.

[0040] The description also describes a method of observing a sample, the sample comprising:

[0041] - a set of organizations,

[0042] - a solid substrate supporting the set of organisms, the method being implemented by an observation system comprising a lensless imaging unit and the method comprising:

[0043] - a phase of obtaining images of the sample,

[0044] - an image analysis phase, the analysis phase including:

[0045] - a step of calculating the probability that at least one organism belongs to a predefined species, the calculation step comprising the application of a neural network to at least one image to obtain the probability that said at least one organism (18) belongs to a predefined species, the predefined species being the set of species that the neural network has been trained to identify in an image,

[0046] - a step of verifying that a reliability condition of the probabilities obtained is met by comparing at least one value from the probabilities obtained with a predefined threshold, and

[0047] - a step of determining at least one piece of information relating to the identity of the at least one organism, the information being the belonging to a species not included in the predefined species or when the reliability condition is not met. According to particular embodiments, the observation method has one or more of the following characteristics, taken in isolation or in all technically possible combinations:

[0048] - the observation system comprises a sample storage space, the imaging unit comprising an optical sensor for which an acquisition zone is defined, the acquisition phase further comprising, for each image obtained, a step of transporting the sample from the storage space to the acquisition zone for the implementation of the acquisition step and a step of transporting the sample to the storage space after the implementation of the acquisition step.

[0049] - the two transport stages are implemented by a robotic arm equipped with gripping grippers.

[0050] - for each pair of neighboring images, the determination step includes determining the distortion between two neighboring images, determining the translational shift between the two overlapping zones of the neighboring images, determining an affine transformation making it possible to move from the overlapping zone of the first image to the second image of the two neighboring images and determining the transformation function to be applied to each image to minimize the distortions between each image.

[0051] - determining the transformation function to be applied to each image to minimize distortions between each image involves solving an overdetermined linear optimization problem.

[0052] - during the fusion step, each overlap area of ​​the image is an average of the overlap areas of the corresponding corrected thumbnails.

[0053] - during the acquisition stage, the images are acquired along successive lines to cover the entire sample.

[0054] - during the acquisition stage, the number of acquired images is between 12 and 15.

[0055] - the solid substrate is capable of growing at least part of the set of organisms and the sample comprises at least one area in which the substrate comprises an antibiotic, the characteristic determined in the determination step being the sensitivity of an organism to the antibiotic.

[0056] - the sensitivity of the organism to the antibiotic is obtained by determining a characteristic dimension of the inhibition circle associated with the growth of at least part of the set of organisms on the area in which the substrate comprises an antibiotic, the determination comprising a step of extracting the contours of the inhibition circle and estimating the characteristic dimension from the extracted contours.

[0057] - a bacterial mat is defined for the sample, the analysis phase comprising a step of obtaining the bacterial mat, the determination step using the bacterial mat obtained to determine at least one characteristic relating to the set of organisms.

[0058] - the time interval between each image acquisition during the implementation of the acquisition phase is adjustable.

[0059] - the time interval depends on at least one determined characteristic.

[0060] - the value resulting from the probabilities is the maximum value of the calculated probabilities.

[0061] - the organisms are organized into colonies, the observation method further comprising a colony detection phase for detecting the colonies and locating them, the neural network of the calculation step also taking as input the location of at least one colony.

[0062] - the analysis phase includes a step of determining at least one property relating to the detected colonies and a step of selecting the colonies to obtain selected colonies, the obtaining step being implemented only for the selected colonies.

[0063] - a plurality of organisms correspond to the same species, the determination step comprising the implementation of an aggregation function of the different probabilities found for each of the organisms.

[0064] - the aggregation function gives as species for the plurality of organisms the maximum of the set of probabilities.

[0065] - the aggregation function gives as species for the plurality of organisms the species for which the sum on each organism of the probabilities that the organism belongs to the species is maximum

[0066] - during the calculation stage, the neural network is a multi-class classification neural network.

[0067] - a phylogenetic tree has been defined for the neural network, the phylogenetic tree comprising a set of nodes distributed according to at least two phylogenetic levels, the lowest phylogenetic level being the species, the nodes being linked together when there is a phylogenetic link between them, the verification step comprising the calculation of a probability for each node from the probabilities calculated in the calculation step and the verification of the conformity condition for each node. - during the determination step, the information determined is the membership at the lowest phylogenetic level respecting a conformity condition.

[0068] - phylogenetic levels are chosen from the list consisting of phylum, class, order, family and genus.

[0069] - the number of phylogenetic levels is greater than or equal to 4.

[0070] - the predefined threshold has a value between 0.7 and 0.9.

[0071] The description also relates to a system for observing a sample, the sample comprising:

[0072] - a set of organisms, and

[0073] - a substrate supporting the set of organisms, the observation system comprising a lensless imaging unit and the observation system being suitable for:

[0074] - obtain images of the sample, the observation system being specific to, for each image obtained:

[0075] - acquire a set of images of a part of the sample, each acquired image being called a thumbnail, each thumbnail having an overlapping area with at least one other thumbnail and each area of ​​the sample being visible in at least one thumbnail,

[0076] - determine a transformation to be applied to each image to correct both translation and rotation offsets using overlap zones, to obtain a transformation specific to each image,

[0077] - apply the transformation to each image to obtain corrected images, and

[0078] - merge the thumbnails to obtain the sample image, and

[0079] - analyze the images, the observation system being, during this analysis, suitable for:

[0080] - determine at least one characteristic relating to the set of organisms by analyzing at least one of the images obtained.

[0081] The description also relates to a system for observing a sample, the sample comprising:

[0082] - a set of organisms, and

[0083] - a solid substrate supporting the set of organisms, the observation system comprising a lensless imaging unit suitable for:

[0084] - obtain images of the sample,

[0085] - analyzing the images, the analysis comprising: - calculating the probability that at least one organism belongs to a predefined species, the calculation comprising applying a neural network to at least one image to obtain the probability that said at least one organism belongs to a predefined species, the predefined species being the set of species that the neural network has been trained to identify in an image,

[0086] - verification that a reliability condition of the probabilities obtained is met by comparison of at least one value from the probabilities obtained with a predefined threshold, and

[0087] - the determination of at least one piece of information relating to the identity of the at least one organism, the information being membership of a species not included in the predefined species when the reliability condition is not met.

[0088] According to particular embodiments, the observation system further comprises an element for regulating the environment of the observation system.

[0089] The description also relates to a sample intended for an observation system as previously described, the sample comprising:

[0090] - a set of organisms, and

[0091] - a solid substrate supporting the set of organisms, the solid substrate being capable of growing at least part of the set of organisms and comprising at least one zone in which the substrate comprises an antibiotic in the form of a pellet having a characteristic dimension less than or equal to 6 millimeters.

[0092] According to particular embodiments, the sample has one or more of the following characteristics, taken in isolation or in all technically possible combinations:

[0093] - the sample has a wall equipped with gripping organs.

[0094] - the characteristic dimension of the pellet is greater than or equal to 1 millimeter.

[0095] - the characteristic dimension of the pellet is less than or equal to 2 millimeters.

[0096] In this description, the expression "suitable for" means indifferently "adapted for", "adapted to" or "configured for".

[0097] Furthermore, in all that follows, the expression "between two values" is understood to mean a framework in the broad sense, that is to say including the values ​​of the limits.

[0098] Characteristics and advantages of the invention will appear on reading the description which follows, given solely by way of non-limiting example, and made with reference to the appended drawings, in which: - figure 1 is a schematic representation of an observation system,

[0099] - Figure 2 is two top and side views of an example sample,

[0100] - Figure 3 is a schematic representation of part of the observation system of Figure 1,

[0101] - Figure 4 is a flowchart of an example of implementation of part of an observation method,

[0102] - Figure 5 schematically illustrates a first acquisition strategy allowing the entire Petri dish to be covered (case with 12 images),

[0103] - Figure 6 schematically represents all the images obtained by implementing the first acquisition strategy,

[0104] - Figure 7 schematically illustrates a first acquisition strategy allowing the entire Petri dish to be covered (case with 15 images),

[0105] - Figure 8 illustrates three images schematically illustrating how to use overlap zones to obtain the translational distortion between two images,

[0106] - figure 9 represents two images, one obtained without implementing a step of the method of figure 4 and the other with implementing this step,

[0107] - Figure 10 is a flowchart of an example of implementation of part of an observation method,

[0108] - Figure 11 is a schematic representation of a sample comprising antibiotic pellets,

[0109] - Figure 12 is a flowchart of another example of implementation of part of an observation method,

[0110] - Figure 13 is a representation of an example of a phylogenetic tree, and

[0111] - Figure 14 is a representation of another example of a phylogenetic tree.

[0112] In the following, the notions of “upstream” and “downstream” are defined with reference to the direction of propagation of light.

[0113] Figure 1 illustrates an observation system 10 allowing the observation of a sample 12.

[0114] Such a sample 12 is visible in both representations of Figure 2.

[0115] The sample 12 comprises a set of organisms 14 and a solid substrate 16 supporting the set of organisms 14.

[0116] Depending on the case, an organism 18 of the set of organisms 14 is acellular or cellular.

[0117] Among cellular organisms, prokaryotic organisms and eukaryotic organisms are usually distinguished. Archaea (also called archaebacteria) and bacteria are examples of prokaryotic organisms.

[0118] Eukaryotic organisms are either unicellular or multicellular. For example, protozoa, amoebas, algae, or yeasts are unicellular eukaryotic organisms.

[0119] Multicellular eukaryotic organisms are cells derived from, for example, human and non-human mammals, fungi, plants, protists or chromists.

[0120] For simplification, the terms bacteria or bacterial culture are used in the remainder of the document, but what is described can be transposed without problem to fungi or germs and more generally any organism 18 capable of growing on the substrate 16.

[0121] The substrate 16 is capable of growing at least a portion of the set of organisms 14. Thus, the substrate 16 is itself a culture medium or is placed in contact with a culture medium.

[0122] Substrate 16 is, for example, an agar medium.

[0123] For example, substrate 16 is a Muller-Hinton 2 agar medium (referred to as MH2 agar medium in the remainder of the description).

[0124] According to one variant, the substrate 16 serves only as a biological glue.

[0125] For example, substrate 16 comprises polylysine or type I collagen.

[0126] Preferably, the agar medium is chosen to allow the culture of any type of organisms 18 while being transparent to allow optical observation of the medium, without diffusing light.

[0127] The agar medium is, for example, BHI, MH or TSA medium.

[0128] The abbreviation BHI stands for Brain-Heart Infusion. BHI is a versatile and rich culture medium for growing bacteria.

[0129] The abbreviation MH stands for "Mueller-Hinton." MH medium is a rich agar for bacterial culture. This makes it an ideal candidate for use in performing an antibiogram.

[0130] The abbreviation TSA refers to the English name "Trypticase Soy Agar" which literally means tryptone soy agar. TSA medium is a non-selective and very versatile agar.

[0131] The previous examples have the advantage of having good optical properties (transparent and non-diffusing) and of being non-selective and therefore suitable for allowing the growth of the most common bacteria. The method can also be used with selective agar media, for example if it is desirable to promote the culture of certain species.

[0132] In the example of Figure 2, sample 12 is in the form of a Petri dish 20.

[0133] Petri dish 20 here has a cylindrical shape with a circle at its base.

[0134] This means that the substrate 16 is delimited by the wall of the Petri dish 20 and that this wall forms a cylinder.

[0135] More specifically, the Petri dish 20 has two parts 22 which fit into each other. Each part 22 is a circle extended by side walls 24, one of the parts 22 comprising the substrate 16 and therefore serving as a support while the other serves as a cover making it possible to isolate the substrate 16 and the set of organisms 14 from the external environment.

[0136] The circle of the cylindrical shape has a diameter between 35 millimeters (mm) and 150 mm, most often equal to 90 mm.

[0137] Alternatively, Petri dish 20 may have a cylindrical shape with a square base.

[0138] In such a case, it is common to find squares with a side of 120 mm.

[0139] Further, as seen in Figure 2 on the right, the Petri dish 20 is inserted into a holding device 25.

[0140] For convenience, the holding device 25 is not shown in the left-hand figure of Figure 2.

[0141] The holding device 25 has an outer cylinder shape and an inner shape having a dimension allowing it to fit onto the Petri dish 20.

[0142] The holding device 25 has gripping members 26.

[0143] The gripping member 26 is, for example, a gripping groove on a projection.

[0144] Without this being limiting, the gripping members 26 are arranged in an equidistant manner.

[0145] One of the gripping members 26 is provided with a keying device 27.

[0146] The holding device 25 is, for example, manufactured using a 3D printer.

[0147] The presence of these gripping members 26 makes it possible to limit the rotations of the Petri dish 20 during its transport.

[0148] The observation system 10 comprises an incubator 30, a transport unit 32, a lensless imaging unit 34 and an analysis unit 36.

[0149] The incubator 30 is used to store the Petri dishes 20 to be analyzed in a controlled environment. For example, the incubator 30 is suitable for ensuring that the temperature and humidity are within a predefined range. The incubator 30 therefore comprises at least one element 37 for regulating an environmental parameter.

[0150] The regulating element 37 is then capable of regulating said environmental parameter (for example, the temperature), in the storage space 38 of the Petri dishes 20.

[0151] Alternatively, the incubator 30 is capable of regulating said environmental parameter throughout the observation system 10.

[0152] For example, the parameter is a parameter related to humidity or temperature.

[0153] For illustration, the regulating element 37 is a passive humidity generating unit.

[0154] More particularly, the regulating element 37 is a water container with a paper diffuser, arranged in the incubator 30.

[0155] Such a regulation element 37 makes it possible to maintain a stable and high humidity while the Petri dishes 20 can be opened regularly, for better control (stability) of the morphotype and to avoid drying out of the agar medium.

[0156] Maintaining good humidity thus helps improve bacterial growth and the way bacteria organize themselves within the colony. It also helps prevent shrinkage of the agar medium, which generates annoying anisotropic movement of the colonies over time.

[0157] In the schematic example of figure 1, the storage space 38 is an open cabinet comprising several housings 39, each housing 39 being suitable for accommodating a Petri dish 20.

[0158] The incubator 30 therefore makes it possible to carry out the simultaneous incubation of a plurality of Petri dishes 20.

[0159] The transport unit 32 is capable of transporting the sample 12 from the incubator 30 to an acquisition area and of carrying out the transport in the reverse direction.

[0160] In the case where the incubator 30 regulates the entire observation system 10, it will then be understood that the transport unit 32 is capable of transporting the sample 12 from the storage space 38 to an acquisition zone and vice versa.

[0161] The transport unit 32 is here a robotic arm 40 equipped with gripping grippers 41.

[0162] The gripping pliers 41 allow the sample 12 to be grasped.

[0163] The robotic arm 40 is capable of performing translations and / or rotations to enable such transport and in particular the positioning of the sample 12 in the desired position. The lensless imaging unit 34 is capable of obtaining transmission images of the sample 12.

[0164] For this, the lensless imaging unit 34 comprises, from upstream to downstream, a light source 42 and a detector 44.

[0165] However, as a variant or in addition, the lensless imaging unit 34 may include other elements such as a diaphragm or an optical fiber at the output of the light source 42.

[0166] The light source 42 comprises a plurality of light-emitting diodes 46 (often referred to by the English acronym LED for “Light Emitting Diode”).

[0167] Each light-emitting diode 46 here provides partially coherent light.

[0168] This means that the light-emitting diode 46 makes it possible to obtain a temporal coherence of the order of 90 nm and a spatial coherence limited by the size of the light-emitting diode 46, typically 200 pm.

[0169] Each light-emitting diode 46 is capable of emitting over a range of wavelengths which is smaller than the size of the organism 18 observed, while allowing the use of conventional detection means. Wavelengths in the visible or near infrared range are then preferred. Alternatively, the wavelength of the laser beam is in a band of different wavelengths. The wavelength depends in particular on the organism 18 to be observed and its sensitivity to illumination by the light source 42.

[0170] As a specific example, the wavelengths used are a wavelength in the red (e.g. 625 nm), a wavelength in the green (e.g. 528 nm), a wavelength in the blue (e.g. 455 nm) and a wavelength in the infrared (e.g. 940 nm).

[0171] Such wavelengths make it possible to improve the analysis of samples, in particular for the applications described below of obtaining the inhibition circle or identifying organisms 18.

[0172] Wavelengths can be sent simultaneously or sequentially.

[0173] The detector 44 is here formed by a housing 48 protecting a sensor 50.

[0174] According to the example shown, the housing 48 has a parallelepiped shape but other shapes could be envisaged.

[0175] The 48 housing is fixed on an optical table to give it good stability.

[0176] The 50 sensor here is a CMOS type optical sensor.

[0177] The abbreviation CMOS refers to a technology for manufacturing electronic components and, by extension, to the components manufactured using this technology. The abbreviation comes from the corresponding English term "Complementary Metal Oxide Semiconductor" which literally means "Complementary Metal Oxide Semiconductor".

[0178] The sensor 50 comprises a set of pixels 52. In this sense, the sensor 50 is a matrix photosensor.

[0179] Each pixel 52 has a size between 3 micrometers (pm) and 4 pm, preferably equal to 3.75 pm.

[0180] Additionally, the 50 sensor has a field of view of 24mm by 36mm.

[0181] Such values ​​correspond to an optical sensor with a large field and good spatial resolution.

[0182] As its name indicates, the lensless imaging unit 34 does not have a magnification optic interposed between the sensor 50 and the light source 42.

[0183] Thus, the images acquired by the sensor 50 are formed by the radiation directly transmitted by the illuminated sample 12.

[0184] The sample 12 is positioned in an acquisition zone defined for the sensor 50 (i.e. in its optical field), at a short distance from the sensor 50, typically at a distance less than a few centimeters, preferably less than 1 centimeter (cm).

[0185] It can be specified that this distance corresponds to the distance between the organisms 14 and the sensor 50. In fact, the Petri dish 20 is placed on the sensor 50 protected by a glass slide.

[0186] The analysis unit 36 ​​is capable of analyzing the images acquired by the imaging unit to obtain information relating to the set of organisms 14.

[0187] The analysis unit 36 ​​is represented more precisely in Figure 3.

[0188] The analysis unit 36 ​​is capable of interacting with a computer program product to enable the implementation of certain steps of an observation method. The associated steps are therefore steps implemented by computer.

[0189] The analysis unit 36 ​​is here a desktop computer. Alternatively, the analysis unit 36 ​​is a rack-mounted computer, a laptop, a tablet, a personal digital assistant (PDA), or a smartphone.

[0190] In the case of Figure 3, the analysis unit 36 ​​comprises a computer 56, a user interface 58 and a communication device 60.

[0191] The calculator 56 is an electronic circuit designed to manipulate and / or transform data represented by electronic or physical quantities in registers of the analysis unit 36 ​​and / or memories into other similar data corresponding to physical data in the memories of registers or other types of display devices, transmission devices or storage devices.

[0192] As specific examples, the computer 56 includes a single-core or multi-core processor (such as a central processing unit (CPU), a graphics processing unit (GPU), a microcontroller, and a digital signal processor (DSP)), a programmable logic circuit (such as an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a programmable logic device (PLD), and programmable logic arrays (PLAs)), a state machine, a logic gate, and discrete hardware components.

[0193] The computer 56 comprises a data processing unit 62 adapted to process data, in particular by performing calculations, memories 64 adapted to store data and a reader 66 adapted to read a computer-readable medium.

[0194] The user interface 58 includes an input device 68 and an output device 70.

[0195] The input device 68 is a device allowing the user of the analysis unit 36 ​​to enter information or commands into the analysis unit 36.

[0196] In Figure 3, the input device 68 is a keyboard. Alternatively, the input device 68 is a pointing device (such as a mouse, a touchpad, and a graphics tablet), a voice recognition device, an eye tracker, or a haptic (motion analysis) device.

[0197] The output device 70 is a graphical user interface 58, i.e., a display unit designed to provide information to the user of the analysis unit 36.

[0198] In Figure 3, the output device 70 is a display screen for visually presenting the output. In other embodiments, the output device 70 is a printer, an augmented and / or virtual display unit, a speaker or other sound generating device for presenting the output in sound form, a vibration and / or odor producing unit, or a unit adapted to produce an electrical signal.

[0199] In a specific embodiment, the input device 68 and the output device 70 are the same component forming human-machine interfaces, such as an interactive display.

[0200] The communication device 60 allows one-way or two-way communication between the components of the analysis unit 36. For example, the communication device 60 is a bus communication system or an input / output interface. The presence of the communication device 60 allows, in certain embodiments, the components of the computer 56 to be remote from each other.

[0201] The computer program product includes a computer-readable medium.

[0202] The computer-readable medium 72 is a tangible device readable by the reader 66 of the computer 56.

[0203] Notably, the computer-readable medium 72 is not a transient signal per se, such as radio waves or other freely propagating electromagnetic waves, such as light pulses or electronic signals.

[0204] Such a computer-readable storage medium 72 is, for example, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any combination thereof.

[0205] As a non-exhaustive list of more specific examples, the computer-readable storage medium 72 is a mechanically encoded device, such as punched cards or raised structures in a groove, a floppy disk, a hard disk drive, a read-only memory (ROM), a random access memory (RAM), an erasable programmable read-only memory (EROM), an electrically erasable readable memory (EEPROM), a magneto-optical disk, a static random access memory (SRAM), a compact disc (CD-ROM), a digital versatile disc (DVD), a USB flash drive, a floppy disk, a flash memory, a solid-state drive (SSD), or a PC card such as a PCMCIA memory card.

[0206] A computer program is stored on the computer-readable storage medium 72. The computer program comprises one or more sequences of stored program instructions.

[0207] Such program instructions, when executed by the data processing unit 62, cause steps of the observation method to be executed.

[0208] For example, the form of the program instructions is a source code form, a computer-executable form, or any intermediate form between source code and a computer-executable form, such as the form resulting from the conversion of the source code via an interpreter, assembler, compiler, linker, or locator. Alternatively, the program instructions are microcode, firmware instructions, state definition data, integrated circuit configuration data (e.g., VHDL), or object code.

[0209] The program instructions are written in any combination of one or more languages, e.g. an object-oriented programming language (FORTRAN, C++, JAVA, HTML, Python), a procedural programming language (e.g. C language). Alternatively, the program instructions are downloaded from an external source via a network, as is the case, in particular, for applications. In this case, the computer program product comprises a computer-readable data carrier on which the program instructions are stored or a data carrier signal on which the program instructions are encoded.

[0210] In each case, the computer program product includes instructions that can be loaded into the data processing unit 62 and adapted to cause execution of the design method when executed by the data processing unit 62.

[0211] Depending on the embodiments, the execution is entirely or partially carried out either on the analysis unit 36, i.e. a single computer, or in a system distributed between several computers (in particular via the use of cloud computing).

[0212] The operation of the observation system 10 will now be described with reference to several implementations of an exemplary observation method.

[0213] The observation method is a method of observing a sample 12 aimed at obtaining information on the sample 12.

[0214] Depending on the applications, the information will differ.

[0215] In what will be described later, according to a first application, the information is the sensitivity of the organism 18 to one or more antibiotic(s). Such a first application corresponds to the performance of an antibiogram.

[0216] According to a second application, the information is identification information of the organization 18.

[0217] Each of these applications have in common the acquisition of images of Petri dish 20 and the analysis of these images to obtain information relating to the phenomena taking place in Petri dish 20.

[0218] The observation process thus comprises two phases, namely an image acquisition phase P1 and an image analysis phase P2.

[0219] The following will first describe the P1 obtaining phase common to each of the two aforementioned applications before describing the P2 analysis phase specific to each application.

[0220] An example of a flowchart of a P1 acquisition phase is presented schematically in Figure 4.

[0221] In the obtaining phase P1, the observation system 10 seeks to obtain images of the sample 12.

[0222] For this, according to the example described, for each image of the sample 12 obtained, the obtaining phase P1 comprises the implementation of several steps, namely an acquisition step E100, a determination step E102, an application step E104 and a fusion step E106.

[0223] During the acquisition step E100, the observation system 10 acquires a set of images of a part of the sample 12.

[0224] This corresponds to the fact that even if the field of the sensor 50 is relatively large, it does not allow to obtain an image of the entire Petri dish 20 in a single acquisition.

[0225] Before detailing these acquisition strategies, it should be emphasized that before the acquisition step E100, each Petri dish 20 is transported from housings 39 to the acquisition area defined for the sensor 50.

[0226] In addition, a Petri dish 20 lid removal operation can be performed using a suction cup in conjunction with a vacuum pump. This makes it easier to take images.

[0227] After image acquisition, sample 12 is then returned to incubator 30.

[0228] In practice, the arm 40 moves the clamps 41 until they grip the sample 12 and then transports it by placing it on the housing 48.

[0229] Similarly, for the return, the arm 40 moves the clamps 41 until they grip the sample 12 and lead it to one of the housings 39.

[0230] Before placing the Petri dish 20 back into the incubator 30, the lid is replaced.

[0231] For completeness, it is also possible not to remove the cover to implement the process which will now be described.

[0232] Figure 5 schematically illustrates a first acquisition strategy allowing the entire Petri dish 20 to be covered.

[0233] In this example, Petri dish 20 has a diameter of 90 mm.

[0234] According to this first example, 12 distinct images are acquired by the sensor 50.

[0235] Each image corresponding to a spatial area smaller than Petri dish 20, these images will be called thumbnails in the following to avoid any ambiguity with an image of Petri dish 20 which can be reconstructed from these thumbnails.

[0236] A thumbnail image is thus a transmission image of an area of ​​the sample 12 obtained by acquisition by the sensor 50 of the light emitted by the light source 42.

[0237] Each image corresponds to the same field of vision and therefore has the same dimension.

[0238] Each of these thumbnails corresponds to a rectangle in Figure 5 and is numbered from 11 to 112 to show the order in which these different thumbnails are acquired. The thumbnails are taken in successive lines from the upper left corner to the lower right corner.

[0239] According to the example described, Petri dish 20 is traversed in four lines, so that each line contains three thumbnails.

[0240] This corresponds to a scan of sample 12 by observation system 10.

[0241] It is the Petri dish 20 which is moved, the light source 42 and the sensor 50 remaining fixed.

[0242] The first image corresponding to rectangle 11 has a spatial overlap with the second image corresponding to rectangle I2.

[0243] The spatial overlap corresponds to an overlap area indicated by reference 120 in Figure 5.

[0244] In the following, such an overlapping area is referred to as a lateral overlapping area 120.

[0245] According to the example of Figure 5, the lateral overlapping zone 120 is rectangular and corresponds to an area between 10% and 30% of the area of ​​an image.

[0246] Each image in the same row has the same lateral overlap area with the adjacent image.

[0247] This implies that the second image has two overlapping zones with the images of the same line, a first lateral overlapping zone 120 with the first image (on its left part) and a second lateral overlapping zone 122 with the third image (on its right part).

[0248] The first image also has an overlapping area with the fourth image.

[0249] The overlap area is indicated by reference 124 in Figure 5.

[0250] In the following, such an overlapping area is referred to as a vertical overlapping area 124.

[0251] The vertical overlap zone 124 is a rectangular zone having a surface area smaller than the surface area of ​​the lateral overlap zone 120 between the first image and the second image.

[0252] However, this could be the top surface, which would be particularly indicated for certain sensor sizes.

[0253] According to the example of FIG. 5, the surface area of ​​the vertical overlap zone 124 is less than 5% of the surface area of ​​a thumbnail.

[0254] As seen in Figure 5, each vertical overlapping zone 124 has a common zone 126 with a lateral overlapping zone 120. Due to the shape of the overlapping zones, the common zone 126 has a rectangular shape.

[0255] Common area 126 is thus a common area with four thumbnails.

[0256] For illustration, the common area 126 of the first image is common to the second image, the fourth image and the fifth image.

[0257] An experimental example of the imagettes obtained by such an acquisition strategy is shown in Figure 6.

[0258] According to the second example of acquisition strategy corresponding to Figure 7, 15 distinct images are acquired along five lines, so that each line also has three images like the first example (left figure).

[0259] In this example, similarly to the previous case, each image has a lateral overlap zone 130 and a vertical overlap zone 132.

[0260] Compared to the first example, the lateral overlap zone 130 corresponds to a relative surface area compared to the surface area of ​​a similar image (always between 10% and 30%) while the vertical overlap zone 132 has a relative surface area compared to the surface area of ​​a larger image, typically of the same order of magnitude as the lateral overlap zone 130.

[0261] This leads to a common area 134 having a square shape.

[0262] Such configurations make it possible to obtain a sufficient overlapping surface between the different images to carry out the following steps.

[0263] In particular, it is advantageous that, during the acquisition step E100, the imagettes are acquired along successive lines to cover the entire sample 12.

[0264] Preferably, the number of acquired images is between 12 and 15.

[0265] The following steps, namely the determination, application and fusion steps, aim to achieve an adjustment of the thumbnails.

[0266] Such an adjustment is sometimes referred to as "stitching."

[0267] As will appear in the description that follows, the overlap zones make it possible in particular to take into account the inaccuracies in the positioning of the Petri dish 20 and to determine the transformations to be applied to replace the images in the same reference frame.

[0268] The determination step E102 aims to obtain a transformation to be applied to each image to correct both the translation and rotation offsets and thus obtain a transformation specific to each image.

[0269] Translational and rotational offsets come from several elements that can combine. Thus, the inaccuracies of mechanical components (grippers and movement arms in particular) can generate such offsets.

[0270] Due to repeated movements of the Petri dish 20, it may rotate relative to whatever is holding it in place.

[0271] Another cause is the intervention of the operator who may have to examine a Petri dish 20 more closely by eye and who does not put the Petri dish 20 exactly back in its original position.

[0272] According to the example described, the determination step E102 comprises several operations.

[0273] A first operation aims to determine the distortion between two neighboring images.

[0274] The thumbnails are denoted A and B in the following and are assumed to be part of the same row. For example, thumbnail A is the first thumbnail and thumbnail B is the second thumbnail.

[0275] In a first operation, the lateral overlapping zone between images A and B is cut into n sub-zones.

[0276] In this case, the sub-zones are identical to facilitate calculations, but it would be possible to implement determination step E102 with sub-zones of different surfaces.

[0277] Furthermore, the sub-zones advantageously have a rectangular shape for the same computational reasons.

[0278] The number n is a natural integer strictly greater than 2.

[0279] The number n is determined so that the shifts induced by rotation errors are locally negligible within the sub-area.

[0280] Rotation errors are considered negligible as long as they correspond to a maximum shift of one pixel within a sub-zone.

[0281] As an order of magnitude, for a lateral zone of size 1914*6380, a number n of 252 guarantees a negligible rotational shift.

[0282] We thus obtain a division of the overlap zone for the image A leading to a set of n sub-zones of the image noted sub-zones Ai, the index i varying between 1 and n.

[0283] Similarly, we also obtain a set of n sub-zones of the image B noted sub-zones Bj.

[0284] The second operation then consists of determining the translational offset between the two overlapping zones of images A and B.

[0285] By construction, each sub-zone Ai corresponds to a respective sub-zone Bi. In the absence of errors, the sub-zones Ai and Bj would be identical two by two.

[0286] Due to the previous division, the sub-zones Ai and Bi only present a translational shift.

[0287] When the distortions are limited to simple translations, it is sufficient to identify a pair of corresponding points in each of the sub-areas, the translation vector being the offset between the two points when the images are superimposed.

[0288] This situation is schematically illustrated in Figure 8 for a sub-area A1 (image on the left) and a corresponding sub-area B1 (center image) with an illustration pattern. Two corresponding points are indicated by a cross on these images.

[0289] When the patterns are substantially superimposed, the two points are not superimposed and the translational shift then appears.

[0290] From a mathematical point of view, this can be written as follows.

[0291] Noting p A l e IR 2 and p B l e IR 2 two corresponding points belonging respectively to the sub-zones Ai and Bj, the translational shift of the sub-zone Ai with respect to the sub-zone Bi verifies the following relation:

[0292] PB = PA + t A l B

[0293] For this, it is possible to use a phase correlation method.

[0294] This gives the set of translational offset values ​​for each point.

[0295] A third operation is then to determine an affine transformation allowing the transition from the overlap zone of image A to image B.

[0296] This means that we are trying to determine an affine transformation M which is composed of a rotation of angle 0 and a translation of vector corresponding to the vector

[0297] For this, the following equation is solved:

[0298] P B = Mr. P A

[0299] Or :

[0300] Any technique that can converge here to a solution for this equation can be considered. In some cases, convergence can be facilitated by the use of an outlier removal technique.

[0301] Such outliers may appear because the thumbnails contain both the interior of Petri dish 20, the periphery of Petri dish 20, and the exterior of Petri dish 20.

[0302] To eliminate outliers, the RANSAC method can be applied.

[0303] The RANSAC method is a method for estimating the parameters of certain mathematical models. More precisely, it is an iterative and non-deterministic method used when the observed data set may contain outliers.

[0304] The abbreviation RANSAC refers to the English term "RANdom SAmple Consensus" which literally means "random sample consensus".

[0305] A similar approach can be applied for any overlap area.

[0306] Thus, in the example described, for the first image, three matrices M will be obtained, one with the second image, one with the fourth image and one with the fifth image.

[0307] At the end of the third operation, the relative distortions between each image are obtained.

[0308] The fourth operation aims to obtain the transformation function to apply to each image to minimize distortions between each image.

[0309] Such an operation can be carried out according to the methodology which will now be explained.

[0310] Let (y AB p BA ) k un set of k pairs of corresponding points in two thumbnails A and B, such that p AB e A and p AB e B where the notations A and B respectively designate the set of points of the images A and B.

[0311] For simplicity, it will be noted P AB = PAB ET PBA = (.PBA) k in the sequel.

[0312] In the absence of rotation and translation errors for imagettes A and B, the points would be superimposed so that P would come AB - P BA = 0.

[0313] In the present operation, we therefore seek affine transformations of the plane such that: APAB ~ M B PBA = 0

[0314] Or :

[0315] M A is the affine transformation matrix applied to the image A, M B is the affine transformation matrix applied to image B. As before, each affine transformation M A or M B which is the compound of a rotation of angle 0 and a translation of vector corresponding to the vector

[0316] \ L y)

[0317] Let M = {M A, A e 7} the set of transformation matrices applied to each of the images of the set of thumbnails noted V.

[0318] To ensure the uniqueness of the solution, an image fl e V is chosen as a reference image. By definition, it is such that its transformation matrix is ​​the identity:

[0319] Or :

[0320] • is the affine transformation matrix applied to the image Q.

[0321] With this definition, the problem to be solved can be expressed mathematically as:

[0322] Or :

[0323] • S A B is an indicator function which is equal to 1 if the images A and B have a common overlapping zone, 0 otherwise.

[0324] This corresponds to an overdetermined linear optimization problem. Any technique that can solve such a problem can be used here.

[0325] For example, the technique used may be a least squares technique.

[0326] At the end of this fourth operation and therefore of the determination step E102, the expression of the set of transformations of the set M is thus obtained. We note the optimal transformation thus obtained for the image A with the notation M° pt .

[0327] During the application step E104, the transformations thus obtained are applied to the original image, that is to say that mathematically for all the images: îj = M° pt . Ij

[0328] Or :

[0329] • Ij designates the original image, the images being identified by an index j, and

[0330] • M° pt the optimal transformation matrix determined in the previous operation, and

[0331] • j denotes the corrected image. By applying the respective optimal transformation for each of the image Ij, all the corrected image j are thus obtained.

[0332] During the fusion step E106, the aim is to carry out the fusion of all the recalibrated images Ç to obtain the image.

[0333] To do this, the images are positioned according to the selected acquisition diagram, here that of figure 5.

[0334] Then, the issue of overlapping areas must be addressed.

[0335] According to the example described, the overlapping areas are merged.

[0336] Such fusion can be achieved by averaging the signal from each of the images.

[0337] Each point in the final overlap area is thus the arithmetic mean of the value of the points in the overlap area in the first image and the points in the overlap area in the second image.

[0338] Such a fusion mode makes it possible to soften residual discontinuities or shifts.

[0339] This is visible in Figure 9 which represents the image obtained by superimposing the images without fusion on the left and the image obtained with the implementation of the fusion mode which has just been described.

[0340] In another example, the closest image signal is chosen.

[0341] In such a case, the overlapping area is cut into two parts of equal size.

[0342] For the special case of a side overlap area, the left half of the overlap area is filled by the left image and the right half of the overlap area is filled by the right image.

[0343] A similar operation is performed for the case of a vertical overlap zone.

[0344] Such a blending mode allows to preserve the background texture for later analyses, as will be explained later in the description.

[0345] Other fusion methods may be considered depending on the treatment that will be done to the overlapping areas.

[0346] A set of techniques has just been presented which allow an image to be obtained from thumbnails.

[0347] This is therefore a spatial reconstruction allowing us to manage the positioning problems of sample 12.

[0348] The analysis unit 36 ​​then implements an analysis phase P2 of the images.

[0349] First, with reference to Figure 10, a P2 analysis phase carried out for an antibiogram is described. An antibiogram is a test of the sensitivity of bacteria to antibiotics.

[0350] Such a test is often referred to by the abbreviation AST, which stands for "Antibiotic Susceptibility Testing."

[0351] It should be noted here that an antibiogram is also used for other organisms such as fungi. In this sense, an antibiogram is a test of each organism's sensitivity to antibiotics.

[0352] The P2 analysis phase of the images thus aims to determine, from the images obtained by the P1 acquisition phase, the sensitivity of each organism 18 to at least one antibiotic.

[0353] For this, it is assumed that the substrate 16 is capable of growing at least a portion of the set of organisms 14 and that the sample 12 comprises at least one area in which the substrate 16 comprises an antibiotic.

[0354] For this, the antibiotic is introduced in the form of a 100 lozenge as shown schematically in Figure 11.

[0355] As visible in this figure 11, a 100 pellet corresponds to a cylinder.

[0356] In other embodiments, the antibiotic may be introduced in the form of a strip.

[0357] According to the example of Figure 10, the analysis phase P2 comprises an obtaining step E150, a detection step E152, and a determination step E154.

[0358] During the obtaining step E150, the analysis unit 36 ​​obtains, for each image, the bacterial mat.

[0359] When the concentration of bacteria is very high on a culture medium, the bacteria grow, forming what is called a bacterial lawn. The bacterial lawn is then a multi-layered film of bacteria covering the surface of the agar medium. This coverage is more or less uniform but does not allow for counting or identifying bacterial colonies.

[0360] As an illustration, the E150 obtaining step involves three operations.

[0361] In a first operation, the calculator 56 detects the 100 antibiotic tablets.

[0362] To implement such detection, the computer 56 implements thresholding on the gray level in the image. Indeed, on a microscopy image without a lens, the pellets, which are very opaque, appear black. A series of morphological operations makes it possible to complete the detection by using the information that the pellets 100 are black discs on the images, which makes it possible to remove dust and other artifacts detected by the thresholding step. Other techniques based on shape recognition could also be used to implement such detection. As an illustration, a CNN neural network could be used by the computer 56.

[0363] During a second operation, the computer 56 detects the edges of the box 20.

[0364] More precisely, the computer 56 detects the holding device 25 which appears in black on the image.

[0365] The holding device 25 is, for example, detected by applying a threshold to the gray level of the image. The holding device 25 delimits the portion outside the Petri dish 20 and the portion inside the Petri dish 20. The outer portion is masked, and the inner portion is retained.

[0366] The third operation is to remove the detected elements from the image.

[0367] This helps isolate the bacterial mat.

[0368] Such isolation is advantageously used here to obtain a gain in robustness for the continuation of the implementation of the analysis phase P2.

[0369] The computer 56 then implements the detection step E152.

[0370] Detection step E152 aims to detect bacterial or fungal growth.

[0371] For this purpose, the fact that an evolution of the texture of the bacterial mat, whether spatial or temporal, is linked to bacterial growth is used.

[0372] A texture is a structured spatial arrangement of pixels.

[0373] In this case, the parts of the image corresponding to the bacterial mat contain a characteristic texture (pattern and spatial frequency) different from the parts of the image where the bacteria have not developed.

[0374] In other words, the difference in texture makes it possible to differentiate the bacterial mat from the inhibition zone.

[0375] This detection step E152 can be implemented by determining the texture in the current image and then comparing it with a previously obtained texture.

[0376] The calculator 56 can, for example, determine the texture using Haralick criteria.

[0377] Haralick criteria are metrics calculated from the gray-level co-occurrence matrix and used to quantify the spatial organization of the different gray levels in the image.

[0378] By calculating these criteria on a sliding window of the image, a local measure of the criterion is obtained, throughout the image.

[0379] According to another example, the calculator 56 uses histograms. More precisely, the calculator 56 produces a gray level histogram which it compares to a reference histogram corresponding to the presence of an inhibition zone, the distance between the two histograms serving as a metric for the texture.

[0380] Alternatively, the reference histogram is that of an image in which no bacteria or fungi have grown.

[0381] In yet another example, a PSD is used.

[0382] Instead of creating a histogram as before, we create a PSD on the image which is compared to a reference PSD.

[0383] The abbreviation "PSD" refers to the corresponding English term "Power Spectrum Density". This provides a representation of the frequency organization of pixels in the image.

[0384] It can also be considered the use of a wavelet transform to obtain the texture.

[0385] The wavelet transform consists of applying a basic function, called a wavelet, to several scales of the image, thus making it possible to characterize the texture of the image at several levels of resolution. Applying the wavelet transform to a part of the image returns a list of wavelet coefficients that characterize the texture at different levels of resolution. These wavelet coefficients are used as texture indicators and can be compared with reference coefficients, or be input to a classifier to classify the texture.

[0386] Another example is the use of support vector machines. Such a tool is more often referred to by the abbreviation SVM, which refers to the corresponding English term "Support-Vector Machine".

[0387] Such an SVM tool can be used as a binary classifier whose two possible outputs would be: Bacterial mat or zone of inhibition.

[0388] The input to the SVM could be a list of pixels, or metrics from the previously detailed methods such as the metric from the Haralick criteria.

[0389] The training of such an SVM is carried out on a set of images containing inhibition zones and bacterial mat.

[0390] The different examples can be combined, the different results obtained by implementing the aforementioned techniques then being the subject of an aggregation, for example the calculation of an average of the results or the use of an SVM which takes as input all the calculated metrics. The texture is thus determined.

[0391] The calculator 56 then makes a comparison between this texture and a previously obtained texture. When the difference is greater than the measurement noise, this means the presence of bacterial or fungal growth.

[0392] The calculator 56 thus detects bacterial or fungal growth.

[0393] After such detection, the calculator 56 implements the determination step E154.

[0394] During the determination step E154, the computer 56 determines at least one characteristic relating to the set of organisms 14 by analyzing at least one of the images obtained.

[0395] The analysis consists of measuring the inhibition diameter based on an image which can be either the original image or the texture image.

[0396] When the inhibition zones are not yet sufficiently visible on the image, the texture image will generally be preferred.

[0397] In the proposed implementation example, the characteristic determined in determination step E154 is the sensitivity of organism 18 to the antibiotic.

[0398] The sensitivity is obtained here by determining a characteristic dimension of the inhibition circle associated with the growth of the set of organisms 14 on the area in which the substrate 16 comprises an antibiotic.

[0399] The circle of inhibition refers to the delimitation of the growth zone of organisms 14, growth which is inhibited by the presence of an antibiotic.

[0400] The organism-free zone 14 is usually called the inhibition zone.

[0401] In the most usual case, the delimitation is a circle but sometimes presents a more complex shape, particularly in the presence of interaction with other antibiotics or near the edge of the Petri dish 20.

[0402] The dimension of the inhibition circle defined as an average dimension of the distance between two opposite points.

[0403] This dimension, which we will call diameter, depends on the sensitivity of the bacterial strain to the antibiotic agent used.

[0404] In the following, a determination of the diameter of the inhibition circle is proposed.

[0405] Estimating the inhibition diameter then makes it possible to estimate the organism's sensitivity to the antibiotic agent, according to the diffusion method in agar medium defined by EUCAST (European reference body), CLSI (American reference body) or CA-SFM (French microbiology body).

[0406] For example, for each species / antibiotic pair, EUCAST defines an inhibition diameter threshold below which the organism is defined as resistant, and above which the organism is defined as sensitive. In the case of Figure 10, the determination step E154 includes an extraction operation and an estimation operation.

[0407] The first operation is performed by a contour extraction technique.

[0408] In the general case, the circle of inhibition is indeed circular.

[0409] In such a case, the contour extraction technique is a logistic fit to the radial mean of the inhibition circle.

[0410] Alternatively, it is also possible to use as a contour extraction technique a contour extraction followed by a determination of the circle inscribed in the inhibition zone.

[0411] In such a case, a Canny detector or gradient detectors, such as a Sobel filter and a Prewit filter, can be used.

[0412] In more complex cases where the zone of inhibition is not circular, the previous technique can be used using as an approximation that the zone of inhibition is defined by circular arcs.

[0413] Alternatively, it is possible to use contour extraction to obtain the exact shape of the inhibition zone and possibly estimate its diameter as the diameter of the inscribed circle.

[0414] In another example, the texture of the bacterial mat is used to obtain these contours.

[0415] The estimation operation consists of making an estimate of the dimension of the contours using an estimation technique.

[0416] In a first example, the estimation technique consists of calculating the radial mean.

[0417] In a second example, the estimation technique consists of making a logistic adjustment.

[0418] The technique involves calculating a radial average of the gray level around the center of the patch.

[0419] More precisely, the technique involves extracting gray-level profiles (sections) orthogonally to the contour of the patch, all around the patch at a predefined angular step. The angular step is, for example, one degree.

[0420] The profiles are then averaged.

[0421] The profile is then approximated by a logistic function.

[0422] Alternatively, any other function that models a gray level transition can be used.

[0423] Additionally, a RANSAC technique can also be used to eliminate outliers from the extracted contour. Implementing both operations allows for a better estimation of the diameter in the case of a non-circular inhibition zone.

[0424] This is particularly useful for detecting interactions between antibiotics since the presence of synergy or antagonism generally leads to non-circular shapes for the zone of inhibition.

[0425] Furthermore, regardless of the shape of the inhibition zone, a gain in robustness is observed for the measurement of the diameter of the inhibition zone.

[0426] The process therefore allows rapid detection of characteristics relating to organisms.

[0427] Such a gain in speed and resolution makes it possible to use samples 12 for which the pellet 100 has a reduced characteristic size.

[0428] For example, the characteristic size may be reduced by a factor of 3 to have a size less than or equal to 6 mm, less than or equal to 4 mm, preferably less than or equal to 2 mm.

[0429] A minimum size of 1 mm is nevertheless desirable.

[0430] The acquisition phase P2 which was presented with reference to figure 4 can also be advantageously used in an identification process.

[0431] The identification method aims to identify each organism 18 from at least one part of the box 20.

[0432] A species is a taxonomic rank in bacterial nomenclature grouping together all strains considered to be sufficiently close to the type strain of the species to be assimilated to it.

[0433] According to the example in Figure 12, the analysis phase P2 comprises a detection sub-phase SP1 followed by an identification sub-phase SP2.

[0434] The SP1 detection sub-phase aims to detect colonies on the images obtained after the P1 acquisition phase and ideally to obtain their location.

[0435] A bacterial colony is a cluster of bacteria resulting from the duplication of a bacterial strain (sometimes several) on an agar medium. It is by dividing that bacteria organize themselves into colonies.

[0436] Such a colony is sometimes referred to by the acronym CFU, which refers to the corresponding English term for “Colony Forming Unit”.

[0437] In practice, the number of colonies on a Petri dish can be up to a few hundred.

[0438] According to the example described, the identification sub-phase SP2 comprises a calculation step E160, a verification step E162 and a determination step E164. During the calculation step E160, the calculator 56 applies a neural network to at least one image.

[0439] Before explaining this E160 calculation step in more detail, it is appropriate to give some general elements concerning the notion of neural network.

[0440] A neural network consists of an ordered succession of layers of neurons, each of which takes its inputs from the outputs of the previous layer.

[0441] More precisely, each layer consists of neurons taking their inputs from the outputs of the neurons in the previous layer, or from the input variables for the first layer.

[0442] Alternatively, more complex neural network structures can be considered with a layer that can be connected to a layer further away than the immediately preceding layer.

[0443] Each neuron is also associated with an operation, that is, a type of processing, to be carried out by said neuron within the corresponding processing layer.

[0444] Each layer is connected to other layers by a plurality of synapses. A synaptic weight is associated with each synapse, and each synapse forms a connection between two neurons. It is often a real number, which takes both positive and negative values. In some cases, the synaptic weight is a complex number.

[0445] Each neuron is capable of performing a weighted sum of the value(s) received from the neurons of the previous layer, each value then being multiplied by the respective synaptic weight of each synapse, or link, between said neuron and the neurons of the previous layer, then applying an activation function, typically a non-linear function, to said weighted sum, and delivering as output of said neuron, in particular to the neurons of the following layer connected to it, the value resulting from the application of the activation function. The activation function makes it possible to introduce non-linearity into the processing carried out by each neuron. The sigmoid function, the hyperbolic tangent function, the Heaviside function are examples of activation functions.

[0446] As an optional addition, each neuron is also able to apply, in addition, a multiplicative factor, also called bias, to the output of the activation function, and the value delivered at the output of said neuron is then the product of the bias value and the value from the activation function.

[0447] In some special cases, the neural network is a convolutional neural network. A convolutional neural network is also sometimes called a convolutional neural network or by the acronym CNN, which refers to the English term for "Convolutional Neural Networks".

[0448] In a convolutional neural network, each neuron in a single layer has exactly the same connection pattern as its neighboring neurons, but at different input positions. The connection pattern is called a convolution kernel, or more commonly, a "kernel."

[0449] Additionally, there are particular neuron layers such as a fully connected neuron layer.

[0450] A fully connected layer of neurons is one in which the neurons in that layer are each connected to all the neurons in the previous layer.

[0451] Such a type of layer is more often referred to as "fully connected" and sometimes referred to as a "dense layer".

[0452] The example of Figure 12 for calculation step E160 is first illustrated for a simple example in which the observed sample 12 is mono-microbial, that is to say that only one species of bacteria or fungus is present in the imaged box 20.

[0453] In such a case, the calculation step E160 comprises an application operation and an association operation.

[0454] During the application operation, the calculator 56 applies a classifier here.

[0455] For example, the classifier is a K-class classification neural network (each class is then a species).

[0456] The classifier is, for example, a CNN network followed by a SASE block.

[0457] The abbreviation SASE refers to the corresponding English term for "Spatially-Adaptive Squeeze-Excitation" which literally means spatially adaptive compression excitation.

[0458] The SASE block is used here to improve the abstraction capabilities of the model corresponding to the CNN network.

[0459] The classifier is trained on a set of 278 Petri dishes of 10 bacterial species.

[0460] Such a neural network provides, from an input image and the location of the colony for each colony, a set of scores associated with each of the classes.

[0461] Preferably, the input is an array of multiple frames.

[0462] For example, the input is an array of size 125x125x36 corresponding to 36 frames (acquisition of an image every 30 minutes for 18h) of a 125x125 pixel clip of a previously detected colony. Of course, the size of the clip can vary, the number of frames also and it is possible that the frames include several types of frames, each type corresponding to a specific wavelength.

[0463] In other words, for each colony i, the neural network gives a set of K scores S L where i is an integer varying between 1 and N the number of colonies.

[0464] Like the scores can be interpreted as probabilities of belonging to the corresponding class, it can be written:

[0465] According to the example considered, the calculator 56 associates the colony considered with a species during the association operation.

[0466] More specifically, calculator 56 considers that the species associated with the colony considered is the species corresponding to the score the highest.

[0467] This corresponds to the following mathematical relationship:

[0468] Or :

[0469] • e t denotes the species associated with colony i, and

[0470] • class is a function that gives the species associated with a probability.

[0471] At the end of the calculation step E160, a set of most probable species-colony associations is thus obtained for each colony.

[0472] During the verification step E162, the computer 56 verifies the reliability of the prediction result.

[0473] In other words, the calculator 56 verifies that a reliability condition of the probabilities obtained is met by comparing at least one value from the probabilities obtained with a predefined threshold.

[0474] This E162 verification step aims to handle the fact that, during their training, neural networks are trained on a predefined set of species. This results in an absence of certain classes.

[0475] Thus, in inference, the classifier does not have the capacity to give a good answer for a colony corresponding to a class unknown to the classifier.

[0476] In this example, the maximum value of the scores associated with a colony is compared to a threshold e.

[0477] When this maximum value is strictly lower than the threshold e, it is considered that it is not possible to make a prediction, i.e. the colony corresponds to an unknown species or that this prediction is not sufficiently reliable.

[0478] The threshold e is a number between 0 and 1 which is chosen according to the specific case. Preferably, the threshold e is chosen between 0.7 and 0.9.

[0479] In practice, a threshold e of 0.8 allows good performance to be obtained.

[0480] First, it is assumed that the species is known.

[0481] The calculator 56 then implements the determination step E164.

[0482] According to the example of monomicrobial sample 12, aggregation can be implemented.

[0483] In such an operation, the different previous associations are aggregated using an aggregation function to obtain a single class (a single species) for all the colonies in box 20.

[0484] Several aggregation functions can be considered at this stage.

[0485] A first aggregation function consists of retaining the most frequent species in box 20.

[0486] Mathematically, this amounts to performing the following operation:

[0487] Or :

[0488] • E designates the species retained for the whole of box 20, and

[0489] • I denotes the function such that B is 1 if e t = k and 0 otherwise.

[0490] A second example of an aggregation function consists of retaining the species whose sum of associated scores is the largest.

[0491] In such an example, the following mathematical function is used:

[0492] Where q t = max p ik is the maximum of the score associated with species e t .

[0493] Such an example of implementation of the E164 determination step makes it possible to obtain a considerable gain in robustness for this case of a mono-microbial sample 12.

[0494] This, in fact, makes it possible to remedy the fact that it is very common to have a few poorly identified colonies on a dish, despite high identification precision.

[0495] For example, with 95% accuracy, a dish of 400 colonies will statistically have 20 misidentified colonies. This implementation thus makes it possible to correct the misidentification of the 20 colonies.

[0496] Ultimately, this technique amounts to a voting technique on a set of 14 organizations (here the whole of box 20).

[0497] Thus, what has just been described can be generalized to a plurality of organisms 18 corresponding to the same species. In this case, the determination step E164 comprises the implementation of an aggregation function of the different probabilities found for each of the organisms 18.

[0498] The first example described then corresponds to an aggregation function giving as species for the plurality of organisms 18 the maximum of the set of probabilities while the second example corresponds to an aggregation function giving as species for the plurality of organisms 18 the maximum of the set of probabilities.

[0499] If the species to be identified is not known, by construction, this means that the reliability condition is not met.

[0500] The calculator 56 then provides information that each organism 18 considered belongs to a species not included in the predefined species.

[0501] This therefore allows the computer 56 to be able to manage new situations including situations not learned during the training of the neural network.

[0502] According to a more elaborate embodiment which will now be described, the calculator 56 is capable of providing additional information on an organism 18 belonging to an unknown species.

[0503] For this, a phylogenetic tree is used.

[0504] By definition, a phylogenetic tree is a tree-like representation of the relationships between different organisms 18 and is constructed from their genetic characteristics and their kinship links.

[0505] Such a tree is shown in Figure 13.

[0506] The tree has several nodes which are distributed according to phylogenetic levels from a root (highest level) to the leaves (lowest level).

[0507] The phylogenetic level is called the depth of the taxon studied in the phylogenetic tree.

[0508] The phylogenetic levels considered are thus based on the taxonomic rank, that is to say the hierarchical level in the classification of living things.

[0509] In some cases, phylogenetic levels include common taxonomic ranks, based on genetics and / or groupings of taxa or species advantageously chosen to address a problem, particularly a specific identification problem such as identifying gram-positive and gram-negative organisms.

[0510] Groupings of taxa or species are based on criteria chosen for the specific case, the criterion being able to be genetic, phenotypic, clinical (symptom, antibiotic sensitivity) or material (type of antibiogram available). According to the example shown, there are 7 levels which are in successively decreasing order: domain, phylum, class, order, family, genus and species.

[0511] However, this is only an example since there are lower level philogroups that could explain differences in morphotypes and resistance within the same species.

[0512] A node of a level is identified by a letter and a number corresponding to the order in which it appears from left to right in Figure 13.

[0513] When a phylogenetic link exists between two nodes, a link is visible in this figure 13. In this view, a node of a higher level is a parent node of a node of a lower level and conversely, a node of a lower level is a child node of a higher level.

[0514] The following table can thus be drawn up: [Table 1]

[0515] The phylogenetic tree in Figure 13 therefore includes 10 species divided into 7 genera belonging respectively to 5 families, themselves divided into 4 orders forming part of 2 classes corresponding to 2 phyla of the domain of bacteria. With biological knowledge, it is possible to draw up such a phylogenetic tree knowing the species.

[0516] In other words, it can be considered that from the 10 species E1 to E10, it is possible to derive the phylogenetic tree of Figure 13.

[0517] Of course, the 10 species E1 to E10 can be associated with a different phylogenetic tree.

[0518] In particular, instead of a 7-level tree, it could be considered a 3- or 4-level tree.

[0519] Furthermore, it is not mandatory that the levels be exactly consecutive as is the case for the tree in Figure 13. One could, for example, form a 4-level tree with the nodes D1 , P1 , P2,

[0520] F1 to F5 and E1 to E10.

[0521] Preferably, however, in practice, levels close to species E1 to E10 will be chosen to obtain the most precise information possible according to one of the techniques which will be described. In this embodiment, the verification step E162 is carried out for each node.

[0522] To do this, during the verification step E162, the calculator 56 calculates a score for each node. The score of each node is calculated as the sum of the scores of each of the child nodes.

[0523] We can thus obtain a value for each of the nodes.

[0524] It follows from such a definition that the root of the phylogenetic tree has a score equal to 1. The calculation and its results will now be illustrated through three examples.

[0525] The first example corresponds to a prediction of a species Enterococcus faecalis.

[0526] In such a case, the resulting scores of the neural network are as follows: [Table 2]

[0527] As a result, the non-zero values ​​of the other nodes are as follows:

[0528] [Table 3]

[0529] The set of nodes verifying the reliability condition (threshold here set at 0.7) are therefore nodes D1, P2, C2, 03, F4, G6 and E6.

[0530] During the determination step E164, the observation system 10 will be able to provide the user with enriched information, namely that not only the species will be known but also the other phylogenetic levels.

[0531] This example shows that using the phylogenetic tree does not interfere with accurate prediction.

[0532] Typically, the 56 calculator should not identify the correct species and not the correct class.

[0533] In the second example, the aim is to predict a species Enterobacter cloacae complex while the scores resulting from the neural network are as follows:

[0534] [Table 4]

[0535] Such a situation can occur when two species are relatively close, which is the case in this example since species E1 and E2 belong to the same genus.

[0536] It can also be noted that, in this hypothesis, the calculator 56 predicts with the highest score the species E2 or the species Escherichia coli, which is erroneous. For a threshold of 0.5, the calculator 56 would output an error or for a higher threshold, typically 0.7, the calculator 56 would not give any prediction.

[0537] By calculating each of the values ​​for the nodes, the non-zero values ​​of the other nodes are as follows:

[0538] [Table 5]

[0539] With a threshold of 0.7, nodes F1, O1, C1, P1 and D1 are reliable (meet the reliability condition) and not the others.

[0540] One way to choose the determined information is to traverse the nodes in the descending direction from the initial node (root) to the corresponding nodes in species by following a path of nodes respecting the reliability condition and by selecting the full denomination associated with the last node.

[0541] Here the path is: node F1, node 01, node 01, node P1 then node D1.

[0542] The information determined is that of node D1.

[0543] This corresponds to determining that organism 18 belongs to the family Enterobacteriaceae.

[0544] It should be noted that the information obtained at the end is correct despite an erroneous prediction at the output. For a threshold of 0.7, the observation system 10 thus provides information that the species of organism 18 is not known or not reliable but that the identified family is the Enterobacteriaceae family.

[0545] Thus, the technique which has just been described with the use of the phylogenetic tree allows a good identification at the family level in the absence of an identification at the species level.

[0546] In the third example, the aim is to identify a species Staphylococcus capitis but the neural network has not been trained to predict this species.

[0547] The species Staphylococcus capitis corresponds to the same genus G7 as nodes E8 to E10. This is indicated by an additional dotted link in Figure 14 which corresponds to the tree in Figure 13 enriched with an EX node associated with the species Staphylococcus capitis.

[0548] The implementation of the E160 calculation step leads to the following scores:

[0549] [Table 6]

[0550] As expected, the neural network predicts species relatively close to the species sought since species E9 and E10 belong to the same genus as the species to be identified.

[0551] It can also be noted that, in this hypothesis, calculator 56 predicts with the highest score species E9, i.e. species Staphylococcus saprophyticus, which is erroneous.

[0552] For a threshold of 0.5, the calculator 56 would output an error or for a higher threshold, typically 0.7, the calculator 56 would not give any prediction. By performing the calculation of each of the values ​​for the nodes, the non-zero values ​​of the other nodes are as follows:

[0553] [Table 7]

[0554] The calculator 56 then performs the same operation of traversing the nodes in the descending direction from the initial node (root) to the corresponding nodes in species by following a path of nodes respecting the reliability condition and by selecting the full name associated with the last node.

[0555] This leads him to determine that the reliable information is that corresponding to node G7.

[0556] This corresponds to determining that the organism 18 to be identified belongs to the genus Staphylococcus.

[0557] It should be noted that the information obtained at the end is correct despite an erroneous prediction at the output.

[0558] For a threshold of 0.7, the observation system 10 thus provides information that the species of the organism 18 is not known or not reliable but that the genus identified is the genus Staphylococcus.

[0559] Furthermore, because of the zero score for species E8, it can be confidently deduced that the species is not Staphylococcus aureus.

[0560] Thus, the technique just described with the use of the phylogenetic tree allows good identification at a phylogenetic level higher than that of the species even when the neural network does not have the capacity to determine the species to be identified.

[0561] From the results obtained, it appears that the use of a phylogenetic tree allows for greatly improved identification.

[0562] This approach thus makes it possible to provide information on higher phylogenetic levels when the score obtained at a lower level is not sufficiently high. This allows for a gain in robustness by constraining the groupings of species and the path of the tree by genetic links.

[0563] Furthermore, without any retraining of the neural network, the observation system 10 is capable of performing identification at a higher phylogenetic level even for unknown species.

[0564] The described approach thus makes it possible to efficiently manage the problem of training the identification neural network.

[0565] Indeed, to recognize a species of bacteria, the neural network must be trained on a large number of strains of this species. Some species of bacteria are very common in clinical analyses, so it is easy to build a training database for these species. On the other hand, many species are much rarer: it is therefore difficult to build a training database large enough to integrate these species.

[0566] Where classical classification approaches at best provide no information, and at worst are systematically wrong about these species not present in the training space, the approach just described makes it possible to predict relevant information about the organism 18 tested.

[0567] The process just described therefore allows not only good identification of the species of organisms 18 but also the provision of reliable information in the case of unknown species or species that are difficult to recognize with certainty.

[0568] Other embodiments of the observation methods described above can advantageously be envisaged.

[0569] In particular, according to one embodiment, the time interval between each obtaining of an image during the implementation of the obtaining phase P1 is adjustable.

[0570] Advantageously, the time interval depends on at least one determined characteristic.

[0571] For example, for the growth of the bacterium Escherichia coli incubated at 37°, it is known that a lag phase of 2 hours takes place, then an exponential growth of 10 hours then a slowdown to reach a stationary phase then a decline phase.

[0572] In such a case, one could consider taking an image every hour during the lag phases and an image every 10 minutes during the growth phase.

[0573] It could even be envisaged that the time interval is determined in real time based on the observed evolution of colony growth.

[0574] To reduce the computational load, it may also be interesting to limit the part of the image to be analyzed. For example, it is possible to determine at least one property relating to the detected colonies and then select colonies, the computational steps (in particular application of the neural network) being implemented only for the selected colonies.

Claims

CLAIMS 1. A method of observing a sample (12), the sample (12) comprising: - a set of organisms (14), - a solid substrate (16) supporting the set of organisms (14), the method being implemented by an observation system (10) comprising a lensless imaging unit (34) and the method comprising: - a phase of obtaining images of the sample (12), - an image analysis phase, the analysis phase including: - a step of calculating the probability that at least one organism (18) belongs to a predefined species, the calculation step comprising the application of a neural network to at least one image to obtain the probability that said at least one organism (18) belongs to a predefined species, the predefined species being the set of species that the neural network has been trained to identify in an image, - a step of verifying that a reliability condition of the probabilities obtained is met by comparing at least one value from the probabilities obtained with a predefined threshold, and - a step of determining at least one piece of information relating to the identity of the at least one organism (18), the information being membership of a species not included in the predefined species when the reliability condition is not met.

2. Observation method according to claim 1, in which the value resulting from the probabilities is the maximum value of the calculated probabilities.

3. Observation method according to claim 1 or 2, in which the organisms (18) are organized into colonies, the observation method further comprising a colony detection phase for detecting the colonies and locating them, the neural network of the calculation step also taking as input the location of at least one colony.

4. Observation method according to claim 3, in which the analysis phase comprises a step of determining at least one property relating to the detected colonies and a step of selecting the colonies to obtain selected colonies, the obtaining step being implemented only for the selected colonies.

5. Observation method according to any one of claims 1 to 4, in which a plurality of organisms (18) correspond to the same species, the determination step comprising the implementation of an aggregation function of the different probabilities found for each of the organisms (18).

6. Observation method according to claim 5, in which the aggregation function gives as species for the plurality of organisms (18) the maximum of the set of probabilities.

7. Observation method according to claim 5, in which the aggregation function gives as species for the plurality of organisms (18) the species for which the sum on each organism (18) of the probabilities that the organism (18) belongs to the species is maximum.

8. Observation method according to any one of claims 1 to 7, wherein, during the calculation step, the neural network is a multi-class classification neural network.

9. Observation method according to any one of claims 1 to 8, in which a phylogenetic tree has been defined for the neural network, the phylogenetic tree comprising a set of nodes distributed according to at least two phylogenetic levels, the lowest phylogenetic level being the species, the nodes being linked together when there is a phylogenetic link between them, the verification step comprising the calculation of a probability for each node from the probabilities calculated in the calculation step and the verification of the conformity condition for each node.

10. Observation method according to claim 9, in which, during the determination step, the information determined is the membership at the lowest phylogenetic level meeting a conformity condition.

11. Observation method according to claim 9 or 10, wherein the phylogenetic levels are chosen from the list consisting of phyllum, class, order, family and genus.

12. Observation method according to any one of claims 9 to 11, in which the number of phylogenetic levels is greater than or equal to 4.

13. Observation method according to any one of claims 1 to 12, in which the predefined threshold has a value between 0.7 and 0.

9.

14. Observation system (10) of a sample (12), the sample (12) comprising: - a set of organisms (14), and - a solid substrate (16) supporting the set of organisms (14), the observation system (10) comprising a lensless imaging unit (34) and suitable for: - obtain images of the sample (12), - analyze the images, the analysis including: - calculating the probability that at least one organism (18) belongs to a predefined species, the calculation comprising applying a neural network to at least one image to obtain the probability that said at least one organism (18) belongs to a predefined species, the predefined species being the set of species that the neural network has been trained to identify in an image, - verification that a reliability condition of the probabilities obtained is met by comparison of at least one value from the probabilities obtained with a predefined threshold, and - the determination of at least one piece of information relating to the identity of the at least one organism (18), the information being membership of a species not included in the predefined species when the reliability condition is not met.

15. Observation system according to claim 14, the observation system (10) further comprising a regulating element (37) for the environment of the observation system (10).

Images