Sample observation method for identification purposes and related devices

Abstract

The invention relates to a method for observing a sample comprising a set of living beings, implemented by an observation system (10) comprising a lensless imaging unit (34) and comprising: - a stage of obtaining an image of the sample; and - an image analysis stage comprising: - a calculation of a probability that a living being belongs to a preset species by applying a neural network to obtain a belonging probability; - a verification that a reliability condition of the belonging probability is satisfied by comparing a value resulting from the probability with a preset threshold; and - a determination of information related to the identity of the living being, which belongs to a species that is among the preset species or, when the reliability condition is not satisfied, to a species that is not among the preset species.

Description

Technical Field

[0001] This invention relates to an observation method and related apparatus. The invention belongs to the field of microbiology, and more specifically relates to the identification of organisms and the study of interactions between organisms and culture media. Background Technology

[0002] It is known to culture organisms on specific culture media, such as agar-based nutrient media. These media can increase the number of organisms sampled, either specifically or through spatial separation to form different types of organisms in the sample. Specific amplification is achieved through the presence of inhibitors that promote the growth of some organisms while inhibiting others. Spatial separation of different types of organisms can only be achieved in the case of multi-microbial samples.

[0003] One of the disadvantages of culturing on such media is the time required to obtain a group of organisms from a single organism that can be observed and sampled by an operator. It typically requires about twenty generations, corresponding to a relatively long period of at least 24 hours.

[0004] Therefore, there is an urgent need for a robust and rapid method for observing organisms. Summary of the Invention

[0005] Therefore, this specification discloses a sample observation method, wherein the sample comprises: - A group of organisms, and - A solid matrix supporting the group of organisms; The method is implemented by an observation system including a lensless imaging unit, and the method includes: - Sample image acquisition stage, which includes the following steps for each acquired image: - Collect multiple images of a portion of the sample. Each collected image is called a sub-image. Each sub-image has an overlapping region that overlaps with at least one other sub-image, and each region of the sample is visible in at least one sub-image. - The steps of determining the transformation to be applied to each sub-image to correct translation and rotation offsets using overlapping regions to obtain the specific transformation for each sub-image; - Apply the transformation to each sub-image to obtain the corrected sub-image; and - The step of fusing sub-images to obtain a sample image; and - Image analysis phase, the analysis phase including: - By analyzing at least one acquired image, determine at least one feature associated with the group of organisms.

[0006] According to the specific implementation plan, the observation method includes one or more of the following features, which can be used individually or in any technically feasible combination: - The observation system includes a sample storage space, and the imaging unit includes an optical sensor defining a collection area; for each acquired image, the collection phase further includes: a step of transporting the sample from the storage space to the collection area to perform the collection step, and a step of transporting the sample back to the storage space after the collection step is completed.

[0007] - The two transport steps are performed by a robotic arm equipped with a gripper.

[0008] - For each pair of adjacent sub-images, the determination step includes: determining the distortion between the two adjacent sub-images, determining the translation offset between the two overlapping regions of the adjacent sub-images, determining the affine transformation that can be transformed from the overlapping region of the first sub-image of the two adjacent sub-images to the second sub-image, and determining the transformation function to be applied to each sub-image to minimize the distortion between each sub-image.

[0009] - Determine the transformation function to be applied to each sub-image to minimize the distortion between each sub-image, including solving an overdetermined linear optimization problem.

[0010] - In the fusion step, each overlapping region of the image is the average value of the overlapping regions of the corresponding corrected sub-image.

[0011] - In the acquisition step, sub-images are acquired row by row to cover the entire sample.

[0012] - In the acquisition step, the number of sub-images acquired is 12 to 15.

[0013] - The solid matrix is ​​adapted to culture at least a portion of the population of organisms, and the sample contains at least one region in which the matrix contains an antibiotic; the characteristic determined in the determining step is the sensitivity of the organism to the antibiotic.

[0014] - The sensitivity of the organism to the antibiotic is obtained by measuring the characteristic size of the inhibition zone, which is associated with the growth of at least a portion of the population of organisms in the antibiotic-containing matrix region; the measurement includes the steps of extracting the contour of the inhibition zone and estimating the characteristic size from the extracted contour.

[0015] - Define a microbial biofilm for the sample; the analysis phase includes the step of obtaining the microbial biofilm, and the determination step uses the obtained microbial biofilm to determine at least one characteristic associated with the group of organisms.

[0016] - During the acquisition phase, the time interval for acquiring each image is adjustable.

[0017] - The time interval depends on the determined at least one feature.

[0018] - The value obtained from the probability is the maximum value of the calculated probability.

[0019] - The organism is divided into colonies; the observation method further includes a colony detection stage to detect and locate colonies; the neural network of the computation step also takes the location of at least one colony as input.

[0020] - The analysis phase includes the steps of determining at least one attribute associated with the detected colonies, and the step of selecting colonies to obtain selected colonies; the obtaining step is performed only on the selected colonies.

[0021] - Multiple organisms correspond to the same species; the determination step includes performing an aggregation function on the probabilities of each organism.

[0022] - The aggregation function takes the maximum value in the probability set as the species of the plurality of organisms.

[0023] - The aggregation function classifies the species of the plurality of organisms as such that the sum of the probabilities of an organism belonging to that species is the largest for each organism.

[0024] - In the calculation step, the neural network is a multi-class neural network.

[0025] - Define a phylogenetic tree for the neural network, the phylogenetic tree containing a set of nodes distributed according to at least two phylogenetic levels, the lowest level being species; nodes with phylogenetic associations are interconnected; the verification step includes calculating the probability of each node based on the probability calculated in the calculation step, and verifying the conformity conditions of each node.

[0026] - In the determination step, the determined information belongs to the lowest phylogenetic level that meets the compliance conditions.

[0027] - The phylogenetic hierarchy is selected from: phylum, class, order, family, and genus.

[0028] - The number of phylogenetic levels is greater than or equal to 4.

[0029] - The preset threshold value is between 0.7 and 0.9.

[0030] This specification also describes a method for observing a sample, wherein the sample comprises: - A group of organisms; - A solid matrix supporting the group of organisms; The method is implemented by an observation system including a lensless imaging unit, and the method includes: - Sample image acquisition stage; - Image analysis phase, the analysis phase including: - A step of calculating the probability that at least one organism belongs to a preset species, the calculation step comprising applying a neural network to at least one image to obtain the probability that the at least one organism (18) belongs to a preset species; the preset species being a set of species that the neural network can be trained to identify in an image; - A step of verifying whether the obtained probabilities meet reliability conditions by comparing at least one value of the obtained probabilities with a preset threshold; and - Determine at least one piece of information related to the identity of the at least one organism; when the reliability condition is not met, the information belongs to a species not included in the preset species.

[0031] Depending on the specific implementation, the observation method includes one or more of the following features, which may be used individually or in a combination of all technically feasible features: - The observation system includes a sample storage space, and the imaging unit includes an optical sensor defining a collection area; for each acquired image, the collection phase further includes: transporting the sample from the storage space to the collection area to perform the collection step, and transporting the sample back to the storage space after the collection step is completed.

[0032] - The two transport steps are carried out by a robotic arm equipped with a gripper.

[0033] - For each pair of adjacent sub-images, the determination step includes: determining the distortion between the two adjacent sub-images, determining the translation offset between the two overlapping regions of the adjacent sub-images, determining the affine transformation that can be transformed from the overlapping region of the first sub-image of the two adjacent sub-images to the second sub-image, and determining the transformation function to be applied to each sub-image to minimize the distortion between each sub-image.

[0034] - Determine the transformation function to be applied to each sub-image to minimize the distortion between each sub-image, including solving an overdetermined linear optimization problem.

[0035] - In the fusion step, each overlapping region of the image is the average value of the overlapping regions of the corresponding corrected sub-image.

[0036] - In the acquisition step, sub-images are acquired row by row to cover the entire sample.

[0037] - In the acquisition step, the number of sub-images acquired is 12 to 15.

[0038] - The solid matrix is ​​adapted to culture at least a portion of the population of organisms, and the sample contains at least one region in which the matrix contains an antibiotic; the characteristic determined in the determining step is the sensitivity of the organism to the antibiotic.

[0039] - The sensitivity of the organism to the antibiotic is obtained by measuring the characteristic size of the inhibition zone, which is associated with the growth of at least a portion of the population of organisms in the antibiotic-containing matrix region; the measurement includes the steps of extracting the outline of the inhibition zone and estimating the characteristic size from the extracted outline.

[0040] - Define a microbial biofilm for the sample; the analysis phase includes the step of obtaining the microbial biofilm, and the determination step uses the obtained microbial biofilm to determine at least one characteristic associated with the group of organisms.

[0041] - During the acquisition phase, the time interval between acquiring each image is adjustable.

[0042] - The time interval depends on at least one of the determined features.

[0043] - The value obtained from the probability is the maximum value of the calculated probability.

[0044] - The organism is divided into colonies; the observation method further includes a colony detection stage to detect and locate colonies; the neural network of the computation step also takes the location of at least one colony as input.

[0045] - The analysis phase includes the steps of determining at least one attribute of the detected colonies and selecting colonies to obtain selected colonies; the obtaining step is performed only on the selected colonies.

[0046] - Multiple organisms correspond to the same species; the determination step includes performing an aggregation function on the probabilities of each organism.

[0047] - The aggregation function takes the maximum value in the probability set as the species of the plurality of organisms.

[0048] - The aggregation function classifies the species of the plurality of organisms as such that the sum of the probabilities of an organism belonging to that species is the largest for each organism.

[0049] - In the calculation step, the neural network is a multi-class neural network.

[0050] - Define a phylogenetic tree for the neural network, the phylogenetic tree containing a set of nodes distributed according to at least two phylogenetic levels, the lowest phylogenetic level being the species; nodes with phylogenetic associations are interconnected; the verification step includes calculating the probability of each node based on the probability calculated in the calculation step, and verifying the conformity conditions of each node.

[0051] - In the determination step, the determined information belongs to the lowest phylogenetic level that meets the compliance conditions.

[0052] - The phylogenetic hierarchy is selected from: phylum, class, order, family, and genus.

[0053] - The number of phylogenetic levels is greater than or equal to 4.

[0054] - The preset threshold value is between 0.7 and 0.9.

[0055] This specification also relates to an observation system for observing a sample, the sample comprising: - A group of organisms; and - The matrix that supports the group of organisms; The observation system includes a lensless imaging unit, and the observation system is adapted to: - Acquire sample images; for each acquired image, the observation system is adapted to: - Collect multiple images of a portion of the sample, each collected image is called a sub-image, each sub-image has an overlapping region that overlaps with at least one other sub-image, and each region of the sample is visible in at least one sub-image; - Determine the transformation to be applied to each sub-image to correct translation and rotation offsets using overlapping regions, so as to obtain a transformation specific to each sub-image; - Apply the transformation to each sub-image to obtain the corrected sub-image; and - Fuse sub-images to obtain sample images; and - Analyze the image; the observation system is adapted during the analysis to: - By analyzing at least one acquired image, determine at least one feature associated with the group of organisms.

[0056] This specification also relates to an observation system for observing a sample, the sample comprising: - A group of organisms; and - A solid matrix supporting the group of organisms; The observation system includes a lensless imaging unit and is adapted to: - Obtain sample images; - Analyze the image, the analysis including: - Calculate the probability that at least one organism belongs to a preset species, the calculation comprising applying a neural network to at least one image to obtain the probability that the at least one organism belongs to the preset species; the preset species is a set of species that the neural network can identify in the image after training; - Verify whether the obtained probabilities meet the reliability conditions by comparing at least one value of the obtained probabilities with a preset threshold; and Determine at least one piece of information related to the identity of the at least one organism; when the reliability condition is not met, the information belongs to a species not included in the preset species.

[0057] According to a specific implementation plan, the observation system also includes adjustment elements for adjusting the environment of the observation system.

[0058] This specification also relates to a sample suitable for the above-described observation system, the sample comprising: - A group of organisms; and - A solid matrix supporting the group of organisms; the solid matrix is ​​adapted to culture at least a portion of the group of organisms and includes at least one region containing an antibiotic in the form of a pellet, the pellet having a characteristic size of less than or equal to 6 mm.

[0059] According to a specific implementation plan, the sample includes one or more of the following features, which may be used individually or in a combination of all technically feasible features: - The sample has a wall configured with a gripping structure.

[0060] - The characteristic size of the pill is greater than or equal to 1 mm.

[0061] - The characteristic size of the pills is less than or equal to 2 mm.

[0062] In this specification, the word “suitable” is used indiscriminately to mean “applicable to,” “adapted to,” or “configured as.”

[0063] Furthermore, in the following text, the phrase "between two values" indicates a broad range, in other words, it includes the limit value.

[0064] The features and advantages of the invention will become apparent from the following description, which is given by way of non-limiting example only, and with reference to the accompanying drawings, in which: Figure 1 The observation system is shown schematically. Figure 2 The sample includes a top view and a side view. Figure 3 schematically shown Figure 1 Observe a part of the system; Figure 4 A flowchart illustrating the implementation of a partial observation method; Figure 5 The illustration shows the first acquisition strategy, which can cover the entire petri dish (12 sub-images). Figure 6 The image set acquired using the first acquisition strategy is illustrated schematically. Figure 7 The illustration shows the first acquisition strategy, which can cover the entire petri dish (15 sub-images). Figure 8 Three images are shown schematically to illustrate how to use overlapping regions to obtain translational distortion between two sub-images; Figure 9 Two images are shown, one of which is not implemented. Figure 4 One method step yields the result, while another step is performed to obtain the result. Figure 10 A flowchart illustrating the implementation of a partial observation method; Figure 11 The illustration shows a sample containing antibiotic pills; Figure 12 A flowchart illustrating the partial observation method for another example; Figure 13 An example of a phylogenetic tree is shown; and Figure 14 Another example of a phylogenetic tree is shown.

[0065] In the following description, the concepts of "upstream" and "downstream" are defined with reference to the direction of light propagation.

[0066] Figure 1 The observation system 10 for the observable sample 12 is shown.

[0067] This sample 12 can be found in Figure 2 Two views.

[0068] Sample 12 contains a group of organisms 14 and a solid matrix 16 supporting the group of organisms 14.

[0069] Depending on the circumstances, organism 18 in this group of organisms 14 may be acellular or cellular.

[0070] Cellular organisms are generally divided into prokaryotes and eukaryotes.

[0071] Archaea (also known as archaea) and bacteria are examples of prokaryotes.

[0072] Eukaryotic organisms can be single-celled or multicellular. For example, protozoa, amoebas, algae, and yeast are single-celled eukaryotic organisms.

[0073] Multicellular eukaryotic organisms are cells derived from, for example, humans and non-human mammals, fungi, plants, protozoa, or chromophores.

[0074] For simplicity, the terms “bacteria” and “bacterial culture” are used below, but the meaning can be readily applied to fungi or microorganisms and more generally any organism 18 that can grow on substrate 16.

[0075] The substrate 16 is suitable for culturing at least a portion of the group of organisms 14. Therefore, the substrate 16 is itself a culture medium or is in contact with a culture medium.

[0076] Matrix 16 is, for example, agar medium.

[0077] As an example, substrate 16 is Muller-Hinton 2 agar medium (hereinafter referred to as MH2 agar medium).

[0078] According to one variant, matrix 16 is used only as a bio-adhesive.

[0079] As an example, matrix 16 contains poly-L-lysine or type I collagen.

[0080] Preferably, agar medium is selected so that all types of organisms can be cultured 18 and it is transparent so that the medium can be observed optically without scattering light.

[0081] The agar medium is, for example, BHI, MH, or TSA medium.

[0082] BHI stands for brain and heart broth extract. BHI is a universal nutrient-rich culture medium used for culturing bacteria.

[0083] MH stands for Mueller-Hinton. MH is a nutrient-rich agar used for culturing bacteria. It is an ideal candidate for antibiotic susceptibility testing.

[0084] TSA stands for trypsin-soybean agar. TSA is a non-selective, versatile agar.

[0085] The above examples have the advantages of good optical properties (transparent and non-scattering) and non-selectivity, and are therefore suitable for the growth of common bacteria.

[0086] This method can also be used in selective agar media, for example, when it is necessary to promote the culture of certain species.

[0087] exist Figure 2 In the example, sample 12 is in the form of petri dish 20.

[0088] In this case, the petri dish 20 is cylindrical with a circular bottom.

[0089] This means that the substrate 16 is defined by the wall of the petri dish 20 and that the wall forms a cylinder.

[0090] More specifically, the petri dish 20 has two mating parts 22. Each part 22 is circular and extends with sidewalls 24. One part 22 contains the substrate 16 and thus serves as a support, while the other serves as a lid, separating the substrate 16 and the group of organisms 14 from the external environment.

[0091] The diameter of the cylindrical circle ranges from 35 mm to 150 mm, typically 90 mm.

[0092] Alternatively, the petri dish 20 may be cylindrical with a square base.

[0093] In this case, the side length of the square is usually 120 mm.

[0094] In addition, such as Figure 2 As shown on the right, the petri dish 20 is placed in the support device 25.

[0095] For convenience, Figure 2 The support device 25 is not shown in the left-side view.

[0096] The support device 25 has a cylindrical shape and its internal dimensions are adapted to fit the petri dish 20.

[0097] The support device 25 has a gripping element 26.

[0098] The grip 26 is, for example, a groove for engaging with a protrusion.

[0099] The gripping elements 26 are arranged in a uniform distribution, but are not limited thereto.

[0100] One of the grips 26 is equipped with a foolproof device 27.

[0101] The support device 25 is made, for example, by 3D printing.

[0102] The presence of the gripper 26 can limit the rotation of the petri dish 20 during transport.

[0103] The observation system 10 includes an incubator 30, a transport unit 32, a lensless imaging unit 34, and an analysis unit 36.

[0104] The incubator 30 is used to store the culture dish 20 to be analyzed in a controlled environment. As an example, the incubator 30 is adapted to ensure that the temperature and humidity are maintained within a preset range.

[0105] Therefore, the incubator 30 includes at least one regulating element 37 for adjusting environmental parameters.

[0106] The regulating element 37 is adapted to regulate the environmental parameters (e.g., temperature) within the storage space 38 of the storage culture dish 20.

[0107] Alternatively, the incubator 30 is adapted to adjust the environmental parameters throughout the observation system 10.

[0108] As an example, the parameter is a humidity or temperature-related parameter.

[0109] As an explanation, the regulating element 37 is a passive humidity generating unit.

[0110] More specifically, the regulating element 37 is a water container with a paper diffuser placed inside the incubator 30.

[0111] The regulating element 37 can maintain a stable high humidity, even if the petri dish 20 is turned on periodically, thereby better controlling (stabilizing) the morphology and preventing the agar medium from drying out.

[0112] Maintaining good humidity levels improves bacterial growth and the arrangement of bacteria within colonies. It also helps prevent agar medium shrinkage, which over time generates anisotropic movements that are detrimental to colonies.

[0113] exist Figure 1 In the illustrative example, storage space 38 is an open cabinet with multiple compartments 39, each compartment 39 being adapted to accommodate petri dishes 20.

[0114] Therefore, the incubator 30 can incubate multiple petri dishes 20 simultaneously.

[0115] The transport unit 32 is adapted to transport the sample 12 from the incubator 30 to the collection area and for reverse transport.

[0116] If the incubator 30 adjusts the entire observation system 10, the transport unit 32 is adapted to transport the sample 12 from the storage space 38 to the collection area, and vice versa.

[0117] The transport unit 32 is here a robotic arm 40 equipped with a gripper 41.

[0118] The gripper 41 is used to grasp sample 12.

[0119] The robotic arm 40 is adapted to perform translation and / or rotation for transport purposes, particularly to place the sample 12 in a desired position.

[0120] The lensless imaging unit 34 is adapted to acquire a transmission image of sample 12.

[0121] Therefore, the lensless imaging unit 34 includes a light source 42 and a detector 44 from upstream to downstream.

[0122] As an alternative or supplement, the lensless imaging unit 34 may include other components, such as an aperture or optical fiber at the output of the light source 42.

[0123] The light source 42 includes multiple light-emitting diodes 46 (LEDs).

[0124] In this case, each light-emitting diode 46 provides partially coherent light.

[0125] This means that the LED 46 can achieve temporal coherence of about 90 nm, while spatial coherence is limited by the size of the LED 46, typically 200 µm.

[0126] Each light-emitting diode 46 is adapted to emit in a wavelength range smaller than the size of the organism 18 to be observed, while allowing the use of conventional detection devices. Therefore, visible or near-infrared wavelengths are preferred. Alternatively, the laser beam wavelength can be located in a different band. The wavelength depends particularly on the organism 18 to be observed and its sensitivity to illumination by the light source 42.

[0127] As specific examples, the wavelengths used are: red light wavelength (e.g., 625 nm), green light wavelength (e.g., 528 nm), blue light wavelength (e.g., 455 nm), and infrared wavelength (e.g., 940 nm).

[0128] These wavelengths can improve sample analysis, and are particularly suitable for applications such as obtaining inhibition zones or identifying organisms.

[0129] The wavelengths can be emitted simultaneously or sequentially.

[0130] The detector 44 is formed by the housing 48 that protects the sensor 50.

[0131] In the example shown, housing 48 has a cuboid shape, but other shapes may be used.

[0132] The housing 48 is fixed to the optical platform to provide good stability.

[0133] Sensor 50 is a CMOS optical sensor in this case.

[0134] The abbreviation CMOS refers to the technology used to manufacture electronic components, and by extension, the components made using this technology. It stands for Complementary Metal-Oxide-Semiconductor.

[0135] Sensor 50 includes a pixel set 52. Therefore, sensor 50 is a matrix optical sensor.

[0136] The size of each pixel 52 is between 3 micrometers (µm) and 4 µm, preferably equal to 3.75 µm.

[0137] In addition, the field of view of sensor 50 is 24 mm × 36 mm.

[0138] These values ​​correspond to optical sensors with large field of view and good spatial resolution.

[0139] As the name suggests, the lensless imaging unit 34 does not have a magnifying lens between the sensor 50 and the light source 42.

[0140] Therefore, the image acquired by sensor 50 is formed by radiation directly transmitted from the irradiated sample 12.

[0141] The sample 12 is placed within the defined acquisition area (i.e., its light field) of the sensor 50, and is close to the sensor 50, usually less than a few centimeters, preferably less than 1 centimeter.

[0142] More specifically, this distance corresponds to the distance between the organism 14 and the sensor 50. In fact, the petri dish 20 is placed on the sensor 50, which is protected by a glass plate.

[0143] The analysis unit 36 ​​is adapted to analyze the images acquired by the imaging unit to obtain information related to the group of organisms 14.

[0144] Analysis unit 36 ​​is shown in more detail. Figure 3 .

[0145] The analysis unit 36 ​​is adapted to interact with a computer program product to implement certain steps of the observation method. These steps are therefore computer-implemented.

[0146] Analysis unit 36 ​​is a desktop computer in this case. Alternatively, analysis unit 36 ​​can be a rack-mount computer, laptop, tablet, personal digital assistant (PDA), or smartphone.

[0147] exist Figure 3 In this scenario, the analysis unit 36 ​​includes a computer 56, a user interface 58, and a communication device 60.

[0148] Computer 56 is an electronic circuit designed to manipulate and / or convert data represented in electronic or physical quantities in the registers and / or memory of analysis unit 36 ​​into other similar data corresponding to physical data in registers, memory or other types of display devices, transmission devices or storage devices.

[0149] As a specific example, computer 56 includes single-core or multi-core processors (such as central processing unit (CPU), graphics processing unit (GPU), microcontroller, digital signal processor (DSP)), programmable logic circuits (such as application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), programmable logic device (PLD), programmable logic array (PLA)), finite state machines, logic gates, and discrete hardware components.

[0150] The computer 56 includes a data processing unit 62 adapted to process data (especially to perform calculations), a memory 64 adapted to store data, and a reader 66 adapted to read computer-readable media.

[0151] The user interface 58 includes an input device 68 and an output device 70.

[0152] Input device 68 allows the user of analysis unit 36 ​​to input information or instructions to analysis unit 36.

[0153] Figure 3 In this embodiment, the input device 68 is a keyboard. Alternatively, the input device 68 may be a pointing device (such as a mouse, touchpad, and drawing tablet), a voice recognition device, an eye-tracking device, or a haptic (motion analysis) device.

[0154] The output device 70 is a graphical user interface 58, which is a display unit designed to provide information to the user of the analysis unit 36.

[0155] Figure 3 In this embodiment, the output device 70 is a display screen used to visually present the output. In other embodiments, the output device 70 is a printer, an augmented reality and / or virtual reality display unit, a speaker or other sound generation device (presenting the output in an auditory form), a vibration and / or odor generation unit, or a unit adapted to generate electrical signals.

[0156] In a specific implementation, the input device 68 and the output device 70 are the same component, forming a human-machine interface, such as an interactive screen.

[0157] The communication device 60 allows for one-way or two-way communication between the components of the analysis unit 36. For example, the communication device 60 is a bus communication system or an input / output interface.

[0158] The presence of communication device 60 enables, in some implementations, components of computer 56 to be remotely configured with each other.

[0159] Computer program products include computer-readable media 72.

[0160] Computer-readable medium 72 is a physical device that can be read by reader 66 of computer 56.

[0161] Specifically, the computer-readable medium 72 itself is not a transient signal, such as radio waves or other freely propagating electromagnetic waves like light pulses or electrical signals.

[0162] The computer-readable storage medium 72 is, for example, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any combination thereof.

[0163] As a more specific example in a non-exhaustive list, computer-readable storage medium 72 is a mechanical encoding device (e.g., a punched card or recessed structure), a floppy disk, a hard disk, a read-only memory (ROM), a random access memory (RAM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magneto-optical disk, a static random access memory (SRAM), an optical disk (CD-ROM), a digital versatile optical disk (DVD), a USB flash drive, a flash memory, a solid-state drive (SSD), or a PC card (e.g., a PCMCIA memory card).

[0164] The computer program is stored on a computer-readable storage medium 72. The computer program includes one or more stored sequences of program instructions.

[0165] When the program instructions are executed by the data processing unit 62, the steps of the observation method are implemented.

[0166] For example, program instructions may be in the form of source code, computer-executable form, or any intermediate form between source code and computer-executable form, such as the form of source code after being transformed by an interpreter, assembler, compiler, linker, or locator. Alternatively, program instructions may be microcode, firmware instructions, state definition data, integrated circuit configuration data (e.g., VHDL), or object code.

[0167] Program instructions are written in any combination of one or more languages, such as object-oriented programming languages ​​(FORTRAN, C++, JAVA, HTML, Python) or procedural programming languages ​​(such as C).

[0168] Alternatively, program instructions may be downloaded from an external source via a network, particularly in the case of software applications. In this case, the computer program product includes a computer-readable data medium storing the program instructions or a data carrier signal encoding the program instructions.

[0169] In each case, the computer program product includes instructions that can be loaded into the data processing unit 62 and are adapted to implement the design method when executed by the data processing unit 62.

[0170] According to the implementation plan, the execution is carried out entirely or partially on the analysis unit 36 ​​(i.e., a single computer) or in a distributed system of multiple computers (especially through cloud computing).

[0171] The operation of the observation system 10 will now be described by referring to various implementations of the example observation method.

[0172] The observation method is to observe sample 12 to obtain relevant information about sample 12.

[0173] Information varies depending on the application.

[0174] As described below, according to the first application, the information is the sensitivity of an organism 18 to one or more antibiotics. This first application corresponds to an antibiotic susceptibility testing.

[0175] According to the second application, the information is for the identification of organism 18.

[0176] Each of these applications involves image acquisition and analysis of the culture dish 20 to obtain relevant information about the culture dish 20.

[0177] Therefore, the observation method includes two stages: the image acquisition stage P1 and the image analysis stage P2.

[0178] The following text first describes the acquisition phase P1 common to both of the aforementioned applications, and then describes the analysis phase P2 specific to each application.

[0179] An example flow diagram for obtaining phase P1 is shown below. Figure 4 .

[0180] In the acquisition phase P1, the observation system 10 aims to acquire an image of sample 12.

[0181] Therefore, according to the example, for each image of the acquired sample 12, the acquisition stage P1 includes multiple steps, namely, acquisition step E100, determination step E102, application step E104 and fusion step E106.

[0182] In acquisition step E100, the observation system 10 acquires multiple images of a portion of sample 12.

[0183] This corresponds to the fact that although sensor 50 has a large field of view, it is impossible to obtain an image of the entire culture dish 20 in a single acquisition.

[0184] Before describing these acquisition strategies in detail, it should be noted that prior to acquisition step E100, each culture dish 20 is transported from compartment 39 to the defined acquisition area of ​​sensor 50.

[0185] Furthermore, the lid of the petri dish 20 can be removed using a suction cup combined with a vacuum pump. This facilitates image acquisition.

[0186] After image acquisition was completed, sample 12 was returned to incubator 30.

[0187] In practice, the robotic arm 40 moves the gripper 41 to grasp the sample 12, then transports the sample and places it on the housing 48.

[0188] Similarly, upon return, the robotic arm 40 moves the gripper 41 to grasp the sample 12 and deliver the sample to a compartment 39.

[0189] Before placing the petri dish 20 back into the incubator 30, cover it with the lid.

[0190] For completeness, the lid may also be retained when implementing the method described below.

[0191] Figure 5 The illustration shows a first sampling strategy that can cover the entire petri dish 20.

[0192] In this example, petri dish 20 has a diameter of 90 mm.

[0193] According to the first example, sensor 50 acquires 12 different images.

[0194] Each image corresponds to a spatial region smaller than the culture dish 20. These images are referred to below as sub-images to distinguish them from the images of the culture dish 20 reconstructed from these sub-images.

[0195] The sub-image is therefore a transmission image of a certain area of ​​sample 12, obtained by sensor 50 collecting light emitted by light source 42.

[0196] Each sub-image corresponds to the same field of view, and therefore has the same size.

[0197] Each of these sub-images is in Figure 5 Each sub-image corresponds to a rectangle, numbered I1 to I12, to indicate the acquisition order of these different sub-images.

[0198] Sub-images are acquired in consecutive rows from the top left corner to the bottom right corner.

[0199] According to the example, the petri dish 20 has four rows, and each row contains three sub-images.

[0200] This corresponds to the observation system 10 scanning sample 12.

[0201] The petri dish 20 moves while the light source 42 and sensor 50 remain fixed.

[0202] The first sub-image of rectangle I1 and the second sub-image of rectangle I2 have spatial overlap.

[0203] Spatial overlap correspondence Figure 5 The overlapping area indicated by mark 120.

[0204] The overlapping area will be referred to as the horizontal overlapping area 120 below.

[0205] exist Figure 5 In the example, the horizontally overlapping area 120 is a rectangle, and the corresponding surface area accounts for 10% to 30% of the surface area of ​​an image.

[0206] Each sub-image within the same row has the same horizontal overlap area that overlaps with the adjacent images.

[0207] That is, the second sub-image and the same row sub-image have two overlapping areas: the first horizontal overlapping area 120 overlaps with the first sub-image (on the left), and the second horizontal overlapping area 122 overlaps with the third sub-image (on the right).

[0208] The first sub-image also has an overlapping area that overlaps with the fourth sub-image.

[0209] The overlapping area is Figure 5 It is shown in the middle with the mark 124.

[0210] The overlapping region will be referred to as the longitudinal overlapping region 124 below.

[0211] The vertically overlapping region 124 is a rectangular region with a surface area smaller than that of the horizontally overlapping region 120 between the first and second sub-images.

[0212] However, a larger surface area can also be used, especially recommended for certain sensor sizes.

[0213] exist Figure 5 In the example, the surface area of ​​the vertically overlapping region 124 is less than 5% of the surface area of ​​a sub-image.

[0214] like Figure 5 As shown, each vertically overlapping region 124 and the horizontally overlapping region 120 share a common region 126.

[0215] Due to the shape of the overlapping areas, the common area 126 is rectangular.

[0216] The common area 126 is therefore the common area of ​​the four sub-images.

[0217] As an example, the common area 126 of the first sub-image is shared with the second, fourth, and fifth sub-images.

[0218] Experimental examples of the sub-images obtained by this acquisition strategy are shown in Figure 6 .

[0219] According to the example of the second data collection strategy (corresponding) Figure 7 Fifteen different sub-images were collected and divided into five rows, with each row still containing three sub-images (same as the first example, left image).

[0220] In this example, as in the previous case, each sub-image has a horizontal overlapping region 130 and a vertical overlapping region 132.

[0221] Compared to the first example, the relative surface area of ​​the horizontal overlap region 130 is similar to that of the sub-image surface area (still 10% to 30%), while the relative surface area of ​​the vertical overlap region 132 is larger and is generally on the same order of magnitude as that of the horizontal overlap region 130.

[0222] This forms a square public area 134.

[0223] These configurations allow for sufficient overlap surface area between sub-images to implement subsequent steps.

[0224] Of particular advantage is that in acquisition step E100, sub-images are acquired row by row to cover the entire sample 12.

[0225] The preferred number of sub-images to be acquired is 12 to 15.

[0226] Subsequent steps (i.e., determining, applying, and blending) are designed to adjust the sub-images.

[0227] This adjustment is sometimes referred to as "stitching".

[0228] As is evident below, the overlapping regions, in particular, can take into account the positioning error of the petri dish 20 and determine the transformations to be applied to place the sub-images in the same reference frame.

[0229] The purpose of step E102 is to obtain the transformation to be applied to each sub-image in order to correct for translation and rotation offsets, thereby obtaining the specific transformation for each sub-image.

[0230] Translational and rotational offsets are caused by a variety of factors and can be superimposed.

[0231] Therefore, defects in mechanical components (especially grippers and moving arms) can cause such offsets.

[0232] As the culture dish 20 moves repeatedly, it may rotate relative to its positioning support.

[0233] Another reason is operator intervention: the operator may need to visually inspect the petri dish 20 at a closer distance but fails to return it to its precise original position.

[0234] According to the example, step E102 is determined to include multiple operations.

[0235] The first operation aims to determine the distortion between two adjacent sub-images.

[0236] The sub-images will be referred to as A and B below, and are assumed to be in the same row. For example, sub-image A is the first sub-image, and sub-image B is the second sub-image.

[0237] In the first operation, the horizontally overlapping region between sub-images A and B is divided into n sub-regions.

[0238] In this specific case, the sub-regions are identical for ease of calculation, but sub-regions with different surface areas can also be used to perform the determination step E102.

[0239] Furthermore, for the same computational reasons, the sub-region is advantageously rectangular.

[0240] The number n is strictly a natural number greater than 2.

[0241] Determine a number n such that the offset caused by rotational error within the subregion can be locally ignored.

[0242] The rotation error is considered negligible as long as it corresponds to the maximum offset of a pixel within the sub-region.

[0243] As an example of scale, for a horizontal region of size 1914×6380, the number n=252 can ensure that the rotational offset is negligible.

[0244] The overlapping regions of the sub-image A obtained by such segmentation result in a set of n sub-regions of the sub-image, denoted as sub-region Ai, with indices i from 1 to n.

[0245] Similarly, we obtain a set of n sub-regions of sub-image B, denoted as sub-region Bi.

[0246] The second operation involves determining the translation offset between the two overlapping regions of sub-images A and B.

[0247] According to the structure, each subregion Ai corresponds to a corresponding subregion Bi.

[0248] When there is no error, subregions Ai and Bi are identical in pairs.

[0249] Due to the above segmentation, sub-regions Ai and Bi only exhibit translational shifts.

[0250] When the distortion is only translation, it is only necessary to identify a pair of corresponding points in each sub-region, and the offset between the two points when the images are superimposed is the translation vector.

[0251] This situation is caused by Figure 8 The illustrations depict exemplary patterns for sub-region A1 (left image) and corresponding sub-region B1 (middle image). Two corresponding points are marked with a cross in these images.

[0252] When the patterns are basically superimposed, the two points do not coincide, and the translational shift is obvious.

[0253] From a mathematical perspective, this can be represented as follows.

[0254] remember and Let Ai and Bi be two corresponding points belonging to subregions Ai and Bi respectively. Then, the translational offset of subregion Ai relative to subregion Bi satisfies the following relationship: Therefore, the phase correlation method can be used.

[0255] This gives each point a set of translation offset values.

[0256] The third operation involves determining the affine transformation that can be used to transform the overlapping region from sub-image A to sub-image B.

[0257] That is, to determine affine transformations It is rotated by angle θ and corresponds to the vector It consists of vector translations.

[0258] Therefore, solve the following equation: in: • , and • , and • Any convergent solution to the equation can be used.

[0259] In some cases, outlier removal techniques can facilitate convergence.

[0260] Such outliers may occur because the sub-image contains the interior of culture dish 20, the edge of culture dish 20, and the exterior of culture dish 20.

[0261] To eliminate outliers, the RANSAC method can be used.

[0262] RANSAC is a method for estimating parameters of certain mathematical models. More specifically, it is an iterative nondeterministic method used to observe cases where datasets may contain outliers.

[0263] RANSAC stands for Random Sampling Consistency (RANSAC) RANdom SAmple Consensus ).

[0264] A similar method can be applied to any overlapping region.

[0265] Therefore, in the example described, three matrices will be obtained for the first sub-image. M One uses the second sub-image, one uses the fourth sub-image, and one uses the fifth sub-image.

[0266] At the end of the third operation, the relative distortion between each sub-image is obtained.

[0267] The fourth operation aims to obtain the transformation function to be applied to each sub-image in order to minimize the distortion between sub-images.

[0268] This operation can be performed using the following method.

[0269] set up Let A and B be two sub-images, A and B, with k pairs of corresponding points satisfying... and ,symbol and Let A and B represent the sets of points in sub-images A and B, respectively.

[0270] For the sake of simplicity, the following text will be recorded as follows: and .

[0271] When sub-images A and B have no rotation or translation errors, the points will coincide. .

[0272] This operation aims to find a planar affine transformation such that: in: The affine transformation matrix applied to sub-image A The affine transformation matrix applied to sub-image B As mentioned above, each affine transformation or Rotation by angle θ and corresponding vector It consists of vector translations.

[0273] set up V is the set of transformation matrices applied to each image in the sub-image set V.

[0274] To guarantee a unique solution, a sub-image is selected. As a reference sub-image, its transformation matrix is, by definition, the identity matrix: in Let be the affine transformation matrix applied to the subimage Ω.

[0275] Therefore, the problem to be solved can be mathematically expressed as: in: This is an indicator function; it is 1 when sub-images A and B have a common overlapping area, and 0 otherwise.

[0276] This corresponds to an overdetermined linear optimization problem. Any technique can be used to solve this problem.

[0277] For example, the least squares method can be used.

[0278] When the fourth operation (i.e., determining step E102) ends, the set is obtained. The transformation set expression. Thus, the optimal transformation of subimage A is obtained, denoted as... .

[0279] In step E104, the resulting transformation is applied to the original sub-image, mathematically for all sub-images: in: This represents the original sub-image, identified by index j. Let represent the optimal transformation matrix determined by the previous operation, and This represents the corrected sub-image.

[0280] For each sub-image Applying their respective optimal transformations yields all corrected sub-images. .

[0281] The fusion step E106 includes fusion of all adjusted sub-images. To obtain the image.

[0282] Therefore, based on the selected acquisition layout (here it is) Figure 5 (Location sub-image)

[0283] Then, the issue of overlapping regions is addressed.

[0284] According to the example, overlapping regions are merged.

[0285] This fusion can be achieved by averaging the signal from each sub-image.

[0286] Each point in the final overlapping region is the arithmetic mean of the values ​​of the overlapping region points of the first sub-image and the overlapping region points of the second sub-image.

[0287] This fusion method helps to smooth out residual offsets or discontinuities.

[0288] This can be seen from Figure 9 It can be seen that it shows the image obtained by the unfused superimposed image on the left, and the image obtained by performing the above fusion method.

[0289] In another example, the signal is selected from the most recent image.

[0290] In this case, the overlapping area is divided into two parts of equal size.

[0291] For horizontally overlapping regions, the left half of the overlapping region is filled by the left sub-image, and the right half of the overlapping region is filled by the right sub-image.

[0292] Perform similar operations on vertically overlapping regions.

[0293] This fusion method preserves the background texture for subsequent analysis, as detailed below.

[0294] Other fusion methods can be considered depending on the processing applied to overlapping areas.

[0295] The above text introduced a series of techniques for obtaining images from sub-images.

[0296] Therefore, spatial reconstruction can address the localization problem that occurred in sample 12.

[0297] Analysis unit 36 ​​then performs image analysis phase P2.

[0298] The following text is for reference only. Figure 10 Describe the analytical phase P2 used for antibiotic susceptibility testing.

[0299] Antibiotic susceptibility testing is a test of bacteria's sensitivity to antibiotics.

[0300] This test is commonly referred to as AST (Antibiotic Susceptibility Testing).

[0301] It should be noted that antibiotic susceptibility testing is also used for other organisms such as fungi. Therefore, antibiotic susceptibility testing involves testing the susceptibility of each organism to 18 pairs of antibiotics.

[0302] The image analysis phase P2 aims to determine the sensitivity of each organism 18 to at least one antibiotic from the images obtained in the acquisition phase P1.

[0303] Therefore, it is assumed that the matrix 16 is suitable for culturing at least a portion of the organism population 14, and that the sample 12 contains at least one region in which the matrix 16 contains antibiotics.

[0304] Therefore, antibiotics are introduced in the form of pills, such as... Figure 11 As shown schematically.

[0305] like Figure 11 As shown, pill 100 is cylindrical.

[0306] In other embodiments, antibiotics may be introduced in the form of test strips.

[0307] according to Figure 10 For example, the analysis phase P2 includes the acquisition step E150, the detection step E152, and the determination step E154.

[0308] In step E150, the analysis unit 36 ​​acquires microbial moss for each image.

[0309] When the bacterial concentration on a culture medium is extremely high, bacteria grow and form what is known as a microbial biofilm. A microbial biofilm is a multi-layered bacterial film covering the surface of an agar medium. This biofilm is generally uniform, but bacterial colonies cannot be counted or identified.

[0310] As an example, obtaining step E150 involves three operations.

[0311] In the first operation, computer 56 detected 100 antibiotic pills.

[0312] To perform this detection, the computer 56 performs thresholding segmentation on the image grayscale. Specifically, in the lensless microscope image, the pellet is very opaque and appears black. A series of morphological operations supplement the detection, utilizing the information that the pellet 100 in the image is a black disk, to remove dust and other artifacts detected by the thresholding segmentation step.

[0313] Other pattern recognition-based techniques can also be used to perform this detection. As an example, computer 56 can use a convolutional neural network (CNN).

[0314] In the second operation, computer 56 detects the edge of culture dish 20.

[0315] More specifically, computer 56 detects the support device 25, which appears as black in the image.

[0316] For example, threshold segmentation is performed on the image grayscale to detect the support device 25. The support device 25 defines the outer and inner portions of the culture dish 20. The outer portion is masked, while the inner portion is preserved.

[0317] The third operation involves removing the detected elements from the image.

[0318] This allows for the isolation of microbial flora.

[0319] This separation is beneficial in obtaining a robust gain for the subsequent implementation analysis phase P2.

[0320] Computer 56 then performs detection step E152.

[0321] Detection step E152 is designed to detect bacterial or fungal growth.

[0322] Therefore, we can utilize the fact that changes in microbial biofilm texture (spatial or temporal) are related to bacterial growth.

[0323] Texture is a structured spatial arrangement of pixels.

[0324] In this case, the portion of the image corresponding to the microbial moss contains characteristic textures (spatial frequencies and patterns), which differ from the portion of the image where no bacteria have grown.

[0325] In other words, texture differences can distinguish between microbial moss and inhibition zones.

[0326] This detection step E152 can be implemented by determining the current image texture and comparing it with previously acquired textures.

[0327] For example, computer 56 can use Haralick features to determine textures.

[0328] Harallik features are a metric calculated based on the gray-level co-occurrence matrix, which can quantify the spatial distribution structure of each gray level in an image.

[0329] By computing these features over a sliding window of the image, a local measure of the features of the entire image is obtained.

[0330] According to another example, computer 56 uses histograms.

[0331] More specifically, computer 56 generates a grayscale histogram and compares it with a reference histogram showing the presence of the corresponding inhibition zone. The distance between the two histograms is used as a texture metric.

[0332] Alternatively, the reference histogram is a histogram of an image without bacterial or fungal growth.

[0333] In yet another example, PSD is used.

[0334] Instead of creating the histogram mentioned above, a PSD is prepared for the image and compared with a reference PSD.

[0335] PSD stands for Power Spectral Density, which plots the frequency distribution structure of image pixels.

[0336] Wavelet transform can also be used to obtain textures.

[0337] Wavelet transform involves applying a basic function (called a wavelet) to multiple scales of an image, thereby characterizing image texture at multiple resolution levels. Applying wavelet transform to a local area of ​​an image returns a set of wavelet coefficients that characterize the texture at different resolution levels. These wavelet coefficients are used as texture indices, which can be compared with reference coefficients, or as input to a classifier to classify textures.

[0338] In another example, a support vector machine is used. This tool is often abbreviated as SVM, which stands for Support Vector Machine. Support Vector Machine .

[0339] This SVM tool can be used as a binary classifier, outputting two possibilities: microbial flora or inhibition zone.

[0340] The input to the SVM can be a list of pixels or a metric obtained by the methods described above (such as the Hararik feature metric).

[0341] The SVM is trained on an image set containing antibacterial regions and microbial moss.

[0342] The different examples described above can be combined to aggregate various results obtained by implementing the techniques described above, for example, by calculating the average of the results or by using an SVM that computes all input metrics. This is used to determine the texture.

[0343] Computer 56 then compares the texture with the previously acquired texture.

[0344] When the difference is greater than the measurement noise, it indicates the presence of bacterial or fungal growth.

[0345] Computer 56 thus detects bacterial or fungal growth.

[0346] After the detection, computer 56 performs the determination step E154.

[0347] In step E154, computer 56 determines at least one feature associated with organism group 14 by analyzing at least one acquired image.

[0348] The analysis includes measuring the diameter of the inhibition zone based on an image (which can be a raw image or a textured image).

[0349] When the antibacterial zone in the image is not clear enough, a textured image is usually preferred.

[0350] In the proposed implementation example, the characteristic determined in step E154 is the sensitivity of organism 18 to antibiotics.

[0351] In this case, sensitivity is obtained by measuring the characteristic size of the inhibition zone, which is related to the growth of the organism population 14 in the region containing the antibiotic matrix 16.

[0352] The inhibition zone refers to the boundary of the growth area of ​​an organism, where growth is inhibited by the presence of antibiotics.

[0353] The area devoid of organisms is usually called the inhibition zone.

[0354] Most commonly, the boundary is circular, but sometimes the shape is more complex, especially when interacting with other antibiotics or near the edge of the culture dish 20.

[0355] The size of the inhibition zone is defined as the average distance between two pairs of points.

[0356] This size (hereinafter referred to as diameter) depends on the sensitivity of the bacterial strain to the antibiotic agent used.

[0357] The following section presents a method for determining the diameter of the inhibition zone.

[0358] The diameter of the inhibition zone can be estimated using the agar diffusion method defined by EUCAST (European Reference Institute), CLSI (American Reference Institute), or CA-SFM (French Society for Microbiology) to estimate the sensitivity of an organism to antibiotic agents.

[0359] As an example, EUCAST defines an inhibition diameter threshold for each bacterial species / antibiotic pair; organisms below this threshold are defined as resistant, and organisms above this threshold are defined as susceptible.

[0360] exist Figure 10 In this scenario, step E154 is determined to include an extraction operation and an estimation operation.

[0361] The first operation uses contour extraction technology.

[0362] Typically, the inhibition zone is an effective circular shape.

[0363] In this case, the contour extraction technique is a logical adjustment of the radial mean of the inhibition zone.

[0364] Alternatively, contour extraction technology can also be used to determine the inscribed circle of the inhibition zone after contour extraction.

[0365] In this case, the Canny edge detector or gradient-based edge detectors such as the Sobel filter and the Prewitt filter can be used.

[0366] In more complex cases where the inhibition zone is not circular, the above technique can be used to approximate the inhibition zone as defined by a circular arc.

[0367] Alternatively, contour extraction can be used to obtain the precise shape of the inhibition zone and its diameter can be estimated as the diameter of the inscribed circle.

[0368] According to another example, these contours are obtained using microbial moss texture.

[0369] The estimation operation includes using estimation techniques to estimate the profile dimensions.

[0370] According to the first example, the estimation technique includes calculating the radial mean.

[0371] According to the second example, the estimation technique includes performing logic adjustments.

[0372] This technique involves calculating the radial mean of the gray level around the center of the pill.

[0373] More specifically, the technique involves extracting grayscale profiles (slices) along the entire surface of the pill, at a preset angular step size, in a direction orthogonal to the pill's outline. The angular step size is, for example, 1 degree.

[0374] Then take the average of the cross-sections.

[0375] The profile was then fitted using a logistic function.

[0376] Alternatively, any other grayscale transition modeling function can be used.

[0377] In addition, RANSAC technology can be used to eliminate outliers from the extracted contours.

[0378] For non-circular inhibition zones, performing both procedures can yield a more accurate diameter estimate.

[0379] This is particularly useful for detecting interactions between antibiotics, as synergistic or antagonistic effects often result in non-circular inhibition zones.

[0380] Furthermore, the robustness of inhibition zone diameter measurement was improved regardless of the shape of the inhibition zone.

[0381] Therefore, this method can quickly detect features related to organisms.

[0382] This speed and resolution improvement can be achieved using pellet 100 on a sample 12 with a reduced feature size.

[0383] For example, the feature size can be reduced by a factor of 3, less than or equal to 6 mm, less than or equal to 4 mm, preferably less than or equal to 2 mm.

[0384] However, a minimum size of 1 mm is still desired.

[0385] refer to Figure 4 The acquisition phase P2 described can also be advantageously used for identification methods.

[0386] The identification method aims to identify each organism 18 from at least a portion of the petri dish 20.

[0387] Species is a classification level in bacterial nomenclature, used to group all strains that are sufficiently similar to the type strain of a species and can be classified into that species.

[0388] according to Figure 12 For example, the analysis phase P2 includes the detection sub-phase SP1 and the identification sub-phase SP2.

[0389] The detection sub-stage SP1 aims to detect colonies in the image obtained after the acquisition stage P1 and ideally obtain their location.

[0390] Bacterial colonies are clusters of bacteria that form on an agar medium from the replication of an initial set of bacteria (sometimes several). Bacteria form colonies through division.

[0391] This colony is sometimes referred to as a CFU (colony forming unit).

[0392] In practice, the number of colonies on petri dish 20 can reach several hundred.

[0393] According to the example, the identification sub-stage SP2 includes a calculation step E160, a verification step E162, and a determination step E164.

[0394] In calculation step E160, computer 56 applies a neural network to at least one image.

[0395] Before explaining the computational step E160 in more detail, it is necessary to outline some general aspects of the concept of neural networks.

[0396] A neural network consists of ordered, continuous layers of neurons, with each neuron taking input from the output of the previous layer.

[0397] More specifically, each layer includes neurons that receive outputs from neurons in the previous layer or inputs from input variables in the first layer.

[0398] As an alternative, a more complex neural network structure could be considered, where a layer could connect to a more distant layer that is not immediately adjacent to the preceding layer.

[0399] Each neuron is also associated with the operation (i.e., the processing type) that the neuron in the corresponding processing layer needs to perform.

[0400] Each layer is connected to other layers via multiple synapses. Each synapse is associated with a synaptic weight, and each synapse forms a connection between two neurons. These weights are typically real numbers, which can be positive or negative. In some cases, the synaptic weights are complex numbers.

[0401] Each neuron can compute a weighted sum of values ​​received from neurons in the preceding layer (each value multiplied by the corresponding synaptic weight of each synapse or connection between that neuron and the preceding neurons), then apply an activation function (usually a non-linear function) to the weighted sum, and output the value obtained by applying the activation function at the neuron's output (especially to the next layer of neurons connected to it). The activation function introduces non-linearity into the processing performed by each neuron. The Sigmoid function, hyperbolic tangent function, and Heaviside function are examples of activation functions.

[0402] As an optional addition, each neuron may also apply a multiplier factor (also known as a bias) to the output of the activation function, and the value passed to the neuron's output is the product of the bias value and the activation function value.

[0403] In certain specific cases, the neural network is a convolutional neural network.

[0404] It is also abbreviated as CNN, which stands for Convolutional Neural Network.

[0405] In a convolutional neural network, each neuron in the same layer has the exact same connection pattern as its neighboring neurons, but they are located at different input locations. This connection pattern is called the convolution kernel.

[0406] In addition, there are specific neuronal layers, such as fully connected neuronal layers.

[0407] A fully connected neuron layer refers to a layer where all neurons are connected to all neurons in the previous layer.

[0408] This layer type is often called "fully connected" or sometimes "dense layer".

[0409] Figure 12 The example of calculation step E160 is first explained for a simple case: the observed sample 12 is a single microorganism, that is, only one species of bacteria or fungus exists in the imaging petri dish 20.

[0410] In this case, calculation step E160 includes application operations and associated operations.

[0411] In the application operation, computer 56 applies a classifier here.

[0412] For example, the classifier is a K-classification neural network (each category is a species).

[0413] The classifier is, for example, a CNN followed by a SASE module.

[0414] SASE stands for Spatially-Adaptive Squeeze-Excitation.

[0415] The SASE module is used here to enhance the abstraction capabilities of the corresponding CNN model.

[0416] The classifier was trained on a set of 278 petri dishes containing 10 bacterial species.

[0417] The neural network takes the input image and the colony location, and provides a set of scores for each colony associated with each category.

[0418] The preferred input is an array combining multiple frames.

[0419] For example, the input is an array of size 125×125×36, corresponding to 36 frames of 125×125 pixel cropped from the previously detected colonies (one image is acquired every 30 minutes for 18 hours).

[0420] Of course, the cropping size and frame count can vary, and the frames can contain multiple types, each corresponding to a specific wavelength.

[0421] In other words, for each colony i, the neural network gives a K score. Set, where i is an integer from 1 to N (number of colonies).

[0422] Due to the score This can be interpreted as the probability of belonging to the corresponding category, and can be written as: ,satisfy According to the example, in the association operation, computer 56 associates the colony to be tested with a species.

[0423] More specifically, Computer56 considers the species associated with the tested bacterial colony to be the highest score. The corresponding species.

[0424] This corresponds to the following mathematical relationship: in: Indicates the species associated with colony i, and To give a function for probabilistically associated species Therefore, at the end of calculation step E160, the set of most likely species-colony associations is obtained for each colony.

[0425] In verification step E162, computer 56 verifies the reliability of the prediction results.

[0426] In other words, the computer verifies that the obtained probability meets the reliability condition by comparing at least one of the obtained probabilities with a preset threshold.

[0427] The verification step E162 aims to address the following problem: the neural network is trained on a pre-defined set of species. This results in the absence of certain categories.

[0428] Therefore, it can be inferred that the classifier is not capable of giving the correct answer for colonies corresponding to categories unknown to the classifier.

[0429] Based on this example, the maximum score associated with the colony is compared with the threshold. Compare.

[0430] When the maximum value is strictly below the threshold ε, it is considered that a prediction cannot be made, that is, the colony corresponds to an unknown species or the prediction reliability is insufficient.

[0431] threshold The value is between 0 and 1, and should be selected based on the specific situation.

[0432] Preferred selection threshold Between 0.7 and 0.9.

[0433] In practice, threshold A value of 0.8 yields good performance.

[0434] The species were initially assumed to be known.

[0435] Computer 56 then performs the determination step E164.

[0436] Aggregation can be implemented based on 12 examples of single microbial samples.

[0437] In this operation, an aggregation function is used to aggregate the aforementioned associations to obtain a single category (single species) for all colonies in petri dish 20.

[0438] At this point, various aggregate functions can be considered.

[0439] The first aggregation function involves selecting the most frequent species in culture dish 20.

[0440] Mathematically equivalent to performing the following operations: in: This indicates the species selected for the entire petri dish (20), and For the indicator function, such that hour It is 1 if it is 1, otherwise it is 0. The second example of an aggregation function includes selecting the species with the highest association score.

[0441] In this example, the following mathematical function is used: in In order to be with species The highest score related to the association.

[0442] The implementation example of this determination step E164 significantly improves robustness for the single microbial sample 12 scenario.

[0443] This is because it overcomes the following common problem: despite high identification accuracy, some misidentified colonies still exist on the petri dish.

[0444] For example, with an accuracy rate of 95%, a culture dish with 400 colonies will statistically have 20 incorrectly identified colonies. This implementation method can correct for these 20 incorrectly identified colonies.

[0445] Ultimately, this is equivalent to a voting technique for organism group 14 (here, the entire petri dish 20).

[0446] Therefore, the above content can be extended to multiple organisms corresponding to the same species 18.

[0447] In this case, determining step E164 involves performing an aggregation function on the probabilities of each organism 18.

[0448] The first example above corresponds to an aggregation function that uses the species corresponding to the maximum value of the probability set as the species of the plurality of organisms 18, while the second example corresponds to an aggregation function that uses the species corresponding to the maximum value of the probability set as the species of the plurality of organisms 18.

[0449] If the species to be identified is unknown, then the reliability criteria are not met.

[0450] Computer 56 then outputs the following information: Each of the tested organisms 18 belongs to a species not included in the preset species.

[0451] This enables the computer to handle new situations, including those not learned during neural network training.

[0452] According to the more refined implementation described below, computer 56 is able to provide additional information about organism 18 belonging to an unknown species.

[0453] Therefore, a phylogenetic tree is used.

[0454] By definition, a phylogenetic tree is a tree-like representation of the relationships between different organisms18, constructed based on their genetic characteristics and interrelationships.

[0455] The tree is shown Figure 13 .

[0456] The tree has multiple nodes, distributed according to the phylogenetic hierarchy from the root (highest level) to the leaf (lowest level).

[0457] Phylogenetic hierarchy refers to the depth of the taxonomic units studied in a phylogenetic tree.

[0458] Therefore, the phylogenetic hierarchy is based on taxonomic levels, that is, the hierarchy of biological classification.

[0459] In some cases, phylogenetic hierarchies include conventional taxonomic ranks, taxonomic units, or species groupings that are advantageously chosen based on genetics and / or to address specific identification problems (e.g., identifying Gram-positive versus Gram-negative bacteria).

[0460] Taxonomic units or species groupings are based on criteria selected according to specific circumstances. These criteria can be genetic, phenotype, clinical (symptoms, antibiotic sensitivity), or material (using antibiotic susceptibility testing types).

[0461] According to the example shown, there are 7 levels, arranged in descending order as follows: domain, phylum, class, order, family, genus, and species.

[0462] However, this is only an example, because there are lower-level phylogenetic groups that can explain morphological and resistance differences within the same species.

[0463] Nodes at a certain level are identified by letters and numbers, corresponding to... Figure 13 The order in which they appear from left to right.

[0464] When there is a phylogenetic association between two nodes, Figure 13 The connection is visible in the middle.

[0465] In this view, higher-level nodes are the parent nodes of lower-level nodes, and vice versa.

[0466] The following table can be established as follows: Table 1

[0467] therefore, Figure 13 The phylogenetic tree contains 10 species belonging to 7 genera and 5 families, which in turn belong to 4 orders and 2 classes, corresponding to 2 phyla of the domain Bacteria.

[0468] With the help of biological knowledge, a phylogenetic tree can be drawn based on known species.

[0469] In other words, it can be assumed that from 10 species E1 to E10, we can deduce... Figure 13 The phylogenetic tree.

[0470] Of course, the 10 species E1 to E10 can be associated with different phylogenetic trees.

[0471] Specifically, a 3- or 4-level tree can be considered instead of a 7-level tree.

[0472] Furthermore, the hierarchy need not be as... Figure 13 It is completely continuous, like a tree.

[0473] For example, a 4-level tree can be composed of nodes D1, P1, P2, F1 to F5 and E1 to E10.

[0474] However, in practice, it is preferable to select a level close to species E1 to E10 to obtain the most accurate information possible using one of the following techniques.

[0475] In this implementation, verification step E162 is performed on each node.

[0476] Therefore, during verification step E162, computer 56 calculates the score for each node.

[0477] The score of each node is calculated as the sum of the scores of all its child nodes.

[0478] This allows you to obtain a value for each node.

[0479] According to this definition, the root fraction of a phylogenetic tree is equal to 1.

[0480] The following three examples illustrate the calculation and its results.

[0481] The first example corresponds to the predicted species Enterococcus faecalis.

[0482] In this scenario, the neural network yields the following scores: Table 2

[0483] Therefore, the other non-zero values ​​for nodes are as follows: Table 3

[0484] Therefore, the set of nodes that meet the reliability condition (the threshold is set to 0.7 here) is nodes D1, P2, C2, O3, F4, G6 and E6.

[0485] In step E164, the observation system 10 can provide the user with additional information: not only known species, but also other phylogenetic levels.

[0486] This example demonstrates that the use of phylogenetic trees does not interfere with accurate predictions.

[0487] Typically, computer 56 should identify the correct species rather than the correct class.

[0488] In the second example, the objective was to predict the Enterobacter cloacae complex, and the scores obtained by the neural network are as follows: Table 4

[0489] This situation may occur when two species are close to each other; in this example, species E1 and E2 belong to the same genus.

[0490] It should also be noted that under this assumption, computer 56 predicted species E2, i.e., Escherichia coli, with the highest score, which was incorrect.

[0491] When the threshold is 0.5, the computer 56 outputs an error, or when the threshold is higher (typically 0.7), the computer 56 does not provide a prediction.

[0492] Calculate the values ​​for each node. The non-zero values ​​for other nodes are as follows: Table 5

[0493] When the threshold is 0.7, nodes F1, O1, C1, P1, and D1 are reliable (meeting the reliability conditions), while other nodes are unreliable.

[0494] One way to select the determined information is to traverse downwards from the initial (root) node to the corresponding species node, and along the node path that satisfies the reliability condition, select the complete name associated with the last node.

[0495] In this case, the path is as follows: node F1, node O1, node C1, node P1, and then node D1.

[0496] The identified information is node D1.

[0497] This corresponds to the determination that organism 18 belongs to the Enterobacteriaceae family.

[0498] It should be noted that although the output may be an incorrect prediction, the final information obtained is correct.

[0499] At a threshold of 0.7, the observation system 10 provides information that the species of organism 18 is unknown or unreliable, but the family is identified as Enterobacteriaceae.

[0500] Therefore, the above-mentioned phylogenetic tree technique can achieve good identification at the family level when species-level identification is not possible.

[0501] In the third example, the aim was to identify the species *Staphylococcus capillus-veneris* (…). Staphylococcus capitis However, the neural network was not trained to predict the species.

[0502] Staphylococcus capitulata corresponds to G7 of the same genus at nodes E8 to E10. Figure 14 The dashed lines in the middle indicate the corresponding connections. Figure 13 Tree plus node EX (related to the species Staphylococcus capitella).

[0503] The scores for the implementation of step E160 are as follows: Table 6

[0504] As expected, the neural network predicted species that were closely related to the species to be identified, since species E9 and E10 belong to the same genus as the species to be identified.

[0505] It should also be noted that under this assumption, computer 56 predicted species E9, namely the species *Staphylococcus saprophyticus*, with the highest score, which was incorrect.

[0506] When the threshold is 0.5, the computer 56 outputs an error, or when the threshold is higher (typically 0.7), the computer 56 does not provide a prediction.

[0507] Calculate the values ​​for each node. The non-zero values ​​for other nodes are as follows: Table 7

[0508] Computer 56 then performs the same operation: traversing down from the initial (root) node to the corresponding species node, along the node path that satisfies the reliability condition, and selecting the full name associated with the last node.

[0509] This allows it to identify the node G7 as the source of reliable information.

[0510] The organism to be identified was determined to belong to the genus Staphylococcus.

[0511] It should be noted that although the output may be an incorrect prediction, the final information obtained is correct.

[0512] At a threshold of 0.7, the observation system 10 provides information that the species of organism 18 is unknown or unreliable, but the genus is identified as Staphylococcus.

[0513] Furthermore, since the species has an E8 score of 0, it can be definitively concluded that the species is not Staphylococcus aureus.

[0514] Therefore, the above-mentioned phylogenetic tree technique can achieve good identification at a phylogenetic level higher than the species level, even when the neural network is unable to identify the species to be identified.

[0515] The results show that the use of phylogenetic trees significantly improves identification efficiency.

[0516] This method can provide information at higher phylogenetic levels when the scores obtained at lower levels are not high enough.

[0517] This improves robustness by restricting species grouping and tree paths based on genetic associations.

[0518] Furthermore, without any retraining of the neural network, the observation system 10 can also identify unknown species at a higher phylogenetic level.

[0519] Therefore, the method effectively solves the problem of identifying neural network training.

[0520] Specifically, to identify bacterial species, a neural network needs to be trained on a large number of strains of that species. Some bacterial species are very common in clinical analyses, making it easy to build a training database for them. However, many species are much rarer, making it difficult to build a sufficiently comprehensive training database to include them.

[0521] Traditional classification methods at most produce no information output, and in the worst case, they systematically make mistakes for species outside the training space, while the above method can predict relevant information about the organism to be tested.

[0522] Therefore, the method can not only effectively identify the species of organism 18, but also provide reliable information when the species is unknown or difficult to identify precisely.

[0523] Other implementation schemes of the above observation method may be advantageously considered.

[0524] Specifically, according to one implementation scheme, the time interval for acquiring each image during the acquisition phase P1 is adjustable.

[0525] Advantageously, the time interval depends on at least one of the determined characteristics.

[0526] For example, for the growth of E. coli incubated at 37°C, it is known that there is a 2-hour lag phase, followed by 10 hours of exponential growth, then a slowdown to a stationary phase, and then a decline phase.

[0527] In this case, it is advisable to collect one image per hour during the lag phase and one image every 10 minutes during the growth phase.

[0528] It is even conceivable that time intervals could be determined in real time based on observed changes in colony growth.

[0529] To reduce computational load, it is also beneficial to limit the portion of the image to be analyzed.

[0530] For example, one could envision determining at least one attribute associated with the colonies being detected, then screening the colonies, with computational steps (especially in neural network applications) performed only for the selected colonies.

Claims

1. A method for observing a sample (12), said sample (12) comprising: - Group of organisms (14); - A solid matrix (16) supporting the organism group (14); The method is implemented by an observation system (10) including a lensless imaging unit (34) and the method includes: -The stage of acquiring images of sample (12), - Image analysis stage, the analysis stage includes: - A step of calculating the probability that at least one organism (18) belongs to a preset species, said calculation step comprising applying a neural network to at least one image to obtain the probability that the at least one organism (18) belongs to a preset species, said preset species being a set of species that the neural network can be trained to identify in an image. - The step of verifying that the obtained probability meets the reliability condition by comparing at least one value of the obtained probability with a preset threshold, and - The step of determining at least one piece of information related to the identity of the at least one organism (18), wherein, if the reliability condition is not met, the information belongs to a species not included in a preset species.

2. The observation method according to claim 1, wherein the value obtained from the probability is the maximum value of the calculated probability.

3. The observation method according to claim 1 or 2, wherein the organism (18) is arranged in colonies, the observation method further includes a colony detection stage to detect and locate colonies, and the neural network of the calculation step also takes the location of at least one colony as input.

4. The observation method of claim 3, wherein the analysis phase includes the step of determining at least one attribute associated with the detected colony, and the step of selecting colonies to obtain selected colonies, the obtaining step being performed only on the selected colonies.

5. The observation method according to any one of claims 1 to 4, wherein a plurality of organisms (18) correspond to the same species, and the determining step comprises performing an aggregation function on the probabilities of each organism (18).

6. The observation method according to claim 5, wherein the aggregation function takes the species corresponding to the maximum value in the probability set as the species of the plurality of organisms (18).

7. The observation method according to claim 5, wherein the aggregation function assigns the plurality of organisms (18) to species such that the sum of the probabilities of each organism (18) belonging to that species is maximized.

8. The observation method according to any one of claims 1 to 7, wherein, In the calculation step, the neural network is a multi-class neural network.

9. The observation method according to any one of claims 1 to 8, wherein, Define a phylogenetic tree for the neural network, the phylogenetic tree containing a set of nodes distributed according to at least two phylogenetic levels, the lowest phylogenetic level being the species, and nodes that have phylogenetic associations are connected together. The verification step includes calculating the probability of each node based on the probability obtained in the calculation step and verifying the conformity conditions of each node.

10. The observation method according to claim 9, wherein, In the determination step, the information determined belongs to the lowest phylogenetic level that meets the compliance criteria.

11. The observation method according to claim 9 or 10, wherein the phylogenetic hierarchy is selected from: phylum, class, order, family, and genus.

12. The observation method according to any one of claims 9 to 11, wherein the number of phylogenetic levels is greater than or equal to 4.

13. The observation method according to any one of claims 1 to 12, wherein the value of the preset threshold is between 0.7 and 0.

9.

14. An observation system (10) for observing a sample (12), the sample (12) comprising: - Groups of organisms (14), and - A solid matrix (16) supporting the organism group (14); The observation system (10) includes a lensless imaging unit (34) and is adapted to: - Obtain the image of sample (12); - Analyze the image, the analysis including: - Calculate the probability that at least one organism (18) belongs to a preset species, the calculation comprising applying a neural network to at least one image to obtain the probability that the at least one organism (18) belongs to the preset species, the preset species being a set of species that the neural network can identify in the image after training. - By comparing at least one of the obtained probabilities with a preset threshold, the reliability condition is verified. - Determine at least one piece of information related to the identity of the at least one organism (18), wherein, if the reliability condition is not met, the information belongs to a species not included in a preset species.

15. The observation system according to claim 14, wherein the observation system (10) further comprises an adjustment element (37) for adjusting the environment of the observation system (10).

Images