METHOD FOR IN SITU ANALYSIS OF OBJECTS SUSPENDED FROM A SUBSTANCE

Abstract

The present invention relates to a computer-implemented method for training a neural network for classifying images of objects suspended in a substance, preferably crystals of that substance, and to a computer-implemented method that uses this trained neural network for analyzing objects suspended in said substance, particularly crystals suspended in said substance, especially for real-time monitoring of the progress of a crystallization process of that substance. Figure 7 to be published

Description

Title of the invention: METHOD FOR IN SITU ANALYSIS OF OBJECTS SUSPENDED IN A SUBSTANCE technical field

[0001] The invention relates to a computer-implemented method for training a neural network for classifying images of objects suspended in a substance, preferably crystals suspended in a substance, and to a computer-implemented method that uses this trained neural network for analyzing objects suspended in said substance, preferably crystals suspended in said substance, in particular for real-time monitoring of the progress of a crystallization process of this substance. Previous technique

[0002] Crystallization is a fundamental process in chemistry, used for the purification and separation of molecules in a variety of industries, such as pharmaceuticals, fine chemicals, food processing, and materials. This process involves changing a substance, by altering its solubility, from a liquid or gaseous state to a solid state in the form of crystals, thus enabling the isolation of specific compounds with high purity. The molecules in question may include pharmaceutical active ingredients, polymers, chemical intermediates, precursors of inorganic materials, or even food additives.

[0003] Real-time monitoring of crystallization is essential to guarantee the quality of the final product. This control allows for the optimization of crystal size, shape, and purity, parameters crucial for the operability of crystallization processes and the subsequent performance of the manufactured products. For example, in the pharmaceutical industry, the size and morphology of an active ingredient's crystals can influence its bioavailability, while in the polymer industry, these characteristics can determine the mechanical and thermal properties of the final material. The operability of the solid chain (unit operations of crystallization, solid / liquid separation, crystal transport, and drying) is also dependent on crystal size and shape.

[0004] In the literature, known technologies for real-time crystallization monitoring include various analytical methods, such as in situ microscopy, light scattering, the use of FBRM (Focused Beam Reflectance Measurement) technology, or spectroscopic techniques (such as infrared spectroscopy). These techniques make it possible to monitor the morphological transformations of crystals. and to identify critical points in the process, such as the timing of germination, crystal growth, and / or crystal shape. Precise control of these steps is indeed of paramount importance to obtain a crystallized product that meets the purity, performance, and operability requirements of the process.

[0005] Advanced crystallization monitoring is increasingly sought after to improve the efficiency and quality of industrial processes, while minimizing losses and energy costs.

[0006] To perform population analyses of crystals, there are three main types of methods. First, there are ex situ methods. These allow for study outside the reaction environment and require the collection of crystal suspensions and the preparation of dedicated samples according to the operating constraints. Ex situ methods are suitable for characterizing crystals obtained at the end of crystallization at room temperature and without risk of altering the system, but not always for characterizing crystals obtained at temperature during the process. Indeed, the collection and preparation of samples may alter the objects to be characterized, for example, if samples are taken at a temperature other than room temperature, or under conditions where the system is likely to evolve rapidly.

[0007] Among existing ex situ methods, the system developed by Sympatec, called "QICPIC," is typically used on samples of crystal suspensions, or crystals taken from samples and then resuspended in a saturated solvent. This system involves circulating the suspension of dilute crystals: the suspended crystals are thus photographed as they pass in front of a sensor with a high-resolution camera. The main drawback of this system lies in its viewing window: without edge effect management, it ignores crystals that are too large to fit within the frame. Indeed, the morphological operations performed are too simple to allow for a truly in-depth study of the size of a crystal population.

[0008] Other ex situ methods consist of observing crystals by optical microscopy or electron microscopy, but require the use of algorithms or image processing methods to provide access to precise measurements and sufficient statistics to be representative.

[0009] Simpler ex situ methods, in their implementation, are based on deep learning. Rather than passing the crystals through a liquid stream in front of a camera as with the "QICPIC" system, several shots in front of a microscope optic are sufficient. Then, a trained model can be used to process the data acquired ex situ. For example, one can cite the work of Jaiming Han, etc., and their "S2aNet" model, which can be used to perform regressions and extract what he thinks is each crystal (Jiaming Han. s2anet CNN model, GitHub repository, 2021).

[0010] The work of Daniel Bischoff also gives interesting results in terms of automatic detection of crystals in images, even if orientations are present on the crystals (see Daniel Bischoff, Brigitte Walla, Dirk Weuster-Botz. Machine learning-based protein crystal detection for monitoring of crystallization processes enabled with large-scale synthetic data sets of photorealistic images, Analytical and Bioanalytical Chemistry, 2022, 414,21, 6379-6391).

[0011] In this context, patent application WO03087326A2 describes an apparatus that automatically captures, stores, and analyzes images of crystallization experiments contained in systems referred to in the document as "crystallization plates." The apparatus uses a support capable of accommodating different types of crystallization plates and includes image acquisition optics, a light source, a support positioning controller, and a database for storing experimental information. The apparatus captures images of each crystallization site located in the cells of the plates using different light source and polarization conditions, and then processes these images to form extended fused images of each crystallization site.

[0012] Patent application CN114358175A describes a system for precisely identifying the endpoint in a wastewater evaporation crystallization process using ammonia. It uses a sampling platform to collect crystals, and then an image recognition system to determine the endpoint.

[0013] US patent application 2013286180A1 describes a system for measuring the crystallization rate of crystalline-amorphous mixtures. It uses a sample holder, a heating device to melt an ink composition and maintain it at a specified temperature, a cooling device to cool the molten ink, a microscope and a video recorder to capture images of the cooled ink, and a video processing computer to extract crystallization parameters from the images, enabling the identification of fast-solidifying crystalline inks.

[0014] All the aforementioned ex situ crystallization monitoring methods therefore require removing samples from the reaction system, on which measurements are performed, and processing the crystal images obtained outside the reaction system. Ex situ image processing tends to simplify the problem by eliminating, for example, issues related to turbidity or suspension concentration in the reaction medium. This type of technique therefore does not allow for truly monitoring crystallization and capturing its kinetics. By recovering crystals at the end of the experiment, it is possible to model populations a posteriori. However, estimating information a priori remains impossible or completely biased. We can therefore conclude that an ex situ analysis of a crystallization is insufficient to summarize its entire kinetics, and only provides information on the evolution of a system and the final point reached a posteriori.

[0015] There are also the so-called in situ crystal analysis methods. This is referred to as intra-reactional analysis, which is carried out, for example, using a measuring or imaging probe placed directly in the medium where crystallization occurs.

[0016] Several known in situ methods exist. One of the most widely used is the Focused Beam Reflectance Measurement (FBRM) technique, which consists of measuring the chord length parameter on crystals (see Hishamuddin E. Real-time Monitoring of the Crystallization of Palm Oil and its Products by Focused Beam Reflectance Measurement (FBRM), 2014, Palm Oil Developments, 61, pl2-13, 21-25, 2014). This type of analysis can provide qualitative information on the evolution of systems and is better suited to monitoring isotropic crystals (spheres, cubes). However, it is not possible to relate the measured chord lengths to the morphological characteristics of the crystals (i.e., their effective shape and dimensions). The chord length values ​​obtained can also be more difficult to interpret for anisotropic crystals (needles, platelets, etc.).

[0017] Imaging probes also exist (for example, the EasyViewer probe manufactured by Mettler Toledo, the Sopat probe manufactured by SOPAT GmbH, or the probes manufactured by BlazeMetrics) that allow the acquisition of images of crystals in suspension in situ. However, image processing tools are necessary in this case. The EasyViewer probes produced by Mettler Toledo and the associated image processing software "iC Vision" provide access to a simplified version of the FBRM algorithm applicable to images. The previously mentioned S2aNet model can also be used in situ. If the particle concentration is sufficiently low, or if they are not entangled, it is possible that an algorithm such as this one could recognize them (see, for example, the work of Yuanyi Wu, Zhenguo Gao, and Sohrab Rohani).Deep leaming-based oriented object detection for in situ image monitoring and analysis: A process analytical technology (PAT) application for taurine crystallization, Chemical Engineering Research and Design, 2021, 170, 444-455). .

[0018] Patent application CN114418990A describes an in situ crystallization monitoring system using a high-resolution imaging probe. The classification of the detected crystal types is performed by deep learning. In particular, the characterization method employed includes extracting imagelets, each containing a crystal to be classified, extracting statistics, and then classifying the crystals based on knowledge of their morphological characteristics (shape, circularity, size, etc). Even though this type of system can work at low concentrations, it is not capable of giving reliable results as soon as the concentration of the reaction medium exceeds a certain limit.

[0019] Patent application CN104991984A describes a system and method for monitoring data for a sugar crystallization process by boiling. The method consists of obtaining real-time and historical data, establishing a data model for the crystallization process, optimizing this model, and then measuring and calculating various process parameters.

[0020] Furthermore, patent application CN112730174A describes a method for real-time monitoring of the concentration of spheroidal particles in a crystallization process. This method involves preparing suspensions of spheroidal particles at different concentrations, observing these suspensions in motion using a two-dimensional optical imaging system, calculating the total number of pixels of spheroidal particles in a two-dimensional image, and then using this curve to obtain real-time concentration data during the crystallization process. This method provides accurate monitoring of the concentration of spheroidal particles in crystallization, also allowing information to be obtained on crystal aggregation as a function of real-time variations in concentration.

[0021] It is therefore observed that in situ crystallization monitoring methods are performed in the reaction medium and that they employ analytical crystal characterization methods or deep learning. In both cases, the methods are not sufficiently robust to excessive turbidity or to overly complex crystal images, such as, for example, images with entanglement.

[0022] The last type of method known for monitoring crystallization is called "pseudo in situ" because it involves an intra-reaction study but conducted outside the crystal growth medium. Typically, 3D imaging methods employ a binocular imaging probe system placed outside the crystallizer. With this type of system, it is possible to reconstruct, albeit at the cost of significant computation time, the 3D morphological appearance of the crystals present inside the crystallizer.

[0023] The work of Huo, Liu et al. is based, for example, on a similar system consisting of a binocular probe placed outside the crystallizer, allowing double image acquisition of the crystals in the mixture (see Huo Y., Liu T., Yang Y., Ma CY, Wang XZ, Ni X. In Situ Measurement of 3D Crystal Size Distribution by Double-View Image Analysis with Case Study on l-Glutamic Acid Crystallization, Industrial & Engineering Chemistry Research, 2020, 59,10, 4646-4658). A 3D reconstruction of the crystals is therefore possible by interpolation.

[0024] The limitation of pseudo-in situ methods lies in the readability of the images resulting from shooting through the wall. Although the system is proven, if the concentration of the mixture becomes too high, the opacity increases and it becomes virtually impossible to recover information on each of the crystals, despite adjusting the angle between the two cameras.

[0025] In addition to the aforementioned methods, there are also methods for studying single crystals. For example, US patent application 5746825A describes a method and apparatus for measuring the diameter of a growing single crystal at the crystallization boundary during its extraction from molten material. It works by capturing an image of a portion of the crystallization boundary reflected on at least one mirror, and then determining the diameter of the single crystal based on the relative position of the crystallization boundary observed on the mirror image.

[0026] However, the study of single crystals does not take into account the problems encountered with crystals that may be entangled and for which images cannot be processed with current segmentation algorithms.

[0027] It is therefore observed that the prior art primarily uses traditional image recognition methods to analyze crystal images, including multi-scale detection algorithms, model-based recognition algorithms, multivariate statistical models, or synthetic image analysis algorithms. Although these methods give good results in some simple cases, their accuracy is affected by numerous factors, such as image quality, solids concentration, clustering and overlap between crystals, etc., and conventional image recognition methods are therefore unsuitable for analyzing crystal images in a batch or continuous industrial crystallization process. Furthermore, traditional methods cannot handle images of suspensions with high in situ crystal concentrations.

[0028] One of the objectives of the present invention is to remedy the disadvantages of the prior art mentioned above by proposing a method for the analysis of objects in suspension of a substance of interest, preferably crystals of said substance, which method being capable of providing information on characteristics of interest of said objects, even at a high concentration and in real time for example in situ during a crystallization. Objectives and Summary of the Invention

[0029] The invention proposes to achieve the aforementioned goal by means of a computer-implemented method for training a neural network for image classification. of objects suspended from a substance, preferably crystals of said substance, comprising the following steps: i. Acquire a training dataset comprising a. at least five classes of belonging to objects of said substance, each class being related to a single reference population of objects of said substance which is statistically different from that of the other classes; and b. a plurality of reference images of samples of suspended matter of said substance for each of said membership classes, said samples being previously prepared by suspending matter belonging to one of said membership classes in a liquid medium, so as to obtain samples of suspended matter at at least two different entrainment concentrations for each of said classes, which plurality of reference images is acquired by a suspended matter imaging technique and comprises at least 100 images for each sample at different concentrations for each of said classes; ii. Provide said training dataset to an image classification neural network, in order to train the neural network to classify reference images of samples of suspended objects of said substance, according to membership classes and for each of said at least two training concentrations.

[0030] Thus, the method according to the present invention provides a training, or learning, process for a neural network that first classifies reference images visualizing populations of objects suspended in a substance of interest, particularly crystals, according to their class membership among a set of known or previously characterized membership classes. According to the invention, the membership classes selected for training the neural network are chosen so that they are independent of each other; that is, each of these classes relates to a single reference population of objects of said substance that is statistically different from that of the other classes.

[0031] In other words, this means that each class chosen in the process according to the invention relates to a single population of objects whose distribution of one or more specific characteristics or properties is known, such as size (for example, length, width and / or average equivalent diameter) and / or morphology (for example, shape, slenderness, rectangularity), which distribution is significantly different from those representative of the populations of objects of the substance belonging to the other classes considered, as estimated for example by analysis of at least one distribution metric and / or by at least one appropriate statistical test.

[0032] The inventors have discovered that the neural network thus trained can effectively be used to analyze "unknown" samples of suspended objects of said substance, preferably crystals of said substance, and obtain information on one or more characteristics of interest of these objects, for example of crystals at the end of a crystallization process, but also in real time during this crystallization process.

[0033] The invention therefore also relates to a computer-implemented method for analyzing objects suspended in a substance, preferably crystals suspended in said substance, comprising the following steps: i. To obtain one or more actual images of objects suspended in said substance; and ii. Provide the real images obtained in the previous step to an image classification neural network, said neural network being trained by a computer-implemented method according to any of the embodiments of the present invention to give a probability that each of said real images visualizes a population of objects belonging to a class from a predefined set comprising at least five different membership classes as defined in this application.

[0034] In particular, the real images of suspended objects are acquired with the same suspended object imaging technique as that used for the acquisition of the reference training images of the neural network employed.

[0035] Advantageously, the use of the same image acquisition technique during the process of analyzing suspended objects and during the training of the neural network used, allows the neural network trained according to the present invention to provide reliable output information on the belonging of the objects present on the real images to a certain class according to the characteristics of the suspended objects analyzed, for example in terms of the distribution of their sizes or their morphologies.

[0036] While prior art methods are capable of characterizing crystals in reaction mixtures with a relatively low concentration (i.e., less than 2%) and allowing each crystal to be visualized independently of the others, the method according to the present invention makes it possible to characterize a set of populations of suspended objects, and in particular suspended crystals, even under conditions ranging from 2% to 35% concentration in a suitable liquid medium. This represents a considerable advantage because, depending on the size and / or the shape of the crystals considered, from concentrations of 2 to 5%, it often becomes impossible to distinguish one crystal from another according to the techniques known in the prior art.

[0037] As already mentioned, another advantage of the present invention is to provide a method which allows for real-time and in situ monitoring, i.e. in a reaction medium, in particular of the crystallization of a substance of interest, as well as post-crystallization monitoring or analysis.

[0038] According to a much preferred embodiment of the invention, the method therefore makes it possible to follow in real time the progress of a crystallization process of a substance of interest from several real images of the crystals in suspension of the substance in their growth medium, i.e. in situ, acquired at one or more times of the crystallization process.

[0039] The process of the invention therefore makes it possible, to a certain extent, to avoid taking samples, the associated uncertainties (evolution of the system), and the processing time for ex situ analysis.

[0040] By simply using previously acquired reference images as training data for the neural network, the method of the invention makes it possible to obtain, by classification and estimation, real-time data on the crystallization of a certain substance.

[0041] All parameters of the method of the invention are also adjustable. For example, for real-time monitoring, it is possible to adjust the number of images to be analyzed according to the progress of the crystallization process. It is also possible to add real-time concentration data to the neural network training dataset, for example, using monitoring methods based on chemical composition analysis of the substance in the crystallizer.

[0042] In other words, the method according to the present invention advantageously allows for continuous adaptation to the data it must process. As long as corresponding object membership classes are added for training the neural network, the versatility of the classification is ensured.

[0043] Another advantage of the method according to the present invention is its relatively reasonable data processing time. Thanks to a simple classification, rather than a regression chain, it saves computation time, thus enabling real-time processing. Indeed, depending on the sequence size in number of images, the processing time can vary from 1 to 4 seconds. Since a sequence is processed as soon as it is complete (every "X" images), the processing time is carried out in parallel with the continuous image capture.

[0044] The invention also relates to a computer program comprising instructions which, when the program is executed by a computer, lead the computer to execute the steps of a process according to any one of the embodiments described herein.

[0045] Another object of the present invention relates to a computer-readable storage device or medium storing instructions which, when executed by a computer, cause the computer to carry out the steps of a process according to any of the embodiments described herein.

[0046] The invention also relates to a computer system comprising at least one computer and one or more computer-readable storage devices or media, which devices or media store instructions which, when executed by the computer, cause the computer to carry out the steps of a process according to any one of the embodiments described herein.

[0047] The present invention also relates to a device comprising a computer system according to any one of the embodiments described herein and an image acquisition system linked or connected to this computer system, which image acquisition system is preferably adapted to be immersed in a liquid medium comprising suspended objects of a substance and in particular crystals of said substance.

[0048] Other objects and advantages of the invention will become apparent from the following description of particular embodiments of the invention, given by way of non-limiting examples, the description being made with reference to the attached figures described below. List of figures

[0049] [Fig.1A]

[0050] Fig. 1A illustrates cumulative frequency histograms (on the ordinate axis) for 7 crystal size classes of 4-aminobenzoic acid (samples 1-7), as a function of the widths of said crystals (abbreviated as L1 and expressed in pm, on the abscissa axis).

[0051] [Fig.1B]

[0052] Fig. 1B illustrates cumulative frequency histograms (on the ordinate axis) for 7 size classes of 4-aminobenzoic acid crystals (samples 1-7), as a function of the lengths of said crystals (abbreviated as L2 and expressed in pm, on the abscissa axis).

[0053] [Fig.2]

[0054] Fig. 2 illustrates topographic plots (width in pm, on the x-axis / length in pm, on the y-axis) of different size classes of 4-aminobenzoic acid crystals (samples 1-4).

[0055] [Fig.3A]

[0056] Figure 3 schematically illustrates a data processing chain for ex-situ 4-aminobenzoic acid crystals, for the characterization of a set of crystal membership classes as training data for a neural network according to an example of a preferred embodiment of the present description.

[0057] [Fig.3B]

[0058] Fig. 3B illustrates a cumulative frequency histogram (on the ordinate axis) for an illustrative class of 4-aminobenzoic acid crystal size, as a function of the widths of said crystals (abbreviated as L1 and expressed in pm, on the abscissa axis), as well as the corresponding distribution curve of the widths of the population of crystals considered, obtained by interpolation according to a gamma law.

[0059] [Fig.3C]

[0060] Fig. 3C illustrates a cumulative frequency histogram (on the ordinate axis) for an illustrative class of 4-aminobenzoic acid crystal size, as a function of the lengths of said crystals (abbreviated as L2 and expressed in pm, on the abscissa axis), as well as the corresponding distribution curve of the lengths of the population of crystals considered, obtained by interpolation according to a gamma law.

[0061] [Fig.3D]

[0062] Fig. 3D illustrates a cumulative frequency histogram (on the ordinate axis) for an illustrative class of 4-aminobenzoic acid crystal size, as a function of the width / length ratio of said crystals (abbreviated as L1 / L2, on the abscissa axis), as well as the corresponding distribution curve of the lengths of the population of crystals considered, obtained by interpolation according to a gamma law.

[0063] [Fig.4]

[0064] Fig. 4 schematically illustrates a data processing chain for 4-aminobenzoic acid crystals in situ, to obtain a plurality of reference images as training data for a neural network according to an example of a preferred embodiment of the present description.

[0065] [Fig.5]

[0066] Figure 5 schematically illustrates the architecture of the Inception classification neural network used in a process according to an example of a preferred embodiment of the present description (Szegedy et al. “Rethinking the Inception Architecture for Computer Vision” 2015, arXiv: 1512.00567, https: / / paperswithcode.com / method / inception-v3).

[0067] [Fig.6]

[0068] Figure 6 schematically illustrates a real-time data processing chain from the crystallization of 4-aminobenzoic acid according to an example of a preferred embodiment of the present description.

[0069] [Fig.7]

[0070] Figure 7 illustrates the results of monitoring the crystallization of 4-aminobenzoic acid obtained by a crystal image analysis method according to a preferred embodiment of this description, and in particular the evolution of the mean or maximum of the weighted estimated distribution curves of widths (L1, in pm), lengths (L2, in pm), and the width / length ratio (L1 / L2) obtained for each actual image analyzed, as a function of the times in the crystallization process at which said images were acquired (i.e., image batch). Only one weighted estimated distribution curve of crystal widths is shown for illustrative purposes in Figure 7. Description of the implementation methods

[0071] According to the present invention, the expressions "between ... and ..." and "between ... and ..." are equivalent and mean that the limit values ​​of the interval are included in the range of values ​​described. If this were not the case and the limit values ​​were not included in the range described, such clarification will be provided by the present invention.

[0072] In the following, particular embodiments of the invention may be described. They may be implemented separately or in combination with each other, without limitation of combinations where technically feasible.

[0073] For the purposes of the present invention, the terms "object" or "objects" of a certain substance mean any of the solid objects of said substance, in particular crystalline or non-crystalline solid particles, preferably crystals, which can be detected and measured using any device and / or technique for imaging suspended objects known to those skilled in the art, for example by an imaging probe according to any of the variants described in this application.

[0074] The terms "object" or "objects" of a certain substance as used in the present invention can therefore cover a wide range of different sizes and / or morphologies, provided that they can be detected and measured by means of any device and / or technique for imaging suspended objects known to a person skilled in the art.

[0075] According to a highly preferred embodiment of the invention, the objects of any substance considered according to any of the variants illustrated in this description or in the claims are solid particles of said substance and more preferably crystals of said substance. According to this highly preferred embodiment of the invention, the terms "object" or "objects" may be replaced by the terms "crystal" or "crystals" at each occurrence in the description and in the claims.

[0076] The expression "suspended object imaging technique" is to be understood in the context of the invention as including any of the "in situ" characterization techniques, that is to say, the characterization of any type of solid object or particle, in particular crystals, which are suspended in a liquid medium.

[0077] In the context of the present invention, the expression "statistically different" means that a population of objects, in particular crystals, belonging to a certain class, or more specifically a distribution curve of at least one property (or characteristic) of the objects, in particular crystals, of said population, is significantly different from that of the other classes considered, as evaluated or determined by any method or analysis known in the field of statistical analysis or data science.

[0078] For example, as explained in more detail in the following paragraphs, a distribution curve of at least one property of objects, in particular crystals, of a population belonging to a certain class, can be evaluated or determined to be significantly different from that(s) of populations of objects, in particular crystals, belonging to the other classes considered, by analysis of at least one distribution metric and / or by at least one appropriate statistical test.

[0079] In the context of the present invention, the expressions "distribution curve", "distribution law" and "distribution function" are used interchangeably.

[0080] The expression "greater than..." is understood as strictly greater than, and symbolized by the sign ">", and the expression "less than" as strictly less than, and symbolized by the sign "<".

[0081] In this application, the term “include” is synonymous with (means the same as) “include” and “contain,” and is inclusive or open and does not exclude other unstated elements. It is understood that the term “include” includes the exclusive and closed term “consist.”

[0082] The invention relates firstly to a method implemented by computer, or by a processor, for training a neural network for the classification of images of objects suspended in a substance, which method comprises, preferably consists of, the following steps: i. Acquire a training dataset comprising a. at least five classes of object membership of said substance, each class relating to a single reference population of objects of said substance that is statistically different from that of the other classes; and b. a plurality of reference images of samples of suspended matter of said substance for each of said membership classes, said samples being previously prepared by suspending matter belonging to one of said membership classes in a liquid medium, so as to obtain samples of suspended matter at at least two different entrainment concentrations for each of said classes, which plurality of reference images is acquired by a suspended matter imaging technique and comprises at least 100 images for each sample and at each concentration for each of said classes; ii. Provide said training dataset to an image classification neural network, in order to train the neural network to classify reference images of samples of suspended objects of said substance, according to membership classes and for each of said at least two training concentrations. The substance

[0083] According to the invention, the images of the substance to be classified can be images of objects of any substance or combination of organic or inorganic substances known in the art and capable of forming crystalline or non-crystalline solid particles when passing from a liquid or gaseous state to a solid state.

[0084] Preferably, the images of the substance to be classified are images of crystals of any substance or combination of organic or inorganic substances known in the art and capable of forming crystals when changing from a liquid or gaseous state to a solid state. The images of the substance to be classified and the classification classes considered will therefore in this case be preferably images and classes of "crystals" of said substance, respectively.

[0085] The substance may also be any organic or inorganic compound, preferably having a melting point above ambient, preferably greater than or equal to 20°C at atmospheric pressure (i.e. about exactly 0.1 MPa absolute).

[0086] Without limitation, the substance may be, for example, a polymer or an active ingredient of pharmaceutical or medical interest, as well as any additive used in the food industry, chemical intermediates, precursors of inorganic materials and / or mineral crystals.

[0087] Preferably, the substance can be chosen from organic compounds such as pharmaceutical active ingredients, like aspirin, paracetamol or any other active ingredient of interest.

[0088] The substance can also be chosen from inorganic compounds such as aluminas, silicas, zeolites, and all oxide materials and their precursors used for applications in catalysis, in batteries (materials, cathodes, anodes, ...), optics, etc.

[0089] The substance can also be chosen from organic or mineral salts such as those used in the agricultural industry, such as ammonium sulfate or any other compound of interest in this sector. Step i) Acquisition of training data: Membership classes

[0090] The training data set used in the process according to the invention, referred to as training or learning data, comprises at least five classes of membership of objects of the substance of interest considered, preferably at least 6, 7, 8, 9 or 10 classes of membership.

[0091] According to the invention, the classes (or membership classes) chosen from the training dataset are independent or distinct from each other, that is to say, each of these classes relates to a single reference population of objects of the substance in question which is statistically different from that of the other classes.

[0092] In particular, the classes to which the objects belong are not redundant, that is to say, they are chosen so as to represent populations of objects, preferably crystals, having at least one or more properties or characteristics, in particular of size and / or morphology and / or composition, which are different from each other.

[0093] Preferably, a reference population of objects of the substance belonging to a specific class of the training data set is considered to be statistically different from those of the other classes if the distribution curve of at least one specific property or characteristic of the objects of said reference population is determined to be significantly different from the distribution curves of the same property of the objects of the reference populations belonging to the other classes, by analysis of at least one distribution metric and / or by at least one appropriate statistical test.

[0094] Each of the classes chosen in the training dataset is therefore preferably relative to a single reference population of objects, preferably crystals, having at least one distribution curve of a specific property or characteristic of the latter which is statistically different from the distribution curves of the same property of the objects, in particular crystals, of the populations belonging to the other classes.

[0095] More specifically, a reference population of objects, preferably crystals, belonging to a specific class in the training dataset can be considered statistically different from those of other classes if the distribution curve of at least one specific property or characteristic chosen from size, including width, length, width / length ratio and / or equivalent diameter, and morphology, including shape, slenderness, rectangularity, circularity, and / or faceting of objects, in particular crystals, of said population, is determined to be significantly different from the distribution curves of the same property of objects, in particular crystals, of populations belonging to other classes, by analysis of at least one distribution metric and / or by at least one appropriate statistical test.

[0096] In other words, each of the classes chosen in the training dataset is preferably relative to a single reference population of objects, in particular crystals, having at least one distribution curve of a specific property chosen from among size, in particular width, length, width / length ratio and / or mean equivalent diameter, and morphology, in particular shape, slenderness, rectangularity, circularity, and / or faceting of objects, in particular crystals, of said population, which is significantly different from the distribution curves of the same property of objects, in particular crystals, of populations belonging to the other classes.

[0097] The "equivalent diameter" of objects, in particular crystals, is a measure that describes the average size of objects in the population of objects of the substance under consideration and is generally calculated as the diameter of a sphere that would have the same volume as the crystal.

[0098] The "shape" of objects, particularly crystals, of the substance in question refers to the external geometric structure that the objects or particles of that substance assume when they form or crystallize. This shape can be determined by the internal molecular arrangement and the chemical forces involved. Solid particles, particularly crystals, can adopt various shapes such as cubic, hexagonal, orthorhombic, plate-like, spherical, and needle-like, among others, depending on their atomic structure and the crystallization conditions used.

[0099] Crystal faceting refers to the process of formation of planar faces and distinct geometric shapes on crystals. This phenomenon is influenced by the crystal growth conditions, the molecular structure of the substance, and the interactions between molecules. Several methods exist for measuring the degree of crystal faceting, such as scanning electron microscopy (SEM), X-ray diffraction, image analysis, and surface roughness measurement. These techniques allow for the precise evaluation of the facets of the crystals, their organization, and their quality.

[0100] According to a preferred embodiment, a population of objects, in particular crystals, belonging to a specific class in the training data set is considered to be statistically different from those of other classes if the width / length ratio between the average width and the average length of the objects, in particular crystals, belonging to said population differs by at least 10% from the width / length ratios between the average widths and lengths of the objects, in particular crystals, of the populations belonging to other classes.

[0101] According to a particularly preferred embodiment, a population of objects, in particular crystals, belonging to a specific class in the training data set is considered to be statistically different from those of the other classes if the size distribution curve of the objects, in particular crystals of said population, in particular the width distribution curve, the length distribution curve, and / or the width / length ratio distribution curve of the objects, in particular crystals, of said population, is / are determined to be significantly different from the distribution curves of the same property of the objects, in particular crystals, of the populations belonging to the other classes, by analysis of at least one distribution metric and / or by at least one appropriate statistical test.

[0102] This means that each class chosen in the training dataset is, even more preferably, relative to a single reference population of objects, in particular crystals, having at least one size distribution curve, in particular a width distribution curve, a length distribution curve and / or a width / length ratio distribution curve, which is statistically different from the distribution curves of the same property of objects, in particular crystals, of populations belonging to the other classes.

[0103] Of course, it is possible that each membership class provided in the neural network training dataset relates to a single population of objects, in particular crystals, having a distribution of a property or characteristic of objects, in particular crystals, other than size or morphology which represents a distinctive feature of said population and which is statistically different from those of populations belonging to other classes.

[0104] The distribution curves of at least one specific property or characteristic of the objects, in particular of the crystals, of each population considered are preferably cumulative frequency distribution curves or functions (CDF for "Cumulative Distribution Function" in English).

[0105] As already mentioned, the difference(s) between the distribution curves of the properties of objects, in particular crystals, of each population considered can(s) generally be evaluated or determined as significant relative to each other by analysis of at least one distribution metric and / or by at least one appropriate statistical test.

[0106] A distribution metric is understood to mean any quantitative measure that assesses certain characteristics of the distribution curves under consideration in order to compare their shapes, deviations, central tendencies, and / or overall distributions. These metrics are known in statistical analysis or data science for evaluating the similarity or differences between two or more distributions based on the statistical properties of the populations under consideration.

[0107] Preferably, the difference between the distribution curves considered can be determined by analyzing, in particular comparing, at least one criterion or distribution metric chosen from among the Kolmogorov-Smirnov (KS) distance, the Euclidean distance, the Kullback-Leibler divergence, the Pearson correlation coefficient, the difference in area under the curve (AUC), or a combination thereof. In particular, the difference between two distribution curves among those considered can be determined to be significant if the selected distribution metric(s) is / are less than or exceeds a predetermined threshold.

[0108] By way of example and not by way of limitation, the difference between two width distribution curves of crystals belonging to two populations among those considered can be determined to be significant if the KS distance between the cumulative probability distributions of widths of said crystals is equal to or greater than 50 and / or if the Euclidean distance between the cumulative probability distributions of widths of said crystals is equal to or greater than 800. This means that the two curves considered are not redundant and represent statistically distinct characteristics of the crystal populations.

[0109] Again, by way of example and not by way of limitation, the difference between two length distribution curves of crystals belonging to two populations among those considered can be determined to be significant if the KS distance between the cumulative probability distributions of widths of said crystals is equal to or greater than 200 and / or if the Euclidean distance between the cumulative probability distributions of widths of said crystals is equal to or greater than 5300. This means that the two curves considered are not redundant and represent statistically distinct characteristics of the crystal populations.

[0110] According to another embodiment of the invention, the difference between the distribution curves considered can be determined by a non-parametric statistical test chosen from the Kolmogorov-Smirnov test, the Wilcoxon test and the Mann- Whitney U. These two tests use the same format for the compared samples: a cumulative distribution function. However, the scope of the present invention is obviously not limited to a particular choice of parametric or nonparametric test used to compare statistical distributions. Other similar types of statistical tests can therefore be applied.

[0111] Preferably, the distribution curves under consideration are determined to be statistically different on the basis of the Kolmogorov-Smirnov test. This is advantageous insofar as such a test allows for the direct comparison of two data samples to determine whether they come from the same distribution law or curve, regardless of knowledge of that law.

[0112] In particular, the test always looks for the maximum distance "Ds" between two cumulative distribution functions; from the metric "Ds", it is possible to calculate the probability "ps", called p-Value, that the two cumulative distribution functions tested come from the same distribution law.

[0113] By using one of the tests described above, we therefore have a measure to highlight distances between two populations of objects of the substance, in particular of crystals, and thus determine whether they are significantly different from each other.

[0114] The distribution curves of at least one specific property of the objects of each reference population according to any of the variants mentioned herein can be obtained through the acquisition, extraction and ex situ analysis of data from samples of objects belonging to said object populations of the substance, i.e. outside of a reaction environment.

[0115] It can therefore be said that each class in the training dataset used in the process according to the invention is preferably pre-characterized. This means that each of these classes relates to a single reference population of objects, in particular crystals, of the substance in question, from which information has been previously obtained relating to a particular property or characteristic of the latter, in particular at least one distribution curve of at least one specific property chosen from among the size, e.g. width, length, width / length ratio and / or mean equivalent diameter, and the morphology, e.g. shape, slenderness, rectangularity, circularity, and / or faceting, of the objects, in particular crystals, and all this thanks to the acquisition, extraction and ex situ analysis of data from samples of objects belonging to said population of objects.

[0116] According to the invention, each membership class in the training dataset used in the method according to the invention is therefore preferably associated with at least one curve or law or function that is representative of the distribution of a particular property or characteristic of the population of objects, in particular of crystals, belonging to said class, which curve or law or function is statistically different from the curves / laws / functions of distribution of the same property or characteristic of objects, in particular of crystals, of populations belonging to other classes.

[0117] Preferably, each of these classes is associated with at least one distribution curve of sizes, e.g. of widths, lengths, width / length ratios and / or equivalent diameters, or of morphology, e.g. of shapes, slenderness, rectangularity, circularity, and / or faceting, of said objects, in particular of said crystals, in the reference population belonging to the class considered, which curve is statistically different from the distribution curves of the same property of the objects, in particular of the crystals, of the populations belonging to the other classes.

[0118] More preferably, each membership class in the training dataset used in the method according to the invention is therefore preferably associated with at least one size distribution curve of objects, in particular crystals, in the reference population belonging to the class considered, which curve is statistically different from the size distribution curves of objects, in particular crystals, of populations belonging to other classes.

[0119] According to a particularly preferred embodiment of the invention, each of these membership classes is associated with at least one curve that is representative of the distribution of widths, at least one curve that is representative of the distribution of lengths and optionally at least one curve that is representative of the distribution of the width / length ratio of objects, in particular crystals, in the population belonging to said class, which curves are statistically different from the distribution curves of the same property or characteristic of size of objects, in particular crystals, of populations belonging to the other classes.

[0120] Each distribution curve mentioned in this application can be obtained by characterizing objects belonging to a certain population using one or more ex-situ object characterization methods, such as, for example, optical microscopy, laser granulometry or the QICPIC system, using samples from processes of formation and / or growth of the objects considered, or preferably from crystallization processes, the parameters of which (e.g. crystallization solvent, temperature curve, seeding, sieving, grinding) have conditioned the objects, in particular the crystals, to take on different properties, for example a different size and / or morphology.

[0121] The images of the objects, in particular the crystals, obtained by this characterization can thus be analyzed by any method known to those skilled in the art, to extract the desired data concerning at least one of the properties of the aforementioned objects, in particular their size and / or morphology. In particular, Data extraction can be performed using a learning model capable of performing multi-regression to find bounding boxes of objects present in each image.

[0122] Once the data relating to the properties of the objects have been retrieved from each image, data analysis methods can be applied to obtain the distribution curves of a certain object property for each population, that is, a frequency curve or histogram as a function of the object property considered. Interpolation to a multivariate mathematical distribution can then be applied to reduce the parameter space to only the parameters of the distribution used. Reference images

[0123] The training dataset used in the method according to the invention also includes a plurality of reference images of samples of objects in suspension of said substance, in particular of crystals in suspension of said substance, for each of said membership classes, i.e. a plurality of distinct reference images, each of which visualizes objects, in particular crystals, from a reference population belonging to at least one specific class among the membership classes considered.

[0124] According to the invention, each of the samples used for obtaining the reference images is previously prepared by suspending objects, in particular crystals, belonging to one of the at least five classes of membership considered in a suitable liquid medium, so as to obtain samples of objects, in particular crystals, in suspension at at least two different entrainment concentrations for each of the classes considered, preferably at least 3, 4 or 5 different entrainment concentrations for each of the classes considered.

[0125] Preferably, the liquid medium used for preparing samples of objects, in particular crystals, of the substance and for acquiring reference images is a non-solvent for the substance in question, that is to say, it does not allow the dissolution of objects, in particular crystals, of the substance. In other words, the objects and in particular the crystals of the substance remain suspended in the liquid medium in question.

[0126] Depending on the substance chosen, the liquid medium can be an organic medium, for example an organic solvent or a mixture of organic solvents, or an aqueous medium, in particular an aqueous solution.

[0127] Preferably, the liquid medium is determined by the solvent used in the process of forming the solid objects or particles of the substance, in particular by the solvent used in the crystallization process of the substance.

[0128] Preferably, the liquid medium is composed of the solvent saturated with the substance.

[0129] Most preferably, the liquid medium is a saturated aqueous solution.

[0130] The use of reference images of samples of objects of the substance, in particular crystals, at different concentrations is very advantageous because it allows the selected neural network to be trained to classify images of objects, in particular crystals, of the substance of interest considered from data similar to those acquired under real conditions of formation of said objects or of crystallization in a reaction medium, for example during the process of germination and growth of crystals in situ.

[0131] In the context of the present invention, the expression "different training concentrations" means that the two or more samples used for the acquisition of the reference images are prepared by suspending objects of the class considered, and contain quantities of objects, in particular crystals, belonging to said class which are not the same, preferably contain quantities of objects, and in particular crystals, which have an absolute difference between each other of at least 1.5% or at least 2%, even more preferably at least 5%, said quantities being in particular expressed in weight.

[0132] For example, for each class, the quantity of objects in a certain sample may be greater or less than the quantity of objects of the same class in another sample.

[0133] Again, for each class, the concentration or quantity of objects in a sample can be at least about 1.5, 2, 3, 4, or more times greater or less than the concentration or quantity of objects of the same class in another sample.

[0134] Preferably, all training concentrations chosen are less than or equal to 35% weight or 30% weight, the percentages being given here in weight relative to the total weight of the suspension.

[0135] According to one embodiment, all chosen training concentrations vary between 2% and 35% by weight, preferably between 5% and 30% by weight, more preferably between 5% and 25% by weight of objects, in particular crystals, of the substance, the percentages being given here in weight relative to the total weight of the suspension.

[0136] Knowing the chemical nature of the selected substance, a person skilled in the art may select a series of different concentrations, preferably less than or equal to 35% by weight or 30% by weight of the substance, which is representative of the variation in concentration of said substance during the formation process of said objects or preferably of crystallization. The percentages are given here in weight relative to the total weight of the suspension. According to another embodiment, each of the samples used for obtaining the reference images is previously prepared by suspending solid particles, preferably crystals, belonging to one of the at least five classes considered, in a suitable liquid medium, so as to obtain samples of suspended solid particles, in particular suspended crystals, at least two different entrainment concentrations less than or equal to 35%, for example between 2% and 35% for each of the classes considered, preferably at least 3, 4 or 5 different entrainment concentrations between 2% and 35% for each of the classes considered, the percentages being given here in weight relative to the total weight of the suspension.

[0137] According to a highly preferred embodiment, the samples are pre-prepared by suspending objects, in particular crystals, belonging to one of the at least five selected membership classes in a suitable liquid medium, so as to obtain samples of suspended objects, in particular suspended crystals, at at least three different concentrations being equal respectively to 2%, 10% and 20% by weight of objects of the substance, in particular crystals of the substance, relative to the total weight of the suspension, for each of said classes.

[0138] According to another embodiment, the samples are previously prepared by suspending objects, in particular crystals, belonging to one of the at least five selected membership classes in a suitable liquid medium, so as to obtain samples of suspended objects, in particular suspended crystals, at at least five different concentrations being equal respectively to 2%, 5%, 10%, 20%, and 30% by weight of objects, in particular crystals, of the substance relative to the total weight of the suspension, for each of said classes.

[0139] According to the invention, each image of the plurality of reference images is acquired by an object imaging technique, in particular of crystals, in suspension, called an object imaging technique, in particular of crystals, in situ.

[0140] According to a preferred embodiment, the reference images are acquired by an image acquisition system adapted to be immersed in a liquid medium, in particular organic or aqueous, comprising objects of the substance selected according to any of the variants described in this application, most preferably by an optical imaging probe, in particular one with high resolution. Several examples of imaging probes are known to those skilled in the art, for example, the EasyViewer imaging probes produced by Mettler Toledo.

[0141] In another embodiment, the reference images can be acquired by a light scattering tool, in particular by laser granulometry.

[0142] According to the invention, the plurality of reference images in the training dataset comprises at least 100 images for each sample at each concentration for each of the membership classes considered, in particular at least 200, 300, 400, 500 or 1000 images.

[0143] Preferably, the plurality of reference images in the training dataset comprises at least 100 images up to 10,000 images. It is indeed possible to use a very large number of images, especially if one chooses to have more than five classes and thus a more robust trained neural network. Step ii) of training the neural network

[0144] According to the invention, step ii) of the training or learning process comprises providing the selected training dataset to an image classification neural network, in order to train the neural network to classify images of samples of objects suspended in the substance, according to the membership classes and for each of the at least two training concentrations considered.

[0145] Advantageously, the training method of the present invention provides that the neural network is trained so that it can give a probability of belonging to said at least five membership classes for "unknown" or "to be analyzed" samples of objects, in particular crystals, in suspension of the same substance of interest considered.

[0146] The neural network is therefore preferably trained to give a probability that an "unknown" or "to be analyzed" sample of objects, in particular crystals, in suspension of the same substance considered belongs to one of the corresponding membership classes among the classes considered in the training dataset.

[0147] The neural network used in the process according to the present invention is preferably a deep learning neural network, even more preferably it is a convolutional neural network (CNN).

[0148] Preferably, the neural network chosen in the method according to the present invention is adaptive, that is to say, it is possible to add layers to make it more robust if it is desired to add more than five classes in the training dataset.

[0149] Examples of neural networks that can be used in the method according to the present invention include the convolutional image classification model of the "Inception" family, in particular the "Inception-V2" or "Inception-V3" model, developed by Google in 2014. These models can be used for image classification and for transfer learning, which allows for pre-trained weights and thus reduces the material weight and time required for training.

[0150] Alternatively, other neural networks could be used. For example, simpler models such as a convolutional neural network of the "MobileNet" family are also conceivable with few classes and reference images acquired at three different concentrations.

[0151] According to a preferred embodiment, the chosen neural network is a model as described above which includes at least one fully connected classification layer, as well as Softmax (or equivalent) type activation layers or operators, in order to generate as output a set of scores or probability coefficients by membership class which, summed, give 1 (100%).

[0152] Method for analyzing objects suspended in a substance

[0153] The invention also relates to a method implemented by computer, or by a processor, for the analysis of suspended objects, preferably suspended crystals, of a substance according to any of the variants mentioned in this application, which method comprises, preferably consists of, the following steps: i. To obtain one or more real images of objects, in particular crystals, suspended in said substance; and ii. Providing the real images obtained in the previous step to an image classification neural network, said neural network being trained by a computer-implemented method according to any of the embodiments as defined in this description or in the claims, to give a probability that each of said real images visualizes a population of objects, in particular crystals, belonging to a class from a predefined set comprising at least five different membership classes as defined in this description or in the claims. Step i) Acquisition of real images

[0154] The first step i) of the analysis method according to the invention therefore includes obtaining or acquiring one or more real images of objects in suspension of the selected substance of interest, i.e. one or more images of one or more samples of objects, in particular crystals, in suspension "unknown" or "to be classified" of the same substance considered in the training phase of the neural network according to the present invention.

[0155] In particular, the real images of suspended objects are acquired using the same suspended object imaging technique as that used for acquiring the reference images for training the neural network employed. The use of the same imaging technique makes it possible to use the trained neural network to classify the real images of the substance, according to the membership classes considered, using the information learned during the training phase, and advantageously to thus reliably determine the probability of belonging to at least one of said at least five membership classes.

[0156] One or more real images is / are acquired by any imaging technique used to obtain the reference images used in the training process, preferably by an optical imaging probe, especially a high-resolution one, for example the EasyViewer imaging probes produced by Mettler Toledo, or by a light-scattering tool.

[0157] According to the invention, all real images of objects of the substance in question are images of objects, preferably crystals, of said substance in suspension, i.e. in a suitable liquid medium and in particular a liquid medium which is a non-solvent of the substance in question, for example according to any of the variants described in this application.

[0158] Depending on the substance chosen, the liquid medium can be organic or aqueous. Preferably, the liquid medium is a saturated solution prepared from the reference solvent used in the crystallization process. Preferably, the liquid medium is a saturated aqueous solution.

[0159] Advantageously, one or more actual images of the objects of the substance in question can be acquired by any appropriate imaging technique directly in their medium or environment of in situ growth, that is to say in the course of their process of formation or, preferably, of crystallization.

[0160] In a particularly preferred embodiment, step i) of the analysis method according to the invention comprises obtaining one or more actual images of the suspended crystals of the substance in question in their formation and / or growth medium at one or more times during a crystallization process of the substance, for example, at different times between the beginning and the end of the crystallization process in question. This makes it possible to monitor in real time the changes in properties or particular characteristics of the crystals during their formation and / or growth in situ.

[0161] Preferably, step i) of the analysis method according to the invention comprises obtaining or acquiring at least 2, 3, 5, 10, 15, 20, 25, 30, 35, 40, 45, 50, 55, 60, 65, 70, 75, 80, 85, 90, 95, 100, 150, 200, 250, 300 or 500 actual images of the substance of interest considered, even more preferably obtained at different consecutive or successive moments of a crystallization process of the substance in question.

[0162] In other words, step i) of the analysis method according to the invention may include obtaining a sequence of actual crystallization images of the substance of interest considered in its in situ formation and / or growth medium. Step ii) of classifying the actual images

[0163] According to the invention, step ii) of the analysis process comprises providing the real images obtained in step i) to an image classification neural network, which neural network is a neural network trained by a method implemented by a computer or by a processor according to any of the embodiments of the invention described in this application.

[0164] In accordance with the training method implemented according to the present invention, the trained neural network used in the analysis method is capable of giving a probability that each of the real images provided to it visualizes a population of objects, preferably crystals, belonging to a class from a predefined set comprising at least five different membership classes, preferably at least 6, 7, 8, 9, or 10 different membership classes.

[0165] The at least five different membership classes considered in step ii) are any membership class as defined in this application by reference to the dataset used in the neural network training process. This means that each of these membership classes relates to a single reference population of objects, in particular crystals, of the substance that is statistically different from that of the other classes.

[0166] Preferably, each of the membership classes considered in step ii) is associated with at least one curve which is representative of the distribution of a particular property or characteristic of the reference population of objects, in particular crystals, belonging to said class.

[0167] In particular, each of these classes is associated with at least one distribution curve of sizes, e.g. of widths, lengths, width / length ratios and / or mean equivalent diameters, or of morphology, e.g. of shapes, slenderness, rectangularity, circularity, and / or faceting, of said objects, in particular of said crystals, in the reference population belonging to the class considered.

[0168] According to a particularly preferred embodiment of the invention, each of these membership classes is associated with at least one curve that is representative of the distribution of widths, at least one curve that is representative of the distribution of lengths and optionally at least one curve that is representative of the distribution of the width / length ratio of objects, in particular crystals, in the reference population belonging to said class.

[0169] The trained neural network used in step ii) of the analysis method according to the invention is therefore configured to receive at least one real-world input image of objects of the substance of interest and to process it according to the current values ​​of the parameters of the classification neural network, in order to generate a classification output that is indicative of the membership of the real-world image of the objects in one of the classes considered. The neural network thus generates a respective classification output for each real-world input image.

[0170] In particular, the classification output generated using the neural network by processing a real input image includes a set of coefficients or scores, each coefficient or score representing a predicted probability that the analyzed real image visualizes a population of objects of the substance belonging to one of the considered membership classes.

[0171] Preferably, the trained neural network produces a set of coefficients for each real input image, each of whose coefficients indicates the probability that said real image visualizes a population of objects, in particular crystals, belonging to a class among said predefined set of membership classes considered.

[0172] Optional step ii') calculation of a quality bias in predictions

[0173] Advantageously, the analysis method according to the present invention may include an additional step ii') of calculating a quality bias for each of the real images from the set of coefficients generated with the trained neural network.

[0174] Preferably, the value of the quality bias is: - equal to 1 if a coefficient indicating the highest probability among all said coefficients has a value greater than 60%, - equal to 0.5 if a coefficient indicating the highest probability among all said coefficients has a value greater than 30% and if a coefficient indicating the second highest probability among all said coefficients has a value less than 20%, - equal to 0.1 if a coefficient indicating the highest probability among all said coefficients has a value greater than 30% and if a coefficient indicating the second highest probability among all said coefficients has a value greater than 20%, - equal to 0.05 if a coefficient indicating the highest probability among all said coefficients has a value less than 30% and if a coefficient indicating the second highest probability among all said coefficients has a value less than 20%, - equal to 0.025 if a coefficient indicating the highest probability among all said coefficients has a value less than 30% and if a coefficient indicating the second highest probability among all said coefficients has a value greater than 20%, - equal to 0.1 if we do not find ourselves in any of the previous cases.

[0175] Optional step iii) of selecting classes with predominant probabilities

[0176] Advantageously, the analysis method according to the present invention can also include a step iii) of selection, for each of the real images provided to the trained neural network and on the basis of the coefficients generated as output by the neural network for each of the real images, between one and three classes from the predefined set of classes considered, to which classes are associated the highest probabilities of belonging.

[0177] In other words, step iii) provides for selecting or retaining the most probable membership classes in the predefined set of membership classes considered, based on the values ​​of the probability coefficients provided by the trained neural network output of the classification.

[0178] Preferably, step iii) includes selecting the two or three (if the first two are not predominant over the third) coefficients or scores representative of the highest probabilities of belonging for each real image analyzed by the neural network.

[0179] Preferably, step iii) further includes the selection of one or more distribution curves associated with each of the classes selected as the most probable. Advantageously, this selection makes it possible to associate with each real image analyzed by the neural network one or more curves, laws, or functions that are representative of the distribution of one or more particular properties or characteristics of objects, in particular crystals, from reference populations belonging to the most probable classes among all the classes considered.

[0180] As will be explained in the following paragraphs, this allows us to estimate, for each real input image, a curve or law or distribution function of a specific property of interest of the objects, in particular of the crystals, visible on the real image in question, from the curves / laws / functions of the reference populations belonging to the resulting classes as the most probable.

[0181] Optional step iv) of developing an estimated distribution curve for each real image

[0182] Advantageously, the analysis method according to the present invention may also include a subsequent step (iv) from step (iii) of developing at least one estimated distribution curve for each of the real images analyzed by the trained neural network. The development of each estimated distribution curve can be based on the coefficients generated by the neural network for each said real images and distribution curves selected for each of said real images in step iii), according to the resulting classes as the most probable.

[0183] In particular, the elaboration is carried out by weighting the distribution curve(s) selected in step iii) on the basis of the coefficients generated with the neural network for each of said real images and optionally on the basis of the quality bias calculated in step ii').

[0184] This allows us to generate, for each real image analyzed, an estimated distribution curve which is indicative of the distribution of a specific property of interest of the objects, in particular of the crystals, visible on the real image in question, notably chosen from size or morphology.

[0185] In a preferred embodiment, the optional step iv) includes the development of at least one estimated width distribution curve and at least one estimated length distribution curve for each of the real images analyzed by the trained neural network, based on the coefficients generated with the neural network for each of said real images as well as the width and length distribution curves selected for each of said real images in step iii), according to the resulting classes as the most probable.

[0186] Optional step v) of determining the progression or advancement of a formation and / or growth process of objects of the substance considered, and in particular of a crystallization process of said substance

[0187] The analysis method according to the present invention is particularly suitable for monitoring in real time a process of formation and / or growth of objects of the substance of interest considered and in particular a process of crystallization of said substance, that is to say for obtaining in real time information relating to one or more properties or characteristics of the objects and particularly of the crystals of the substance considered during their process of formation and / or growth in situ, i.e. in an appropriate reaction medium.

[0188] Another object of the present invention therefore relates to an analysis method according to any of the embodiments illustrated in this application, for the real-time monitoring of the progress or advancement of a formation and / or growth process of objects of said substance, preferably of a crystallization process of said substance.

[0189] When the analytical method according to the present invention comprises, in step i), obtaining several actual images of the objects, in particular of the crystals suspended in the substance in their in situ formation and / or growth medium at one or more times during a formation and / or growth process, preferably a crystallization process, of said substance, the method can therefore advantageously include an additional step v) of determining the value of one or more quantities indicating the progress of the formation and / or growth process, preferably crystallization, in question.

[0190] One or more quantities indicating the progress of the formation and / or growth process, preferably crystallization, can be determined on the basis of the estimated distribution curve(s) as developed in step iv), in particular on the basis of the estimated crystal size distribution curve(s) as developed in step iv).

[0191] For example, said quantity or several quantity(ies) indicating the progress of the crystallization process may be represented by the maximum or by the average of the estimated distribution curve(s) as developed in step iv).

[0192] In a preferred embodiment, said quantity or several quantities indicating the progress of the formation and / or growth process, preferably crystallization, is / are represented by the maximum and / or the average of the width distribution curve of the objects, in particular of the crystals, estimated and by the maximum and / or the average of the length distribution curve of the objects, in particular of the crystals, estimated, as developed in step iv).

[0193] The analysis method according to the present invention may also include a step vi) of generating a graph visualizing the evolution of one or more quantities thus determined as a function of the moments of the formation and / or growth process, preferably of crystallization, considered, i.e. as a function of the moments of the formation and / or growth process, preferably of crystallization, at which the actual images were acquired. Other objects of the invention

[0194] The present invention thus relates to a computer program comprising instructions which, when the program is executed by a computer, cause the computer to execute the steps of a training process and / or an analysis process according to any of the embodiments described in this application.

[0195] The present invention also relates to a computer-readable storage medium storing instructions which, when executed by a computer, cause the computer to execute the steps of a training process and / or an analysis process according to any of the embodiments described in this application.

[0196] The present invention also relates to a computer system comprising at least one computer and one or more computer-readable storage media, storing instructions which, when executed by the computer, lead the computer to implement the steps of a training process and / or an analysis process according to any of the embodiments described in this application.

[0197] Preferably, one or more storage media is / are in communication with the computer and presents / have a set of instructions coded on it / them and executable by the computer system to carry out the steps of a training process and / or an analysis process according to any of the embodiments described in this application.

[0198] Preferably, all storage media mentioned herein also store a training data set according to any of the variants described in this application.

[0199] The present invention also relates to a device comprising a computer system according to any one of the embodiments described herein and an image acquisition system linked or connected to this computer system.

[0200] According to the present invention, the image acquisition system of the device is configured to acquire one or more images of suspended objects, in particular suspended crystals, of a substance of interest in a liquid medium, for example an organic or aqueous medium. Without limitation, the substance may be any of the substances mentioned in this application.

[0201] Preferably, the image acquisition system is adapted to be immersed in a liquid medium comprising suspended objects, in particular suspended crystals of said substance.

[0202] According to a highly preferred embodiment, the system is an imaging probe.

[0203] The image acquisition system is preferably in communication, particularly electronic, with the computer system. In particular, it is linked or connected to this computer system by electronic means, for example wired or wireless. Preferably, the system is connected to the computer system by wire. Alternatively, the electronic connection can be made via a wireless protocol such as Wi-Fi, 3G, 4G, 5G and / or Bluetooth. Examples

[0204] The invention is illustrated by the following examples which are in no way limiting.

[0205] Example 1 (according to the invention) - Training a convolutional neural network for the classification of images of 4-aminobenzoic acid crystals

[0206] Selection and characterization of membership classes of 4-aminobenzoic acid crystals

[0207] Seven different reference samples of 4-aminobenzoic acid crystals from crystallization processes, whose parameters (temperature curve, seeding ...) conditioned the crystals to take different sizes, are analyzed in order to acquire, extract and generalize data relating to the sizes of these crystals.

[0208] As shown in Figure 1 and [Fig. 2], the seven different samples are obtained such that each sample comprises a single population of 4-aminobenzoic acid crystals having a predetermined width distribution curve and length distribution curve that are statistically different from those of the other samples. The cumulative frequency histograms shown in Figure 1 for the 7 crystal size classes demonstrate that there is no linear dependence between the crystal classes with respect to the descriptor.

[0209] As shown schematically in [Fig.3A], the acquisition of data relating to the sizes of the crystals of said reference samples, known as "ex-situ" data, can be done by optical microscopy, or with the help of other methods of morphological characterization of crystals such as laser granulometry or the QICPIC system.

[0210] In the present case, data acquisition is done using an optical microscope and consists of a shot on a glass slide of the seven different samples of 4-aminobenzoic acid crystals.

[0211] Once all the images have been collected, the size information of the crystals present in them is extracted. In order to qualitatively determine the length and width of each crystal, a machine learning model capable of performing multi-regression is used to find the bounding boxes of the crystals present in each image. Since width information is also important, it is understood that the bounding boxes can have an orientation parameter denoted by 0 in order to accurately correspond to the edges of the crystals.

[0212] Once the size information has been retrieved, a frequency histogram as a function of the width, length, or width-to-length ratio of the crystals is obtained for each image (see Figures 3B-D). Interpolation to a multivariate mathematical distribution reduces the parameter space to only the parameters of the distribution used. Thus, the output of the ex-situ processing chain yields as many size distribution curves, i.e., mathematical distributions, as there are morphological indicators (width, length, width-to-length ratio) for each previously selected sample of 4-aminobenzoic acid crystals.

[0213] The reference samples of the crystals thus characterized constitute a set of seven different membership classes of crystals of this substance previously characterized, i.e. classes independent of each other, which can be used for training an image classification neural network.

[0214] The Kolmogorov-Smirnov (KS) distance between two cumulative probability distribution curves of crystal widths belonging to two classes among those considered is at least equal to 54, while the KS distance between two cumulative probability distribution curves of crystal lengths belonging to two classes among those considered is at least equal to 227. The seven selected classes are therefore considered to be significantly different from each other.

[0215] In other words, each of the seven samples of 4-aminobenzoic acid crystals thus characterized constitutes a membership class which is associated with length and width distribution curves of these crystals and can therefore be used for training an image classification neural network.

[0216] Acquisition of reference images of 4-aminobenzoic acid crystals for each class of belonging

[0217] As illustrated in [Fig. 4], a reference image acquisition campaign of 4-aminobenzoic acid crystals is carried out by resuspending crystals from each of the seven previously characterized reference samples in a corresponding saturated aqueous medium, so as to obtain a plurality of 4-aminobenzoic acid crystal suspension samples at at least three different concentrations (i.e., 2%, 10%, and 20% wt. relative to the total weight of the suspension). This makes it possible to simulate a 4-aminobenzoic acid crystallization environment and to acquire artificially in situ reference images of crystals of the substance of interest, with a correlation to the previously acquired ex situ data.

[0218] The plurality of reference images of the suspension samples at the different concentrations thus prepared is acquired by an in situ imaging probe and includes at least 100 images for each suspension sample, at different concentrations for each of the membership classes considered (i.e. 100 images for each of the 7x3 = 21 suspension samples).

[0219] Training a neural network by convolutional deep learning

[0220] Reference images of suspended 4-aminobenzoic acid crystals acquired at different concentrations are used, along with their corresponding class memberships, as validation data for a neural network with a deep-learning architecture. More specifically, the Inception-V2 model is used as the neural network, which is well known for its robustness on images with sometimes minute details.

[0221] The selected neural network performs a classification by probability of belonging to one of the 7 previously characterized size classes. To this end, as shown schematically in [Fig. 5], the last connected layer of the selected model includes a "softmax" activation function to retrieve prediction scores per class which, when summed, give 1 (100%).

[0222] Example 2 (according to the invention) - Real-time analysis of suspended 4-aminobenzoic acid crystals during the crystallization process of the substance in a liquid medium

[0223] Generation of crystal size distributions for real image sequences of successive crystallizations

[0224] As schematically shown in [Fig.6], a sequence of consecutive real images is acquired during the crystallization of 4-aminobenzoic acid in an aqueous medium saturated by an in-situ imaging probe, the same imaging probe as that used in Example 1 to acquire the plurality of reference images.

[0225] Each real image is provided to the previously trained neural network, as explained in Example 1, which returns, for each real input image, seven scores or coefficients indicating the probability of belonging to one of the seven classes considered.

[0226] On these probability coefficients, a bias on quality is calculated in order to perform a non-linear weighting at the end of the processing chain.

[0227] The bias is calculated as follows: • If the highest probability is greater than 60%, the weight is 1 • If the highest probability is greater than 30% but the second is less than 20%, the weight is 0.5 • If the highest probability is greater than 30% and the second is greater than 20%, the weight is 0.1 • If the highest probability is less than 30% and the second is less than 20%, the weight is 0.05 • If the highest probability is less than 30% but the second is greater than 20%, the weight is 0.025 • If none of the previous cases apply, the prediction has a weight of 0.01

[0228] The two or three (if the first two are not predominant over the third) most important membership probability coefficients are retained at each prediction.

[0229] Next, the distribution curves obtained from the ex-situ analysis of the seven reference samples of 4-aminobenzoic acid crystals are recovered.

[0230] For each actual image of 4-aminobenzoic acid crystals, a pseudo-random generation of two distribution curves per class is performed. These curves are then linearly weighted by the two, or three, predominant membership probability coefficients. And finally, a weighted curve of the curves The distribution of the reference samples is generated in relation to the classification result.

[0231] To improve the accuracy of the generation, each weighted curve is associated with its previously calculated bias. This results in a second weighting and only one pair of distribution curves, i.e., statistical laws, as output for each real-world input image analyzed by the trained neural network. This double weighting allows us to associate, with each real-world image taken in situ, at least one length distribution curve and one width distribution curve of the analyzed crystals for each image.

[0232] Upon analysis of the sequence of real images acquired during the crystallization of the substance, a graphic animation is generated representing the evolution of the identification of the analyzed crystal population. More specifically, as shown for example in [Fig. 7], the graphic animation visualizes the evolution of one or more quantities, such as the mean or maximum of the weighted distribution curves obtained for each analyzed real image, as a function of the times during the crystallization process at which said images were acquired.

Claims

Demands

1. A computer-implemented method for training a neural network for classifying images of objects suspended in a substance, the method comprising the following steps: i. Acquire a training dataset comprising a. at least five classes of belonging to objects of said substance, each class relating to a single reference population of objects of said substance which is statistically different from that of the other classes; and a. a plurality of reference images of samples of suspended matter of said substance for each of said membership classes, said samples being previously prepared by suspending matter belonging to one of said membership classes in a liquid medium, so as to obtain samples of suspended matter at at least two different entrainment concentrations for each of said classes, which plurality of reference images being acquired by a suspended matter imaging technique and comprising at least 100 images for each sample and at each concentration for each of said classes; b. Provide said training dataset to an image classification neural network, in order to train the neural network to classify reference images of samples of suspended objects of said substance, according to membership classes and for each of said at least two training concentrations.

2. A computer-implemented method according to claim 1, wherein each membership class is relative to a single reference population of objects of the substance having at least one distribution curve of a property or characteristic of said objects chosen from size and morphology, which is significantly different from the distribution curves of the same property or characteristic as the objects of the populations of other classes.

3. A computer-implemented method according to claim 1 or 2, wherein each membership class relates to a single reference population of objects of the substance having at least one distribution curve of a property or size characteristic of said objects chosen from width, length and / or width / length ratio, which is statistically different from the distribution curves of the same property of objects from the populations of other classes.

4. A computer-implemented method according to any one of the preceding claims, wherein said samples are pre-prepared by suspending objects of the substance belonging to one class among said membership classes in a liquid medium, so as to obtain samples of suspended objects at at least 3 or 5 different entrainment concentrations for each of said classes.

5. A computer-implemented method according to any one of the preceding claims, wherein said training concentrations vary between 2% and 35% by weight, preferably between 2% and 30% by weight, of objects of the substance relative to the total weight of the suspension.

6. A computer-implemented method according to claims 4 or 5, wherein said samples are pre-prepared by suspending objects of the substance belonging to one of said classes in a liquid medium, so as to obtain samples of objects in suspension at at least three different concentrations which are less than or equal to 35% by weight of objects of the substance relative to the total weight of the suspension, for each of said classes, preferably at least three or four or five different concentrations between 2% and 35% by weight of objects of the substance relative to the total weight of the suspension, for each of said classes.

7. A computer-implemented method according to any one of the preceding claims, wherein each of the reference images of said samples is acquired by an optical imaging probe.

8. A computer-implemented method according to any one of the preceding claims, wherein said neural network is a neural network for deep learning, preferably is a convolutional neural network (CNN).

9. A computer-implemented method according to any one of the preceding claims, wherein said objects of the substance are crystals of said substance.

10. A computer-implemented method for analyzing suspended objects of a substance, comprising the following steps: i. Obtaining one or more real images of suspended objects of said substance; and ii. Providing the real images obtained in the preceding step to an image classification neural network, said neural network being trained by a computer-implemented method according to any one of claims 1 to 9 to give a probability that each of said real images visualizes a population of objects of the substance belonging to a class from a predefined set comprising at least five different membership classes as defined in any one of claims 1 to 9; said real images of suspended objects being acquired with the same suspended object imaging technique as that used for acquiring the reference images for training said neural network.

11. A computer-implemented method according to claim 10, wherein each of said membership classes relates to a single reference population of objects of said substance which is statistically different from that of the other classes.

12. A computer-implemented method according to the preceding claim, wherein each of said membership classes is associated with at least one property or characteristic distribution curve, in particular size distribution curve, of the objects of the substance in said reference population, which is statistically different from the size distribution curves of the objects in the populations of the other classes, preferably associated with at least one width distribution curve, at least one length distribution curve, or optionally at least one width / length ratio distribution curve of the objects in said population. reference, which are statistically different from the distribution curves of the same property of objects in populations of other classes.

13. A computer-implemented method according to any one of claims 10 to 12, wherein said neural network produces a set of coefficients for each real input image, each of the coefficients indicating the probability that said real image visualizes a population of objects of the substance belonging to a class among said predefined set of membership classes.

14. A computer-implemented method according to claim 13, further comprising the following steps: i. Selecting, for each of said real images and on the basis of the coefficients generated by the neural network for each of said real images, between one and three classes from said predefined set to which are associated the highest probabilities of membership and the distribution curves which are associated with said classes; and ii. Developing at least one estimated distribution curve for each of said real images, by weighting the distribution curve(s) selected in step iii) on the basis of the coefficients generated with the neural network for each of said real images and optionally on the basis of a quality bias.

15. A computer-implemented method according to claim 14, for the real-time monitoring of the progress of a formation and / or growth process of objects of said substance, preferably a crystallization process of said substance, wherein step i) comprises obtaining several real images of the objects suspended in said substance in their in situ formation and / or growth medium at one or more times during a formation and / or growth process of said substance, preferably a crystallization process of said substance, which method further comprises the following step: i. Determining the value of one or more quantities indicating the progress of the formation and / or growth process of objects of said substance, preferably the crystallization process of said substance, based on the estimated object size distribution curve(s) as developed in step iv).

16. A computer-implemented method according to claim 15, wherein said one or more quantities indicating the progress of the formation and / or growth process of objects of said substance, preferably of the crystallization process, are chosen from the maximum and the average of the estimated object size distribution curve(s) as developed in step iv).

17. A computer-implemented method according to any one of claims 10 to 16, wherein said objects are crystals of said substance.

18. A computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out the steps of a process according to any one of claims 1 to 9 and / or 10 to 17.

19. A computer-readable storage medium storing instructions which, when executed by a computer, cause the computer to carry out the steps of a process according to any one of claims 1 to 9 and / or 10 to 17.

20. Computer system comprising at least one computer and one or more computer-readable storage media, storing instructions which, when executed by the computer, cause the computer to carry out the steps of a process according to any one of claims 1 to 9 and / or 10 to 17.

21. Device comprising a computer system according to the preceding claim and at least one image acquisition system linked or connected to said computer system, which image acquisition system is preferably adapted to be immersed in a liquid medium comprising objects suspended from a substance, preferably crystals suspended from a substance.

Images