Semi-Automatic Segmentation System for Particle Measurement from Microscopic Images

Abstract

A computer-based method for measuring small particles for pharmaceutical formulations, wherein a digital microscope image (6) of small particles (10) used in a pharmaceutical formulation is created by an image sensor (5) and provided to a computer (2) that executes measurement software (8), the software (8) segments the small particles (10) in the digital microscope image (6), calculates the small particles (10) according to a specific parameter set, the software (8) samples various candidate parameter sets, automatically applies them to perform segmentation and / or calculation processing, displays the results to a user (1) via a display (4), the user (1) selects the candidate parameter set that shows the best result, the software (8) uses this user feedback to establish and train an internal machine learning model (7), and applies the trained model (7) to repeat the automatic segmentation and / or calculation and user feedback acquisition process until an optimal parameter set is approved by the user (1).

Description

Technical Field

[0001] The invention described in this specification discloses a method and a system for software-based semi-automatic particle measurement. This invention deals with the technical fields of digital image processing and pharmaceutical formulations.

Background Art

[0002] In the case of pharmaceutical formulations, especially solid formulations, the final products such as the final dosage forms require specific desirable properties such as hardness and surface smoothness. To adjust the manufacturing process to meet these requirements, it is necessary to consider the properties of individual components. For example, the surface smoothness of small API particles affects how they aggregate with other additives. Also, the surface properties of the aggregates affect the properties of compressed tablets. Therefore, it is necessary to measure various properties of small particles and select an appropriate measurement device according to their properties. The structure and fractality of the particle surface can be measured using a scanning electron microscope or a bright-field microscope. Since microscope images usually show many particles, it is necessary to segment and separate each particle before calculating the particle-level measurement values.

[0003] Established particle segmentation workflows are implemented in open-source software packages such as ImageJ / Fiji and Ilastik, and commercial software such as Zeiss ZEN lite. However, these workflows require extensive training and expertise of users. Furthermore, to segment particles with these workflows, for each image to be analyzed, adjustment of various parameters by the user and other user interventions are required, and the whole process takes time.

[0004] For certain properties that can be evaluated from microscopic images, there are other commercial tools that rely on other measurement approaches. For example, particle size distribution can also be obtained from devices of Beckman Coulter Life Sciences or Thermo Fisher that use laser diffraction. However, with these, surface properties such as surface smoothness cannot be directly and visually analyzed.

Summary of the Invention

Problems to be Solved by the Invention

[0005] Therefore, the object of this patent application is to provide a more efficient particle measurement approach for pharmaceutical formulations, which does not require much specific user expertise regarding the measurement process.

[0006] This problem is solved by a computer-based method for measuring small particles for pharmaceutical formulations, in which a digital microscopic image of the small particles used in the pharmaceutical formulation is created by an image sensor and provided to a computer that executes measurement software, the software segments the small particles in the digital microscopic image, calculates the properties of the small particles according to a specific parameter set, the software samples various candidate parameter sets, automatically applies them to perform segmentation and / or calculation processing, displays the results to the user via a display, the user selects the candidate parameter set that shows the best results, the software uses this user feedback to establish and train an internal machine learning model, and repeats the automatic segmentation and / or calculation and user feedback acquisition process until the optimal parameter set is approved by the user, using the trained model.

[0007] What is proposed is a general framework for optimizing the parameters of a workflow. Specifically, the system samples various candidate parameter sets and automatically executes the workflow. The user checks each result, such as particle segmentation corresponding to the parameter set, and selects the optimal one. Next, the system learns from this feedback and constructs an internal model about which combinations of parameters lead to good results. Then, new candidate parameter sets that are more promising are sampled. Again, the user selects the best result, and the system learns from this feedback and repeats this process several times until the system converges to an appropriate parameter set. From the user's perspective, this process is very intuitive because the user only needs to compare the final results instead of adjusting the individual workflow parameters.

[0008] Advantageous and thus preferred further developments of the invention become apparent from the relevant subclaims, description and associated drawings. One preferred further development of the disclosed method is to use pharmaceutical active ingredient (API) particles as small particles. However, the present invention is not limited to API - particles. Other suitable types of small particles can also be measured. Another preferred development of the disclosed method is that the measured properties include the structure and fractality of the surface of small API particles for determining the surface smoothness and particle size distribution of the small API particles. These properties obtained from digital images are two of several properties that can be used. Other methods are also suitable, but the most preferred options for application in the segmentation process are the structure and fractality of the surface of small API particles, and the particle size distribution.

[0009] Another preferred further development of the disclosed method is that the small particles consist of very variable particle shapes and sizes. Therefore, it is very difficult to perform a reliable segmentation process and thus measurements. The ability to handle these different particle shapes and sizes is one of the advantages of the method of the present invention. Another preferred further development of the disclosed method is that the digital microscope images are created under variable illumination conditions. If the measurement software does not operate with a specific illumination setting, another setting can be used. Alternatively, digital images can always be created with a specific set of illumination conditions and the different exposure digital images can be used by the software. By installing suitable lamps, different illumination conditions can be accommodated. For this purpose, it must be possible to illuminate a test sample with the small particles to be measured. A variety of illumination devices with different light sources are possible, ranging from the installation of standard LEDs to specific experimental devices that emit not only standard white light but also light of a specific wavelength.

[0010] Another solution for the assigned task is a system for performing computer-based small particle measurements for pharmaceutical formulations, the system including a computer that executes measurement software, a display for displaying information to a user, means for the user to input data and / or instructions into the software, and an image sensor, the system being configured to create digital microscope images of small particles used in pharmaceutical formulations via the image sensor, segment small particles within the digital microscope images, and calculate characteristics of the small particles according to a specific parameter set via the software, the software sampling different candidate parameter sets, automatically applying them to the segmentation and / or calculation process, displaying the results to the user via the display, the user selecting the candidate parameter set that yields the best results, and the software using this user feedback to establish and train an internal machine learning model and applying the trained model to repeat the automatic segmentation and / or calculation and user feedback acquisition process until an optimal parameter set is approved by the user, said system being included.

[0011] In the case of this system, the computer is usually a stand-alone device, but it is also possible to execute the measurement software using an optical sensor with an embedded computing device. In either case, it is necessary to link various devices via a data connection, such as a suitable data network like LAN, Wi-Fi, Bluetooth, etc. The digital images can be stored in the memory belonging to the computer or in another database connected via the data network.

[0012] One preferred further development of the disclosed system is that the image sensor is a scanning electron microscope or a bright-field microscope that creates microscope images as digital images. These digital images are then stored in memory and provided to the measurement software, which analyzes them according to the invented method. Another preferred further development of the disclosed system is that the software consists of two connected software components, one responsible for particle segmentation and the other for characteristic calculation, or one software component that performs both tasks. Ultimately, which embodiment is preferred depends on the programming language applied, the software libraries and / or frameworks available and in use, the hardware used, and other operating environments. The proposed solution enables the generation of particle measurement values in an intuitive way with minimal user operation and little training time.

Brief Description of the Drawings

[0013] The method and system according to the present invention, as well as their functionally advantageous developments, will be described in more detail below with reference to the related drawings using at least one preferred exemplary embodiment. In the drawings, the same reference numerals are assigned to corresponding elements. The following are shown in the drawings.

[0014]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

[0015] The system of the scanning electron microscope (SEM) 11 in the preferred embodiment is composed of several components shown in FIG. 1. These components include a control unit 2 in the form of any suitable computer 2 that can access the memory 3. A microscope 5, preferably a scanning electron microscope or a bright-field microscope, is connected to the computer 2, and a digital microscope image 6 of small particles 10, particularly active pharmaceutical ingredient (API) particles 10 used in pharmaceutical preparations such as compressed tablets, can be generated. The digital microscope image 6 is preferably stored in the memory after being created. It can also be stored in other available memories such as a server like local memory or the cloud. Also, the local memory preferably stores software 8 in the form of a control program 8 that provides a machine learning model (AI model) 7 that can be trained and processed with the digital microscope image 6. Furthermore, the system 11 includes a display that displays a user interface 9, preferably a GUI 9, to the user 1, and the user 1 can be displayed process-related information such as the digital microscope image 6 and the calculation results from the software 8 via the display 4. The user 1 can also input commands and other data to the software 8 via the user interface 9.

[0016] The method invented in one preferred embodiment is shown in FIGS. 2 to 5 and is described in more detail in the following chapters. The software application 8 executed by the described computer 2 implements the method of the present invention. Therefore, the basic workflow of this application mainly consists of the following three steps. 1. Examine a test sample containing API particles 10 with a digital microscope 5 to create a microscope image 6 stored in the memory 3 and upload it via the user interface 9. Here, the basic information of the uploaded image 6 is automatically extracted (see FIG. 2) 2. As outlined in the previous section, through iterative "best candidate" selection, an appropriate segmentation of the microscopic image 6 is derived. For this purpose, the user 1 is presented with candidate segmentations and selects the candidate that seems most appropriate (see Figure 3). The progress of the internal optimization is visualized via the display 4 as shown in Figure 4. For example, the user 1 provides feedback on which parts of the parameter space are recognized and an appropriate segmentation is being generated. If the user 1 is satisfied with the segmentation, proceed to the next step. 3. In the user interface 9, as shown in Figure 5, the background segment can be filtered based on the convexity, area, and texture changes of the segments. When the user 1 is satisfied with the final result, various particle measurements such as size, diameter, and elongation are calculated, and the results can be exported and further analyzed via an Excel sheet or the like for export.

[0017] More specifically, step 2 of the above workflow is based on Bayesian optimization in the parameter space of the segmentation algorithm. Preferably, the Felzenszwalb segmentation algorithm is used, but other segmentation algorithms with a reasonable amount of parameters can also be used. In the Bayesian optimization framework, a Gaussian process in the parameter space is used to model some form of utility of the parameter set. In this example, the quality of the segmentation generated by the Gaussian process is modeled. Since pairwise comparison is desired as user feedback for learning the utility, the preference learning Gaussian process proposed by Chu & Ghahramani in their 2005 paper "Preference Learning with Gaussian Processes" is utilized. In that case, the goal of the optimization is to find a set of parameters of the segmentation algorithm that leads to an appropriate segmentation of the physical particles in the digital input image 6. The optimization procedure for that is as follows.

[0018] 1. The Gaussian process of Bayesian optimization is initialized with a prior distribution with little information content. 2. The following procedure is repeated until the user finds a sufficiently accurate segmentation. a. Use the current posterior probability for the segmentation parameters (given by the current Gaussian process) together with an acquisition function to randomly sample a small set of new parameter settings. Here, any acquisition function may be used, and it is preferable to use an acquisition function that promotes a certain degree of sample diversity. For example, improvement can be expected in batches. b. Each sampled parameter set is used to create a segmentation of the physical particles in the input image 6. c. The current optimal segmentation is obtained by examining all previously used parameter sets and obtaining the segmentation of the most useful parameter set modeled by the current Gaussian process. d. In the user interface, each segmentation, that is, the segmentation related to the newly sampled parameter set and the current best segmentation, is displayed to User 1. User 1 is asked to specify the segmentation that he / she thinks is the best among the displayed segmentations. e. When User 1 selects the optimal segmentation from the displayed segmentations, a series of pairwise comparisons are made in the sense that the selected segmentation is superior to the other segmentations. These pairwise comparisons update the Gaussian process.

[0019] Reference Signs 1 User 2 Computer / Control Unit 3 Memory 4 Display 5 Image Sensor / Microscope 6 Digital Microscope Image 7 AI / Machine Learning Model 8 Software / Control Program 9 User Interface (GUI) 10 Active Pharmaceutical Ingredient Particles (API) 11 Scanning Electron Microscope (SEM) System

Claims

1. A computer-based method for measuring small particles for pharmaceutical formulations, wherein a digital microscope image (6) of small particles (10) used in a pharmaceutical formulation is created by an image sensor (5) and provided to a computer (2) running measurement software (8); the software (8) segments the small particles (10) in the digital microscope image (6), calculates the properties of the small particles (10) according to a specific set of parameters (10); the software (8) samples various candidate parameter sets, automatically applies them to perform segmentation and / or calculation processing, displays the results to a user (1) via a display (4), the user (1) selects the candidate parameter set that shows the best results, the software (8) uses this user feedback to establish and train an internal machine learning model (7), applies the trained model (7) and repeats the automatic segmentation and / or calculation and user feedback acquisition process until the optimal parameter set is approved by the user (1).

2. The method according to claim 1, wherein the active pharmaceutical ingredient particles (API) (10) are used as small particles (10).

3. The method according to claim 1 or 2, wherein the measured characteristics consist of the surface structure and fractality of the small API particles, and determine the surface smoothness and particle size distribution of the small API particles.

4. The method according to claim 1, wherein the small particles (10) consist of highly variable particle shapes and sizes.

5. The method according to claim 1, wherein a digital microscope image (6) is created under variable illumination conditions.

6. A system for performing computer-based small particle measurement for pharmaceutical formulations, comprising a computer (2) running measurement software (8), a display (4) for displaying information to a user (1), means for the user (1) to input data and / or instructions to the software (8), and an image sensor (5), wherein the system (11) is configured to create a digital microscope image (6) of small particles (10) used in pharmaceutical formulations via the image sensor (5), segment the small particles (10) in the digital microscope image (6), and calculate the properties of the small particles (10) according to a specific set of parameters via the software (8), and the software The system comprises a user (8) that samples different candidate parameter sets, automatically applies them to a segmentation and / or computation process, displays the results to the user (1) via a display (4), the user (1) selects the candidate parameter set that yields the best results, the software (8) uses this user feedback to establish and train an internal machine learning model (7), applies the trained model (7), and repeats the automated segmentation and / or computation and user feedback acquisition process until the optimal parameter set is approved by the user (1).

7. The system according to claim 6, wherein the image sensor (5) is a scanning electron microscope or a bright-field microscope that generates a microscope image as a digital image (6).

8. The system according to claim 6 or 7, wherein the software (8) comprises either two connected software components, one of which is responsible for particle segmentation and the other for calculating properties, or one software component that performs both tasks.

Images