Semi-automatic segmentation system for particle measurement from microscope images
Through the computer-based small particle measurement method, the image sensor and machine learning model are used to automatically segment and calculate the particle attributes, the problem of professional knowledge and time consumption in the measurement process of small particles in drug preparations is solved, and efficient and intuitive measurement results are achieved. Suitable for particles of varying shapes and sizes.
Patent Information
- Application Number
- CN202380036894.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-04-29
- Filing Date
- 2023-04-26
- Publication Date
- 2024-12-10
AI Technical Summary
The prior art requires a lot of user expertise and time in the measurement of small particles in pharmaceutical preparations, and it is difficult to directly analyze surface properties such as smoothness, and it is difficult to deal with particle segmentation of varying shapes and sizes.
Using computer-based small particle measurement method, digital microscope images are created through image sensors, software is used to automatically segment and calculate particle properties, internal machine learning models are established through iterative processes, parameter sets are optimized, user interaction and training time are reduced, suitable for different lighting Particles with varying conditions and shapes.
The particle measurement value is generated with minimal user interaction and less training time, which improves measurement efficiency and accuracy, and is particularly suitable for measuring the surface structure, fractality and size distribution of small particles in pharmaceutical preparations.
Smart Images
Figure CN119110947A_ABST
Abstract
Description
[0001] The invention described herein discloses a method and system for software-based semi-automatic particle measurement. Technical Field
[0002] The present invention relates to the technical fields of digital image processing and pharmaceutical preparations. Background Art
[0003] In pharmaceutical formulations, especially for solid dosage products, the final product (e.g., final pill) should have certain desired properties, such as hardness, surface smoothness, etc. In order to customize the production process so that these requirements are met, the characteristics of the individual components need to be considered. For example, the surface smoothness of small API particles will affect how they agglomerate with other excipients. And the surface properties of the agglomerates will affect the properties of the compressed pills. Therefore, it is necessary to measure the various properties of the small particles, and the appropriate measuring equipment should be selected based on the properties. Scanning electron microscopes or bright field microscopes can be used to measure the structure and fractal properties of the particle surface. Since microscope images usually show many particles, each particle must be segmented and isolated before calculating the particle level measurement value.
[0004] In open software packages such as ImageJ / Fiji or Ilastik as well as commercial software such as Zeiss ZENlite, established particle segmentation workflows are implemented. However, these workflows require extensive training and expertise of the user. Furthermore, segmenting particles with these workflows requires adjustment of various parameters by the user, as well as other user intervention for each image to be analyzed, making the entire process time consuming.
[0005] For some properties that can be assessed from microscope images, there are other commercial tools that rely on other measurement methods. For example, particle size distribution can also be obtained from instruments from Beckman Coulter Life Sciences or Thermofisher that employ laser diffraction. However, these do not allow direct visual analysis of surface properties such as surface smoothness. Summary of the invention
[0006] It is therefore the task of the present patent application to provide a more efficient particle measurement method for pharmaceutical preparations which also requires less specific user expertise regarding the measurement process.
[0007] This task has been solved by a method for computer-based small particle measurement for pharmaceutical preparations, wherein a digital microscope image of small particles for pharmaceutical preparations is created by an image sensor and provided to a computer executing measurement software, the software segments small particles in the digital microscope image and calculates properties of the small particles according to a specific parameter set, wherein the software samples different candidate parameter sets, automatically applies them to the segmentation and / or calculation process, and shows the results to a user via a display, wherein the user picks the candidate parameter set with the best results, the software uses this user feedback to build and train an internal machine learning model and applies the trained model to repeat the automatic segmentation and / or calculation and user feedback acquisition process until the optimal parameter set has been recognized by the user. A general framework for optimizing parameters of a workflow is proposed. Specifically, the system samples different candidate parameter sets and automatically runs the workflow. The user sees the corresponding results, such as particle segmentation corresponding to the parameter set, and picks the best one. The system then learns from this feedback and builds an internal model about what parameter combination leads to good results. Then, new, more promising candidate parameter sets are sampled. Again, the user picks the best result, the system learns from this feedback, and the process continues for several iterations until the system has converged on a good set of parameters. From the user's perspective, the process is very intuitive because the user only needs to compare the final results instead of having to adjust individual workflow parameters.
[0008] Advantageous and therefore preferred further developments of the invention emerge from the associated dependent claims as well as from the description and the associated drawings.
[0009] One of those preferred further developments of the disclosed method comprises that active pharmaceutical ingredient (API) particles are used as small particles. However, the present invention is not limited to API particles. Other suitable types of small particles can also be measured with it.
[0010] Another of those preferred further developments of the disclosed method comprises that the measured attributes consist of the structure and fractality of the small API particle surface to determine the surface smoothness and particle size distribution of the small API particles. Those attributes obtained from the digital image are two of several possible attributes that can be used. Other attributes are also suitable, but the structure and fractality of the small API particle surface and the particle size distribution are the most preferred options to be applied in the segmentation process.
[0011] Another of those preferred further developments of the disclosed method includes that small particles consist of highly variable particle shapes and sizes. This is why it is so difficult to perform a reliable segmentation process and therefore measurement. One of the advantages of the method of the present invention is that it can handle those different particle shapes and sizes.
[0012] Another of those preferred further developments of the disclosed method consists in that the digital microscope images are created under variable lighting conditions. If the measurement software cannot operate with a specific lighting setting, different settings can be used. Alternatively, it is possible to always create the digital images with a specific set of different lighting conditions and then use those different exposure digital images for the software. The different lighting conditions can be provided by a suitable lamp arrangement, which must be able to illuminate the test sample with the small particles to be measured. Different lighting devices with different light sources are possible - from standard LED devices up to specific laboratory equipment, which emit standard white light, but also emit light with specific wavelengths.
[0013] Another solution to the assigned task consists of a system for performing computer-based small particle measurement of a pharmaceutical preparation, the system consisting of a computer executing measurement software, a display showing information to a user, means for the user to input data and / or instructions, and an image sensor, wherein the system is arranged to create a digital microscope image of the small particles of the pharmaceutical preparation via the image sensor, to segment the small particles in the digital microscope image and to calculate the properties of the small particles via the software according to a specific parameter set, and wherein the software samples different candidate parameter sets, automatically applies them to the segmentation and / or calculation process, and shows the results to the user via the display, wherein the user picks the candidate parameter set with the best result, the software uses the user feedback to build and train an internal machine learning model and applies the trained model to repeat the automatic segmentation and / or calculation and user feedback obtaining process until the optimal parameter set has been approved by the user. For this system, the computer is usually a separate device, but it is also possible to use an optical sensor with an integrated computing device, which is then able to execute the measurement software. In any case, the different devices must be linked via a data connection (e.g. a suitable data network, such as LAN, wifi, Bluetooth or other). The digital images may be stored in a memory belonging to the computer, or in a separate database connected via a data network.
[0014] One of the preferred further developments of the disclosed system comprises that the image sensor is a scanning electron microscope or a bright field microscope, which creates the microscope images as digital images. Then, those digital images are stored in a memory and provided to a measurement software, which analyzes them according to the method of the invention.
[0015] Another preferred further development of the disclosed system comprises that the software consists of two connected software components, one component is responsible for particle segmentation and the other component is responsible for attribute calculation, or the software consists of one software component that performs both tasks. Which embodiment is preferred ultimately depends on the usage environment, such as the programming language of the application, the available and used software libraries and / or frameworks, the used hardware, etc.
[0016] The proposed solution allows particle measurements to be generated in an intuitive manner with minimal user interaction and less training time.
[0017] Detailed description of the invention
[0018] The method and system according to the invention and their functionally advantageous developments are described in more detail below using at least one preferred exemplary embodiment with reference to the associated drawings. In the drawings, mutually corresponding elements have the same reference numerals. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] Figure 1 : Schematic overview of the system components used
[0020] Figure 2 : Step 1: Upload microscope image via the user interface
[0021] Figure 3 : Step 2a: Export suitable segmentation of microscopy images
[0022] Figure 4 Step 2b: Visualize the progress of internal optimization
[0023] Figure 5 : Step 3: Filter the background fragment
[0024] The system of the scanning electron microscope (SEM) 11 in its preferred embodiment consists of Figure 1. These components include a control unit 2 in the form of any kind of suitable computer 2, which has access to a memory 3. Connected to the computer 2 is a microscope 5, preferably a scanning electron microscope or a bright field microscope, which is capable of creating digital microscope images 6 of small particles 10 (in particular, active pharmaceutical ingredient (API) particles 10 of pharmaceutical preparations for compressed pill drugs, etc.). The digital microscope images 6 are preferably stored on the memory after creation. They can also be stored on any other available memory, whether it is a local memory or any server (such as a server in the cloud). Preferably, but not exclusively, software 8 in the form of a control program 8 is also stored on the local memory, which also provides a machine learning model (AI model) 7, which can be trained with digital microscope images 6 and process digital microscope images 6. In addition, the system 11 includes a display, which shows a user interface 9, preferably a GUI 9, to the user 1, and any process-related information, such as digital microscope images 6 or any calculation results from the AI model 7 corresponding to the software 8, can be shown to the user 1 via the display 4. The user 1 can also input commands or any other data to the software 8 via the user interface 9.
[0025] The method of the present invention in a preferred embodiment is now Figures 2 to 5 and described in more detail in the following sections.
[0026] The software application 8 executed by the described computer 2 implements the method of the present invention. Therefore, the basic workflow of this application consists of three main steps:
[0027] 1. Upload a microscope image 6 through the user interface 9, which has been created by examining a test sample with API particles 10 with a digital microscope 5 and is stored on the memory 3. Here, basic information of the uploaded image 6 is automatically extracted (see Figure 2 )
[0028] 2. Derive a suitable segmentation of the microscope image 6 by iterative "best candidate" selection as outlined in the previous section. To this end, the user 1 is presented with candidate segmentations and he selects the candidate he considers to be the best (see Figure 3 ).
[0029] The progress of the internal optimization is visualized via the display 4, such as Figure 4 As shown in , for example, user 1 has feedback, i.e., which parts of the parameter space are identified as producing good segmentations. Once user 1 is satisfied with the segmentation, he proceeds to the next step.
[0030] 3. User interface 9 allows filtering background segments based on segment convexity, area and texture variation, as in Figure 5 As can be seen in .
[0031] Once the user 1 is satisfied with the final result, different particle measurements, such as size, diameter, elongation etc. are calculated and the results can be exported, for example via an Excel spreadsheet, for further analysis.
[0032] In more detail, step 2 of the workflow mentioned above is based on Bayesian optimization in the parameter space of the segmentation algorithm. Preferably, the Felzenszwalb segmentation algorithm is used, but any other segmentation algorithm with an appropriate number of parameters may be used. In the Bayesian optimization framework, a Gaussian process on the parameter space is used to model some form of utility of a parameter set. In this example case, the Gaussian process models the quality of the resulting segmentation. Because it is desired to learn the utility using pairwise comparisons as user feedback, the preference learning Gaussian process proposed by Chu and Ghahramani in their 2005 paper "Preference Learning with Gaussian Processes" is utilized. The goal of the optimization is then to find a parameter set of the segmentation algorithm that leads to a good segmentation of the physical particles in the digital input image 6. To this end, the optimization process operates as follows:
[0033] 1. The Gaussian process for Bayesian optimization is initialized with an uninformative prior.
[0034] 2. Iterate the following steps until the user finds a sufficiently accurate segmentation
[0035] a. The current posterior likelihood of the segmentation parameters (given by the current Gaussian process) is used together with the acquisition function to randomly sample a small set of new parameter settings. Any acquisition function can be used here, preferably an acquisition function that promotes a certain degree of diversity in the samples, such as batch expected improvement.
[0036] b. Each sampled set of parameters is used to create a segmentation of the physical particles in the input image 6.
[0037] c. Find the current best segmentation by looking at all previously used parameter sets and taking the segmentation that achieves the highest utility as modeled by the current Gaussian process.
[0038] d. In the user interface, show each of the segmentations, i.e. those associated with the newly sampled parameter set and the current best segmentation, to user 1. User 1 is asked to indicate which one she thinks is the best among those displayed segmentations.
[0039] e. Once user 1 has selected the best segmentation among those displayed, this triggers a set of pairwise comparisons, i.e. the selected segmentation is better than each of the other segmentations. With the help of these pairwise comparisons, the Gaussian process is updated.
[0040] List of References
[0041] 1 user
[0042] 2Computer / Control Unit
[0043] 3 Memory
[0044] 4 Display
[0045] 5Image Sensor / Microscope
[0046] 6 Digital microscope images
[0047] 7AI / Machine Learning Models
[0048] 8. Software / Control Program
[0049] 9 User Interface (GUI)
[0050] 10Active Pharmaceutical Ingredient Granules (API)
[0051] 11Scanning Electron Microscope (SEM) System
Claims
1. A method for computer-based small particle measurement of a pharmaceutical preparation, wherein a digital microscope image (6) of small particles (10) of the pharmaceutical preparation is created by an image sensor (5) and provided to a computer (2) executing measurement software (8), the software (8) segmenting the small particles (10) in the digital microscope image (6) and calculating the properties of the small particles (10) according to a specific parameter set, wherein the software (8) samples different candidate parameter sets, automatically applies them to the segmentation and / or calculation process, and shows the results to a user (1) via a display (4), wherein the user (1) selects the candidate parameter set with the best results, the software (8) uses this user feedback to build and train an internal machine learning model (7) and applies the trained model (7) to repeat the automatic segmentation and / or calculation and user feedback acquisition process until the optimal parameter set has been recognized by the user (1).
2. The method according to claim 1, wherein active pharmaceutical ingredient particles (API) (10) are used as small particles (10).
3. The method according to one of the preceding claims, wherein the measured properties consist of the structure and fractal properties of the small API particle surface to determine the surface smoothness and the particle size distribution of the small API particles.
4. The method according to one of the preceding claims, wherein the small particles (10) consist of highly variable particle shapes and sizes.
5. The method according to one of the preceding claims, wherein the digital microscope image (6) is created under variable lighting conditions.
6. A system for performing computer-based small particle measurement of a pharmaceutical preparation, comprising a computer (2) executing measurement software (8), a display (4) for showing information to a user (1), means for the user (1) to input data and / or instructions to the software (8), and an image sensor (5), wherein the system (11) is arranged to create a digital microscope image (6) of small particles (10) of the pharmaceutical preparation via the image sensor (5), to segment the small particles (10) in the digital microscope image (6) and to calculate the properties of the small particles (10) via the software (8) according to a specific parameter set, and wherein the software (8) samples different candidate parameter sets, automatically applies them to the segmentation and / or calculation process, and shows the results to the user (1) via the display (4), wherein the user (1) selects the candidate parameter set with the best results, the software (8) uses this user feedback to build and train an internal machine learning model (7) and applies the trained model (7) to repeat the automatic segmentation and / or calculation and user feedback acquisition process until the optimal parameter set has been recognized by the user (1).
7. System according to claim 6, wherein the image sensor (5) is a scanning electron microscope or a bright field microscope, which creates a microscope image as a digital image (6).
8. A system according to claim 6 or 7, wherein the software (8) consists of two connected software components, one component being responsible for particle segmentation and the other component being responsible for property calculation, or the software (8) consists of one software component performing both tasks.