Coating film thickness detection method and system based on visual imaging and deep learning

Abstract

The invention relates to the technical field of quality control, and provides a coating film thickness detection method and system based on visual imaging and deep learning. The method comprises the following steps: acquiring a visual image of the whole coating process of the fluidized bed, and marking target particles in the visual image to draw the edges of the target particles; based on the marked visual image, training a Mask R-CNN model; detecting a real-time visual image by using the trained Mask R-CNN model to obtain a target particle image, and processing the target particle image to obtain a black and white rendering image of target particles; calculating the particle size of the pellet based on the black and white rendering graph, and calculating the difference value between the particle size of the pellet and the size of the blank particle to obtain the thickness of the pellet coating film. According to the method, the thickness of the pellet coating film can be accurately calculated, and a stable and efficient solution is provided for on-line monitoring of the thickness of the pellet coating film.

Description

Technical Field

[0001] The present invention relates to the technical field of quality control, and in particular to a coating film thickness detection method and system based on visual imaging and deep learning. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Coating of oral solid dosage forms such as tablets and granules is a critical step in pharmaceutical manufacturing and is typically performed in a coating pan or fluidized bed equipment. The main purposes of film coating include controlling the site and rate of drug release (e.g., in the case of enteric coating, the drug is insoluble in the stomach but dissolves in the intestine), improving the appearance of the drug, masking unpleasant tastes such as bitterness, and improving drug stability by providing a barrier against environmental and physiological factors that may degrade the active pharmaceutical ingredient. Therefore, film thickness is a critical quality attribute that significantly affects coating performance. Even small changes in film thickness can affect the quality, solubility, and stability of the drug, highlighting the need for real-time monitoring during the coating process.

[0004] Common process analytical technologies (PAT) for measuring film thickness or particle size are generally divided into direct and indirect methods. Direct methods include optical coherence tomography (OCT), visual imaging, spatial filter velocimetry (SFV), and focused beam reflectometry (FBRM). Visual imaging is a promising tool with great potential in particle size detection, especially when combined with image analysis algorithms in pharmaceutical manufacturing. Compared with other methods, visual imaging can measure and analyze the shape factor and size distribution of particles, providing a more intuitive measurement method. Traditional image segmentation methods include threshold processing and edge segmentation. Threshold processing distinguishes, splits, and merges corresponding regions based on pixel intensity. Edge segmentation identifies boundaries by detecting discontinuities in pixel intensity.

[0005] However, fluidized bed coating is a dynamic, multiphase process. Traditional imaging technology has a certain delay in real-time dynamic monitoring, especially in the case of high-frequency data acquisition and real-time processing. How to ensure both imaging speed and accuracy remains a challenge. In addition, even if imaging technology can capture high-definition images, subsequent image processing and data analysis are still very complicated. Due to the dynamic characteristics of the fluidized bed environment, the shape, size, and motion state of the particles change frequently, making it difficult for automated image processing algorithms to cope with diversity and uncertainty. Therefore, it is necessary to combine deep learning algorithms to more accurately and effectively analyze complex background images. Summary of the invention

[0006] In order to solve the technical problems existing in the above-mentioned background technology, the present invention provides a coating film thickness detection method and system based on visual imaging and deep learning. The present invention adopts a mask region-based convolutional neural network (MaskR-CNN) to segment visual imaging pictures online, so as to accurately calculate the thickness of the micropill coating film, providing a robust and efficient solution for online monitoring of the particle coating film thickness.

[0007] In order to achieve the above object, the present invention adopts the following technical solution:

[0008] The first aspect of the present invention provides a coating film thickness detection method based on visual imaging and deep learning.

[0009] A coating film thickness detection method based on visual imaging and deep learning, comprising:

[0010] Obtain a visual image of the entire fluidized bed coating process, mark the target particles in the visual image and outline the edges of the target particles;

[0011] Train the Mask R-CNN model based on labeled visual images;

[0012] The trained Mask R-CNN model is used to detect the real-time visual image to obtain the target particle image, and the target particle image is processed to obtain the black and white rendering of the target particle;

[0013] Based on the black and white rendering, the pellet particle size was calculated, and the difference between the pellet particle size and the blank particle size was calculated to obtain the pellet coating film thickness.

[0014] Furthermore, the model training process includes: obtaining samples at the time points corresponding to the visual images, and measuring the thickness of the microcapsule coating film offline; calculating the loss function based on the offline measured microcapsule coating film thickness and the microcapsule coating film thickness calculated during the model training process, and training the hyperparameters of the Mask R-CNN model.

[0015] Furthermore, the offline measurement of the micropellet coating film thickness includes: subtracting the equivalent diameter size of blank particles from the equivalent diameter size of particles measured by the Malvern laser method to obtain the micropellet coating film thickness.

[0016] Furthermore, the offline measurement of the pellet coating thickness includes: using an ultraviolet spectrophotometer method to calculate the pellet coating thickness; using the following formula to describe:

[0017]

[0018] In the formula, C API is the API concentration of each group; W pellet is the average weight of each pellet; PAPI is the proportion of API in the coating agent; ρ coating is the coating solid content density; d is the average particle size of the blank pills.

[0019] Furthermore, the difference between the pellet particle size and the blank particle size is calculated to obtain the pellet coating film thickness; the following formula is used to express it:

[0020]

[0021] Among them, ACT jb represents the average coating film thickness of the particle image taken at the jth time point, represents the equivalent average diameter of the particles in the image at time point j, Represents the equivalent mean diameter of the particles in the blank pellet image.

[0022] Furthermore, the method of calculating the particle size of the micropellets based on the black and white rendering image includes: calculating the particle size of the micropellets based on the black and white rendering image using the cv.contourArea() function.

[0023] The second aspect of the present invention provides a coating film thickness detection system based on visual imaging and deep learning.

[0024] A coating film thickness detection system based on visual imaging and deep learning, comprising:

[0025] A data acquisition and processing module is configured to: acquire a visual image of the entire fluidized bed coating process, mark the target particles in the visual image and outline the edges of the target particles;

[0026] A model training module, which is configured to: train a Mask R-CNN model based on the labeled visual images;

[0027] A detection module is configured to: detect the real-time visual image using the trained Mask R-CNN model to obtain a target particle image, and process the target particle image to obtain a black-and-white rendering of the target particle;

[0028] The output module is configured to: calculate the pellet particle size based on the black and white rendering, calculate the difference between the pellet particle size and the blank particle size, and obtain the pellet coating film thickness.

[0029] A third aspect of the present invention provides a computer-readable storage medium.

[0030] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the coating film thickness detection method based on visual imaging and deep learning as described in the first aspect above.

[0031] A fourth aspect of the present invention provides a computer device.

[0032] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the coating film thickness detection method based on visual imaging and deep learning as described in the first aspect above are implemented.

[0033] A fifth aspect of the present invention provides a computer program product or a computer program.

[0034] The present invention provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the steps in the coating film thickness detection method based on visual imaging and deep learning as described in the first aspect above.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] The present invention provides a coating film thickness detection method and system based on visual imaging and deep learning, which uses a mask region-based convolutional neural network (Mask R-CNN) to segment visual imaging images online, so that the coating film thickness of micropills can be accurately calculated, providing a robust and efficient solution for online monitoring of particle coating film thickness. The present invention can accurately predict the coating film thickness under complex backgrounds, greatly improves the prediction accuracy of the model, and provides technical support for subsequent fluidized bed coating quality control, thereby improving the safety and effectiveness of drugs.

[0037] The present invention combines online imaging technology with deep learning algorithms to monitor the thickness changes of coating films in real time and accurately online during fluidized bed coating. The establishment of the technical solution of the present invention helps to timely monitor the dynamic behavior of the coating process during the fluidized bed coating process, ensure the consistency of the quality of the coating process, thereby improving the safety and effectiveness of the drug, and therefore has good practical application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0039] Figure 1 is a flow chart of a coating film thickness detection method based on visual imaging and deep learning shown in the present invention;

[0040] Figure 2 1 is a diagram of a fluidized bed coating experimental device according to the present invention; in the diagram, (a) is a NIR sensor, (b) is an imaging probe, (c) is a NIR sensor control interface, and (d) is an imaging probe control interface;

[0041] Figure 3 (a) is a graph showing the number of particles in the first batch, (b) is a graph showing the number of particles in the second batch, and (c) is a graph showing the number of particles in the third batch;

[0042] Figure 4 : The polygonal marking particles composed of many points shown in the present invention; (a) are clear particles, (b) are agglomerated particles, (c) are incomplete particles at the image boundary, (d) are powders stuck on the window, and (e) are out-of-focus or blurred particles;

[0043] Figure 5 is a graph showing the variation of total loss with iteration steps shown in the present invention;

[0044] Figure 6 (a) is the accuracy evaluation result of the Mask R-CNN model, (b) is the recall evaluation result of the Mask R-CNN model, (c) is the precision evaluation result of the Mask R-CNN model, and (d) is the F1 score evaluation result of the Mask R-CNN model.

[0045] Figure 7 : is a picture segmentation diagram shown in the present invention; (a) is the original image, (b) is the boundary map predicted by the Mask R-CNN model, and (c) is the black and white background rendering;

[0046] Figure 8 1 is a comparison chart of film thickness of visual imaging, laser diffraction and UV-visible spectrophotometer shown in the present invention. DETAILED DESCRIPTION

[0047] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0048] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0049] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0050] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the methods and systems according to various embodiments of the present disclosure. It should be noted that each box in the flowchart or block diagram can represent a module, a program segment, or a part of a code, and the module, program segment, or a part of a code may include one or more executable instructions for implementing the logical functions specified in each embodiment. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, or they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the flowchart and / or block diagram, and the combination of boxes in the flowchart and / or block diagram can be implemented using a dedicated hardware-based system that performs a specified function or operation, or can be implemented using a combination of dedicated hardware and computer instructions.

[0051] As mentioned earlier, common process analytical technologies (PAT) currently used to measure film thickness or particle size include optical coherence tomography (OCT), visual imaging, spatial filter velocimetry (SFV), and focused beam reflectometry (FBRM). Visual imaging is a promising tool with great potential for particle size detection, especially when combined with image analysis algorithms in pharmaceutical manufacturing. Compared with other methods, visual imaging can measure and analyze the shape factor and size distribution of particles, providing a more intuitive measurement method. In addition, when combined with the deep learning method Mask R-CNN, it can accurately and effectively analyze the changes in particle coating film thickness under complex background images.

[0052] In view of this, the present application provides a coating film thickness detection method and system based on visual imaging and deep learning, which uses a mask region-based convolutional neural network (Mask R-CNN) to segment visual imaging images online, so that the thickness of the micropill coating film can be accurately calculated. The present invention is described in detail below through several embodiments:

[0053] Embodiment 1

[0054] like Figure 1As shown, this embodiment provides a coating film thickness detection method based on visual imaging and deep learning.

[0055] Step (1): collecting visual imaging image data of the entire fluidized bed coating process;

[0056] Step (2): Collect samples at different time points during the fluidized bed coating process (corresponding to the time points of the imaging pictures) and measure the thickness of the pellet coating film offline;

[0057] Step (3): Use the marking tool to mark the target particles in the image, create polygons to outline the edges of the target particles, and effectively describe the shapes of various objects;

[0058] Step (4): The label file after labeling, which contains the information of all particles, is used for the subsequent training of the Mask R-CNN model;

[0059] Step (5): Evaluate the performance of the model using accuracy, precision, recall, and F1 value indicators;

[0060] Step (6): Use the optimized Mask R-CNN model to segment particles in unknown sample images to generate clear particle boundaries, and then remove other backgrounds to generate a black and white rendering of the target particles;

[0061] Step (7): Use cv.contourArea() function to calculate the particle size from the black and white rendering image, and then calculate the size of the particles in the corresponding image at each time point, and then subtract the blank particle size to obtain the increased coating film thickness;

[0062] Step (8): Compare with the offline Malvern laser method and UV spectrophotometry to verify the accuracy of the technology.

[0063] In another specific embodiment of the present invention, the step (1) collects visual imaging picture data of the whole process of fluidized bed coating, and the specific method includes:

[0064] During the fluidized bed coating process, an online two-dimensional visualization probe is installed on the outer wall of the fluidized bed window, and the camera shooting frequency is controlled by the imaging system and can be adjusted to a maximum of 10 frames per second. In the present invention, the sampling frequency is one frame every 10 seconds, and the image is displayed and recorded on a computer.

[0065] In another specific embodiment of the present invention, in step (2), samples at different time points during the fluidized bed coating process (corresponding to the time points of the imaging pictures) are collected to measure the thickness of the particle coating film offline:

[0066] The particle size was measured using a Mastersizer 3000 (Malvern Panalytical, Malvern, UK) equipped with an Aero S dry powder dispersion device. The Mastersizer 3000 uses laser diffraction to measure the scattering pattern of the particles. The software analyzes the scattering data to determine the particle size distribution and performs three measurements to ensure accuracy and repeatability. The film thickness measured by visual imaging and laser diffraction can be obtained by subtracting the equivalent diameter of the blank particles from the equivalent diameter of the particles at different time points, as shown in formulas (8)-(9).

[0067] The coating film thickness was measured using a UV spectrophotometer as follows:

[0068] First, a certain weight of blank pills was weighed and placed under a microscope to ensure proper dispersion. The number of pills was counted to calculate the average weight of each pill, and this process was repeated three times to obtain the average value. Secondly, a certain weight of pills was randomly weighed from each pill sample collected during the coating process. Each group of pills was mixed with pure water and ultrasonically treated to ensure that the API was completely dissolved in the aqueous solution. The solution was then filtered through a 0.45 μm filter and the corresponding absorbance was measured using a UV-visible spectrophotometer. Prior to sample analysis, a calibration curve was established to determine the API concentration in the solution. Data associated with each pellet core was obtained by knowing the following information: 1) the proportion of coating ingredients; 2) the coating density (measured using an automatic true density analyzer, BSD-TD); 3) the average diameter of the blank pills. Therefore, the average film thickness (δ thickness,UV ) is calculated as shown in formula (1):

[0069]

[0070] In the formula, C API is the API concentration of each group (w / w); W pellet is the average weight of each pellet; P API is the proportion of API in the coating agent; ρ coating is the coating solid content density; d is the average particle size of the blank pills.

[0071] In another specific embodiment of the present invention, step (3) uses a marking tool to mark the target particles in the image, creating polygons to outline the edges of the target particles, and effectively describing the shapes of various objects;

[0072] Using the VIA 2.0 (Visual-Geometry-Group, 2021) marking tool, polygons are created to outline the edges of the target particles, effectively describing the shapes of various objects. This involves manually placing discrete points along the curved edges of each particle, and lines automatically connect adjacent points to form polygons. The number of points required to accurately mark each particle depends on its size. Larger particles typically require 20-30 points to describe their shape with a polygon. After marking, a label file is generated containing the location information of all particles and sorted by image file name for subsequent training of the Mask R-CNN model. Note that in order to reflect the true size of the particles as much as possible, agglomerated pellets, incomplete pellets at the image boundaries, powder stuck to the window, out-of-focus and blurred pellets, etc. contained in the image are not marked, which will affect the calculation of the reagent particle size.

[0073] In another specific embodiment of the present invention, step (4) uses the label file after labeling, which contains information of all particles, for subsequent training of the Mask R-CNN model.

[0074] The Mask R-CNN model includes a backbone network, a region candidate network, a region of interest alignment layer and a regional convolutional neural network. The backbone network is used to extract feature maps. The backbone network includes a residual network and a feature pyramid. Based on the feature map, a domain candidate network is used to obtain the region of interest and propose a candidate object bounding box. Based on the region of interest and the feature map, a region of interest alignment layer is used to perform matching, complete feature map feature aggregation and pooling to a fixed size, and output to the regional convolutional neural network through a fully connected layer. The regional convolutional neural network includes a first branch, a second branch and a third branch. The first branch realizes particle classification through a softmax classifier, the second branch realizes more accurate target positioning through a bounding box regressor, and the third branch completes the contour segmentation of the target particle through a fully convolutional network and generates a mask. Finally, the output information of each branch is integrated to obtain a target particle image containing a category, a positioning bounding box, and a segmentation mask, thereby realizing accurate positioning and recognition of the target particle.

[0075] The configuration settings for model training are as follows. The number of steps per round is set to 100 and the number of validation steps is set to 30. The confidence of detection is set to 0.7, which means that the Mask R-CNN model will detect an object instance only when the confidence in the existence of the object instance is more than 70%. The learning rate and momentum of the gradient descent algorithm are set to 0.001 and 0.9 respectively to ensure the stability of weight optimization and faster convergence behavior. The loss function can be expressed as:

[0076] L total =L cls +Lbox +L mask (2)

[0077] Among them, L cls , L box and L mask They are classification loss, bounding box regression loss, and mask loss.

[0078] In another specific implementation of the present invention, step (5) judges the performance of the model by accuracy, precision, recall and F1 value indicators.

[0079] The accuracy, precision, recall, and F1 value indicator formulas are as follows:

[0080]

[0081] Among them, TP represents the number of samples that correctly identify the pellets as positive; FN represents the number of samples that incorrectly identify the pellets as negative; FP represents the number of samples that incorrectly identify the pellets as positive; TN represents the number of samples that correctly identify the pellets as negative.

[0082] In another specific embodiment of the present invention, step (6) uses the optimized model for particle detection in an unknown sample image to obtain a target particle image, and then removes other backgrounds to generate a black and white rendering of the target particles.

[0083] In another specific embodiment of the present invention, step (7) uses the cv.contourArea() function to calculate the particle size from the black and white rendering image, and the size of the particles in the image corresponding to each time point is calculated by analogy, and then the blank particle size is subtracted to obtain the increased coating film thickness.

[0084] For the estimation of each particle diameter, the equivalent spherical diameter (ESD) was used. ED is calculated based on the projected area (Area) of the particle:

[0085]

[0086] The equivalent average diameter of the particles in the image It can be further calculated by the following formula:

[0087]

[0088] Among them, ESD i represents the diameter of the i-th particle; N is the total number of particles in the image.

[0089] The average coating thickness (ACT) of the particles in the pictures taken at each time point can be further calculated by the following formula:

[0090]

[0091] Among them, ACT jb represents the average coating film thickness of the particle image taken at the jth time point, represents the equivalent average diameter of the particles in the image at time point j, Represents the equivalent mean diameter of the particles in the blank pellet image.

[0092] In another specific embodiment of the present invention, step (8) is compared with the offline Malvern laser method and ultraviolet spectrophotometry to verify the accuracy of the technology.

[0093] Embodiment 2

[0094] The coating film thickness detection method based on visual imaging and deep learning described in this embodiment comprises the following steps:

[0095] Step (1): Collect visual imaging data of the entire fluidized bed coating process.

[0096] Before each coating batch, preheat the fluidized bed for half an hour to make the fluidized bed granulate in a relatively stable state. After preheating, put the materials in, mix for 10 minutes, and then start spraying the coating liquid.

[0097] During the fluidized bed coating process, the online two-dimensional visualization probe was installed on the outer wall of the fluidized bed window, about 2 mm away from the window (see Figure 2 d). The camera parameters are set as follows: the light source brightness is set to 800%, the current size is set to 0.6A, and the pulse width is set to 0.02ms. The camera shooting frequency is controlled by the imaging system and can be adjusted to a maximum of 10 frames / second. In this study, the sampling frequency is one frame every 10 seconds, and the image is displayed and recorded on the computer. Figure 2 As shown in b, the computer interface displays the acquired particle image in real time.

[0098] Step (2): Collect samples at different time points during the fluidized bed coating process (corresponding to the time points of the imaging pictures) and measure the particle coating film thickness offline.

[0099] The particle size was measured using a Mastersizer 3000 (Malvern Panalytical, Malvern, UK) equipped with an Aero S dry powder dispersion device to analyze the particle size. Approximately 10 grams of sample was loaded into the sample hopper and the particles were dispersed into individual particles using 1 bar of compressed air. Before the sample was introduced, a background measurement was performed using only compressed air to account for any background noise or contamination. The dispersed sample was then introduced into the laser beam path, where the Mastersizer 3000 measured the scattering pattern of the particles using laser diffraction. The software analyzed the scattering data to determine the particle size distribution, and three measurements were performed to ensure accuracy and repeatability. The film thickness measured by visual imaging and laser diffraction can be obtained by subtracting the equivalent diameter size of the blank particles from the equivalent diameter size of the particles at different time points, as shown in formulas (8)-(9) for details.

[0100] The coating film thickness was measured using a UV spectrophotometer as follows:

[0101] First, a certain weight of blank pills was weighed and placed under a microscope (see Figure 3 ) to ensure proper dispersion. The number of pills was counted to calculate the average weight of each pill, and this process was repeated three times to obtain the average value. Secondly, a certain weight of pills was randomly weighed from each pill sample collected during the coating process. Each group of pills was mixed with pure water and ultrasonicated for 20 minutes to completely dissolve the API in the aqueous solution. The solution was then filtered through a 0.45 μm filter and the absorbance was measured at a maximum wavelength of 233 nm (λmax) using a UV-visible spectrophotometer (EU-2800D, Zhongke Ruijie, China). Prior to sample analysis, a calibration curve was established to determine the API concentration in the solution. Third, data related to each pill was obtained by understanding the following information: (i) coating ingredient ratio (ii) coating density (measured using an automatic true density analyzer, Beshide, BSD-TD) (iii) average diameter of the blank pills. Therefore, the average film thickness (δ thickness,UV ) is calculated by formula (1).

[0102] Step (3): Use the marking tool to mark the target particles in the image and create polygons to outline the edges of the target particles, effectively describing the shapes of various objects, such as Figure 4 shown.

[0103] Using the VIA2.0 (Visual-Geometry-Group, 2021) marking tool, polygons are created to outline the edges of the target particles, effectively describing the shapes of various objects. This involves manually placing discrete points along the curved edges of each particle, and lines automatically connect adjacent points to form polygons. The number of points required to accurately mark each particle depends on its size. Larger particles typically require 20-30 points to describe their shape with a polygon. After marking, a label file is generated containing the location information of all particles and sorted by image file name for subsequent training of the Mask R-CNN model. Note that in order to reflect the true size of the particles as much as possible, agglomerated pellets, incomplete pellets at the image boundaries, powder stuck to the window, out-of-focus and blurred pellets, etc. contained in the image are not marked, which will affect the calculation of the reagent particle size.

[0104] Step (4): The label file after labeling, which contains the information of all particles, is used for the subsequent training of the Mask R-CNN model;

[0105] All 140 images are annotated, with 112 images in the training set and 28 images in the validation set. The configuration settings for model training are as follows. The number of steps per round is set to 100, and the number of validation steps is set to 30. The confidence of detection is set to 0.7, which means that the Mask R-CNN model will detect an object instance only when the confidence in the existence of the object instance is more than 70%. The learning rate and momentum of the gradient descent algorithm are set to 0.001 and 0.9 respectively to ensure the stability of weight optimization and faster convergence behavior. Figure 5 The total loss of the Mask R-CNN model (containing 140 images) is shown as a function of the number of training steps. Clearly, the loss decreases as the number of steps increases and stabilizes after 100 steps. Therefore, this study used the model weights after 100 iterations for the new particle segmentation images.

[0106] Step (5): Evaluate the performance of the model by using the accuracy, precision, recall, and F1 value indicators. Use formulas (3)-(6) to calculate the accuracy, precision, recall, and F1 value respectively. The test set results are as follows: Figure 6 shown.

[0107] The test set consists of 112 images selected from 14 batches (8 images per batch) during the coating process in chronological order. The average number of particles per image is about 20. The Mask R-CNN algorithm is further evaluated on the test set in terms of precision, recall, accuracy, and F1 score. Figure 6The evaluation results of the Mask R-CNN algorithm are shown, with precision ranging from 75% to 100% and an average of 97.9%, recall ranging from 75% to 100% and an average of 98.0%, accuracy ranging from 90.1% to 100% and an average of 98.4%, and F1 scores ranging from 85.7% to 100% and an average of 97.9%. The above results show that the performance of the Mask R-CNN model on the validation set is also excellent.

[0108] Step (6): Use the optimized model to detect particles in unknown sample images and obtain the target particle image, such as Figure 7 As shown in (b), other backgrounds are then removed to generate a black and white rendering of the target particles. Figure 7 (c) as shown.

[0109] Step (7): Use the cv.contourArea() function to calculate the particle size from the black and white rendering image. Similarly, calculate the size of the particles in the image corresponding to each time point, and then subtract the blank particle size to obtain the increased coating film thickness. See formulas (7)-(9) for details.

[0110] Step (8): Compare with the offline Malvern method and UV spectrophotometry to verify the accuracy of the technology.

[0111] Figure 8 The comparison results of three methods (imaging, laser diffraction and UV-visible spectrophotometer) for measuring film thickness are given. For the collected particle samples, the film thickness results measured by online visual imaging are consistent with the offline laser diffraction and UV-visible spectrophotometer measurements. Although the coating film thickness values ​​measured by different methods are slightly different, the trends are consistent, proving that imaging technology can be used for online monitoring of film thickness during the coating process.

[0112] Embodiment 3

[0113] This embodiment provides a coating film thickness detection system based on visual imaging and deep learning.

[0114] A coating film thickness detection system based on visual imaging and deep learning, comprising:

[0115] A data acquisition and processing module is configured to: acquire a visual image of the entire fluidized bed coating process, mark target particles in the visual image, and outline the edges of the target particles;

[0116] A model training module, which is configured to: train a Mask R-CNN model based on the labeled visual images;

[0117] A detection module is configured to: detect the real-time visual image using the trained Mask R-CNN model to obtain a target particle image, and process the target particle image to obtain a black-and-white rendering of the target particle;

[0118] The output module is configured to: calculate the pellet particle size based on the black and white rendering, calculate the difference between the pellet particle size and the blank particle size, and obtain the pellet coating film thickness.

[0119] In some embodiments, the model training module is further configured to: during the model training process, include: obtaining samples at time points corresponding to the visual image, and offline measuring the thickness of the microcapsule coating film; based on the offline measured microcapsule coating film thickness and the microcapsule coating film thickness calculated during the model training process, calculate the loss function, and train the hyperparameters of the Mask R-CNN model.

[0120] In some embodiments, the model training module is further configured to: subtract the equivalent diameter size of the blank particles from the equivalent diameter size of the particles measured by the Malvern laser method to obtain the thickness of the micropellet coating film.

[0121] In some embodiments, the model training module is further configured to: calculate the thickness of the pellet coating film using an ultraviolet spectrophotometer method; described by the following formula:

[0122]

[0123] In the formula, C API is the API concentration of each group; W pellet is the average weight of each pellet; P API is the proportion of API in the coating agent; ρ coating is the coating solid content density; d is the average particle size of the blank pills.

[0124] In some embodiments, the output module is further configured to calculate the thickness of the pellet coating film using the following formula:

[0125]

[0126] Among them, ACT jb represents the average coating film thickness of the particle image taken at the jth time point, represents the equivalent average diameter of the particles in the image at time point j, Represents the equivalent mean diameter of the particles in the blank pellet image.

[0127] In some embodiments, the output module is further configured to calculate the pellet particle size based on the black and white rendering image using the cv.contourArea() function.

[0128] Embodiment 4

[0129] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps in the coating film thickness detection method based on visual imaging and deep learning as described in the above-mentioned embodiment 1 or embodiment 2 are implemented.

[0130] Embodiment 5

[0131] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps in the coating film thickness detection method based on visual imaging and deep learning as described in the above-mentioned Embodiment 1 or Embodiment 2 are implemented.

[0132] Embodiment 6

[0133] This embodiment provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the steps in the coating film thickness detection method based on visual imaging and deep learning described in the above-mentioned embodiment 1 or embodiment 2.

[0134] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0135] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0136] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0137] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0138] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0139] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A coating film thickness detection method based on visual imaging and deep learning, characterized in that: include: Obtain a visual image of the entire fluidized bed coating process, mark the target particles in the visual image and outline the edges of the target particles; Train the Mask R-CNN model based on labeled visual images; The trained Mask R-CNN model is used to detect the real-time visual image to obtain the target particle image, and the target particle image is processed to obtain the black and white rendering of the target particle; Based on the black and white rendering, the pellet particle size was calculated, and the difference between the pellet particle size and the blank particle size was calculated to obtain the pellet coating film thickness.

2. The coating film thickness detection method based on visual imaging and deep learning according to claim 1, characterized in that: The model training process includes: obtaining samples at the time points corresponding to the visual images, and measuring the thickness of the microcapsule coating film offline; calculating the loss function based on the offline measured microcapsule coating film thickness and the microcapsule coating film thickness calculated during the model training process, and training the hyperparameters of the Mask R-CNN model.

3. The coating film thickness detection method based on visual imaging and deep learning according to claim 2, characterized in that: The offline measurement of the micropill coating film thickness includes: subtracting the equivalent diameter size of blank particles from the equivalent diameter size of particles measured by the Malvern laser method to obtain the micropill coating film thickness.

4. The coating film thickness detection method based on visual imaging and deep learning according to claim 2, characterized in that: The offline measurement of the pellet coating thickness includes: using an ultraviolet spectrophotometer method to calculate the pellet coating thickness; using the following formula to describe: In the formula, C API is the API concentration of each group; W pellet is the average weight of each pellet; P API is the proportion of API in the coating agent; ρ coating is the coating solid content density; d is the average particle size of the blank pills.

5. The coating film thickness detection method based on visual imaging and deep learning according to claim 1 or 2, characterized in that: The difference between the pellet particle size and the blank particle size is calculated to obtain the pellet coating film thickness; It is expressed by the following formula: Wherein, ACb represents the average coating film thickness of the particle image taken at the jth time point, represents the equivalent average diameter of the particles in the image at time point j, Represents the equivalent mean diameter of the particles in the blank pellet image.

6. The coating film thickness detection method based on visual imaging and deep learning according to claim 1 or 2, characterized in that: The method for calculating the particle size of micropellets based on the black and white rendering image comprises: calculating the particle size of micropellets based on the black and white rendering image using the cv.contourArea() function.

7. The coating film thickness detection system based on visual imaging and deep learning is characterized by: include: A data acquisition and processing module is configured to: acquire a visual image of the entire fluidized bed coating process, mark target particles in the visual image, and outline the edges of the target particles; A model training module, which is configured to: train a Mask R-CNN model based on the labeled visual images; A detection module is configured to: detect the real-time visual image using the trained Mask R-CNN model to obtain a target particle image, and process the target particle image to obtain a black-and-white rendering of the target particle; The output module is configured to: calculate the pellet particle size based on the black and white rendering, calculate the difference between the pellet particle size and the blank particle size, and obtain the pellet coating film thickness.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in the coating film thickness detection method based on visual imaging and deep learning are implemented as described in any one of claims 1 to 6.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the coating film thickness detection method based on visual imaging and deep learning are implemented as described in any one of claims 1 to 6.

10. A computer program product, characterized in that The computer program product includes a computer program, and when the computer program is executed by a processor, the steps in the coating film thickness detection method based on visual imaging and deep learning as described in any one of claims 1 to 6 are implemented.

Images