Methods, electronic devices and computer-readable storage media for testing excipients
By combining the Unet and YOLO network architectures with adaptive thresholding and data augmentation techniques, the problems of misjudgment of container edges and excessive computational load in auxiliary material detection are solved, achieving efficient and accurate impurity detection.
Patent Information
- Application Number
- CN202611321522.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-28
- Publication Date
- 2026-11-17
AI Technical Summary
Existing machine vision inspection solutions are prone to misidentifying container edges, tray textures, and non-extractive areas as impurities in auxiliary material inspection. Furthermore, the uneven texture of solid auxiliary materials and the refraction and reflection of liquid auxiliary materials increase the difficulty of inspection, resulting in excessive computational load and low inspection efficiency.
Image segmentation is performed using a Unet network architecture segmentation model to extract effective regions. Impurity detection is performed using a YOLO network architecture target detection model. An adaptive threshold algorithm is used to mask liquid reflections, Gaussian blur is used to process solid textures, and data augmentation is applied to liquid images. A joint loss function is used to optimize the target detection model, reducing computational load and the risk of false positives.
It enables accurate detection of impurities in auxiliary materials, significantly improving detection accuracy and efficiency, reducing model computational load, isolating container and background interference, and ensuring the accuracy and efficiency of detection.
Smart Images

Figure CN123118531A_ABST
Abstract
Description
Technical Field
[0001] This application generally relates to the field of image processing technology. More specifically, this application relates to a method, electronic device, and computer-readable storage medium for detecting auxiliary materials. Background Technology
[0002] In industrial production, the appearance quality and impurity content of liquid and solid additives are key indicators determining the quality of the final product. Traditional manual visual inspection methods suffer from high subjectivity, low efficiency, and susceptibility to environmental interference, making them unsuitable for the demands of large-scale, high-standard industrial production. In recent years, the introduction of computer vision and deep learning technologies into industrial quality inspection has become an industry trend, effectively improving the level of automation in inspection.
[0003] In existing machine vision inspection solutions, a single target detection model is typically used to directly identify and locate impurities in the raw images of acquired auxiliary material samples. However, in industrial production scenarios, auxiliary material samples are usually photographed inside specific containers. Existing technologies directly input the entire raw image (including background and container) into the target detection model, which easily leads to misclassification of container edges, tray textures, and background features in non-auxiliary material areas as impurities. Furthermore, the uneven texture of solid auxiliary materials and the refraction and reflection of liquid auxiliary materials can cause variations in pixel features within the effective detection area, further increasing the detection difficulty. In addition, directly processing the entire image, which contains a large amount of irrelevant background information, significantly increases the computational load on the model and reduces detection efficiency.
[0004] In view of this, there is an urgent need to provide a solution for the detection of excipients, so as to effectively isolate the container from background interference by accurately extracting the effective detection area of solids or liquids, reduce the computational load of the model, and thus improve the accuracy and efficiency of excipient impurity detection. Summary of the Invention
[0005] In order to at least solve one or more of the technical problems mentioned above, this application proposes a method, electronic device and computer-readable storage medium for detecting auxiliary materials in several aspects.
[0006] In a first aspect, this application provides a method for detecting excipients, comprising: acquiring an original image of the excipient to be detected; segmenting the original image using a segmentation model to extract an effective region in the original image, the effective region including a solid region or a liquid region; and performing impurity detection on the effective region using a target detection model to obtain an impurity detection result for the excipient to be detected.
[0007] In some embodiments, the segmentation model employs a Unet network architecture, including a downsampling network, a bottleneck layer, an upsampling network, and an output layer. The downsampling network includes multiple convolutional layers and pooling layers, and the upsampling network includes multiple deconvolutional layers and skip connection layers. Before performing the segmentation process using the segmentation model, the method further includes: using a sample dataset with effective region annotations, using DiceLoss as the loss function, freezing the downsampling network and the bottleneck layer, and fine-tuning the upsampling network.
[0008] In other embodiments, the object detection model employs a YOLO network architecture, including a feature extraction network, a feature fusion network, and an output network for bounding box regression, category classification, and impurity counting; and before using the object detection model for impurity detection, the method further includes: freezing the feature extraction network, and training the feature fusion network and the output network using a joint loss function, wherein the joint loss function is obtained by weighted summation of bounding box regression loss, category classification loss, and impurity count regression loss.
[0009] In some other embodiments, the joint loss function is calculated using the following formula: ;in, =0.5, β=0.3, γ=0.2; L CIoU For bounding box regression loss, L Focal For category classification loss, L L1 This represents the regression loss due to impurity counts.
[0010] In some embodiments, before using the target detection model to detect impurities in the effective region, the method further includes performing at least one of the following preprocessing on the image of the effective region to obtain a preprocessed image, such that the target detection model performs impurity detection on the preprocessed image: (1) when the effective region is a liquid region, locating the reflective region in the effective region and generating a secondary mask by means of an adaptive threshold algorithm to shield the interference of refracted reflection on subsequent detection; (2) performing data augmentation processing on the effective region, wherein in response to the effective region being a solid region, performing Gaussian blur texture perturbation processing on the effective region; and in response to the effective region being a liquid region, performing at least one data augmentation processing including angle adjustment, brightness adjustment, and contrast adjustment.
[0011] In other embodiments, the method further includes: using the color features of a preset standard excipient image as a reference feature, calculating the normalized comprehensive deviation between the color features of the image of the effective region and the reference feature; comparing the normalized comprehensive deviation with a preset deviation threshold to determine whether the chromaticity and / or transparency of the excipient to be detected meet the standard.
[0012] In some other embodiments, when the excipient to be detected is a solid or a liquid, the normalized comprehensive deviation includes a colorimetric comprehensive deviation; and calculating the normalized comprehensive deviation between the color features of the image of the effective region and the reference features includes: determining the RGB mean, RGB range, and CIELAB mean of the pixels in the image of the effective region; determining the colorimetric comprehensive deviation of the excipient to be detected based on the RGB mean, the RGB range, the CIELAB mean, and the reference features; wherein the colorimetric comprehensive deviation of the excipient to be detected is determined in the following manner:
[0013] ; in, Indicates overall color deviation. This indicates the deviation of the effective region from the RGB mean of the reference feature. This indicates the L-value deviation between the effective region and the reference feature. This indicates the AB value deviation between the effective region and the reference feature. F represents the RGB range and deviation between the effective region and the reference feature. ref F represents the reference feature. test The color characteristics of the effective area, This represents the k-mean value in the reference feature. This represents the k-mean of the effective region. This represents the mean L value among the reference features. The mean L value of the effective region is represented. This represents the sum of the RGB ranges in the reference feature. This represents the sum of the RGB ranges of the effective region.
[0014] In some embodiments, when the excipient to be detected is a liquid, the normalized comprehensive deviation includes a transparency comprehensive deviation; and calculating the normalized comprehensive deviation between the color features of the image of the effective region and the reference features includes: converting the image of the effective region into a grayscale image; calculating the mean brightness, standard deviation brightness, and relative standard deviation of the grayscale image to characterize transparency; and determining the transparency comprehensive deviation of the excipient to be detected based on the mean brightness, the standard deviation brightness, the relative standard deviation, and the reference features; wherein the transparency comprehensive deviation of the excipient to be detected is determined in the following manner: ; in, This indicates the overall deviation in transparency. This indicates the deviation of the average brightness of the effective area from that of the reference feature. This represents the relative standard deviation of the effective region from the reference feature. This represents the mean brightness value in the reference feature. This represents the average brightness of the effective area. This represents the relative standard deviation in the reference characteristic. The relative standard deviation of the effective area is represented by the ratio of the brightness standard deviation to the brightness mean.
[0015] In a second aspect, this application provides an electronic device comprising: a processor configured to execute program instructions; and a memory configured to store the program instructions, which, when loaded and executed by the processor, cause the processor to perform a method according to any one of the first aspects of this application.
[0016] In a third aspect, this application provides a computer-readable storage medium storing program instructions that, when loaded and executed by a processor, cause the processor to perform the method according to any one of the first aspects of this application.
[0017] Using the method for detecting excipients provided above, this embodiment of the application first uses a segmentation model to accurately extract the effective region (solid region or liquid region) from the original image of the excipient to be detected, thereby achieving physical isolation between the excipient sample and the container and background. Then, the pure effective region is used as input to the target detection model for targeted impurity detection. This effectively avoids interference from irrelevant regions such as container boundaries and background textures on the target detection task, significantly reduces the computational load and misjudgment risk of the target detection model, and thus significantly improves the accuracy and efficiency of detecting minute impurities in industrial excipients. Attached Figure Description
[0018] The above and other objects, features, and advantages of exemplary embodiments of this application will become readily understood by reading the following detailed description with reference to the accompanying drawings. In the drawings, several embodiments of this application are illustrated by way of example and not limitation, and the same or corresponding reference numerals denote the same or corresponding parts, wherein: Figure 1 An exemplary flowchart of a method for detecting excipients according to some embodiments of this application is shown; Figure 2 Exemplary flowcharts of methods for detecting excipients according to other embodiments of this application are shown; Figure 3 An exemplary flowchart of a method for testing excipients according to some other embodiments of this application is shown; Figure 4 Exemplary block diagrams of electronic devices according to some embodiments of this application are shown. Detailed Implementation
[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0020] It should be understood that the terms "comprising" and "including" used in the specification and claims of this application indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.
[0021] It should also be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application. As used in this specification and claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this specification and claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations.
[0022] As used in this specification and claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."
[0023] The specific embodiments of this application will now be described in detail with reference to the accompanying drawings.
[0024] Figure 1 Exemplary flowcharts of methods for detecting excipients according to some embodiments of this application are shown. Figure 1 As shown, the method 100 may include: in step S102, acquiring an original image of the excipient to be detected; in step S104, using a segmentation model to segment the original image to extract an effective region in the original image, the effective region including a solid region or a liquid region; in step S106, using a target detection model to detect impurities in the effective region to obtain the impurity detection result of the excipient to be detected.
[0025] The aforementioned raw images are typically images taken with an industrial camera, containing the excipient to be tested and its container. The excipient to be tested can be a liquid or a solid excipient. In some embodiments, in step S102, the raw image acquisition process can preferably employ a standardized image acquisition scheme. This image acquisition scheme is performed under controlled environmental conditions, using a specified industrial camera and light source, in conjunction with a standardized container, to acquire raw image data of the sample according to fixed shooting parameters, thereby establishing a detection benchmark at the data source.
[0026] In a specific example, in step S102, the image acquisition device can be, for example, a Hikvision MV-CA050-10GM industrial camera (e.g., with a resolution of up to 5 megapixels) and a Dehong DH-LED-6060 diffuse light source. To further reduce the impact of external environmental fluctuations on image quality, the acquisition environment can be controlled within the range of temperature 20℃~25℃ and humidity 40%~60%, and preferably, shooting can be performed under conditions free from strong external light interference. Through the above environmental and equipment settings, external variables can be effectively controlled, ensuring strong comparability and consistency of the original images acquired between different batches.
[0027] Specifically, for the acquisition of raw images of liquid excipients, the acquisition configuration can be further refined to minimize the interference of liquid refraction and reflection on image features. In some embodiments, the liquid excipient can be placed in a 50mL transparent borosilicate beaker without a background color, and black light-absorbing paper can be attached to the outer wall of the beaker to reduce the reflection of ambient light from the side; preferably, a diffuse integrating sphere light source is used, and images are acquired at a 45° side lighting angle and a fixed shooting distance of 10cm. In this way, the high light reflection of the liquid surface can be effectively reduced.
[0028] The configuration for acquiring raw images of solid excipients differs from that for liquids. Solid excipients can be placed on a 10cm white ceramic tray without any impurities, and the surface of the tray is preferably matte to eliminate specular reflections. In terms of shooting angle, a diffused top light source is preferred, and images are acquired at a vertical shooting angle and a fixed shooting distance of 15cm.
[0029] Through the aforementioned differentiated configurations, the original images can maximize the preservation of the inherent visual features of the auxiliary materials while effectively mitigating the impact of adverse factors such as uneven surface texture of solid auxiliary materials, tray material reflection, and high-gloss reflection of liquid auxiliary materials. Thus, the acquisition of the original images lays a solid data foundation for the subsequent semantic segmentation process.
[0030] After acquiring the original image data, the process proceeds to step S104, where a segmentation model is used to perform segmentation processing on the original image, thereby extracting the effective region from the original image. In this step, the effective region refers to the image region that includes only the solid excipient region or the liquid excipient region. The introduction of the segmentation model aims to accurately separate the background and the container from the original image, thus providing a clean detection input for subsequent impurity detection. By inputting the original image into the segmentation model, a binary mask can be generated (e.g., the pixels of the effective region are set to 1, and the pixels of the background region are set to 0), thereby achieving accurate segmentation of the effective region and the background region (including the container and the background of the original image).
[0031] In some embodiments, the segmentation model may employ a deep learning-based Unet network architecture, including a downsampling network, a bottleneck layer, an upsampling network, and an output layer, wherein the downsampling network includes multiple convolutional layers and pooling layers, and the upsampling network includes multiple deconvolutional layers and skip connection layers; and before using the segmentation model for segmentation processing, method 100 may further include: using a sample dataset with effective region annotations, using DiceLoss as the loss function, freezing the downsampling network and the bottleneck layer, and fine-tuning the upsampling network.
[0032] Specifically, the Unet network structure can include a downsampling network (also called an encoder) for progressively compressing the image spatial dimension and extracting abstract semantic features, a bottleneck layer for connecting contextual information, an upsampling network (also called a decoder) for restoring low-resolution feature maps to the original image resolution, and an output layer for outputting the final binary mask. The downsampling network typically consists of alternating layers of convolutional and pooling layers, while the upsampling network consists of multiple deconvolutional layers, with skip connections introduced between corresponding feature maps of the same scale. The introduction of skip connections allows the upsampling network to fuse the shallow features retained in the downsampling network with deep semantic features, thereby significantly improving the pixel-level segmentation accuracy for locating the effective edge of the detection material.
[0033] In a specific example, the downsampling network may contain four convolutional layers and pooling layers, where the convolutional layers may use 3×3 convolutional kernels and ReLU activation functions, and the pooling layers may use 2×2 max pooling with a stride of 2; the bottleneck layer may be implemented using a single convolutional layer; the upsampling network may contain, for example, four deconvolutional layers and skip connection layers, where the deconvolutional layers may use 2×2 deconvolution; and the output layer may be implemented using a sigmoid function.
[0034] Before the segmentation model is officially deployed, it needs to be fine-tuned to adapt to the industrial auxiliary material detection scenario. The specific implementation of this fine-tuning training is as follows: First, a sample dataset containing various types of auxiliary materials is constructed, and the solid or liquid regions in each sample image in the dataset are labeled at the pixel level (the effective region is set as the foreground, and the remaining regions as the background). Considering that auxiliary material regions often dominate the image, with a relatively balanced foreground and background, but requiring extremely high boundary accuracy, DiceLoss is preferred as the loss function during this fine-tuning training process. In terms of training strategy, based on the principle of transfer learning, this implementation prefers to freeze all parameters of the downsampling network and bottleneck layer used to extract general image features, focusing only on the upsampling network and skip connection layers for training optimization. This layered fine-tuning strategy of "freezing the backbone and fine-tuning the tail" avoids destroying the excellent feature extraction capabilities of the pre-trained model during the fine-tuning process, while enabling the model to quickly adapt to the fine segmentation task of the effective auxiliary material region with extremely low training overhead, outputting a binary mask with the same resolution as the original image.
[0035] In a specific example, firstly, image data covering 21 types of liquid excipients and 7 types of solid excipients can be collected as a sample dataset. To ensure the robustness of the model under different operating conditions, the number of images for each sample type is no less than 300, and these images include different containers, lighting conditions, and background scenes. Then, all the collected images are pixel-level labeled. During the labeling process, the areas containing liquid or solid excipients are marked as foreground categories (i.e., pixel value 1), and the areas containing the background and containers are marked as background categories (i.e., pixel value 0). After labeling, the data is divided into training, validation, and test sets according to a preset data partitioning strategy, for example, allocated in proportions of 70%, 20%, and 10%, respectively, for subsequent model parameter updates, hyperparameter optimization, and objective evaluation of the final model performance.
[0036] Regarding fine-tuning parameter settings, to ensure efficient and stable model convergence, a specific optimization configuration can be adopted in this example. Specifically, the initial learning rate can be set to 0.001 during training, the adaptive moment estimation (Adam) optimizer can be used for parameter updates, and the weight decay coefficient can be set to 0.0001 to constrain model complexity and prevent overfitting. Provided the hardware memory allows, the number of images processed by the model each time (i.e., the batch size) is set to 8. In terms of fine-tuning strategy, this embodiment adopts a hierarchical training method based on transfer learning, specifically: freezing all parameters of the downsampling network (i.e., the backbone network) of the segmentation model, and only performing backpropagation and updating of the network parameters of the upsampling network (including skip connection layers).
[0037] Regarding the design of the loss function during the fine-tuning process, this embodiment considers the distribution characteristics of the effective area of the auxiliary material and the background area in the image. To address the potential class imbalance problem in the segmentation task (although the effective area is relatively large, boundary accuracy requirements must also be considered), Dice Loss is used as the optimization objective of the segmentation model. The mathematical expression of this loss function is as follows: (Formula 1); In the above formula, Y represents the image mask with real annotation (i.e., the pixel-level real result annotated by humans). The symbol represents the predicted mask output by the segmentation model, ∩ represents the pixel-level intersection operation between the ground truth mask and the predicted mask, and ∣ represents the pixel-level intersection operation between the ground truth mask and the predicted mask. | indicates the number of pixels within the calculated mask region. This loss function guides the segmentation model to maintain high accuracy at edges by directly optimizing the overlap between the predicted and ground truth regions, thereby maximizing the intersection-union ratio (IUU) between the predicted and ground truth masks.
[0038] After training on the specific dataset and fine-tuning strategy described above, the segmentation model exhibits excellent performance metrics. Validation results on the test set show that the pixel-level segmentation accuracy of this model can reach over 99.5%, and the Intersection over Union (IoU) ratio is not less than 0.98, enabling reliable pixel-level accurate extraction of liquid or solid regions of the excipient to be detected.
[0039] During the actual inference and output stages, the segmentation model directly outputs a binary mask matrix with the same resolution as the input original image. In this binary mask matrix, regions with a pixel value of 1 represent the effective liquid or solid regions to be extracted, while regions with a pixel value of 0 correspond to the background or container regions that have been removed. All subsequent impurity detection, counting, and other processing steps will be performed based on this binary mask matrix, only targeting the effective regions with a mask value of 1.
[0040] Before the segmentation model in this embodiment of the application is fine-tuned, its basic network parameters can be obtained by pre-training on a large-scale image classification task on a general massive image dataset (such as the ImageNet dataset) so that it has the ability to extract the underlying general semantic features of the image.
[0041] Subsequently, in step S106, this embodiment of the application abandons the conventional method of directly analyzing the entire original image, and instead inputs the extracted clean and effective area into the target detection model, which then performs impurity detection on the effective area to finally obtain the impurity detection result of the excipient to be detected, such as the individual count or region location of the impurities.
[0042] Because impurities are typically small in size and irregular in shape, target detection models need to maintain high accuracy within a limited effective area. One implementation method for a target detection model will be described below.
[0043] In some embodiments, the above-described object detection model may employ a YOLO network architecture, including a feature extraction network, a feature fusion network, and an output network for bounding box regression, category classification, and impurity counting; and before using the object detection model to perform impurity detection, method 100 may further include: freezing the feature extraction network, training the feature fusion network and the output network using a joint loss function, the joint loss function being obtained by weighted summation of bounding box regression loss, category classification loss, and impurity counting regression loss.
[0044] Specifically, the object detection model can adopt the YOLO lightweight network architecture, which consists of three main parts: a feature extraction network (Backbone) for extracting effective region image features, a feature fusion network (Neck) for performing multi-scale feature fusion, and an output network (Head) for finally outputting bounding box regression information, category classification information, and impurity count information. More specifically, in this example, the feature extraction network can use structures such as Cross Stage Partial (CSP) and Spatial Pyramid Pooling (SPP) for feature extraction, while the feature fusion network preferably adopts a structure combining a Path Aggregation Network (PAN) and a Feature Pyramid Network (FPN) to ensure cross-scale feature response to impurities of different sizes.
[0045] Before performing impurity detection, the target detection model needs to be fine-tuned specifically for the task of detecting minute impurities in the excipients under test, using an impurity dataset with effective region impurity annotations. The preferred fine-tuning strategy is to construct an impurity dataset based on images of the effective region, annotating the impurity bounding boxes (covering ≥90% of the impurity area), impurity category labels, and image-level impurity count labels. During training, the feature extraction network (Backbone) used to extract general image low-level features is frozen, and parameters are updated only for the feature fusion network (Neck) and output network (Head) responsible for impurity localization, classification, and counting. This significantly reduces the number of parameters that need to be trained and effectively prevents the risk of overfitting the model on small training sets.
[0046] During fine-tuning training, the object detection model is optimized using a multi-task joint loss function. This joint loss function is obtained by weighted summation of three components: bounding box regression loss, class classification loss, and impurity count regression loss. Specifically, the bounding box regression loss preferably uses the CIoU (Complete Intersection over Union) loss function, which considers not only the intersection-over-union ratio of the predicted and ground truth boxes but also the consistency of center point distance and aspect ratio, thereby improving the accuracy of locating small impurity bounding boxes. The class classification loss preferably uses the Focal loss function, which guides the model to focus on difficult-to-classify impurity samples by reducing the weight of easily classified negative samples, effectively solving the sample imbalance problem between impurities and background. The impurity count regression loss preferably uses the L1 (Mean Absolute Error) loss function, which significantly improves the regression accuracy of the total number of impurities by directly constraining the deviation between the model's predicted impurity count and the ground truth label.
[0047] In some specific embodiments, the weighting coefficients for the three losses can be 0.5, 0.3, and 0.2, respectively, to balance the optimization magnitude of each task. Specifically, the joint loss function is calculated using the following formula: (Formula 2); Where α, β, and γ are the weighting coefficients of each sub-loss function, and their preferred values are respectively... =0.5, β=0.3, γ=0.2; L CIoU For bounding box regression loss, L Focal For category classification loss, L L1 This represents the regression loss due to impurity counts.
[0048] The bounding box regression loss mentioned above can be determined by the following formula: (Formula 3); Where IoU represents the intersection-union ratio, The predicted bounding box represents the Euclidean distance between the center points of the predicted bounding box and the ground truth bounding box; c represents the length of the diagonal of the smallest rectangle enclosing the two boxes; v represents the aspect ratio consistency parameter; α v This represents the balance coefficient.
[0049] The classification loss for the above categories can be determined by the following formula: (Formula 4); Where, p i γ represents the predicted category probability of the i-th impurity in the effective region, γ represents the focusing coefficient (usually 2), and N represents the number of impurities.
[0050] The above-mentioned impurity count regression loss can be determined by the following formula: (Formula 5); Where N represents the number of impurities predicted by the target detection model, N gt This indicates the actual number of impurities.
[0051] In a specific example, an impurity dataset is constructed containing images of impurities from 21 types of liquid excipients and 7 types of solid excipients, with the ratio of qualified to unqualified sample images controlled at 2:1. After completing the impurity labeling (including impurity bounding boxes, impurity category labels, and impurity count labels), the dataset is divided into a training set, a validation set, and a test set according to a preset ratio, preferably 70%, 20%, and 10%, respectively.
[0052] Regarding fine-tuning parameter settings and training strategies, to ensure the model can converge stably and efficiently to the industrial impurity detection task, this embodiment adopts a phased decay learning rate optimization scheme. Specifically, the initial learning rate for this fine-tuning training can be set to 0.00065, and it decreases in three stages during the training iteration process: at the 30th training epoch, the learning rate drops to 0.0005; at the 60th training epoch, the learning rate further decreases to 0.00038. Furthermore, the training batch size is set to 8, the weight decay coefficient is 0.0001, and a gradient clipping strategy is adopted, setting the gradient clipping threshold to 1.0 to prevent gradient explosion and ensure the stability of the training process. In terms of the fine-tuning strategy, based on the principle of transfer learning, all parameters of the feature extraction network (Backbone) are frozen, and only the parameters of the feature fusion network (Neck) and the output network (Head) are optimized and updated. This hierarchical fine-tuning strategy can effectively utilize the excellent general feature extraction capabilities of the pre-trained model, while concentrating training resources on specific tasks such as impurity detection, localization, and counting.
[0053] After fine-tuning and training using the aforementioned dataset configuration and parameter optimization strategies, the object detection model demonstrated excellent overall performance on the test set. Testing verified that the model's mean accuracy (mAP50, i.e., the average accuracy when the intersection-union ratio threshold is 0.5) is no less than 0.26, and the mAP50-95 (i.e., the average accuracy in the range of intersection-union ratio thresholds from 0.5 to 0.95) is no less than 0.07; its precision is no less than 0.28, and its recall is no less than 0.39. In terms of counting accuracy, the object detection model's prediction error for the number of impurities is controlled within one, while the inference computation time for a single image does not exceed 20 milliseconds, balancing the high accuracy and high efficiency requirements of industrial detection scenarios.
[0054] In the impurity detection and counting inference stage for practical detection tasks, the image of the aforementioned effective region is input into the finely tuned target detection model. This model extracts a 512-dimensional image feature vector through its feature extraction network (Backbone), and then performs multi-scale feature fusion through the feature fusion network (Neck) to accommodate the feature representation of impurities of different sizes. Finally, the output network (Head) outputs results including impurity bounding box coordinates, class probabilities, and the predicted number of impurities. In the output stage, the target detection model performs confidence filtering on all prediction results, retaining only detection boxes with a confidence score of at least 0.5 as valid detection results. Finally, it outputs the total number N of impurities in the excipient to be detected, along with spatial information such as the location and size of each impurity, for subsequent comprehensive judgment.
[0055] Understandably, before the fine-tuning training described above, the basic network parameters of the object detection model can be obtained by performing basic pre-training on a large general object detection dataset (such as the MS COCO dataset), so that it has preliminary general image feature extraction and basic object perception capabilities before fine-tuning.
[0056] The target detection model, after being finely tuned and optimized as described above, can accurately output the number of impurities and the position of the bounding box within the effective area in milliseconds, thus completing the automated detection of impurities in auxiliary materials.
[0057] The solution provided in this application first introduces a region segmentation mechanism at the process level, using a fine-tuned Unet model to accurately obtain physically meaningful effective detection regions. Then, a YOLO target detection model specifically fine-tuned for tiny impurities is introduced, limiting the impurity detection task to a clean, effective region. This dual-model collaborative processing architecture effectively isolates irrelevant interference items such as container boundaries and background regions, avoiding redundant calculations and inferences across the entire image. While significantly reducing model computational overhead, it ensures the accuracy of subsequent key indicators such as impurity detection and counting from the data processing source.
[0058] Figure 2 Exemplary flowcharts of methods for detecting excipients according to other embodiments of this application are shown. Figure 2 As shown, the method 200 includes: in step S201, acquiring an original image of the excipient to be detected; in step S202, using a segmentation model to segment the original image to extract the effective region in the original image; in step S203, performing preprocessing on the image of the effective region to obtain a preprocessed image; and in step S204, using a target detection model to detect impurities in the preprocessed image to obtain the impurity detection result of the excipient to be detected.
[0059] Steps S201 and S202 are combined with those in the preceding text. Figure 1 Steps S102 and S104 are described as being the same or similar, and will not be repeated here. The following will combine... Figure 2 Steps S203 and S204 are described in detail.
[0060] The specific implementation of step S203 involves performing further refined preprocessing on the image of the effective region based on the binary mask output by the aforementioned segmentation model. All operations in this preprocessing are strictly limited to the effective liquid or solid regions where the mask value is 1, thereby fundamentally preventing the reintroduction of data noise from irrelevant regions such as the background or container due to preprocessing operations. Specifically, this preprocessing operation includes, but is not limited to, one or more of the following: image normalization, mask-based background removal, generation of reflective region masks for liquid excipients, and data augmentation processing to address the differences in characteristics between solid and liquid excipients.
[0061] In some embodiments, step S203 may include image normalization and background removal operations. To ensure strict consistency in the spatial dimension and numerical distribution of image data in the subsequent object detection model, the effective region's image can first undergo a standardized format conversion, including but not limited to adjusting the image resolution to the target pixel (e.g., 640×640 pixels), ensuring its color space is in standard RGB format, and normalizing the pixel value range to the [0,1] interval. Based on this, combined with the binary mask output by the aforementioned segmentation model, pixel-level multiplication or mask indexing operations are used to directly remove background and container region pixels with a mask value of 0, retaining only the effective liquid or solid region pixels with a mask value of 1. This combined operation effectively eliminates interference from the shooting device and physical environment, providing unified and clean input data for subsequent detection.
[0062] In other embodiments, when the effective area is a liquid area (i.e., for the detection scenario of liquid excipients), step S203 may include: locating the reflective area in the effective area and generating a secondary mask by means of an adaptive threshold algorithm to shield the interference of refracted reflection on subsequent detection.
[0063] Adaptive thresholding algorithms are image processing methods that utilize local thresholds instead of global thresholds. In this embodiment, an adaptive thresholding algorithm (e.g., Otsu's method) is used to automatically identify and locate bright reflective areas within the effective liquid region of the image. Subsequently, a two-level binary mask matrix is constructed, setting the pixel values corresponding to reflective areas to 0 and the pixel values of effective non-reflective areas to 1. By superimposing this two-level mask, potential refracted reflections in the effective region can be shielded from the source during feature extraction and data calculation stages, preventing them from being misjudged as impurities in subsequent target detection model detection processes. This ensures the accuracy of subsequent detection counting and the objectivity of appearance indicators such as color and transparency.
[0064] Furthermore, in some other embodiments, step S203 may include: performing data augmentation processing on the effective region, wherein in response to the effective region being a solid region, a Gaussian blur texture perturbation process is performed on the effective region; and in response to the effective region being a liquid region, at least one data augmentation process including angle adjustment, brightness adjustment, and contrast adjustment is performed.
[0065] When the effective region is a solid material, given the inherent texture distribution characteristics of solid surfaces, a slight Gaussian blur (e.g., Gaussian kernel standard deviation σ=0.01) can be applied to the image to prevent the model from overfitting specific texture patterns, thus simulating subtle perturbations in solid textures under external conditions. When the effective region is a liquid material, considering that the liquid form is easily affected by factors such as container position and light source angle, multi-angle feature fusion enhancement techniques can be used. Specifically, this may include, for example, performing rotation within a ±15° angle range and fine-tuning brightness and contrast within a ±10% range. Through the above differentiated enhancement strategies, various working conditions that may actually be encountered in industrial production can be effectively simulated, significantly improving the robustness and detection stability of the model when facing complex production scenarios.
[0066] Following the preprocessing in step S203, in step S204, the target detection model can be used to detect impurities in the preprocessed image of the effective region. Because the preprocessing operation in step S203, in addition to isolating the liquid-solid effective region from the background, further utilizes key technologies such as reflective shielding, data standardization, and differential enhancement to comprehensively purify the image features within the effective region and significantly expand the data distribution space of the limited samples. This ensures that the data subsequently input into the target detection model and appearance inspection possess both high purity and strong generalization ability, laying a solid data foundation for the final realization of high-precision, high-stability fully automated detection of industrial auxiliary materials.
[0067] Combined with the preceding text Figure 1 and Figure 2 This application details the basic procedures for extracting the effective area of excipients, pretreatment, and impurity detection in its embodiments. In actual industrial production quality control, the appearance quality indicators of excipients, such as chroma (degree of color deviation) and transparency (uniformity of liquid light transmission), are also key factors determining product qualification. Traditional detection methods rely on manual visual observation and description using natural language (such as "colorless and transparent," "light yellow"), lacking objective numerical support and easily leading to misjudgments due to human differences. To further address this issue, this application embodiment further provides... Figure 3 The method shown achieves objectivity, standardization, and automation in the appearance inspection of auxiliary materials.
[0068] Figure 3Exemplary flowcharts of methods for detecting excipients according to further embodiments of this application are shown. Figure 3 As shown, method 300 may include: in step S301, acquiring the original image of the excipient to be detected; in step S302, using a segmentation model to segment the original image to extract the effective region in the original image; in step S303, using a target detection model to detect impurities in the effective region to obtain the impurity detection result of the excipient to be detected; in step S304, using the color features of a preset standard excipient image as a reference feature, calculating the normalized comprehensive deviation between the color features of the effective region image and the reference features; in step S305, comparing the normalized comprehensive deviation with a preset deviation threshold to determine whether the chromaticity and / or transparency of the excipient to be detected meet the standard.
[0069] It is understandable that, such as Figure 3 As shown, while obtaining impurity results through target detection in step S303, this embodiment of the application can execute steps S304 and S305 in parallel to perform quantitative analysis of color and / or transparency. In other embodiments, steps S304 and S305 can be executed after step S303. Steps S301 to S303 are respectively combined with the foregoing... Figure 1 Steps S102, S104, and S106 are the same or similar, and will not be repeated here.
[0070] In step S304, the image of the effective region can originate from the mask region directly output by the segmentation model, or it can be obtained through... Figure 2 The preprocessing steps (such as background removal, reflective area masking, data augmentation, etc.) clean the preprocessed image to ensure the quality of the underlying data during feature extraction.
[0071] In this embodiment, the standard excipient image refers to a benchmark sample image taken under standardized image acquisition conditions (such as specific light source, shooting angle, container configuration, and environmental control), and which has been verified by production process quality and whose appearance indicators (color and transparency) fully meet the requirements of qualified product specifications. It serves as the "reference benchmark" or "gold standard" for comparison and judgment in all subsequent testing processes. The color feature is an objective numerical indicator extracted based on the effective area output by the aforementioned segmentation model (after removing background, container, and reflective interference) by performing mathematical statistics and color space transformation on all pixels within the effective area. The color feature may specifically include, for example, the RGB mean (reflecting the overall hue) and RGB range (characterizing the uniformity of color distribution) in the RGB color space, and the mean values of L (brightness), a, and b (chromaticity coordinates) in the CIELAB color space, which is closer to human visual perception; for liquid excipients, it may also include the mean brightness, standard deviation, and relative standard deviation (the ratio of standard deviation to mean) calculated from its grayscale image. This application embodiment defines the color features extracted from the standard excipient image as reference features, and calculates the normalized comprehensive deviation between the features of the sample to be tested and the reference features in subsequent steps, thereby transforming the fuzzy sensory judgment into a quantifiable numerical comparison, and realizing the objectification and standardization of excipient appearance inspection.
[0072] When calculating the normalized overall deviation, the corresponding feature vector and deviation calculation method differ depending on whether the excipient to be tested is solid or liquid. In some embodiments, when the excipient to be tested is solid or liquid, the normalized overall deviation includes the colorimetric overall deviation. Specifically, the colorimetric overall deviation is obtained by simultaneously calculating the deviation between the effective area to be tested and the reference feature in the RGB three-channel mean, the deviation in the L value (lightness) and AB value (chromaticity coordinates) in the CIELAB color space, and the deviation in the RGB range sum (reflecting the uniformity of color distribution). Finally, the above four deviation components are averaged to obtain the colorimetric overall deviation. This colorimetric overall deviation reflects the degree of deviation of the sample in overall color perception.
[0073] Specifically, in some embodiments, calculating the normalized comprehensive deviation between the color features of the effective region image and the reference features includes: determining the RGB mean, RGB range, and CIELAB mean of the pixels in the effective region image; and determining the colorimetric comprehensive deviation of the excipient to be detected based on the RGB mean, RGB range, CIELAB mean, and reference features; wherein the colorimetric comprehensive deviation of the excipient to be detected is determined in the following manner: (Formula 6); in, Indicates overall color deviation. This represents the deviation of the effective region from the RGB mean of the reference feature. This represents the L-value deviation between the effective region and the reference feature. This indicates the AB value deviation between the effective region and the reference feature. F represents the RGB range and deviation between the effective region and the reference feature. ref F represents the reference feature. test The color characteristics representing the effective area, This represents the k-mean in the reference feature. This represents the k-mean of the effective region. This represents the mean L value in the reference feature. The mean L value represents the effective region. This represents the sum of the RGB ranges in the reference feature. This represents the sum of the RGB ranges of the effective region.
[0074] In Formula 6, all deviations are normalized in the [0,1] interval based on the maximum nominal value of the corresponding color space (e.g., RGB is 255, L is 100, AB is 128, and the sum of the RGB ranges is 765), so that color differences under different dimensions can be compared uniformly.
[0075] Assume the set of pixels in the effective region of the liquid / solid is P = p1, p2, ..., p n , where p i =(R i G i B i Let ) be the RGB value of the i-th pixel, and n be the number of pixels in the effective area. The color characteristics can be determined by the following formula: (Formula 7); in, This represents the RGB mean.
[0076] (Formula 8); in, This represents the sum of the RGB ranges, max() represents the maximum value, and min() represents the minimum value.
[0077] The standard conversion formula for RGB to CIELAB (RGB values are first normalized to [0,1]) is as follows: (Formula Nine); (Formula 10); (Formula Eleven); Where Xn=95.047, Yn=100.000, and Zn=108.883 are the tristimulus values of standard white light. The final CIELAB mean was calculated. =( , , ), L∈[0,100] (luminance), a,b∈[-128,128] (chrominance).
[0078] When the excipient to be tested is a liquid, in addition to the colorimetric deviation, a transparency deviation is specifically introduced into the normalized comprehensive deviation. The transparency characteristic is derived by analyzing the grayscale distribution of the liquid region: first, the image of the effective area is converted into a grayscale image; then, the mean brightness, standard deviation of brightness, and relative standard deviation (the ratio of the mean brightness to the standard deviation of brightness), used to characterize transparency, are calculated. The smaller the relative standard deviation, the more uniform the brightness distribution within the liquid region, and the higher its transparency. Based on the mean brightness, standard deviation of brightness, relative standard deviation, and reference characteristics, the comprehensive transparency deviation of the excipient to be tested is determined. For example, by calculating the mean brightness deviation and relative standard deviation between the effective area of the excipient to be tested and the image of the standard excipient, the average of these two deviations can be used to obtain the comprehensive transparency deviation. This indicator can objectively quantify the clarity of the liquid.
[0079] In some embodiments, the overall transparency deviation of the excipient to be tested is determined in the following manner: (Formula 12); in, Indicates the overall deviation in transparency. This indicates the deviation of the mean brightness of the effective area from that of the reference feature. This represents the relative standard deviation of the effective region from the reference feature. This represents the mean brightness value in the reference feature. This represents the average brightness of the effective area. This represents the relative standard deviation in the reference characteristic. This represents the relative standard deviation of the effective region.
[0080] The relative standard deviation is determined by the ratio of the standard deviation of luminance to the mean of luminance, for example, as shown in the following formula: (Formula Thirteen); in, Indicates relative standard deviation. Indicates the standard deviation of brightness. This represents the average brightness.
[0081] Convert an RGB image to a grayscale image G=g1,g2,...,g n g iFor values ∈ [0, 255], the mean luminance and standard deviation of luminance are determined by the following formulas: (Formula Fourteen); (Formula 15); Among them, g i Let represent the grayscale value of the i-th pixel, and n represent the number of pixels in the grayscale image. Indicates the standard deviation of brightness. This represents the average brightness.
[0082] Further, in step S305, the aforementioned overall color deviation and / or overall transparency deviation are compared with preset deviation thresholds to determine whether the appearance of the excipient to be tested meets the standard. Specifically, differentiated acceptance criteria can be set for different types of excipients: for example, for solid excipients, it is only necessary to determine whether the overall color deviation is less than or equal to a preset color deviation threshold (e.g., 0.08); while for liquid excipients, both the overall color deviation and the overall transparency deviation must be less than or equal to the color deviation threshold (e.g., 0.10). If all deviations are within the threshold range, the color and / or transparency of the excipient are determined to meet the standard; otherwise, it is determined to be non-compliant.
[0083] In other embodiments, for liquid excipients, the overall deviation can be determined by weighted summation of color deviation and transparency deviation; and the appearance of the excipient to be tested can be judged to meet the standard by comparing the overall deviation with a preset overall deviation threshold.
[0084] For example, in some embodiments, the overall deviation can be determined by the following formula: (Formula Sixteen); in, Indicates the overall deviation. Indicates overall color deviation. This indicates the overall deviation in transparency.
[0085] By using the above-mentioned objective and numerical quantitative judgment method, the embodiments of this application effectively solve the drawbacks of strong subjectivity and poor consistency caused by traditional manual judgment based on experience. Figure 3 The illustrated method transforms the detection results of color and transparency into quantifiable objective data (such as deviation values) rather than vague textual descriptions. Furthermore, since steps S303 and S304 / S305 can employ parallel processing logic, the target detection model can simultaneously perform the calculation and judgment of appearance features while inferring impurities, further reducing the overall detection time and effectively meeting the dual requirements of high efficiency and standardization for industrial batch inspection.
[0086] In some other embodiments of this application, after completing the appearance judgment of impurity detection counting and color feature quantification of the aforementioned target detection model, the method of this application embodiment may further include the steps of comprehensively evaluating the above-mentioned detection results and automatically generating standardized reports and connecting with the laboratory system, thereby forming a complete auxiliary detection closed loop.
[0087] Specifically, this comprehensive evaluation step is performed based on preset excipient testing standards, which cover impurity count thresholds and appearance judgment thresholds. The system integrates the impurity count results output by the aforementioned target detection model with the color / transparency judgment results output by the appearance quantification module, and performs differentiated compliance judgment logic according to the excipient type. For liquid excipients, if the total number of impurities N is less than the preset threshold (e.g., N<10), and both the color and transparency judgment results meet the standards, the liquid excipient sample is judged as qualified; otherwise, it is judged as unqualified. For solid excipients, if the color judgment meets the standards and there are no significant foreign impurities (i.e., impurity count N=0), the solid excipient sample is judged as qualified; otherwise, it is judged as unqualified.
[0088] After completing the comprehensive qualification assessment, the system automatically generates a structured test report based on the aforementioned data. This report includes the sample's identification information (such as sample name and sample code), the quality standards used in this test, detailed results of impurity testing (including the total number of impurities N and the spatial location information of each impurity), and quantitative results of the appearance inspection (including RGB mean, CIELAB mean, and colorimetric deviation Color). dev Transparency Overall Deviation dev (Including the results of each individual assessment). The report ultimately provides a clear conclusion on the overall conformity of the sample, ensuring the data integrity and traceability of the entire testing process.
[0089] To achieve informatization and digitalization of industrial testing processes, this application also provides a seamless integration mechanism with a Laboratory Information Management System (LIMS). The system uniformly encapsulates the generated structured test reports according to standard interactive data formats (such as JSON or Excel) and automatically pushes them to the LIMS system through the corresponding data interface. Through this automated integration process, test data can be automatically entered, securely stored, and centrally managed, thus providing solid and reliable data support for subsequent quality statistical queries, big data analysis, and problem tracing. This significantly reduces the error rate caused by manual data entry and improves the informatization level of enterprise quality control.
[0090] Figure 4 Exemplary block diagrams of electronic devices according to some embodiments of this application are shown. Figure 4As shown, the electronic device 400 may include: a processor 410 configured to execute program instructions; and a memory 420 configured to store program instructions that, when loaded and executed by the processor 410, cause the processor 410 to perform the aforementioned combination. Figures 1-3 The method described in any one of them.
[0091] The processor 410 serves as the central processing unit or graphics processing unit (e.g., an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a general-purpose central processing unit (CPU)) of the electronic device 400. It is connected to the memory 420 via a data bus and is configured to execute various program instructions. The memory 420 is used for long-term or short-term storage of the aforementioned program instructions, pre-trained network model weight parameters (such as the fine-tuning parameters of the segmentation model and the target detection model mentioned above), preset chromaticity and transparency deviation thresholds, a reference feature library of standard auxiliary material images, and the acquired raw images and intermediate calculation data.
[0092] In actual operation, when the electronic device 400 is powered on and receives an external trigger signal (e.g., receiving the original image of the material to be inspected from an industrial camera), the program instructions stored in the memory 420 are loaded and executed by the processor 410. When running these program instructions, the processor 410 sequentially executes the aforementioned steps according to the time sequence logic: image acquisition, segmentation model inference, effective region extraction, background and reflective interference removal, target detection model impurity counting, color and transparency feature quantification analysis, normalized comprehensive deviation calculation, pass / fail determination, and generation of a structured inspection report. Through this coordinated operation of hardware and software, the processor 410 can quickly convert the complex underlying matrix operations and model inference processes into quantifiable inspection conclusions.
[0093] In summary, the electronic device provided in this application, by embedding software programs including multi-model collaborative algorithms, quantitative analysis processes, and standard judgment rules into its memory and processor, enables the electronic device to automatically and stably complete various testing tasks for industrial auxiliary materials without human intervention. This electronic device not only possesses low single-inference latency but can also directly interface with information systems such as LIMS in industrial settings, thereby ensuring the efficient implementation and operation of the entire intelligent testing method at the hardware level.
[0094] It should also be understood that any module, unit, component, server, computer, terminal, or device of the executable instructions in this application may include or otherwise access computer-readable media, such as storage media, computer storage media, or data storage devices (removable) and / or non-removable) such as disks, optical discs, or magnetic tapes. Computer storage media may include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data.
[0095] Based on the foregoing, this application also provides a computer-readable storage medium storing computer-readable instructions thereon, which, when executed by one or more processors, implement the above-described combination of instructions. Figures 1-3 The method described in any of the embodiments.
[0096] Computer-readable storage media can be any suitable magnetic or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), etc., or any other medium that can be used to store desired information and can be accessed by an application, module, or both. Any such computer storage medium can be part of a device or accessible to or connected to a device. Any application or module described in this invention can be implemented using computer-readable / executable instructions that can be stored or otherwise maintained by such a computer-readable medium.
[0097] While numerous embodiments of this application have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Many modifications, alterations, and alternatives will arise for those skilled in the art without departing from the spirit and intent of this application. It should be understood that various alternatives to the embodiments of this application described herein may be employed in the practice of this application. The appended claims are intended to define the scope of protection of this application and therefore cover equivalents or alternatives within the scope of these claims.
Claims
1. A method for detecting excipients, characterized in that, include: Acquire the original image of the excipient to be tested; The original image is segmented using a segmentation model to extract effective regions from the original image, including solid or liquid regions. The target detection model is used to detect impurities in the effective area to obtain the impurity detection results of the excipient to be tested.
2. The method according to claim 1, characterized in that, The segmentation model adopts the Unet network architecture, which includes a downsampling network, a bottleneck layer, an upsampling network, and an output layer. The downsampling network includes multiple convolutional layers and pooling layers, and the upsampling network includes multiple deconvolutional layers and skip connection layers. And before performing the segmentation process using the segmentation model, the method further includes: Using a sample dataset with valid region annotations, and with DiceLoss as the loss function, the downsampling network and the bottleneck layer are frozen, and the upsampling network is fine-tuned and trained.
3. The method according to claim 1, characterized in that, The target detection model adopts the YOLO network architecture, including a feature extraction network, a feature fusion network, and an output network for bounding box regression, category classification, and impurity counting; Furthermore, before using the target detection model for impurity detection, the method further includes: The feature extraction network is frozen, and the feature fusion network and the output network are trained using a joint loss function, which is obtained by weighted summation of bounding box regression loss, category classification loss and impurity count regression loss.
4. The method according to claim 3, characterized in that, The joint loss function is calculated using the following formula: ; in, =0.5, β=0.3, γ=0.2; L CIoU For bounding box regression loss, L Focal For category classification loss, L L1 This represents the regression loss due to impurity counts.
5. The method according to claim 1, characterized in that, Before using the object detection model to detect impurities in the effective region, the method further includes performing at least one of the following preprocessing steps on the image of the effective region to obtain a preprocessed image, which the object detection model then uses to detect impurities in the preprocessed image: (1) When the effective area is a liquid area, the reflective area is located in the effective area by an adaptive threshold algorithm and a secondary mask is generated to shield the interference of refracted reflection on subsequent detection; (2) Perform data augmentation processing on the effective region, wherein in response to the effective region being a solid region, perform Gaussian blur texture perturbation processing on the effective region; In response to the effective area being a liquid area, at least one data enhancement process including angle adjustment, brightness adjustment, and contrast adjustment is performed.
6. The method according to claim 1 or 5, characterized in that, Also includes: Using the color features of a preset standard auxiliary material image as a reference feature, the normalized comprehensive deviation between the color features of the effective region image and the reference feature is calculated. The normalized overall deviation is compared with a preset deviation threshold to determine whether the color and / or transparency of the excipient to be tested meet the standard.
7. The method according to claim 6, characterized in that, When the excipient to be detected is a solid or a liquid, the normalized comprehensive deviation includes the colorimetric comprehensive deviation; and the calculation of the normalized comprehensive deviation between the color features of the image of the effective region and the reference features includes: Determine the RGB mean, RGB range, and CIELAB mean of the pixels in the image of the effective region; Based on the RGB mean, the RGB range, the CIELAB mean, and the reference feature, the overall color deviation of the excipient to be tested is determined; The overall color deviation of the excipient to be tested is determined in the following manner: ; in, Indicates overall color deviation. This indicates the deviation of the effective region from the RGB mean of the reference feature. This indicates the L-value deviation between the effective region and the reference feature. This indicates the AB value deviation between the effective region and the reference feature. F represents the RGB range and deviation between the effective region and the reference feature. ref F represents the reference feature. test The color characteristics of the effective area, This represents the k-mean value in the reference feature. This represents the k-mean of the effective region. This represents the mean L value among the reference features. The mean L value of the effective region is represented. This represents the sum of the RGB ranges in the reference feature. This represents the sum of the RGB ranges of the effective region.
8. The method according to claim 6, characterized in that, When the excipient to be detected is a liquid, the normalized comprehensive deviation includes the transparency comprehensive deviation; and the calculation of the normalized comprehensive deviation between the color features of the effective region image and the reference features includes: Convert the image of the effective region into a grayscale image; Calculate the mean brightness, standard deviation of brightness, and relative standard deviation used to characterize transparency of the grayscale image; Based on the mean brightness, the standard deviation of brightness, the relative standard deviation, and the reference feature, the overall transparency deviation of the excipient to be tested is determined; The overall transparency deviation of the excipient to be tested is determined in the following manner: ; in, This indicates the overall deviation in transparency. This indicates the deviation of the average brightness of the effective area from that of the reference feature. This represents the relative standard deviation of the effective region from the reference feature. This represents the mean brightness value in the reference feature. This represents the average brightness of the effective area. This represents the relative standard deviation in the reference characteristic. The relative standard deviation of the effective area is represented by the ratio of the brightness standard deviation to the brightness mean.
9. An electronic device, characterized in that, include: A processor, configured to execute program instructions; as well as A memory configured to store the program instructions, which, when loaded and executed by the processor, cause the processor to perform the method according to any one of claims 1-8.
10. A computer-readable storage medium storing program instructions, characterized in that, When the program instructions are loaded and executed by the processor, the processor performs the method according to any one of claims 1-8.
