A method and system for blister capsule segmentation and defect detection based on YOLO and mathematical modeling

Abstract

The application discloses a medicine plate capsule blister segmentation and defect detection method and system based on YOLO and mathematical modeling, and belongs to the technical field of image processing. In view of the capsule positioning accuracy on an aluminum-plastic medicine plate and the inherent edge roughness problem of a segmentation network, the application realizes accurate positioning and image segmentation of the blister by combining a YOLO model and mathematical modeling technology, retains capsule information in the blister, and is convenient for subsequent capsule defect detection. The application preliminarily positions the capsule blister area position by using a YOLO algorithm, then performs parameterization fitting and geometric reconstruction on the blister contour by a mathematical modeling method, fundamentally avoids the inherent edge roughness problem of the segmentation network, and realizes complete, smooth and accurate description of the blister area. In addition, the application also proposes an improved YOLO11n network for defect detection based on the image characteristics of the capsule blister. Experiments prove that the calculation amount of the application is significantly reduced compared with a traditional method, and the accuracy is significantly improved.

Description

Technical Field

[0001] This invention relates to the field of defect detection technology for aluminum-plastic blister packs using deep learning, and particularly to a method and system for blister pack segmentation and defect detection based on YOLO and mathematical modeling. Background Technology

[0002] Capsule appearance defect detection is a crucial step in quality control during pharmaceutical manufacturing, directly impacting drug appearance quality and production line efficiency. With increasing automation in the pharmaceutical industry, traditional manual inspection methods are increasingly unable to meet the demands for high efficiency and consistency, leading to widespread interest in machine vision-based automated inspection methods. Simultaneously, the overall development of defect detection technology in the industrial sector provides a methodological and technological foundation for capsule inspection.

[0003] Early research on capsule defect detection primarily relied on traditional image processing methods. This involved initial extraction of the target region through grayscale thresholding and analysis and discrimination of the defect region based on texture statistical features. Additionally, some studies enhanced defect contour information using edge detection methods to improve detection accuracy. These methods typically offer advantages such as simplicity and low computational complexity, achieving some success in stable imaging environments. However, in real-world production scenarios, factors like varying lighting, capsule surface reflection, and complex backgrounds make traditional methods highly sensitive to parameter settings and imaging conditions, resulting in insufficient detection stability and robustness. With the development of deep learning technology, convolutional neural networks have demonstrated significant advantages in automatic image feature learning. Capsule defect detection methods based on deep convolutional networks have achieved good results in recognizing complex textures and irregular defects.

[0004] In object detection tasks, the two-stage detection algorithm Faster R-CNN, by introducing a region proposal mechanism, achieves unified modeling of object localization and category recognition, offering advantages in detection accuracy. However, its high computational complexity limits its application in industrial inspection scenarios with high real-time requirements. YOLO, a single-stage object detection algorithm, completes object localization and classification simultaneously in a single forward propagation, demonstrating high detection speed in real-time detection tasks. However, defect detection often requires identifying small targets, making the detection of small regions particularly important.

[0005] Introducing attention mechanisms to enhance the model's ability to model key information has become an important direction for improving deep learning models. Squeeze-and-Excitation networks enhance feature representation capabilities by modeling inter-channel dependencies, while the CBAM module weights features in both channel and spatial dimensions, helping to improve the model's attention to fine-grained defect regions. In the field of industrial surface defect detection, deep learning-based segmentation and detection methods have proven to have good detection performance and application potential. For capsule appearance defects, existing research combines deep feature fusion with semantic information guidance for detection, achieving positive progress in detection accuracy and robustness.

[0006] For the task of accurate segmentation of the blister pack area in aluminum-plastic blister packaging, although existing deep learning-based semantic segmentation methods have achieved good results in pixel-level classification accuracy, segmentation boundaries generally suffer from jagged edges and poor contour smoothness, making it difficult to achieve a balance between region integrity and edge smoothness. For example, Deep Semantic Networks Based on Boundary Refinement (DSN-BR) improves the segmentation accuracy of defective edges by introducing an edge refinement module, but under the complex texture of aluminum-plastic surfaces, blister boundaries still exhibit roughness and discontinuity. While region segmentation algorithms based on proportional features can achieve rapid localization of the blister pack ROI, the segmentation accuracy is limited, and jagged edges and localization errors are prone to occur. The root cause of these problems lies in the loss of spatial details during the downsampling process of the pixel-by-pixel classification paradigm and the lack of an effective edge constraint mechanism. Summary of the Invention

[0007] To address the limitations of existing technologies in accurately segmenting blister regions and the inherent roughness of segmentation networks, this invention proposes a method and system for blister segmentation and defect detection in pharmaceutical capsules based on YOLO and mathematical modeling. It abandons the end-to-end semantic segmentation framework and instead adopts a technical approach combining YOLO object detection and mathematical modeling: first, the detection bounding box of the blister region is obtained using the YOLO algorithm; then, the blister contour is parametrically fitted and geometrically reconstructed using mathematical modeling methods. This fundamentally avoids the inherent roughness of the segmentation network, achieving a complete, smooth, and accurate description of the blister region.

[0008] Therefore, the present invention provides the following technical solution:

[0009] A method for segmenting blister packs of medicine capsules based on YOLO and mathematical modeling includes the following steps:

[0010] Step 1: Acquire images of the medicine blister pack capsules;

[0011] Step 2: Initially locate the blister pack position. Input the image of the blister pack into a target detection model based on the YOLO network to perform target detection on the blister pack and obtain a detection box representing the position of the blister pack.

[0012] Step 3: Based on mathematical modeling and the position of the capsule blister, generate a mask for the capsule blister region, and segment the capsule blister based on the mask;

[0013] The mathematical modeling is as follows: each capsule blister region is composed of two semicircles and a rectangle. Then, the semicircular and rectangular regions are calculated using a detection box representing the capsule blister position. Finally, a mask is determined based on the semicircular and rectangular regions, defining the mask for the i-th capsule blister region. for:

[0014]

[0015] In the formula, For image coordinates, , and Let represent the two semicircular and rectangular regions of the i-th capsule blister region, respectively.

[0016] Optionally, the semicircular region characterized by the position of the capsule blister can be used. , and rectangular area for:

[0017]

[0018]

[0019]

[0020] In the formula, , These represent the centers of the upper and lower circles of the capsule blister corresponding to the i-th detection frame, respectively. , The image coordinates; r is the radius. This represents the x-coordinate of the center of the i-th detection box; This refers to the standard width of the capsule blister pack.

[0021] Optionally, the center , The coordinates satisfy:

[0022]

[0023]

[0024]

[0025] In the formula, This refers to the standard height of the capsule blister pack.

[0026] Optionally, the semicircular region can be calculated using the position of the capsule blister. , and rectangular area Previously, the method also included: first correcting the position of the capsule blister pack, specifically:

[0027] Generate uniformly distributed grid points, with the number of grid points equal to the number of detection boxes identified in step 2;

[0028] The center coordinates of each detection box are projected and aligned along the nearest grid point. The center coordinates of the detection box are then corrected using the coordinates of the nearest grid point. The corrected center coordinates of the i-th detection box are then obtained. for:

[0029]

[0030]

[0031] In the formula, C represents the number of columns in the blister pack of the blister pack, j represents the column marker for the blister pack, R represents the number of rows in the blister pack, and p represents the row marker for the blister pack. and Represents the coordinates of grid points. and The coordinates of the center of the i-th detection box before correction;

[0032] This indicates the difference in absolute value. smallest The column containing the data is the nearest column to the detection box, and the coordinates of the nearest column are also specified. The x-coordinate of the corrected detection box center is used as the center of the generated grid closest to it. This indicates the difference in absolute value. smallest Find the nearest row of the bounding box for the current behavior and set its coordinates. As the y-coordinate of the center of the corrected detection box;

[0033] The median of the width and height of the detection frame is used as the standard width for each capsule blister area. and height ;

[0034] The corrected capsule blister uses vectors express.

[0035] Optionally, the medicine blister pack is an aluminum-plastic blister pack.

[0036] This invention also provides a method for detecting defects in blister packs, comprising the following steps:

[0037] The capsule blister region in the blister image is segmented using the YOLO-based and mathematical modeling-based blister segmentation method described above.

[0038] The capsule blister region is input into a neural network-based capsule defect detection model to obtain capsule defect detection results.

[0039] Optionally, the neural network of the capsule defect detection model adopts the YOLO11n network, which includes a backbone network, a neck layer, and a detection head.

[0040] In the backbone network and the neck layer, GhostConv modules are used to replace some ordinary convolutions, and C3EMA modules are used to replace the original C3k2 modules; in the backbone network, parameterless SimAM attention combined with lightweight C3EMA is used to replace the original PSA modules.

[0041] This invention also provides a system based on the above-mentioned YOLO-based and mathematical modeling-based blister pack segmentation method, comprising:

[0042] The image acquisition module is used to acquire images of medicine blister packs and capsules;

[0043] The preliminary positioning module is used to initially locate the position of the capsule blister pack. The image of the medicine plate capsule is input into the target detection model based on the YOLO network to perform target detection on the capsule blister pack and obtain a detection box representing the position of the capsule blister pack.

[0044] The segmentation module is used to generate a mask for the capsule blister region based on mathematical modeling and the position of the capsule blister, and to segment the capsule blister based on the mask.

[0045] The present invention also provides a computer device, comprising: one or more processors and a memory storing one or more computer programs;

[0046] The processor invokes a computer program to achieve the following:

[0047] The steps of the above-mentioned method for segmenting blister packs of medicine blister packs based on YOLO and mathematical modeling, or the steps of the above-mentioned method for detecting defects in medicine blister packs.

[0048] The present invention also provides a computer-readable storage medium storing a computer program that is invoked by a processor to implement:

[0049] The steps of the above-mentioned method for segmenting blister packs of medicine blister packs based on YOLO and mathematical modeling, or the steps of the above-mentioned method for detecting defects in medicine blister packs.

[0050] Compared with the prior art, the present invention achieves the following progress and effects:

[0051] 1. The technical solution of this invention creatively proposes a technical route that combines YOLO object detection with mathematical modeling. First, the initial position of the blister region is obtained through the YOLO algorithm, and then the mathematical modeling method is used to perform parametric fitting and geometric reconstruction of the blister contour, avoiding the edge roughness problem inherent in the segmentation network, and realizing a complete, smooth and accurate description of the blister region.

[0052] In existing pixel-by-pixel segmentation networks, multiple downsampling operations lead to compression of high-resolution edge information, resulting in blurred and jagged output edges and loss of spatial details. Furthermore, purely data-driven segmentation networks lack explicit geometric priors (such as smoothness, continuity, and convexity), making them prone to local breaks, burrs, or abnormal protrusions. The YOLO solution in this invention can coarsely locate the bubble region. The rectangular detection box output by YOLO excludes most background interference and retains sufficient spatial context. It does not perform pixel-by-pixel classification, avoiding the loss of details caused by segmentation downsampling. Although the YOLO feature map is downsampled, its task is to regress bounding box coordinates, making it insensitive to sub-pixel edges and thus avoiding edge roughness. This invention further uses parametric curves to fit the actual boundary of the bubble within the YOLO-bounded area. The parametric model naturally satisfies continuity and smoothness, outputting a smooth contour with sub-pixel accuracy, directly avoiding rough, jagged edges.

[0053] 2. Addressing the characteristics of capsule surface defects—small size, susceptibility to reflections from curved surfaces, and spatial deformation—this invention proposes an optimized network architecture. Specifically, ordinary convolutions are replaced with GhostConv, which generates redundant feature maps through inexpensive linear transformations, maintaining a high channel count without significantly increasing computation. This makes it more sensitive to local texture fractures caused by minute defects like cracks. Secondly, all original C3k2 modules are replaced with C3EMA. C3EMA's multi-scale attention can simultaneously consider local details and overall deformation context, suppressing false positive responses caused by highlights on the capsule surface and adapting to defect stretching or rotation due to curvature changes. Furthermore, this invention adds a parameter-free SimAM attention layer after SPPF. Based on a local energy function, it dynamically enhances anomalous contrast, making defect signals more prominent without needing to learn specific lighting patterns. Combined with the above network architecture optimizations, fully utilizing the local energy anomalies inherent in capsule defects, this model demonstrates more outstanding and significant performance in this field, exhibiting greater resistance to reflections and fewer missed detections. Attached Figure Description

[0054] Figure 1 Schematic diagrams of several typical defects in capsules;

[0055] Figure 2 is a schematic diagram of blister region image segmentation based on YOLO+ mathematical modeling, wherein, Figure 2a This is a preliminary positioning diagram of the capsule blister pack. Figure 2b A schematic diagram of capsule blister modeling;

[0056] Figure 3 This is a diagram of the YOLO11n network architecture in Example 1;

[0057] Figure 4 This is a schematic diagram of the C3EMA module.

[0058] Figure 5 This is the architecture diagram of the EMAttentionLight module. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. The technical features involved in the various embodiments of the invention described below can be combined with each other as long as they do not conflict with each other.

[0060] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0061] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0062] This invention takes aluminum-plastic blister packs as an example. Based on the fixed nature and positional characteristics of the blister pack area, it proposes a technical approach combining mathematical modeling and YOLO target inspection to accurately locate and segment the blister pack area. The segmented blister pack images can then be applied to defect diagnosis. Specifically, this invention creatively proposes a blister pack area segmentation method based on YOLO and mathematical modeling. First, the positions of each blister are initially located. Then, the positions and sizes of the blister packs are corrected, and mathematical modeling is used to accurately describe the blister pack area, ultimately generating a standardized blister pack image with a uniform white background. Simultaneously, the lightweight YOLO11n model is preferentially used for defect detection.

[0063] The core of this invention is to propose a novel mathematical modeling approach based on the inherent row and column arrangement rules of blister packs, and to perform structured calibration on the original detection results of the YOLO target detection model. Secondly, considering the characteristics of capsule defects being small and ambiguous and requiring real-time detection, the network architecture of the YOLO target detection model is optimized. On the one hand, the YOLO11n network structure is preferentially used for both initial target detection and localization and subsequent defect detection. On the other hand, some ordinary convolutions in the original YOLO11n network are replaced with GhostConv, the original C3k2 module is replaced with a C3EMA module, and the parameterless SimAM attention combined with the lightweight C3EMA module replaces the computationally complex original PSA module. This reduces the number of model parameters and computational cost while maintaining the model's detection accuracy and robustness for capsule surface defects. Dataset validation shows that when using mathematical modeling + YOLO11n network for defect detection, the number of parameters is reduced by 50% and the computational cost by 15%. Especially when applied to defect detection, the detection accuracy for five types of defects remains unchanged and is slightly improved by 0.1-0.2 percentage points. This will enhance the application value of the algorithm in real-world industrial scenarios.

[0064] Example 1

[0065] In defect detection applications, the acquired blister pack images contain a lot of useless background information, which can interfere with the target detection algorithm and significantly affect the detection accuracy. Therefore, it is necessary to perform blister image segmentation on the blister pack images. However, traditional segmentation methods use template matching to segment the blister region of aluminum-plastic blister packs. This method is extremely sensitive to changes in lighting and reflections, and has poor adaptability to local occlusion and pattern changes, making it unable to effectively segment the blister region. Deep learning-based image segmentation algorithms, such as MobileSAM, have strong segmentation capabilities, but in the application of blister region image segmentation, they have poor boundary smoothness, generally low output standardization, and high computational cost.

[0066] This embodiment proposes a novel mathematical modeling approach and utilizes the more advanced YOLO11n network to ultimately generate standardized capsule images with a uniform white background. Specifically, this embodiment provides a blister pack segmentation method for pharmaceutical capsules based on YOLO and mathematical modeling, including the following steps:

[0067] Step S1: Original Image Acquisition and Preprocessing. Considering that this technology can also be applied to defect detection, this embodiment acquires images of various defective and normal drug blister packs, i.e., images of aluminum-plastic drug blister capsules, using an imaging device. In other feasible embodiments, defect samples may not be included in the model training of the segmentation method; this invention does not specifically limit this. This embodiment also preferably performs preprocessing on the drug blister pack images; preprocessing methods include, but are not limited to, data augmentation.

[0068] This embodiment uses a top-down perspective to photograph 150 aluminum-plastic blister packs, resulting in 150 images, each containing 10 capsules. The types and quantities of defects are as follows: 1520 cracks, 910 leaking capsules, 620 double white caps, 560 double colored caps, and 340 missing capsules. Figure 1 The images show typical samples with the aforementioned defects, displayed from left to right.

[0069] To improve the model's generalization ability, data augmentation was performed by systematically combining random color perturbation and geometric flipping. This data augmentation method can simulate the differences in image appearance caused by changes in lighting, imaging equipment, and viewing angle in real-world scenes, providing a high-quality data foundation for training a more robust visual model. Both random color perturbation and geometric flipping methods doubled the sample size, expanding it to 600 images and 6000 bubble masks.

[0070] Step S2: Using a target detection model based on YOLO11n, the position of the blister pack of the medicine blister pack is initially located from the acquired medicine blister pack image.

[0071] In some embodiments, this step can employ a traditional YOLO11n network, or a YOLO11n network with optimized structure as described in this invention (its network architecture will be described in detail below). This invention does not impose specific limitations on this. Furthermore, in scenarios where accuracy requirements are relatively low, selecting other networks within the YOLO series also falls within the scope of protection of this invention.

[0072] Step S3: Based on mathematical modeling and the position of the capsule blister, generate a mask for the capsule blister region to achieve capsule blister segmentation.

[0073] The detection boxes output by the YOLO11n model can initially locate the position of the capsule blister, as shown in Figure 2a. The center position and size of each detection box differ from the true value and need further correction to remove detection errors.

[0074] (1) Sort the detection boxes output by YOLO11n: based on the center point coordinates of each detection box. The primary sorting is performed by the y-coordinate (row direction), followed by a secondary sorting by the x-coordinate (column direction) within the same row, thus arranging all detection boxes according to the "row first, column second" rule. After sorting, each detection box is assigned a consecutive unique index i (i=1,2,…,N), where N is the number of detection boxes. This is done using vectors. Let represent the i-th detection box, where: The coordinates of the center point of the detection box. The width and height (in pixels) of the detection box.

[0075] (2) To correct the center coordinates of each detection frame, this embodiment generates N uniformly distributed grid points on the medicine plate, with the number of grid points equal to the number of detection frames. The coordinates of each grid point are determined by the center coordinates of the detection frame and the actual number of rows and columns of the capsule. The x and y coordinates of all grid center points are obtained by linear interpolation using the minimum and maximum coordinates of the detection frames as boundaries:

[0076]

[0077]

[0078] Where C is the number of columns in the blister pack, j is the column identifier, R is the number of rows in the blister pack, and p is the row identifier. and Represents grid coordinates, and These are the x-coordinate of the last detection box in the first row and the y-coordinate of the first detection box in the last row, respectively. and These are the x and y coordinates of the first detection box, respectively.

[0079] For each detection box, its original center coordinates are... Corrected coordinates are obtained by finding the nearest grid center. :

[0080]

[0081]

[0082] In the formula, This represents the difference in absolute values ​​within the theoretical column coordinate set (i.e., grid coordinates). smallest The column containing the detection box is the nearest column, and the theoretical x-coordinate of the nearest column is used as the corrected x-coordinate of the detection box center; similarly, This represents the difference in absolute values ​​within the theoretical column coordinate set (i.e., grid coordinates). smallest The nearest row of the detection box in the current behavior is used, and the theoretical y-coordinate of the nearest row is used as the y-coordinate of the center of the corrected detection box.

[0083] Since capsules on the same blister pack have consistent external dimensions, this invention calculates the median of the width and height of the original detection frame as the standard width for each blister region. and height :

[0084]

[0085]

[0086] Wherein, after the position and size of the i-th detection box are corrected, a vector is used. express.

[0087] (3) Mathematical modeling of the blister region. The blister region of the capsule is composed of two semicircles and a rectangle. This invention performs precise mathematical modeling of the blister region based on the center coordinates and length and width of the corrected detection frame. For example... Figure 2b As shown, for the capsule in the i-th detection frame, the centers of the upper and lower circles are... , The coordinates are respectively , :

[0088]

[0089]

[0090]

[0091] The two circles have the same radius r, which is _____. Therefore, the top and bottom semicircular regions are respectively:

[0092]

[0093]

[0094] In the formula, This indicates the semi-circular area at the top; This indicates the semi-circular area at the bottom.

[0095] Middle rectangular area for:

[0096]

[0097] Based on the above model, the mask for the i-th capsule blister region is defined as follows:

[0098]

[0099] It should be understood that capsule blister segmentation can be achieved by using a mask in the capsule blister area.

[0100] The correction method proposed in this invention is based on the inherent row and column arrangement of blister packs to perform structured calibration on the original detection results of the YOLO model. The principle is as follows: First, based on the coordinate distribution of the detection boxes, a grid with corresponding rows and columns is generated by linear interpolation using the minimum and maximum coordinates of the detection boxes as boundaries; then, the center coordinates of each detection box are corrected to the center coordinates of the generated grid closest to it, and the median width and height of the original detection boxes are uniformly assigned, i.e., the standard width of each blister pack area. and height Finally, a capsule-shaped mask is used to define the tablet area, eliminating interference from aluminum foil reflections and blister edge interference. Compared to conventional methods that directly use detection boxes, this invention uses median estimation of the blister region's width and height to suppress disturbances to mesh parameters caused by outlier boxes. A mesh is dynamically generated based on the coordinate distribution of the detection boxes to adapt to changes in image scale and viewing angle, and a shape mask enhances the signal-to-noise ratio of the target area. Specifically, regular mesh projection eliminates random offsets in the detection box center coordinates, uniform size corrects for width and height fluctuations caused by local texture differences, and masking confines the analysis range to the effective pixels of the tablet, reducing interference from background signals in subsequent processing steps.

[0101] Example 2

[0102] This invention provides a method for detecting defects in blister packs, comprising the following steps:

[0103] The capsule blister region in the capsule image is segmented using the YOLO-based and mathematical modeling-based blister segmentation method described above; and the capsule blister region is input into a neural network-based capsule defect detection model to obtain the capsule defect detection result.

[0104] It should be understood that this invention does not limit the network architecture and model type of the capsule defect detection model. Referring to the prior art, any corresponding model that can realize capsule defect detection falls within the protection scope of this invention.

[0105] In this embodiment, considering the dual requirements of high accuracy and real-time performance for capsule segmentation, as well as the characteristics of blister packs and the need for defect detection, this embodiment preferably employs a structurally optimized YOLO11n network for both the segmentation and defect detection processes. Specifically:

[0106] Three new modules were introduced to replace existing modules in the YOLO11n network, achieving further weight reduction while maintaining accuracy: First, the GhostConv module (GhostConvolution) replaced some ordinary convolutions. The GhostConv module significantly reduced the number of parameters and computational cost by first extracting a small number of essential features and then deriving similar features through linear transformation. Second, the C3EMA module replaced the original C3k2 module. The C3EMA module enhanced the ability to perceive small targets without introducing significant computational burden. Third, the parameterless SimAM attention combined with the lightweight C3EMA module replaced the original PSA module in the backbone network. The SimAM attention can adaptively enhance defect-related features and suppress irrelevant information, thereby improving detection accuracy.

[0107] The YOLO11n network architecture in this embodiment is as follows: Figure 3 As shown, the system comprises a backbone network, a neck layer, and a head. The backbone network consists of two standard convolutional layers, multiple alternating GhostConv-C3EMA modules, an SPPF module, a SimAM layer, and a C3EMA module. In this embodiment, the input image is preprocessed before entering the backbone network. The standard convolutional layers of the backbone network are used for 2x downsampling, compressing the resolution to 1 / 4 of the original image and expanding the number of channels from 3 to 128. The GhostConv module replaces conventional convolution with a low-cost method to generate redundant feature maps. The C3EMA module introduces EMA attention units within the C3 structure, enhancing global perception capabilities by interactively encoding channel features across spatial dimensions. The SPPF module has a kernel size of 5 and is used to fuse multi-scale receptive field information. The SimAM layer does not introduce additional parameters and adaptively infers the three-dimensional attention weights by calculating the energy function of each neuron. The final C3EMA layer integrates and outputs the features.

[0108] In this embodiment, the backbone network has 5 C3EMA modules. The 2nd, 3rd and 5th C3EMA modules all have output paths that connect to the neck layer.

[0109] The neck layer follows the design principle of a bidirectional feature pyramid. In the top-down path, the deep features output by the 5th C3EMA module in the backbone network are upsampled in the neck layer and concatenated with the features input to the neck layer from the 3rd C3EMA module in the channel dimension. This allows high-level semantics to guide the recovery of low-level details. After being upsampled by the C3EMA module, these features are concatenated with the features input to the neck layer from the 2nd C3EMA module in the backbone network in the channel dimension, entering the bottom-up path. The bottom-up path then uses convolution or GhostConv to downsample the enhanced shallow features, recompressing their resolution and concatenating them with previously saved feature maps of the same size to strengthen the transmission of localization information. The three sets of output feature maps at different resolutions (corresponding to the scales of the 4th and 6th layers of the backbone and subsequent layers of SPPF, respectively) are finally fed into the detection head, each responsible for predicting bounding box coordinates, confidence scores, and class probabilities, achieving differentiated detection of large, medium, and small targets.

[0110] The C3EMA module enhances features by improving the original C3 structure. Its core is replacing the last bottleneck block with a lightweight channel attention module, EMAttentionLight. Figure 4 As shown, this module adopts a dual-branch architecture: the main branch uses 1×1 convolutions for dimensionality reduction before connecting to the sequence processing module, while the side branch uses 1×1 convolutions to preserve the original features. The two are then concatenated and fused with another 1×1 convolution before output. In the sequence processing module, the first n-1 layers use 3×3 standard convolutions combined with batch normalization and SiLU activation for spatial feature extraction. The last layer introduces the EMAttentionLight module to complete channel recalibration. This design integrates the attention mechanism with minimal computational cost, significantly improving feature selection capability while maintaining parameter efficiency. The EMAttentionLight module structure is as follows: Figure 5 As shown, this module first performs global average pooling on the input features to obtain channel descriptors. Then, it generates attention weights through two fully connected layers: the first layer reduces dimensionality by 32x compression and applies ReLU activation; the second layer restores the original dimensionality and normalizes the weights using the Sigmoid function. Finally, the weights are multiplied channel-by-channel with the input features to complete recalibration. This design effectively enhances the model's response to key feature channels with a concise structure, improving feature representation capabilities with almost no increase in computational overhead.

[0111] It should be understood that the network architecture described above in this embodiment is designed to address the characteristics of capsule surface defects, such as small size, susceptibility to surface reflection interference, and spatial deformation. First, in the deep downsampling stage and some feature fusion paths (layers 3, 5, and 7 of the backbone network and layer 21 of the detection head (downsampling fusion layer)), ordinary convolutions are replaced with GhostConv. It generates redundant feature maps through inexpensive linear transformations, maintaining a high number of channels without significantly increasing computation, thus being more sensitive to local texture breaks caused by minute defects such as cracks. Second, all the original C3k2 modules are replaced with C3EMA modules. The multi-scale attention of the C3EMA module can simultaneously consider local details and the overall deformation context, suppressing false positive responses caused by highlights on the capsule surface and adapting to defect stretching or rotation caused by curvature changes. Third, a parameter-free SimAM attention is added after SPPF. It dynamically enhances anomalous contrast based on local energy functions, making defect signals more prominent without needing to learn specific lighting patterns. In summary, these modifications ultimately address the fundamental nature of capsule defects as localized energy anomalies, making them more resistant to reflection and less prone to missed detections than the original model.

[0112] It should also be understood that when the above network is applied to different applications such as target localization and defect detection, the network training data is different, and the input and output layers are adaptively adjusted.

[0113] Example 3

[0114] This invention also provides a system based on the above-described segmentation method, which includes an image acquisition module, a preliminary positioning module, and a segmentation module connected in sequence or to each other.

[0115] The system includes an image acquisition module for acquiring images of the blister packs; a preliminary localization module for initially locating the blister pack position by inputting the blister pack image into a target detection model based on a YOLO network to perform target detection on the blister pack and obtain a detection box representing the position of the blister pack; and a segmentation module for generating a mask for the blister pack region based on mathematical modeling and the position of the blister pack to achieve blister pack segmentation.

[0116] In some embodiments, the system may also include a defect detection module for inputting the capsule blister region into a neural network-based capsule defect detection model to obtain capsule defect detection results.

[0117] It should be understood that the specific implementation process of each module is described in the above method. This invention will not repeat the details here. The above division of functional modules is only for illustrative purposes. In some embodiments, some functional modules can be combined and some functional modules can be separated. Each functional module can be implemented in software, hardware, or a combination of software and hardware. The software and hardware devices include, but are not limited to, general-purpose computer equipment, programmable gate arrays, digital signal processors, microprocessors and their corresponding programming or burning software.

[0118] Example 4

[0119] This invention also provides a computer device, including: one or more processors and a memory storing one or more computer programs;

[0120] The processor invokes a computer program to achieve the following:

[0121] The steps of the above-mentioned method for segmenting blister packs of medicine blister packs based on YOLO and mathematical modeling, or the steps of the above-mentioned method for detecting defects in medicine blister packs.

[0122] Please refer to the explanation of the method above for the specific implementation process of each step.

[0123] It should be understood that, in the embodiments of the present invention, the processor may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor. The memory may include read-only memory and random access memory, and provides instructions and data to the processor. A portion of the memory may also include non-volatile random access memory. For example, the memory may also store device type information.

[0124] Example 5

[0125] This invention also provides a computer-readable storage medium storing a computer program that is invoked by a processor to implement:

[0126] The steps of the above-mentioned method for segmenting blister packs of medicine blister packs based on YOLO and mathematical modeling, or the steps of the above-mentioned method for detecting defects in medicine blister packs.

[0127] Please refer to the explanation of the method above for the specific implementation process of each step.

[0128] The computer-readable storage medium can be an internal storage unit of the hardware and software device in any of the foregoing embodiments (e.g., the hard disk or memory of the controller), or it can be an external storage device of the controller, such as a plug-in hard disk, smart memory card (SMC), secure digital card (SD card), flash memory card, etc., equipped on the controller. Furthermore, the storage medium can also simultaneously include both the controller's internal storage unit and external storage device.

[0129] Based on the above understanding, the core part of the technical solution of this invention that contributes to the prior art, or all or part of the content of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and contains several instructions to cause a computer device (such as a personal computer, server, or network device) to execute all or part of the steps of the methods described in the various embodiments of this invention. Available storage media include, but are not limited to: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.

[0130] Those skilled in the art will understand that embodiments of this application can be provided in the form of a method, system, or computer program product. Therefore, this application can be implemented entirely in hardware, entirely in software, or a combination of hardware and software. Furthermore, this application can also be implemented as a computer program product containing computer-usable program code on a computer-readable storage medium (such as a disk storage device, CD-ROM, optical storage, etc.). Embodiments of this application are described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products. It should be understood that the function of each step in the flowchart and / or each block in the block diagram can be implemented by computer program instructions. These computer program instructions can be executed by a processor of a general-purpose computer, special-purpose computer, or other programmable data processing device to generate means for implementing the functions specified in the flowchart and / or block diagrams. These instructions can also be stored in a computer-readable storage medium to cause a computer or other programmable device to operate in a particular manner, thereby producing an article of manufacture containing instruction means to implement the functions specified in the flowchart and / or block diagrams. Furthermore, the instructions can be loaded onto a computer or other programmable device to form a computer-implemented processing flow by performing a series of operational steps, so that the instructions executed on the computer or other programmable device can implement the functional steps specified in the flowchart and / or block diagrams.

[0131] It should be emphasized that the examples described in this invention are illustrative rather than limiting. Therefore, this invention is not limited to the examples described in the specific embodiments. Any other embodiments derived by those skilled in the art based on the technical solutions of this invention, without departing from the spirit and scope of this invention, whether modifications or substitutions, are also within the protection scope of this invention.

Claims

1. A method for segmenting blister packs of pharmaceutical capsules based on YOLO and mathematical modeling, characterized in that: Includes the following steps: Step 1: Acquire images of the medicine blister pack capsules; Step 2: Initially locate the blister pack position. Input the image of the blister pack into a target detection model based on the YOLO network to perform target detection on the blister pack and obtain a detection box representing the position of the blister pack. Step 3: Based on mathematical modeling and the position of the capsule blister, generate a mask for the capsule blister region, and segment the capsule blister based on the mask; The mathematical modeling is as follows: each capsule blister region is composed of two semicircles and a rectangle. Then, the semicircular and rectangular regions are calculated using a detection box representing the capsule blister position. Finally, a mask is determined based on the semicircular and rectangular regions, defining the mask for the i-th capsule blister region. for: ； In the formula, For image coordinates, , and Let represent the two semicircular and rectangular regions of the i-th capsule blister region, respectively.

2. The method according to claim 1, characterized in that: The semicircular region characterized by the position of the capsule blister , and rectangular area for: ； ； ； In the formula, , These represent the centers of the upper and lower circles of the capsule blister corresponding to the i-th detection frame, respectively. , The image coordinates; r is the radius. This represents the x-coordinate of the center of the i-th detection box; This refers to the standard width of the capsule blister pack.

3. The method according to claim 2, characterized in that: center , The coordinates satisfy: ； ； ； In the formula, This refers to the standard height of the capsule blister pack.

4. The method according to claim 1, characterized in that: Calculate the semi-circular region using the position of the capsule blister , and rectangular area Previously, the method also included: first correcting the position of the capsule blister pack, specifically: Generate uniformly distributed grid points, with the number of grid points equal to the number of detection boxes identified in step 2; The center coordinates of each detection box are projected and aligned along the nearest grid point. The center coordinates of the detection box are then corrected using the coordinates of the nearest grid point. The corrected center coordinates of the i-th detection box are then obtained. for: ； ； In the formula, C represents the number of columns in the blister pack of the blister pack, j represents the column marker for the blister pack, R represents the number of rows in the blister pack, and p represents the row marker for the blister pack. and Represents the coordinates of grid points. and These are the center coordinates of the i-th detection box before correction; This indicates the difference in absolute value. smallest The column containing the data is the nearest column to the detection box, and the coordinates of the nearest column are used. As the x-coordinate of the center of the corrected detection box; This indicates the difference in absolute value. smallest Find the nearest row of the bounding box for the current behavior and set its coordinates. As the y-coordinate of the center of the corrected detection box; The median of the width and height of the detection frame is used as the standard width for each capsule blister area. and height ; The corrected capsule blister uses vectors express.

5. The method according to claim 1, characterized in that: The medicine blister pack is an aluminum-plastic blister pack.

6. A method for detecting defects in blister packs, characterized in that: Includes the following steps: The capsule blister region in the blister pack image is segmented using the method described in any one of claims 1-5; The capsule blister region is input into a neural network-based capsule defect detection model to obtain capsule defect detection results.

7. The method according to claim 6, characterized in that: The neural network of the capsule defect detection model adopts the YOLO11n network, which includes a backbone network, a neck layer, and a detection head. In the backbone network and the neck layer, GhostConv modules are used to replace some ordinary convolutions, and C3EMA modules are used to replace the original C3k2 modules; in the backbone network, parameterless SimAM attention combined with lightweight C3EMA is used to replace the original PSA modules.

8. A system based on the YOLO-based and mathematical modeling-based blister pack segmentation method for pharmaceutical capsules as described in any one of claims 1-6, characterized in that: include: The image acquisition module is used to acquire images of medicine blister packs and capsules; The preliminary positioning module is used to initially locate the position of the capsule blister pack. The image of the medicine plate capsule is input into the target detection model based on the YOLO network to perform target detection on the capsule blister pack and obtain a detection box representing the position of the capsule blister pack. The segmentation module is used to generate a mask for the capsule blister region based on mathematical modeling and the position of the capsule blister, and to segment the capsule blister based on the mask.

9. A computer device, characterized in that: include: One or more processors; A memory that stores one or more computer programs; The processor invokes a computer program to achieve the following: The steps of the blister pack segmentation method based on YOLO and mathematical modeling as described in any one of claims 1-6, or the steps of the defect detection method for blister packs as described in claim 7.

10. A computer-readable storage medium, characterized in that: The computer program is stored and is invoked by the processor to implement: The steps of the blister pack segmentation method based on YOLO and mathematical modeling as described in any one of claims 1-6, or the steps of the defect detection method for blister packs as described in claim 7.

Images