A method of counting a pharmaceutical package

Abstract

The application relates to a kind of improved YOLOv5-based charge detection counting method, it includes S1 obtains original image data set, and the image in original image data set is weakly non-target area WNA data enhancement processing, and the enhanced data set is constructed;S2 improved YOLOv5s model is constructed, and the improved YOLOv5s model is trained using the enhanced data set to obtain trained YOLOv5s model;S3 network pruning is carried out light weight processing to the trained YOLOv5s model to obtain optimal model;S4 an original picture to be predicted is input into optimal model, and the output is a picture with multiple detection frames, each detection frame corresponds to a confidence, after the detection frame below the confidence threshold in the picture is deleted, the picture is input into NMS method processing, and the final detection graph is output, the number of detection frames in the detection graph is the number of charges.

Description

Technical Field

[0001] This invention relates to target inspection methods, and more particularly to a drug package detection and counting method based on an improved YOLOv5. Background Technology

[0002] Traditional object detection algorithms consist of three stages: region proposal, feature extraction, and region classification / boundary regression. Region proposal involves identifying regions in the input image where objects may exist, known as Regions of Interest (ROIs). Since objects can appear at any size and location in the image, traditional methods often use sliding windows of different scales to traverse the image and obtain ROIs. The feature extraction stage commonly uses methods such as scale-invariant feature transformation to extract feature vectors from the ROIs. Finally, a pre-trained classifier is used to detect the category of objects within the ROIs.

[0003] Traditional object detection algorithms often rely on prior knowledge in the feature extraction stage, which has certain limitations. Since 2012, convolutional neural networks (CNNs) have entered the researchers' field of vision with their powerful feature extraction capabilities, resulting in a wealth of research results, among which deep learning object detection algorithms based on CNNs are particularly noteworthy. In the years that followed, two major categories of object detection methods gradually emerged: two-stage detection methods and one-stage detection methods.

[0004] Two-stage detection methods are based on proposed candidate boxes. First, region candidate boxes are generated, then features of each candidate box are extracted, and finally, location boxes are generated and categories are detected. This detection method has high accuracy but is slow.

[0005] One-stage detection methods are based on bounding box regression, which involves classifying and regressing candidate boxes simultaneously with their generation. Compared to two-stage detection methods, this approach is faster but less accurate. Zhu et al. proposed the YOLOv5 algorithm, a typical one-stage detection method. Compared to YOLOv4, YOLOv5 incorporates adaptive image scaling, making it applicable to images of any size. Furthermore, it replaces CSPDarkNet with a CSP structure in the network skeleton, enhancing the network's feature fusion capabilities.

[0006] Object detection and counting methods are mainly divided into three categories: detection-based methods, regression-based methods, and density map-based methods. Detection-based methods primarily include traditional HOG-based detectors or deep learning-based detectors such as YOLO or R-CNN. A drawback of these methods is their susceptibility to occlusion. Regression-based methods capture partial features from the original image and use machine learning models to map the relationship between features and counts. However, traditional image processing and machine learning struggle to capture effective features, resulting in low accuracy and leading to their gradual replacement by deep learning methods. Density map-based methods are currently the mainstream approach for object detection and counting, providing not only the number of objects but also their density distribution. Summary of the Invention

[0007] In view of the above-mentioned problems in the existing technology, the technical problem to be solved by the present invention is how to accurately count the medicine packets.

[0008] To solve the above technical problems, the present invention adopts the following technical solution: a method for detecting and counting medicine packets, comprising the following steps:

[0009] S1: Obtain the original image dataset, perform WNA data augmentation on the images in the original image dataset to weaken non-target regions, and construct the augmented dataset;

[0010] S2: Construct an improved YOLOv5s model and train the improved YOLOv5s model using the augmented dataset to obtain a trained YOLOv5s model.

[0011] S3: Network pruning involves lightweighting the trained YOLOv5s model to obtain the optimal model.

[0012] S4: Input an original image to be predicted into the optimal model, and output an image with multiple detection boxes. Each detection box corresponds to a confidence level. After deleting the detection boxes in the image that are below the confidence level threshold, input the image into the NMS method for processing and output the final detection image. The number of detection boxes in the final detection image is the number of medicine packs.

[0013] Preferably, the process of constructing the enhanced dataset in S1 is as follows:

[0014] WNA data augmentation comprises two parts: target region extraction and non-target region weakening. Target region extraction utilizes existing digital image processing methods, while non-target region weakening involves setting the pixel values ​​of non-target regions to a random value α multiplied by their original pixel values, while the target region retains its original pixel values. The formula is as follows:

[0015]

[0016] Where P(i,j) represents the pixel value at position (i,j) in the image, σ represents the target region, and α∈[0,1] is a random value.

[0017] Preferably, in S2, the improved YOLOv5s model inserts a CBS+C3 module before the last layer CBS+SPP+C3 of the backbone network, and connects the 40×40×512, 20×20×1024, and 10×10×2048 feature maps output by the backbone to the enhanced feature extraction network FPN, while discarding the output of the 80×80×256 feature map.

[0018] Preferably, the process of pruning the trained YOLOv5s model in S3 to reduce its weight is as follows:

[0019] In the YOLOv5 model, batch normalization is performed after each convolution to make the output values ​​more stable. According to the BN layer model formula:

[0020]

[0021] Here, ε is a positive value close to 0, to prevent variance σ from being... B A value of 0 results in invalid calculations. z is an intermediate variable with no actual meaning. in and z out For the input and output of the BN layer, μ B and σ B γ and β are the mean and standard deviation of the input activations on the current mini-batch B, and γ and β are the weights and biases of the BN layer. Each channel outputs z. out The coefficient γ is positively correlated with the network pruning scale. When γ approaches 0, the output is also very small. This is achieved by adding an L1 regularization constraint to γ ​​and applying it to the loss function.

[0022]

[0023] Where x, W, y, and Γ represent the neural network input vector, neural network weight matrix, ideal output vector, and the set of weights of all BN layers in the network, respectively.

[0024] L is the sparse training loss function, l(f(x,W),y) is the normal training loss function, g(γ)=|γ| is the L1 regularization constraint, and λ is the regularization coefficient. During backpropagation, each channel will have two consecutive scaling factors γ. After sparse training, all input-output connections and corresponding weights are removed to prune channels where γ is close to 0. The network accuracy loss is compensated through subsequent fine-tuning. Finally, a compact model is obtained through repeated iterations.

[0025] Preferably, the steps in S4 for inputting the image into the NMS method for processing are as follows:

[0026] S4-1: Sort all detection boxes in descending order of confidence level;

[0027] S4-2: Select the detection box with the highest confidence level, confirm it as the correct detection box, and calculate the IoU between the correct detection box and other detection boxes:

[0028]

[0029] Where iou(a,b) c ) represents detection boxes a and b c The IOU values, where a represents the correct bounding box, and b represents the correct bounding box. c Let C be the c-th detection box, where C is the number of detection boxes remaining in the image after deleting detection boxes below the confidence threshold.

[0030] S4-3: Based on all iou(a,b) calculated in S4-2 c ), delete iou(a,b) c Detection boxes with a value greater than the NMS threshold are processed until all detection boxes have been processed.

[0031] Preferably, the NMS threshold selection step is as follows:

[0032] Step 1: Use the LabelImg annotation tool to label individual medicine packets, only labeling medicine packets 1, 2, and 3, Ω=(ω i ) n Let ω represent the set of n original images. i Let σ represent the i-th image. i =(θ ij ) m Represents ω i The set consisting of m independent real frames of medicine packages.

[0033] Step 2: Set the NMS threshold to 0.9 and call the optimal model to predict ω. i This yields a set δ consisting of s prediction boxes. i =(λ ik ) s , λ ik s represents the k-th (k=1,2,...,s) predicted box for the medicine pack in the i-th image and the total number of medicine pack detection boxes in the i-th image, respectively.

[0034] Step 3: If iou(θ) ij ,λ ik If )>0, then λ ik Add to set Λ = (b t ) uSort the boxes by confidence level and calculate the maximum confidence level of the detection box M and the remaining detection boxes b in Λ. t The intersection-union ratio, i.e., iou(M,b) t ), take the NMS threshold as the minimum intersection-union ratio, that is, th=min{iou(M,b t )}.

[0035] Step 4: Repeat Step 3 m times and update th.

[0036] Step 5: Repeat Step 4 n times and update th.

[0037] Step 6: Set the final NMS threshold as N th = [0.95, 1] ​​× th.

[0038] Compared with the prior art, the present invention has at least the following advantages:

[0039] 1. In this invention, a data augmentation method is used to obtain a higher mAP with a smaller dataset.

[0040] 2. The improved YOLOv5s model structure in this invention is adapted to the detection of large targets in drug packages, thereby improving detection accuracy, which is reflected in the improvement of mAP.

[0041] 3. The method of the present invention obtains a smaller model while maintaining mAP (model accuracy), which is beneficial for deployment on embedded devices. It also obtains an accurate NMS threshold to improve counting accuracy, which is reflected in the improvement of counting accuracy Pc. Attached Figure Description

[0042] Figure 1 This is a simplified flowchart of the method of the present invention.

[0043] Figure 2 The detection results are compared with the independent target at an NMS threshold of 0.9.

[0044] Figure 3 These are the relevant evaluation metrics for YOLOv5s.

[0045] Figure 4 An improved structure for YOLOv5s. Detailed Implementation

[0046] The present invention will now be described in further detail.

[0047] See Figure 1-4 A method for detecting and counting medicine packets, comprising the following steps:

[0048] S1: Obtain the original image dataset, perform WNA data augmentation on the images in the original image dataset to weaken non-target regions, and construct the augmented dataset; The suction cup's medicine-grabbing action is controlled by the extension and retraction of a cylinder. After the suction cup grabs the medicine packet, the cylinder resets, and the suction cup returns to its original position. The camera is fixed on the main frame of the medicine cabinet below the suction cup, meaning the relative positions of the suction cup and the camera are fixed, and the background remains basically unchanged, only the medicine packet changes. However, because the suction cup is a vertically moving mechanism, due to the force applied, it is difficult for the suction cup to completely return to its original position after each medicine packet is picked up. The suction cup will have random slight offsets. In addition, the offset of other parts of the device will also cause background changes. Therefore, traditional image processing methods cannot be used to strictly separate the image background.

[0049] S2: Construct an improved YOLOv5s model and train the improved YOLOv5s model using the enhanced dataset to obtain a trained YOLOv5s model.

[0050] S3: Network pruning involves lightweighting the trained YOLOv5s model to obtain the optimal model.

[0051] S4: Input an original image to be predicted into the optimal model, and output an image with multiple detection boxes. Each detection box corresponds to a confidence level. After deleting the detection boxes in the image that are below the confidence level threshold, input the image into the NMS method for processing and output the final detection image. The number of detection boxes in the final detection image is the number of medicine packs.

[0052] Specifically, the process of constructing the enhanced dataset in S1 is as follows:

[0053] WNA data augmentation comprises two parts: target region extraction and non-target region weakening. Target region extraction employs existing digital image processing methods, primarily including image grayscale conversion, background removal, masking, and noise filtering. Non-target region weakening involves setting the pixels in non-target regions to their original pixel values ​​multiplied by a random value α, while keeping the target region's pixel values ​​unchanged. The formula is as follows:

[0054]

[0055] Where P(i,j) represents the pixel value at position (i,j) in the image, σ represents the target region, and α∈[0,1] is a random value.

[0056] Specifically, in S2, the improved YOLOv5s model inserts a CBS+C3 module before the last layer CBS+SPP+C3 in the backbone network. Simultaneously, the 40×40×512, 20×20×1024, and 10×10×2048 feature maps output from the backbone are connected to the enhanced feature extraction network FPN, while the 80×80×256 feature map output is discarded. The resulting model is better suited to large targets, specifically the detection of medicine packets in this paper. This improves the model's mAP while adapting to the medicine packet dataset. Considering the current detection target is a medicine packet, the captured image size is 1280×1280 pixels, and the size of various medicine packets ranges from approximately 80×80 to 500×500 pixels. Table 1 provides the distinction between small, medium, and large targets in the COCO dataset. It can be seen that the detected targets are mainly large targets, with a small portion being medium targets, and no small targets are detected. Considering the YOLOv5 model structure, to adapt to the detection of targets of different sizes, a feature fusion technique is employed. Three feature maps of different sizes obtained from the backbone feature extraction network are input into the enhanced feature extraction network. The sizes of these three feature maps are 80×80×256, 40×40×512, and 20×20×1024, corresponding to small, medium, and large targets, respectively. Therefore, to meet the target detection requirements of this paper, which primarily targets large targets, the model structure is improved as follows: Figure 4 .

[0057] Table 1 Target Size Definition

[0058]

[0059] Specifically, the process of lightweighting the trained YOLOv5s model through network pruning in S3 is as follows:

[0060] Network pruning refers to introducing a scaling factor for each channel, multiplying the output of each channel by the scaling factor, then jointly training the network weights and these scaling factors, applying sparsity regularization to the scaling factors, pruning channels with smaller outputs, and fine-tuning the network to obtain a lightweight model.

[0061] In the YOLOv5 model, batch normalization is performed after each convolution to make the output values ​​more stable. According to the BN layer model formula:

[0062]

[0063] Here, ε is a positive value close to 0, to prevent variance σ from being... B A value of 0 results in invalid calculations. z is an intermediate variable with no actual meaning. in and z out For the input and output of the BN layer, μ Band σ B γ and β are the mean and standard deviation of the input activations on the current mini-batch B, and γ and β are the weights and biases of the BN layer. Each channel outputs z. out The coefficient γ is positively correlated with the network pruning scale. When γ approaches 0, the output is very small, and pruning this channel has little impact on the model. This is achieved by adding an L1 regularization constraint to γ ​​and applying it to the loss function.

[0064]

[0065] Where x, W, y, and Γ represent the neural network input vector, neural network weight matrix, ideal output vector, and the set of weights for all BN layers in the network, respectively; L is the sparse training loss function, l(f(x,W),y) is the normal training loss function, g(γ)=|γ| is the L1 regularization constraint, and λ is the regularization coefficient, adjusted according to the dataset. During backpropagation, each channel will have two consecutive scaling factors γ. After sparse training, many scaling factors will approach 0. All input-output connections and their corresponding weights are removed to prune channels where γ is close to 0. The network accuracy loss is compensated through subsequent fine-tuning. Finally, a compact model is obtained through repeated iterations.

[0066] Considering that the detection object is only one class, medicine package, the model adjusts the number of channels in the last layer output by adjusting the channels from 512, 1024, and 2048 to 15 = 3 × (4 + 1) channels. That is, each cell has 3 detection boxes, 4 parameters are used for bounding box regression, and 1 parameter represents the confidence score of the detection box. No classification parameters are needed. The model only needs to perform bounding box regression. Compared with the original model, it has multi-class classification capability but has a lot of channel redundancy. Network pruning can be used to make the model lightweight. The resulting model is only 20M, which is 20% smaller and more suitable for deployment on embedded devices.

[0067] Specifically, the steps in S4 for inputting the image into the NMS method for processing are as follows:

[0068] S4-1: Sort all detection boxes in descending order of confidence level;

[0069] S4-2: Select the detection box with the highest confidence level, confirm it as the correct detection box, and calculate the IoU between the correct detection box and other detection boxes:

[0070]

[0071] Where iou(a,b) c ) represents detection boxes a and b c The IOU values, where a represents the correct bounding box, and b represents the correct bounding box. cLet C be the c-th detection box, where C is the number of detection boxes remaining in the image after deleting detection boxes below the confidence threshold.

[0072] S4-3: Based on all iou(a,b) calculated in S4-2 c ), delete iou(a,b) c Detection boxes with a value greater than the NMS threshold are processed until all detection boxes have been processed.

[0073] Specifically, the NMS threshold selection steps are as follows:

[0074] The selection of the non-maximum suppression threshold is crucial to the accuracy of target detection and counting. If the threshold is too small, it will lead to missed detections; if the threshold is too large, it will lead to false detections. This invention has high requirements for the counting accuracy of drug packets, and there is a large overlap after the drug packets are grasped. Therefore, a suitable and accurate NMS threshold must be selected. This invention requires that the detection frames of individual drug packets be sufficiently suppressed, while the detection frames of drug packets in the overlapping area are fully preserved. Therefore, the principle for selecting the NMS threshold is: while ensuring that each individual drug packet can only be selected by one detection frame, the NMS threshold should be as large as possible.

[0075] Independent pillbox: A pillbox has no overlap with any other pillbox and is appropriately far away from any other pillbox. Avoid overlap between the prediction boxes of different independent pillboxes.

[0076] Step 1: Use the LabelImg labeling tool to label the individual medication packets, such as... Figure 2 Only medicine packets 1, 2, and 3 are labeled, Ω = (ω i ) n Let ω represent the set of n original images. i Let σ represent the i-th image. i =(θ ij ) m Represents ω i The set consisting of m independent real frames of medicine packages.

[0077] Step 2: Set the NMS threshold to 0.9 and call the optimal model to predict ω. i This yields a set δ consisting of s prediction boxes. i =(λ ik ) s , λ ik s represents the k-th (k=1,2,...,s) predicted box for the medicine pack in the i-th image and the total number of medicine pack detection boxes in the i-th image, respectively.

[0078] Step 3: If iou(θ) ij ,λ ik If )>0, then λ ik Add to set Λ = (bt ) u Sort the boxes by confidence level and calculate the maximum confidence level of the detection box M and the remaining detection boxes b in Λ. t The intersection-union ratio, i.e., iou(M,b) t ), take the NMS threshold as the minimum intersection-union ratio, that is, th=min{iou(M,b t )}.

[0079] Step 4: Repeat Step 3 m times and update th.

[0080] Step 5: Repeat Step 4 n times and update th.

[0081] Step 6: Set the final NMS threshold as N th = [0.95, 1] ​​× th.

[0082] The NMS threshold obtained through the above process is more accurate than the threshold obtained empirically. It can ensure that no false detection of the target occurs, that is, that each individual drug pack is strictly suppressed and only one detection box remains. It can also minimize the suppression of the detection boxes of overlapping drug packs due to the NMS threshold being too large, thus preventing missed detections and maximizing the accuracy of drug pack counting.

[0083] Furthermore, this method is also applicable to other NMS algorithms that rely on the NMS threshold; if the calculation of IOU in the method is replaced with the calculation of ciou, diou, etc., this method can also be applied to NMS algorithms under different IOU values.

[0084] Experiment and Results Analysis

[0085] 1. Model runtime environment

[0086] The environment configuration and some related parameter settings are shown in Table 1. Other parameters are set to their default values.

[0087] Table 1 Configuration and Parameters

[0088] 1) Evaluation indicators for model performance

[0089] In the current field of object detection, commonly used evaluation metrics include mean precision, precision, recall, and F1 score. Before proceeding, we will first introduce four related parameters: true positives (TP), false positives (FP), true negatives (TN), and false negatives (FN), which represent the number of samples correctly detected as positive, the number of incorrectly detected as positive, the number of correctly detected as negative, and the number of incorrectly detected as negative, respectively.

[0090] Precision (P): The probability that the model correctly detects a positive sample.

[0091]

[0092] Recall (R): Measures the ability of a classifier to identify all positive cases.

[0093]

[0094] Mean Average Precision (mAP): AP represents the average precision for a single class, and is represented by the area under the PR curve. Since this paper only involves the drug packaging class, the mean average precision is equal to the average precision for a single class.

[0095]

[0096] In this experiment, the only target is the medicine pack, so TN = 0.

[0097] 2) NMS threshold evaluation index

[0098] mAP is used to compare the performance of different models. To fully evaluate the effect of the threshold calculated by LITST on model counting, the NMS threshold selection evaluation index is defined as: counting accuracy P. C The formula is as follows:

[0099]

[0100] Wherein, P and R represent precision and recall, respectively, and the calculation formulas are given by (5) and (6);

[0101] The model output is N = TP + FP, which represents the final number of detected drug packets. The closer TP is to N... gt The closer FP is to 0, the more accurate the counting; where TP is closer to N... gt The closer R is to 1, the closer FP is to 0; the closer P is to 1, the closer FP is to 1. Therefore, the NMS threshold N t The selection should ensure that the values ​​of P and R are both relatively large, and that the variation of P and R follows the same pattern as N. t As the threshold increases, P decreases, and R increases; the Euclidean distance between P, R, and 0 is defined to characterize the counting accuracy. When the NMS threshold is chosen to be too large or too small, P... C Both will be relatively small; therefore, the NMS threshold should be selected to maximize counting accuracy. Evaluation index P C Compared to mAP, it can more effectively reflect the effect of detection box suppression and retention, and intuitively reflect the difference in the effectiveness of NMS threshold on counting accuracy.

[0102] ② Experimental methods

[0103] 1) Constructing the WNA augmentation dataset. A smart medicine cabinet continuously dispenses medicine, and 612 original images are captured, including three types of medicine packets: cassia seed, angelica root tail, and rehmannia root. The original images are annotated using the LabelImg tool, generating TXT format annotation files. To fully compare the differences between WNA-augmented and unaugmented training models, the original dataset is divided into two parts: one part, consisting of 512 images and their annotation files, serves as the training set; the other part, consisting of 100 images and their annotation files, serves as the test set. The training set is augmented using the WNA data augmentation method, resulting in 1536 images and 1536 TXT annotation files for model training. Since the algorithm optimization involves modifying the model structure, no pre-training weights are loaded during the training process of any model; the training generations are set to 150, and other parameters remain at their default values.

[0104] 2) WNA Data Augmentation. After setting the parameters, the dataset was imported into the YOLOv5s model for training. To compare the effectiveness of WNA data augmentation, the original image training set and the WNA-augmented training set were used for training respectively. After obtaining the two models, they were tested with the same parameters (NMS threshold of 0.5) using the same test set to obtain relevant metrics.

[0105] 3) Structural Improvement. To verify the effectiveness of the improved YOLOv5s model structure, the improved model is denoted as YOLOv5s+. The same WNA augmented dataset is fed into the improved model for training, and the model is compared with the YOLOv5s trained on the WNA dataset.

[0106] 4) Lightweighting. To verify the lightweighting effect of the model, the original YOLOv5s model, the improved YOLOv5s+ model, and the YOLOv5s+ model after network pruning (denoted as YOLOv5s-) were all exported as ONNX deployment files, and the model size and number of parameters of the three were compared.

[0107] 5) LITST. After model training, the test set data was processed, and the corresponding program was run to obtain an NMS threshold of 0.393. To compare the effectiveness of LITST, a YOLOv5s-model trained with WNA augmented data was selected. Evaluation metrics were calculated and compared on the same test set using different NMS thresholds, with thresholds of 0.1, 0.2, 0.3, 0.393, 0.4, 0.5, 0.6, 0.7, 0.8, and 0.9 being compared respectively.

[0108] ③ Comparison of experimental results

[0109] Table 2 Comparison of test results for different data augmentation methods

[0110]

[0111] Table 3 Comparison of different model sizes

[0112]

[0113] Based on the experimental results shown in Table 2:

[0114] 1) Comparing the model test results on the un-augmented dataset and the WNA augmented dataset, the model's precision (P) and recall (R) both improved, increasing from 0.967 to 0.985 and from 0.918 to 0.977, respectively, and mAP... 0.5 The value increased from 0.952 to 0.971, mAP 0.5:0.95 The enhancement effect was the most significant, with an improvement of nearly 30%, and this experimental result fully demonstrates the effectiveness of WNA data enhancement.

[0115] 2) Comparing YOLOv5s+ and YOLOv5s models, both trained using WNA data augmentation, mAP 0.5 Increased by approximately 1%, mAP 0.5:0.95 An improvement of 2.5%. This demonstrates that the improved YOLOv5 model further enhances model accuracy.

[0116] 3) For YOLOv5s- and YOLOv5s+ models that are both trained using WNA data augmentation, mAP 0.5 Basically unchanged, mAP 0.5:0.95 An improvement of 3%. While maintaining model accuracy, according to the experimental results in Table 3, the YOLOv5s- model reduces the actual model deployment file onnx by more than 20% compared to the YOLOv5s+ model, achieving lightweight deployment while ensuring detection accuracy.

[0117] Since this paper deals with a detection scenario where the camera is fixed and the background similarity is extremely high, the trained model's mAP... 0.5 All were relatively high, remaining above 0.95.

[0118] Table 4 Comparison of test results for different NMS threshold models

[0119]

[0120]

[0121] Table 4 compares the model test results for different NMS thresholds, where N gt N represents the number of real drug packets in the test set, N represents the number of drug packets tested in the test set, TP represents the number of correctly detected positive samples, FP represents the number of incorrectly detected positive samples, P represents precision, and R represents recall. C To represent the counting accuracy, and for easier and more intuitive comparison of counting accuracy, parameter P is set. CnN represents the counting accuracy after normalization using equation (4.30) via the range transformation method. The range normalization method will be introduced in section 4.3.3 of the next chapter; t Represents the NMS threshold, N t =0.393 is the threshold obtained by LITST. As can be seen from the table, when N... t As N increases from 0.1 to 0.9, P decreases and R increases. t When = 0.393, TP, FP and N gt P has the highest degree of proximity. Cn maximum.

[0122] In addition, N in the comparison table t =0.393 and N t Two rows of data with N = 0.8 t The number of detection boxes N=855 corresponding to a value of 0.8 is relatively large, resulting in the highest mAP; at this point, the corresponding TP is close to N. gt However, FP deviates too much from 0, indicating that mAP is difficult to accurately characterize the counting accuracy; while N t =0.393 corresponds to good TP and FP values, indicating that the NMS threshold obtained by LITST is effective for counting accuracy. It can also be seen that the NMS threshold selection has a certain range, approximately [0.3, 0.4]. The more sufficient the test set, the more accurate the threshold obtained by this method.

[0123] Based on the medicine cabinet dispensing process, the quantity of medicine packets is first determined, and then the medicine is dispensed according to the detection results. The accuracy of counting has a higher priority on improving dispensing efficiency than model precision. Furthermore, the difference in model precision is relatively small; therefore, N is selected. t =0.393 is more suitable for smart medicine cabinets.

[0124] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method of counting pills, characterized by, Comprising the following steps: S1: Obtain an original image dataset, and perform weak non-target region (WNA) data enhancement processing on the images in the original image dataset to construct an enhanced dataset; The process of constructing the enhanced dataset in S1 is as follows: The WNA data enhancement includes target region extraction and weak non-target region extraction; wherein the target region extraction adopts an existing digital image processing method, and the weak non-target region extraction refers to setting the non-target region pixels to a random value alpha multiplied by the original pixel value, and the target region remains unchanged, and the formula is as follows: Wherein, P(i,j) represents the pixel value of (i,j) position in the image, sigma represents the target region, and alpha is a random value in [0,1]; S2: Construct an improved YOLOv5s model, train the improved YOLOv5s model using the enhanced dataset to obtain a trained YOLOv5s model; The improved YOLOv5s model in S2 inserts a CBS+C3 module before the last layer CBS+SPP+C3 of the backbone network, and connects the feature maps with outputs of 40x40x512, 20x20x1024 and 10x10x2048 of the backbone to a strengthened feature extraction network FPN, and discards the output of 80x80x256 feature map; S3: Network pruning is performed on the trained YOLOv5s model to obtain an optimal model; S4: input an original picture to be predicted into the optimal model, and output a picture with multiple detection boxes, each detection box corresponds to a confidence, after deleting the detection boxes with confidence lower than a threshold from the picture, input the picture into an NMS method for processing, and output a final detection picture, the number of detection boxes in the final detection picture is the number of medicine packages; The NMS threshold selection step in the NMS method is as follows: Step 1: Use the LabelImg labeling tool to label the independent medicine bag, only label medicine bags 1, 2, 3, Ω = (ω i ) n represents a set of n original pictures, ω i represents the ith picture, and σ i = (θ ij ) m represents a set of m independent medicine bag real boxes in ω i ; Step 2: Set the NMS threshold to 0.9, call the optimal model to predict ω i , get a set of s prediction boxes δ i =(λ ik ) s , λ ik , s respectively represent the k (k = 1, 2,..., s) of the i picture The k (k = 1, 2,..., s) of the i picture contains the total number of medicine bag detection boxes Step 3: If iou(θ) ij ,λ ik If ) > 0, then λ ik Add to set Λ = (b t ) u Sort the boxes by confidence level and calculate the maximum confidence level of the detection box M and the remaining detection boxes b in Λ. t The intersection-union ratio, i.e., iou(M,b) t ), take the NMS threshold as the minimum intersection-union ratio, that is, th=min{iou(M,b t )}; Step 4: execute Step 3 for m times to update th; Step 5: execute Step 4 for n times to update th; Step 6: Take the final NMS threshold as N th = [0.95, 1] x th.

2. A method of counting pills as claimed in claim 1, wherein: The process of network pruning in S3 to make the trained YOLOv5s model lightweight is as follows: In the YOLOv5s model, batch normalization processing is performed after each convolution to make the output value of the convolution more stable, according to the BN layer model formula: where ε is a positive value close to 0, to prevent invalid calculation when variance σ B is 0, is an intermediate variable, with no actual meaning, z in and z out are the input and output of BN layer, μ B and σ B are the mean and standard deviation of input activations on current mini-batch B, γ and β are the weights and bias of BN layer, each channel output z out is scaled by coefficient γ, which is positively correlated, and is selected as the scaling factor of network pruning, when γ tends to 0, the output is also very small, by adding L1 regularization constraint to γ and acting on the loss function: wherein x, W, y, Γ respectively represent the neural network input vector, the neural network weight matrix, the ideal output vector, and the set of all BN layer weights in the network. L is a sparse training loss function, l(f(x,W),y) is a normal training loss function, g(γ)=|γ| is an L1 regularization constraint, and λ is a regularization coefficient; when performing back propagation, each channel will have two consecutive scaling factors γ, after sparse training, all input-output connections and the corresponding weights are deleted to prune the channels with γ close to 0; the network accuracy loss is compensated through subsequent fine-tuning process; finally, a compact model is obtained through repeated multiple iterations.

3. A method of counting pills as claimed in claim 2, wherein: The step of inputting the picture into the NMS method for processing in S4 is as follows: S4-1: sort all detection boxes in descending order of confidence; S4-2: select the detection box with the highest confidence as the correct detection box, and calculate the iou between the correct detection box and other detection boxes: wherein iou(a, b c ) is the iou value of the detection box a and the detection box b c , a is the correct detection box, b c is the cth detection box, and C is the number of detection boxes remaining in the picture after deleting the detection boxes below the confidence threshold. S4-3: delete the detection box with the value of iou(a, b c ) greater than the NMS threshold according to all iou(a, b c ) values calculated in S4-2 until all detection boxes are processed.

Images