Paper medicine box steel seal character recognition method based on improved YOLOV5 model

Abstract

The application discloses a paper medicine box steel seal character recognition method based on an improved YOLOV5 model, and the method comprises the following steps: collecting the steel seal character image of the medicine box by using an image acquisition device; performing image enhancement and other pretreatments on the image and labeling the recognition area; inputting into the improved YOLOV5 model for training to obtain the trained YOLOV5 model; the improved YOLOV5 model comprises the following steps: adding an efficient position attention mechanism CA (Coordinate attention) in the backbone network of YOLOV5, so that the model network can pay attention to a wide range of position information without bringing too much calculation amount, which is helpful to improve the model performance and better locate and identify the target; using SimSPPF to replace SPPF fast pooling layer to improve the training speed; for the dense small target, an additional smaller prior box is used in the model, and a smaller detection head is additionally added; inputting the data set to be recognized into processing to obtain the final prediction result.

Description

Technical Field

[0001] This invention relates to the field of medicine box target recognition, and in particular to a method for recognizing stamped characters on paper medicine boxes based on an improved YOLOv5 model. Background Technology

[0002] Embedded characters are frequently found on various medicine boxes, food products, machine nameplates, and industrial materials. They primarily include information such as serial numbers, production dates, and operating conditions. Compared to inkjet and printed characters, the embossing process is relatively simple and less susceptible to environmental contamination, physical wear, and fading over time, thus maintaining legibility. For medicine boxes, the production date and expiration date are paramount, hence the widespread use of embossed characters. The accuracy of the embossed characters is crucial to ensuring consumers can use the medication within the specified timeframe; therefore, recognizing these characters on medicine boxes is a critical step.

[0003] Initially, manufacturers typically relied on hired labor to inspect stamped characters on product lines. This method was not only slow but also prone to errors and missed detections due to human subjectivity—each person's judgment of character clarity varied—and the high cost of hiring labor. Furthermore, with advancements in technology, machine-based character recognition has emerged, with OCR text recognition and template matching becoming increasingly sophisticated. For manufacturers using printed or inkjet printing, characters like production dates and expiration dates are easier to identify due to their high clarity and clear contrast with the background color, making them readily identifiable using conventional image processing methods. However, stamped characters, with their similar color to the background and less contrast, are difficult to recognize. Additionally, cameras capturing images of rapidly moving medicine boxes on the production line may experience focusing issues, resulting in blurry images that hinder traditional image processing. Environmental factors such as background lighting and the fact that the target is not on a single plane further complicate stamped character recognition, leading to less than ideal accuracy. In recent years, object detection has garnered significant attention in the field of machine vision and has been widely adopted by many manufacturers for pipeline inspection and recognition. Numerous object detection methods have been proposed, and many excellent neural networks, such as CNN, R-CNN, and VGG, have been utilized. Besides considering whether there is room for improvement in detection accuracy, detection speed is also a crucial factor to consider to meet the real-time recognition requirements of pipelines. Summary of the Invention

[0004] Given the above background, to improve the accuracy and efficiency of detecting various stamped characters on paper medicine boxes, a method for recognizing stamped characters on paper medicine boxes based on an improved YOLOv5 model is proposed. This method adds an efficient Coordinate Attention (CA) mechanism to the YOLOv5 model, enabling the model network to focus on a wide range of positional information without incurring excessive computational costs, thus contributing to improved model performance and better target localization and recognition. The SimSPPF fast pooling layer replaces SPPF, improving training speed. For densely packed small targets, a smaller prior box is used within the model, and a correspondingly smaller detection head is added.

[0005] The technical solution adopted in this invention is as follows:

[0006] A method for recognizing stamped characters on paper medicine boxes based on an improved YOLOv5 model includes the following steps:

[0007] S1. Use a camera to acquire clear and blurry images of the stamped characters on paper medicine boxes. After image augmentation and image enlargement, manually label the categories to create a YOLO format dataset.

[0008] S2. Divide the dataset into a validation set, a training set, and a test set according to the proportions.

[0009] S3. Set the parameters batch_size, workers, and Epoch according to the actual configuration information of the computer's memory, graphics card, and video memory.

[0010] S4. Load the pre-trained model, randomly initialize the network weights, and input the dataset into the improved YOLOV5 model for training.

[0011] S5. The YOLOv5 model consists of a Backbone network, a Neck network, and a Head network. The Backbone network extracts features from the dataset, the Neck network uses a dual-tower structure to sample and further extract features, and the Head network uses the extracted features to predict targets based on features of different sizes of anchors in the grid.

[0012] S6. The training prediction map obtained from one round of network training is used to adjust the prior boxes to obtain the predicted boxes. Then, the IoU is calculated with the ground truth boxes of the target, which represents the intersection-union ratio of the predicted boxes and the ground truth boxes. The localization loss is obtained based on the CIoU of the target regression function. Then, the classification loss, confidence loss and localization loss are weighted to obtain the total loss of the network.

[0013] S7. Backpropagate the loss to the improved YOLOv5 model and update the network weight parameters using the SGD stochastic gradient descent method to obtain a new weight model.

[0014] S8. After setting the Epoch value in S3), repeat steps S4)-S7). The network model parameters updated in each round are used as the pre-trained model for the next round of training. Based on the metric information obtained in each round, the best model within the set number of training rounds is finally obtained.

[0015] S9. Input the test images in the test set into the best model in step S8) for training, output the predicted bounding boxes, and group all predicted bounding boxes according to the label of each category based on NMS maximum suppression. Sort the confidence scores in the groups from largest to smallest to obtain the rectangle with the highest score. Then, iterate through the remaining rectangles and calculate the ratio of the intersection to the union of the rectangle with the highest score. If it is greater than a preset threshold, remove it. Then, repeat the above operation on the remaining detection boxes until the final predicted bounding box is obtained, thus realizing the recognition of the stamped characters on the medicine box.

[0016] Furthermore, the image enhancement and dataset augmentation methods in step S1) include: image contrast, image blur, image noise addition, image scaling, and image rotation.

[0017] Furthermore, the online web tool MakeSense was used to annotate the obtained images. The categories to be identified were pre-defined, and the production dates to be identified were selected with a rectangle. The corresponding category labels were then added. After labeling all images, the YOLO labels were output in TXT format. The label content included the category (class), the normalized x and y coordinates of the rectangle center (x_center, y_center), and the width and height of the rectangle.

[0018] Furthermore, the improved YOLOv5 in step S5) consists of three networks: the Backbone network, the Neck network, and the Head network.

[0019] Furthermore, step S4) of the YOLOv5 model includes the following improvements:

[0020] a. Add a CA position attention mechanism module before the fast pooling layer. This module divides the input feature map into two directions, extracts features along one direction, and retains position information along the other direction. Global average pooling is then performed on each direction, and the feature maps obtained from the two directions are encoded.

[0021] b. The fast pooling layer replaces SPPF with the faster SimSPPF and replaces the SiLU activation function with the ReLU activation function;

[0022] c. For small object detection, a smaller prediction box is added: [5, 6, 8, 14, 15, 11]. Correspondingly, a smaller detection head is added to the Head network. The original Neck network is further upsampled to expand the feature map. The resulting feature map is then fused with the feature map in the Backbone network for small object detection.

[0023] Furthermore, the formula for the total loss weight of YOLOv5 in step S6) is as follows:

[0024] LOSS=

[0025] In this formula: For classification loss, only the classification loss of positive samples is calculated;

[0026] The confidence loss is calculated for all samples.

[0027] For localization loss, CIoU_loss is used, and only the localization loss of positive samples is calculated;

[0028] These are the weighting coefficients;

[0029] Furthermore, in step S6), the CIoU loss function is used as the regression loss function for the predicted bounding boxes in the improved YOLOv5 model algorithm, and its relevant formula is as follows:

[0030]

[0031] CIoU_LOSS=1-CIoU=1-( )

[0032]

[0033] in:

[0034] b -- the center point of the prediction box;

[0035] b gt --The center point of the actual bounding box;

[0036] c -- the diagonal distance of the smallest bounding rectangle that can simultaneously contain both the predicted bounding box and the ground truth bounding box;

[0037] w, h -- width and height of the prediction box;

[0038] --Weighting coefficient;

[0039] -- Aspect ratio similarity coefficient.

[0040] Compared with the prior art, the present invention has the following advantages:

[0041] 1. This invention provides a method for recognizing stamped characters on paper medicine boxes based on an improved YOLOv5 model, by adding a CA position attention mechanism module before the fast pooling layer. This helps the network model to better locate and recognize the stamped characters on the medicine box, thus improving recognition accuracy.

[0042] 2. The fast pooling layer uses the faster SimSPPF instead of SPPF and replaces the SiLU activation function with the ReLU activation function, which improves the training speed and better meets the real-time recognition needs of the pillbox manufacturer's production line.

[0043] 3. For small object detection, a smaller prior box is added, and a smaller detection head is correspondingly added to the Head network. The original Neck network undergoes further upsampling and other processing to expand the feature map to adapt to complex backgrounds, reducing the missed detection of small objects such as stamped characters on medicine boxes and improving the algorithm's robustness. Attached Figure Description

[0044] Figure 1 A flowchart illustrating a method for recognizing stamped characters on medicine boxes, provided as an embodiment of the present invention;

[0045] Figure 2 A schematic diagram of the improved YOLOv5 model structure provided for an embodiment of the present invention;

[0046] Figure 3 This is a graph showing the mAP_0.5 results of an embodiment of the present invention. Detailed Implementation

[0047] To make the technical means and objectives of this invention readily understandable, the invention is further described below in conjunction with specific embodiments. It should be noted that the step numbers are for descriptive purposes only and should not be construed as indicating or implying relative importance. Exemplary embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings, but should not be limited to the embodiments set forth herein.

[0048] Based on the above, this invention provides a method for recognizing stamped characters on paper medicine boxes based on an improved YOLOv5 model, the process of which is as follows: Figure 1 As shown, the specific steps include the following:

[0049] S1. Use a camera to acquire a certain number of clear and blurry images of the embossed characters on paper medicine boxes. Enhance and expand the images by adjusting image contrast, blurring, adding noise, scaling, and rotating the images. On average, there are 200 images for each category. Use the online tool MakeSense to label the embossed characters with rectangles and create a dataset in YOLO format.

[0050] S2. Divide the medicine box stamp image dataset into training, validation, and test sets in an 8:1:1 ratio. Place the TXT format files into the corresponding train, val, and test folders under the labels folder, and place the medicine box stamp character images into the corresponding train, val, and test folders under the images folder.

[0051] S3. The computer used in this example has 16GB of memory and an NVIDIA GTX1060 graphics card with 6GB of video memory. Based on this, set batch_size = 4, workers = 2, Epoch = 200, create a configuration file (.yaml) and set it to the file path corresponding to step S2).

[0052] S4. Set the path for the weights parameter, load the pre-trained model (.pt), randomly initialize the network weights, and input the dataset into the improved YOLOV5 model for training.

[0053] S5. This network model consists of a Backbone network, a Neck network, and a Head network, with the network structure as follows: Figure 2 As shown:

[0054] a. The Backbone network extracts features. The input image is 3*640*640 pixels. To address the imbalance of different image sizes in the dataset, a Mosaic data augmentation structure is first used to randomly scale, crop, and stitch the four images, increasing the receptive field and reducing the loss of initial image information. The output image is 64*320*320 pixels. Then, the image passes through a Conv layer. The specific calculation formula for Conv[ch_out, kernel, stride, padding] is as follows:

[0055] Output=(Input-kernel+2*padding) / stride+1

[0056] Therefore, the output image size is 128*160*160. It then passes through layer C3 for feature fusion, learning the residual features without changing the number of input and output channels. The Backbone network goes through four rounds of Conv and C3 modules, ultimately obtaining four feature maps of different sizes: (20*20*1024, 40*40*512, 80*80*256, and 160*160*128). Then, through the CA positional attention mechanism, the input feature map is divided into width and height directions. Features are extracted along one direction while positional information is preserved along the other. Global average pooling is performed on both directions, and the feature maps obtained in both directions are then encoded using the following formula:

[0057]

[0058]

[0059] Z C It is the output associated with the C channel, and H and W are the height and width of the input image.

[0060] Finally, the SimSPPF fast pooling layer uses three identical 5×5 max pooling layers to fuse feature maps with different receptive fields, enriching the expressive power of the feature maps without changing the feature map size or the number of channels.

[0061] b. The Neck network can better extract features from the Backbone, thus improving network performance. It employs a dual-tower structure—a top-down feature fusion using an FPN structure and a bottom-up feature fusion using a PAN structure. The feature maps obtained from the fast pooling layers are upsampled by PyTorch's built-in Upsample module to obtain feature maps of size 40*40*512. Then, the Concat module concatenates the upsampled feature maps with the corresponding feature maps from the Backbone network, and finally, the C3 module performs feature fusion. Since the Backbone network has already completed the extraction of the main feature information, the Neck network uses the C3 module without residuals. This allows for better fusion of large-scale semantic information and small-scale detail information, thereby enhancing the localization capability at multiple scales. The improved YOLOv5 model adds an extra upsampling round, for a total of 4 rounds, resulting in 4 layers of feature maps for better detection of small targets.

[0062] c. The four detection layers of the improved Head network correspond to the (20*20, 40*40, 80*80, 160*160) feature maps obtained in the Neck. For predicting and regressing the target, each grid on the feature map has an initially defined prior box to store its position and classification information.

[0063] S6. The total loss obtained after one round of network training consists of classification loss, confidence loss, and localization loss:

[0064] a. The predicted bounding boxes are adjusted based on the training prediction map to obtain the predicted bounding boxes, and then the IoU is calculated between the predicted bounding boxes and the ground truth bounding boxes of the target:

[0065] This represents the ratio of the intersection to the union of the predicted bounding box and the ground truth bounding box, and is used to calculate the localization loss based on the target regression function CIoU.

[0066]

[0067] CIoU_LOSS=1-CIoU=1-( )

[0068]

[0069] b. Use the binary cross-entropy function to calculate the classification loss:

[0070]

[0071] c. Similar to the loss calculation method described above, the binary cross-entropy function is used to calculate the confidence loss. ;

[0072] Then, the classification loss, confidence loss, and localization loss are weighted to obtain the total loss of the network:

[0073] LOSS=

[0074] S7. Backpropagate the loss to the YOLOv5 model and update the network weight parameters using the SGD stochastic gradient descent method to obtain a new weight model.

[0075] S8. After setting the Epoch value in S3), repeat steps S4)-S7). The network model parameters updated in each round are used as the pre-trained model for the next round of training. Each training round yields bounding box loss, object detection loss, classification loss, P (precision), R (recall), AP (single-class precision), and mAP (maximum accuracy).

[0076] Indicators such as average accuracy for each category are used; the final trained improved YOLOv5 network yields the best model within a set number of training epochs (best.pt);

[0077] S9. Input the test images in the test set into the best model in S8) for training, output the predicted bounding boxes, and group all predicted bounding boxes according to the label of each category based on NMS maximum suppression. Sort the confidence scores in the groups from largest to smallest to obtain the rectangle with the highest score. Then, iterate through the remaining rectangles and calculate the ratio of the intersection and union with the rectangle with the highest score. If it is greater than the preset threshold, remove it. Then, repeat the above operation on the remaining detection boxes until the final predicted bounding box is obtained, thus realizing the recognition of the stamped characters on the medicine box.

[0078] S10. In order to judge the model's running speed, computational load, accuracy, loss and other parameters, as well as to understand the model's generalization ability, the indicators in S8) need to be used for measurement. Based on the comparison of the indicators of different models, the parameters of the network model are further adjusted and the structure is optimized, and finally it is confirmed whether the actual ability of the network meets the detection standard.

[0079] Based on the results obtained from the above embodiments, the SimSPPF fast pooling layer improved the average running speed of the original SPPF single layer by 17.16%, as shown in Table 1. This example compares the mAP index of YOLOv5 model training before and after the improvement, as shown in Table 2.

[0080]

[0081] Table 1 Average running speed of single layer SPPF and SimSPPF

[0082]

[0083] Table 2 Comparison of mAP metrics for YOLOv5 model training before and after improvement

[0084] As described in the above embodiments, this invention provides a method for recognizing stamped characters on paper medicine boxes based on an improved YOLOv5 model. The backbone network is optimized by embedding a CA position attention module within it, which helps the model better locate and recognize targets, improving the detection accuracy of small targets. SimSPPF replaces the original SPPF, accelerating the recognition speed and better adapting to the needs of real-time pipeline detection. Adding a smaller prior box and detection head enhances the model's ability to recognize small targets, reducing missed detections of small targets. Even in more complex backgrounds, this method can effectively improve the problems of missed and false detections of stamped characters on paper medicine boxes.

[0085] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. The general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, and various modifications and improvements to the technical solutions of the present invention should fall within the protection scope of the present invention.

Claims

1. A method for recognizing stamped characters on paper medicine boxes based on an improved YOLOv5 model, characterized in that, Includes the following steps: S1. Use a camera to acquire clear and blurry images of the stamped characters on paper medicine boxes. After image augmentation and image enlargement, manually label the categories to create a YOLO format dataset. S2. Divide the dataset into a validation set, a training set, and a test set according to the proportions. S3. Set the parameters batch_size, workers, and Epoch according to the actual configuration information of the computer's memory, graphics card, and video memory. S4. Load the pre-trained model, randomly initialize the network weights, and input the dataset into the improved YOLOv5 model for training; the YOLOv5 model includes the following improvements: a. A CA (Category A) positional attention mechanism module is added before the fast pooling layer. This module divides the input feature map into two directions, extracts features along one direction, and retains positional information along the other direction. Global average pooling is then performed on each direction. Finally, the feature maps obtained from the two directions are encoded, as shown in the following formula: Z C This is the output associated with the C channel, where H and W are the height and width of the input image; b. The fast pooling layer replaces SPPF with the faster SimSPPF and replaces the SiLU activation function with the ReLU activation function; c. For small object detection, a smaller prediction box is added: [5, 6, 8, 14, 15, 11]. Correspondingly, a smaller detection head is added to the Head network. An upsampling process is performed on the original Neck network to further expand the feature map. The resulting feature map is then fused with the feature map in the Backbone network for small object detection. S5. The YOLOv5 model consists of a Backbone network, a Neck network, and a Head network. The Backbone network extracts features from the dataset, the Neck network uses a dual-tower structure to further extract features, and the Head network uses the extracted features to predict targets based on features of different sizes of anchors in the grid. S6. The training prediction map obtained from one round of network training is used to adjust the prior boxes to obtain the predicted boxes. Then, the IoU is calculated with the ground truth boxes of the target, which represents the intersection-union ratio of the predicted boxes and the ground truth boxes. The localization loss is obtained based on the CIoU of the target regression function. Then, the classification loss, confidence loss and localization loss are weighted to obtain the total loss of the network. S7. Backpropagate the loss to the improved YOLOv5 model and update the network weight parameters using the SGD stochastic gradient descent method to obtain a new weight model. S8. After setting the Epoch value in S3), repeat steps S4)-S7). The network model parameters updated in each round are used as the pre-trained model for the next round of training. Based on the metrics obtained in each round, the optimal model within the set number of training rounds is finally obtained. S9. Input the test images in the test set into the best model in step S8) for training, output the predicted bounding boxes, and group all predicted bounding boxes according to the label of each category based on NMS maximum suppression. Sort the confidence scores in the groups from largest to smallest to obtain the rectangle with the highest score. Then, iterate through the remaining rectangles and calculate the ratio of the intersection to the union of the rectangle with the highest score. If it is greater than a preset threshold, remove it. Then, repeat the above operation on the remaining detection boxes until the final predicted bounding box is obtained, thus realizing the recognition of the stamped characters on the medicine box.

2. The method for recognizing stamped characters on paper medicine boxes based on an improved YOLOv5 model according to claim 1, characterized in that: The image enhancement and dataset augmentation methods in step S1) include: image contrast, image blur, image noise addition, image scaling, and image rotation.

3. The method for recognizing stamped characters on paper medicine boxes based on an improved YOLOv5 model according to claim 2, characterized in that: The images were labeled using the online web tool MakeSense. The categories to be identified were pre-defined, and the production dates to be identified were selected with a rectangle. The corresponding category labels were then added. After labeling all images, the YOLO labels were output in TXT format. The label content included the category (class), the normalized x and y coordinates of the rectangle center (x_center, y_center), and the width and height of the rectangle.

4. The method for recognizing stamped characters on paper medicine boxes based on an improved YOLOv5 model according to claim 1, characterized in that: The improved YOLOv5 in step S5) consists of three networks: Backbone network, Neck network, and Head network.

5. The method for recognizing stamped characters on paper medicine boxes based on an improved YOLOv5 model according to claim 1, characterized in that: The formula for the total loss weight of YOLOv5 in step S6) is as follows: LOSS= In this formula: For classification loss, only the classification loss of positive samples is calculated; The confidence loss is calculated for all samples. For localization loss, CIoU_loss is used, and only the localization loss of positive samples is calculated; These are the weighting coefficients.

6. The method for recognizing stamped characters on paper medicine boxes based on an improved YOLOv5 model according to claim 1, characterized in that: In step S6), the CIoU loss function is used as the regression loss function for the predicted bounding boxes in the improved YOLOv5 model algorithm, and its relevant formula is as follows: CIoU_LOSS=1-CIoU=1-( ) in: b -- the center point of the prediction box; b gt --The center point of the actual bounding box; c -- the diagonal distance of the smallest bounding rectangle that can simultaneously contain both the predicted bounding box and the ground truth bounding box; w, h -- width and height of the prediction box; --Weighting coefficient; -- Aspect ratio similarity coefficient.

Images