Box cover counting method and device based on YOLOv8 model

Abstract

The invention relates to a box skin counting method and device based on a YOLOv8 model, and the method comprises the steps: carrying out the data processing of a box skin image, lightening the YOLOv8 model, and replacing a part of standard convolution in a shallow convolution layer in a backbone network of the YOLOv8 model with deep separable convolution; a PANet structure in the YOLOv8 model is optimized; feature fusion is improved, a weighted fusion mode is adopted, and weights are distributed according to the importance of feature maps with different scales; after knowledge distillation, a lightweight YOLOv8 model is obtained; and performing real-time reasoning acceleration, detection and counting. The case cover counting device comprises a mobile terminal and case cover counting software installed on the mobile terminal, and the mobile terminal is a mobile phone or a PAD and is provided with a camera and a display screen; the box cover counting software comprises an operation module, an image acquisition module, an image data preprocessing module and a detection and counting module. According to the invention, under the scenes of complex illumination, shielding and the like, the detection accuracy of the case leather can reach 96.5% or above.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of deep learning and logistics and warehousing, in particular to a box skin counting method and device based on a YOLOv8 model. BACKGROUND

[0002] In the logistics, warehousing and other industries, box skin as a common goods packaging and storage container, the accurate statistics of its quantity is an important basis for goods management, scheduling and transportation. At present, the box skin quantity statistics mainly relies on manual completion, which has many drawbacks. First, the manual counting efficiency is very low, and a large amount of manpower and time cost is needed when facing a large number of box skins; second, long-term repetitive work can easily cause visual fatigue of the workers, resulting in counting errors and affecting the accuracy of goods management; third, manual counting is difficult to meet the demand of modern logistics and warehousing industry for efficient and automated management.

[0003] Some existing automatic counting technologies use traditional image processing algorithms such as edge detection and contour extraction. However, these algorithms perform poorly in complex scenarios, such as uneven lighting, box skin stacking or partial occlusion, resulting in a significant decrease in detection accuracy and failing to meet the needs of practical applications.

[0004] With the rapid development of deep learning in the field of target detection, although there are target detection methods based on deep learning applied to object counting, most of the existing solutions are designed for fixed scenes or special equipment, which have the problems of large model calculation and high hardware requirements, and are difficult to run efficiently on mobile terminals such as mobile phones and PAD. Mobile terminals have wide application potential in logistics and warehousing scenarios as portable devices, but how to realize high-precision and real-time box skin counting under the limited computing resources and storage conditions of mobile terminals is still a technical problem to be solved. SUMMARY

[0005] In view of the deficiencies of the prior art, the present application provides a box skin counting method and device based on a YOLOv8 model, which realizes fast and accurate counting of box skins in complex scenes by optimizing the deep learning model and improving the data processing process, reduces the dependence on hardware devices, improves the portability and practicality of the system, and meets the actual needs of the logistics, warehousing and other industries.

[0006] To achieve the above purpose, the present application provides the following technical solutions: On the one hand, a box skin counting method based on a YOLOv8 model is proposed, which includes: Box skin image data processing, including: box skin image data preprocessing, mixed data enhancement and labeled data of box skin image data; Lightweight YOLOv8 model, including: Replace part of the standard convolution in the shallow convolution layer of the backbone network of the YOLOv8 model with a depth separable convolution. Optimize the PANet structure in the YOLOv8 model, reduce the number of channels of the intermediate feature map, and replace the bilinear interpolation upsampling with nearest neighbor upsampling; improve feature fusion by using a weighted fusion method, and assign weights according to the importance of different scale feature maps; Knowledge distillation, build a teacher-student model training architecture, use a large pre-trained model as the teacher model and a lightweight model as the student model; when training the student model, calculate the hard label loss and the soft label loss simultaneously, and sum them as the final loss function, so that the student model learns the decision information of the teacher model; Get the lightweight YOLOv8 model; Real-time inference acceleration, including: INT8 quantization of the weights and activation values of the lightweight YOLOv8 model, mapping FP32 data to INT8 range by calculating the scaling factor and zero point; model pruning, using channel pruning method, calculating the importance score of each channel of the convolution layer, removing channels with low scores according to the set pruning ratio; multi-thread parallel processing, in mobile terminal application, creating image acquisition thread and detection thread, image acquisition thread puts the collected images into the queue, detection thread takes images from the queue for detection, and data transmission between threads is realized through the queue; Detection and counting, input the preprocessed container image data into the lightweight YOLOv8 model for target detection, and output the detection box of the container; for overlapping containers, use the clustering algorithm based on intersection over union to merge and correct the detection box; through historical frame data fusion, eliminate the counting error caused by short-term jitter, and finally count and output the number of containers.

[0007] As a preferred technical solution, part of the standard convolution in the shallow convolution layer of the backbone network of the YOLOv8 model is replaced with a depth separable convolution, specifically 20% to 30% of the standard convolution is replaced with a depth separable convolution.

[0008] As a preferred technical solution, when performing knowledge distillation, a multi-scale feature fusion mechanism is introduced into the student network to match the teacher network's recognition ability of container targets at different scales; by constructing FPN or BiFPN structure in the student model, the perception ability of different size targets is enhanced; in the distillation process, feature alignment loss or MSE loss is used.

[0009] As a preferred technical solution, the box skin image data is subjected to hybrid data augmentation, including using Mosaic data augmentation, randomly selecting four images to stitch together a new image, increasing the diversity and density of targets in the image; and using CutMix data augmentation, randomly cropping a region of one image and replacing it at a random position in another image, while adjusting the label information, so that the model can learn the contextual information of the target image.

[0010] As a preferred technical solution, the annotation data is specifically updated as follows: when performing preprocessing of container skin image data and hybrid data augmentation of container skin image data, the corresponding annotation data is updated synchronously to ensure that the annotation box accurately defines the transformed container skin target.

[0011] As a preferred technical solution, the Labellmg annotation tool is used to annotate the images, draw bounding boxes for each box skin and label them with the category "box", and generate the corresponding annotation file; the annotated dataset is divided into training set, validation set and test set in a ratio of 7:1.5:1.5.

[0012] As a preferred technical solution, the YOLOv8 model is used with the Adam optimizer, the learning rate is set to 0.001, and the training is performed for 100 epochs.

[0013] As a preferred technical solution, the preprocessing of the box hull image data includes data collection, image cleaning, adaptive brightness adjustment, and geometric transformation; image cleaning includes blurred image screening, duplicate image detection, and abnormal image removal; among them, blurred image screening adopts the Laplacian variance algorithm, and images with a Laplacian variance lower than the set threshold of 100 are judged as blurred images and removed; duplicate image detection adopts the perceptual hash algorithm to hash the collected images, calculate the hash distance between images, and when the hash distance is less than the set threshold of 5, it is judged as a duplicate image, and only one of them is retained.

[0014] As a preferred technical solution, channels with low scores are removed according to a set pruning ratio, which is 20% to 25%.

[0015] On the other hand, a container counting device implementing the above method is proposed, comprising: a mobile terminal and container counting software installed on the mobile terminal; the mobile terminal is a mobile phone or a tablet, equipped with a camera and a display screen; the container counting software includes: The runtime module is used to run the box counting software and perform data processing and model inference tasks. The image acquisition module connects to the camera of the mobile terminal, controls the camera to acquire images, and transmits the images to the preprocessing module; Image data preprocessing module: performs cleaning, adaptive brightness adjustment, and geometric transformation normalization on the original image; Detection and counting module: Includes a lightweight YOLOv8 object detection model to perform object detection and output detection boxes; performs post-processing for counting by fusing IoU-based clustering algorithm and historical frame data to count the number of boxes; The display module marks the test results on the screen and displays the number of boxes in real time.

[0016] Compared with the prior art, the technical solution of this application has the following beneficial effects: (1) Improve detection accuracy. Through data preprocessing and enhancement, lightweight model optimization and other technologies, the accuracy of box skin detection can reach more than 96.5% in complex lighting and occlusion scenarios, which is significantly higher than traditional methods and unoptimized deep learning models.

[0017] (2) Fast real-time performance: By combining lightweight models, quantization techniques, model pruning and multi-threaded processing, the single counting processing time can be ≤0.25 seconds on ordinary mobile terminals, which meets the real-time counting requirements and significantly improves efficiency compared with traditional methods.

[0018] (3) Low cost and portability: Based on general mobile terminals, no special hardware equipment is required, which reduces the system deployment cost; the equipment is easy to carry and can count boxes anytime and anywhere, and is suitable for multiple scenarios such as logistics and warehousing.

[0019] (4) Strong adaptability: Through diverse data augmentation and model optimization strategies, this invention has good detection effect on boxes of different materials, sizes and placement states, and has strong generalization ability and adaptability. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the process for a box counting method based on the YOLOv8 model proposed in this invention; Figure 2 This is a schematic diagram of a box counting device proposed in this invention.

[0021] Figure descriptions: Operation module 10; Image acquisition module 20; Image data preprocessing module 30; Detection and counting module 40; Display module 50. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Please seeFigure 1 The present invention proposes a box counting method based on the YOLOv8 model, comprising: Step 1: Box skin image data processing, including: box skin image data preprocessing, mixed data augmentation of box skin image data, and annotation data.

[0024] Preferably, image data preprocessing includes data collection, which involves collecting container images from multiple sources, such as different logistics warehouses, transportation scenarios, and storage environments. These scenarios cover indoor warehouses (with varying light intensities and angles, such as spotlights on the warehouse ceiling and side window lighting), outdoor loading and unloading areas (affected by natural light, such as sunny days, cloudy days, and different times of day, such as dusk), and the interior of transport vehicles (where vibrations and bumps may cause image blurring), to ensure that the collected data has broad representativeness. Diverse coverage: Collecting images of different types of container skins, including different materials (such as paper, plastic, and metal), different sizes (large, medium, and small), different colors (such as common white, brown, and blue), and different patterns (with text, logos, and no patterns), while also considering different placement states of the container skins, such as single placement, stacked placement, and partial obstruction placement.

[0025] Preferably, image data preprocessing includes image cleaning, including blurry image filtering, duplicate image detection, and abnormal image removal.

[0026] Preferably, the Laplacian variance algorithm is used to filter blurry images. The variance of the image after processing by the Laplacian operator is calculated, and a variance threshold is set, for example, a threshold of 100. Images below the threshold are judged as blurry and are removed.

[0027] The Laplacian operator is a second-order derivative operator used to detect edges and details in images. The more blurred an image is, the fewer edges and details it has, and the smaller the variance of the image after processing with the Laplacian operator. Therefore, the degree of blurriness of an image can be determined by calculating the variance of the image after processing with the Laplacian operator.

[0028] Specifically, the calculation formula is as follows: Let the input image be \(I(x,y)\), where x and y represent the row and column coordinates of the image, respectively. The Laplacian operator on a two-dimensional image is defined as: \(\nabla^{2}I(x,y)=\frac{\partial^{2}I(x,y)}{\partialx^{2}}+\frac{\partial^{2}I(x,y)}{\partial y^{2}}\) In discrete images, the second derivative is typically approximated by convolution operations using the following 3x3 Laplacian template: \(L = \begin{bmatrix} 0&1&0 \\ 1&-4&1 \\ 0&1&0 \end{bmatrix}\) Convolving image I with Laplacian template L yields the convolved image \(L_{I}\): \(L_{I}(x,y)=(I * L)(x,y)=\sum_{i=-1}^{1}\sum_{j=-1}^{1}I(x + i,y + j)L(i + 1,j +1)\) Calculate the variance \(\sigma^{2}\) of the convolved image \(L_{I}\): \(\mu=\frac{1}{M\timesN}\sum_{x = 0}^{M - 1}\sum_{y = 0}^{N - 1}L_{I}(x,y)\) \(\sigma^{2}=\frac{1}{M\times N}\sum_{x = 0}^{M - 1}\sum_{y = 0}^{N - 1}(L_{I}(x,y)-\mu)^{2}\) where M and N are the number of rows and columns of the image, respectively, and \(\mu\) is the mean of the convolved image.

[0029] Preferably, a perceptual hashing algorithm (taking mean hashing as an example) is used to detect duplicate images. By calculating the hash value and Hamming distance of the image, if the distance is less than a set threshold, such as 5, it is considered a duplicate image.

[0030] Perceptual hashing (taking mean hashing as an example) generates a fixed-length hash value by performing steps such as image downsizing, grayscale conversion, calculating the mean, and comparing pixel values ​​with the mean. If two images have highly similar hash values, they are considered duplicate images.

[0031] Specific steps and formulas: Step 121, Image Scaling: Scaling the image to a fixed size (e.g., 8x8). Let the scaled image be \(I_{s}(x,y)\), where \(x = 0,1,\cdots,7\) and \(y = 0,1,\cdots,7\).

[0032] Step 122, Grayscale conversion: Convert the scaled color image to a grayscale image.

[0033] Step 123, Calculate the mean: Calculate the mean of the grayscale image \(\mu\): \(\mu=\frac{1}{8\times8}\sum_{x = 0}^{7}\sum_{y = 0}^{7}I_{s}(x,y)\) Step 124: Compare pixel values ​​with the mean: For each pixel \(I_{s}(x,y)\), if \(I_{s}(x,y)\geq\mu\), then the hash value at that position is 1; otherwise, it is 0. This yields a 64-bit binary hash value H.

[0034] Step 125: Calculate the hash distance: For two hash values ​​\(H_1\) and \(H_2\), use the Hamming distance d to measure their similarity. The Hamming distance is the number of different characters at corresponding positions in two strings of equal length.

[0035] \(d=\sum_{i=0}^{63}[H_1[i]\neqH_2[i]]\) , where \([H_1[i]\neq H_2[i]]\) is a Boolean expression, which is 1 if \(H_1[i]\) is not equal to \(H_2[i]\) and 0 otherwise.

[0036] Preferably, the isolated forest algorithm is used to remove abnormal images. The abnormality score is obtained by calculating the average path length of data points on the isolated trees. Images with scores higher than a set threshold are judged as abnormal images.

[0037] The Isolation Forest algorithm is an unsupervised anomaly detection algorithm based on decision trees. It constructs multiple decision trees (isolation trees) by randomly selecting features and split points to isolate data points. Anomalies are typically isolated in fewer splitting steps, resulting in shorter path lengths. Whether a data point is an anomaly can be determined by calculating the average path length of each data point across all isolation trees.

[0038] Specific steps and formulas: Step 131: Construct an isolation tree: Randomly select n samples from the original dataset as subsamples to construct an isolation tree. At each internal node of each isolation tree, randomly select a feature f and a split point p to divide the samples into two subsets. Repeat this process until each subset contains only one sample or the maximum tree depth is reached.

[0039] Step 132: Calculate the path length: For a data point x, calculate its path length h(x) on each isolated tree. The path length is the number of edges from the root node to the leaf node containing the data point.

[0040] Step 133: Calculate the average path length: Calculate the average path length of data point x across all isolated trees \(E(h(x))\).

[0041] Step 134: Calculate the outlier score: The outlier score \(s(x,n)\) is calculated as follows: \(s(x,n)=2^{-\frac{E(h(x))}{c(n)}}\), where \(c(n)\) is a constant related to the sample size n, and its calculation formula is: \(c(n)=2H(n-1)-\frac{2(n-1)}{n}\). \(H(k)\) is the harmonic number, which can be approximately expressed as \(H(k)=\ln(k)+\gamma\), and \(\gamma\approx0.5772\) is Euler's constant.

[0042] Preferably, image data preprocessing includes adaptive brightness adjustment, comprising the following steps: Step 141: Brightness evaluation. Calculate the average brightness value of the image by iterating through each pixel, accumulating its brightness value, and dividing by the total number of pixels. For color images, convert them to grayscale before performing brightness calculations.

[0043] Step 142: Adaptive Adjustment. Based on the calculated average brightness value, the image brightness is adjusted using histogram equalization or adaptive histogram equalization (CLAHE). If the average brightness value is lower than the set lower threshold (e.g., 50), the image brightness is increased; if it is higher than the upper threshold (e.g., 200), the image brightness is decreased to bring the image brightness into a suitable range and improve the clarity of the box skin features in the image.

[0044] Preferably, image data preprocessing includes geometric transformation, comprising the following steps: Step 151: Rotate the image, using the image center as the rotation point, and randomly select a rotation angle (e.g., between -45° and 45°). During the rotation, use bilinear interpolation or nearest neighbor interpolation to calculate the rotated pixel values ​​to ensure the continuity and integrity of the image. The rotation operation can simulate different placement angles of the box skin in a real-world scene, increasing the diversity of the data.

[0045] Step 152: Scaling. Randomly select a scaling ratio (e.g., between 0.8 and 1.2) to scale the image. Use an appropriate interpolation method during scaling to ensure no significant distortion occurs. Scaling simulates images of box hulls taken at different distances, improving the model's ability to recognize box hulls at different scales.

[0046] Step 153: Translation. Randomly select a translation distance (e.g., between -10% and 10% of the image width and height) in the horizontal and vertical directions to translate the image. After translation, any parts exceeding the image boundaries can be padded (e.g., zero-padding) to maintain the image size. Translation operations can simulate different positions of the box skin in the image, enhancing the robustness of the model.

[0047] Preferably, the box-shell image data undergoes hybrid data augmentation, including: Mosaic data augmentation: four images are randomly selected and combined into a new image according to a certain ratio (e.g., 2×2 stitching). During the stitching process, the size and position of each image are adjusted to ensure that the four images are seamlessly stitched together. Mosaic augmentation can increase the diversity and density of targets in the image, improving the model's detection capability in complex scenes.

[0048] CutMix data augmentation: Randomly select two images, randomly crop a rectangular region from one image, and then replace this rectangular region at a random position in the other image. Simultaneously, adjust the image label information based on the area ratio of the cropped region. CutMix augmentation allows the model to learn contextual information between different targets, improving the model's generalization ability.

[0049] Preferably, the annotation data specifically involves updating the corresponding annotation data synchronously during image preprocessing and enhancement operations. For example, when performing geometric transformations such as image rotation, scaling, and translation, the coordinates of the annotation boxes are transformed accordingly based on the transformation parameters to ensure that the annotation boxes still accurately define the box skin targets. For hybrid data enhancement operations such as Mosaic and CutMix, the annotation information needs to be recombinated and adjusted to reflect the position and category of the box skin in the new image.

[0050] Step 2: Lightweighting the YOLOv8 Model. To ensure efficient bin and tin counting on mobile devices, the YOLOv8 algorithm requires lightweighting the model structure. The following details the optimization process and provides an example of lightweighting the YOLOv8 model. This includes: Step 211: Replace standard convolution with depthwise separable convolution. Principle: Standard convolution performs convolution operations on all channels of the input feature map during computation, resulting in a large computational load. Depthwise separable convolution, on the other hand, splits the standard convolution into two steps: depthwise convolution and pointwise convolution, thus significantly reducing the computational load.

[0051] Depthwise convolution: A convolution operation is performed on each channel of the input feature map, with each channel corresponding to a convolution kernel. Assuming the number of channels in the input feature map is \(C_{in}\) and the kernel size is \(K\times K\), then the computational cost of depthwise convolution is \(C_{in}\times H\times W\times K\times K\), where H and W are the height and width of the feature map, respectively.

[0052] Pointwise convolution: Convolves the output of a depthwise convolution with a 1×1 kernel, adjusting the number of channels from Cin to Cout. The computational cost of pointwise convolution is Cin×Cout×H×W.

[0053] Computational complexity comparison: The computational complexity of standard convolution is \(C_{in}\times C_{out}\times H\times W\times K\times K\), while the computational complexity of depthwise separable convolution is \(C_{in}\times H\times W\times K\times K+C_{in}\times C_{out}\times H\times W\). When \(C_{out}\) is large and \(K>1\), the computational complexity of depthwise separable convolution is much smaller than that of standard convolution.

[0054] Application in the YOLOv8 model: In the YOLOv8 backbone network, some standard convolutional layers are replaced with depthwise separable convolutional layers. Preferably, 20%–30% of the standard convolutional layers are replaced with depthwise separable convolutional layers. More preferably, the shallow convolutional layers in the backbone network are replaced, because the shallow feature maps have relatively few channels, and the impact on model performance is minimal after replacement.

[0055] Step 212: Optimize the Feature Pyramid Structure. Principle: The Feature Pyramid Network (FPN) is used to fuse feature maps of different scales to improve object detection performance on targets at different scales. Traditional FPN structures may introduce significant computational overhead during feature fusion; optimizing the feature pyramid structure can reduce computational overhead while maintaining detection accuracy.

[0056] Improvements to the Path Aggregation Network (PANet): YOLOv8 employs the PANet architecture. To further optimize it, we can reduce the number of channels in intermediate feature maps and adjust the upsampling and downsampling methods. For example, we can use a lighter-weight upsampling method (such as nearest neighbor upsampling) instead of bilinear interpolation upsampling to reduce computational cost.

[0057] Feature fusion strategy optimization: In the feature fusion process, a weighted fusion method is adopted, which assigns different weights according to the importance of feature maps at different scales, avoiding information redundancy caused by simple addition fusion.

[0058] Step 213: Knowledge distillation technique. Principle: Knowledge distillation is a technique that transfers knowledge from a large pre-trained model (teacher model) to a small model (student model). Teacher models typically have high performance but require a large amount of computation and parameters; student models are relatively lightweight but may not perform as well as teacher models. Through knowledge distillation, student models can learn the decision information from teacher models, thereby improving performance while maintaining lightweight design.

[0059] Distillation Loss Function: The distillation loss function typically consists of two parts: hard label loss and soft label loss. Hard label loss uses the traditional cross-entropy loss function to measure the difference between the student model's output and the true label; soft label loss uses the KL divergence loss function to measure the difference between the student model's soft output (temperature-scaled output) and the teacher model's soft output.

[0060] Distillation process: During student model training, both hard-label loss and soft-label loss are calculated simultaneously, and their weighted sum is used as the final loss function. By adjusting the weights of the soft-label loss, the degree to which the student model learns from the teacher model can be controlled.

[0061] Preferably, during knowledge distillation, a multi-scale feature fusion mechanism is introduced into the student network to match the teacher network's ability to recognize box-shell targets at different scales; the ability to perceive targets of different sizes is enhanced by constructing an FPN or BiFPN structure in the student model; and feature alignment loss or MSE loss is used during the distillation process.

[0062] By applying the techniques of depthwise separable convolutional replacement, feature pyramid structure optimization, and knowledge distillation, the computational load and number of parameters of the box hull counting model can be significantly reduced while maintaining its detection accuracy, thus achieving a lightweight model that can run efficiently on mobile devices. This results in a lightweight YOLOv8 model.

[0063] Step 3: Refine the real-time inference acceleration process.

[0064] Step 311, Quantization Technique (INT8 Quantization): Perform INT8 quantization on the model's weights and activation values. This is done by calculating the scaling factor \(S\) and zeros \(Z\). In deep learning models, 32-bit floating-point numbers (FP32) are typically used to represent weights and activation values. The FP32 data is then mapped to the INT8 range. The range is (-128 to 127), but this representation consumes a lot of memory and has a high computational cost.

[0065] Preferably, the steps include: Step 3111: Determine the scaling factor and zero point. For weights and activation values, collect their dynamic range (maximum and minimum values).

[0066] The scaling factor S and the zero point Z are calculated using the following formulas: \(S=\frac{r_{max}-r_{min}}{q_{max}-q_{min}}\) \(Z=q_{min}-\text{round}(\frac{r_{min}}{S})\); Where \(r_{max}\) and \(r_{min}\) are the maximum and minimum values ​​of the FP32 data, and \(q_{max}=127\) and \(q_{min}=-128\) are the maximum and minimum values ​​of the INT8 data.

[0067] Step 3112: Quantize weights and activation values.

[0068] Quantize the FP32 weights \(W_{fp32}\) into INT8 weights \(W_{int8}\): \(W_{int8}=\text{round}(\frac{W_{fp32}}{S_w})+Z_w\); Where \(S_w\) and \(Z_w\) are the scaling factor and zero point of the weights.

[0069] During inference, the input FP32 activation value \(A_{fp32}\) is quantized into an INT8 activation value \(A_{int8}\): \(A_{int8}=\text{round}(\frac{A_{fp32}}{S_a})+Z_a\) where \(S_a\) and \(Z_a\) are the scaling factor and zero point of the activation value.

[0070] Step 3113, INT8 calculation.

[0071] Using INT8 data for calculations such as convolution and matrix multiplication is typically faster on hardware.

[0072] Step 3114: Dequantize and output.

[0073] Dequantize the output \(O_{int8}\) calculated by INT8 to the output \(O_{fp32}\) of FP32: \(O_{fp32}=S_o(O_{int8}-Z_o)\) where \(S_o\) and \(Z_o\) are the scaling factor and zero point of the output.

[0074] Step 312, Model Pruning Strategy. Principle: Model pruning reduces the number of parameters and computational cost by removing connections or neurons that have a small impact on the output, while maintaining model performance as much as possible. Common pruning methods include structured pruning and unstructured pruning. This section mainly introduces channel pruning within structured pruning. Steps include: Step 3121, Calculating Channel Importance. For convolutional layers, calculate the importance score for each channel. A common method is to calculate the L1 norm of the channels, which is the sum of the absolute values ​​of all weights within a channel. Channels with lower scores contribute less to the model output.

[0075] Step 3122: Determine the pruning ratio. Determine the proportion of channels to be pruned based on requirements; preferably, it can be set to 20%–25%.

[0076] Step 3123: Pruning. Sort the channels by importance score from smallest to largest, and prune a certain percentage of the channels with the lowest scores. Update the model structure, remove the pruned channels, and adjust the number of input channels in subsequent layers accordingly.

[0077] Step 3124: Fine-tune the model. Fine-tune the pruned model to restore the performance lost due to pruning.

[0078] Step 313: Parallel processing using multithreading technology. Principle: Multithreading technology distributes image acquisition and detection tasks across different threads for parallel execution, avoiding the blocking caused by serial processing and thus improving system real-time performance. On mobile terminals, image acquisition typically takes time. If detection tasks are performed simultaneously with image acquisition, the CPU's multi-core resources can be fully utilized, reducing overall processing time. Specifically, this includes the following steps: Step 3131: Create threads, including an image acquisition thread and a detection thread.

[0079] Step 3132: Inter-thread communication is achieved using a queue to transfer data between threads. The image acquisition thread places the acquired images into the queue, and the detection thread retrieves images from the queue for detection.

[0080] Step 3133: Thread synchronization. Use locks or semaphores to ensure synchronization between threads and avoid data races and inconsistencies.

[0081] Step 4: Detection and Counting. The preprocessed box skin image data is input into the lightweight YOLOv8 model for target detection, and the detection boxes of the box skins are output. For overlapping box skins, a clustering algorithm based on intersection-union ratio is used to merge and correct the detection boxes. By fusing historical frame data, the counting error caused by short-term jitter is eliminated, and finally the number of box skins is counted and output.

[0082] Please see Figure 2 A container counting device for implementing the method described above includes: a mobile terminal and container counting software installed on the mobile terminal; the mobile terminal is a mobile phone or a tablet, equipped with a camera and a display screen; the container counting software includes: The runtime module is used to run the box counting software and perform data processing and model inference tasks. The image acquisition module connects to the camera of the mobile terminal, controls the camera to acquire images, and transmits the images to the preprocessing module; Image data preprocessing module: performs cleaning, adaptive brightness adjustment, and geometric transformation normalization on the original image; Detection and counting module: Includes a lightweight YOLOv8 object detection model to perform object detection and output detection boxes; performs post-processing for counting by fusing IoU-based clustering algorithm and historical frame data to count the number of boxes; The display module marks the test results on the screen and displays the number of boxes in real time.

[0083] The above methods, through quantization techniques, model pruning strategies, and multi-threading, can significantly accelerate the real-time inference process of the YOLOv8-based box counting device, thereby improving the system's response speed and processing efficiency.

[0084] Examples This embodiment provides a specific application of this method for counting, and a comparative test with manual counting, the original YOLOv8 model, and the method of this invention. It includes: (I) Data Preparation and Model Training Data collection: Collect 10,000 images of box skins in different scenarios, covering logistics warehouses, transport vehicles and other scenarios, including box skins of different materials, sizes, colors and placement.

[0085] Data annotation: Use the LabelImg annotation tool to annotate the image, draw bounding boxes for each box and label them with the category "box", and generate the corresponding annotation file.

[0086] Dataset partitioning: The labeled dataset is divided into training set, validation set and test set in a ratio of 7:1.5:1.5.

[0087] Model training: Based on the YOLOv8 model, the Adam optimizer was used with a learning rate of 0.001 for 100 epochs. During training, knowledge distillation was employed, using a large pre-trained model as the teacher model to transfer its knowledge to a lightweight student model.

[0088] (II) Implementation of Lightweight Model Optimization Depthwise separable convolution replacement: In the shallow convolutional layers of the YOLOv8 backbone network, 30% of the standard convolutions are replaced with depthwise separable convolutions, reducing the number of model parameters by about 40%.

[0089] Feature pyramid structure optimization: The number of channels in the intermediate feature map of PANet structure is reduced by 20%, nearest neighbor upsampling is used instead of bilinear interpolation upsampling, and the feature fusion weights are optimized.

[0090] Knowledge distillation implementation: The weights of hard label loss and soft label loss in the distillation loss function are set to 0.5 and 0.5 respectively. Through training, the student model learns the decision information of the teacher model, and the validation set mAP decreases by only 1.2%.

[0091] (III) Accelerated Implementation of Real-Time Reasoning Application of quantization techniques: The trained model is quantized using PyTorch's torch.quantization module, reducing the model's memory usage to 30% of the original model.

[0092] Model pruning: Channel pruning is performed on the YOLOv8 model, removing low-scoring channels by a pruning ratio of 20%, and the pruned model is fine-tuned to restore model performance.

[0093] Multithreaded implementation: In Android mobile terminal applications, Java multithreading technology is used to create image acquisition threads and detection threads, data transfer between threads is achieved through queues, and a locking mechanism is used to ensure thread synchronization, achieving a single counting processing time of ≤0.25 seconds.

[0094] (iv) Real-world scenario testing The method and apparatus of this invention were tested in actual logistics warehouses and transport vehicles, and compared with manual counting, traditional image processing methods, and an unoptimized YOLO model: Table 1: Comparison of the Three Methods

[0095] The above figures for manual single-count processing time are only average values ​​under normal conditions, as they are affected by various environmental factors. In complex environments, single-count processing will take longer. Test results show that this invention has significant advantages in both accuracy and processing speed, and can meet the needs of the logistics and warehousing industries for rapid and accurate counting of boxes.

[0096] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A box counting method based on the YOLOv8 model, characterized in that, include: Box skin image data processing includes: box skin image data preprocessing, mixed data augmentation of box skin image data, and annotation data; Lightweight YOLOv8 models, including: Replace some standard convolutions in the shallow convolutional layers of the backbone network of the YOLOv8 model with depthwise separable convolutions; The PANet structure in the YOLOv8 model is optimized to reduce the number of channels in the intermediate feature maps and to replace bilinear interpolation upsampling with nearest neighbor upsampling. Feature fusion is improved by using a weighted fusion method, which assigns weights according to the importance of feature maps at different scales. Knowledge distillation is used to construct a teacher-student model training architecture, with a large pre-trained model as the teacher model and a lightweight model as the student model. When training the student model, both hard label loss and soft label loss are calculated simultaneously, and the weighted sum of the two is used as the final loss function, so that the student model learns the decision information of the teacher model. The lightweight YOLOv8 model is obtained; Real-time inference acceleration includes: INT8 quantization of weights and activation values ​​in a lightweight YOLOv8 model, mapping FP32 data to the INT8 range by calculating scaling factors and zeros; model pruning, employing channel pruning to calculate the importance score of each channel in the convolutional layer and removing low-scoring channels according to a set pruning ratio; and multi-threaded parallel processing, creating image acquisition and detection threads in mobile applications. The image acquisition thread places acquired images into a queue, and the detection thread retrieves images from the queue for detection, using the queue to achieve data transfer between threads. For detection and counting, the preprocessed container skin image data is input into a lightweight YOLOv8 model for target detection, and the detection boxes of the container skins are output. For overlapping container skins, a clustering algorithm based on cross-union ratio is used to merge and correct the detection boxes. By fusing historical frame data, the counting error caused by short-term jitter is eliminated, and finally the number of container skins is counted and output.

2. The box counting method based on the YOLOv8 model according to claim 1, characterized in that, The step of replacing some standard convolutions in the shallow convolutional layers of the YOLOv8 model backbone network with depth-separable convolutions specifically involves replacing 20% ​​to 30% of the standard convolutions with depth-separable convolutions.

3. The box counting method based on the YOLOv8 model according to claim 1, characterized in that, The knowledge distillation process includes introducing a multi-scale feature fusion mechanism into the student network to match the teacher network's ability to identify box-shell targets at different scales; enhancing the student model's ability to perceive targets of different sizes by constructing an FPN or BiFPN structure; and using feature alignment loss or MSE loss during the distillation process.

4. The box counting method based on the YOLOv8 model according to claim 1, characterized in that, The box-shell image data undergoes hybrid data augmentation, including using Mosaic data augmentation, randomly selecting four images and stitching them together to create a new image, increasing the diversity and density of targets in the image; and using CutMix data augmentation, randomly cropping a region from one image and replacing it at a random position in another image, while adjusting the label information to allow the model to learn the contextual information of the target image.

5. The box counting method based on the YOLOv8 model according to claim 1, characterized in that, Specifically, the annotation data is updated synchronously during the preprocessing of the container skin image data and the hybrid data augmentation of the container skin image data to ensure that the annotation box accurately defines the transformed container skin target.

6. The box counting method based on the YOLOv8 model according to claim 5, characterized in that, The images were labeled using the Labellmg annotation tool. Bounding boxes were drawn for each box and labeled with the category "box", generating the corresponding annotation files. The labeled dataset was then divided into training, validation, and test sets in a ratio of 7:1.5:1.

5.

7. The box counting method based on the YOLOv8 model according to claim 6, characterized in that, Based on the YOLOv8 model, the Adam optimizer was used with a learning rate of 0.001, and the training lasted for 100 epochs.

8. The box counting method based on the YOLOv8 model according to claim 1, characterized in that, The preprocessing of the box hull image data includes data collection, image cleaning, adaptive brightness adjustment, and geometric transformation; the image cleaning includes blurred image filtering, duplicate image detection, and abnormal image removal; among which, blurred image filtering adopts the Laplacian variance algorithm, and images with a Laplacian variance lower than a set threshold of 100 are identified as blurred images and removed; the duplicate image detection adopts the perceptual hash algorithm to hash the collected images, calculate the hash distance between images, and when the hash distance is less than a set threshold of 5, it is identified as a duplicate image, and only one of them is retained.

9. The box counting method based on the YOLOv8 model according to claim 1, characterized in that, The process involves removing channels with low scores according to a set pruning ratio, which is 20% to 25%.

10. A box counting device implementing the method according to any one of claims 1-9, Its features are, Includes: a mobile terminal and container counting software installed on the mobile terminal; the mobile terminal is a mobile phone or tablet equipped with a camera and a display screen; the container counting software includes: The running module is used to run the box counting software and perform data processing and model inference tasks; The image acquisition module is connected to the camera of the mobile terminal, controls the camera to acquire images, and transmits the images to the preprocessing module; Image data preprocessing module: performs cleaning, adaptive brightness adjustment, and geometric transformation normalization on the original image; Detection and counting module: Includes a lightweight YOLOv8 object detection model to perform object detection and output detection boxes; performs post-processing for counting by fusing IoU-based clustering algorithm and historical frame data to count the number of boxes; The display module marks the test results on the screen and displays the number of boxes in real time.

Images