Deep learning-based empty tray remnant detection method and system, terminal and medium

Abstract

The invention discloses an empty tray remnant detection method based on deep learning. The method comprises the following steps: acquiring a to-be-detected empty tray image acquired by an image acquisition device; preprocessing the image of the empty tray to be detected to obtain preprocessed image data; performing target detection on the preprocessed image data by adopting a trained improved YOLOv8n model to obtain a detection result and a confidence coefficient thereof; and comparing the confidence coefficient with a preset confidence coefficient threshold value to obtain a comparison result, if the comparison result is that the confidence coefficient is higher than the confidence coefficient threshold value, judging that the tray has remnants, and if the comparison result is that the confidence coefficient is lower than the confidence coefficient threshold value, judging that the tray is an empty tray. The method can overcome the interference caused by illumination change, tray difference and remnant material diversity, realizes real-time online detection with high accuracy and high robustness, has good expandability and adaptive ability, and is suitable for unmanned quality inspection requirements in various production line environments.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, specifically to a method, system, terminal, and medium for detecting empty tray residues based on deep learning. Background Technology

[0002] In automated production lines for pharmaceuticals, medical devices, and food, pallets (or carriers) are widely used to carry products (such as medicine bags) for conveying, testing, and packaging. After production is completed, empty pallets need to be returned to the starting station for the next cycle of use. However, in actual production, due to reasons such as robotic arm failure, control system malfunctions, or human error, some medicine bags are often left in the pallet without being successfully removed.

[0003] Failure to promptly detect and remove these residual products will pose serious quality and safety risks: on the one hand, residual medicine bags may be mixed with new batches of products in subsequent processes, causing cross-contamination or dosage errors; on the other hand, when empty pallets re-enter the filling stage, they may cause equipment jamming, damage, or trigger alarm shutdowns, affecting production efficiency and the stability of the automation system. Summary of the Invention

[0004] The purpose of this invention is to provide a method, system, terminal, and medium for detecting abandoned items in empty pallets based on deep learning, which can perform online detection of abandoned items in empty pallets with high accuracy and high robustness.

[0005] This invention is achieved through the following technical solution: In a first aspect, embodiments of the present invention provide a method for detecting empty pallet residue based on deep learning, comprising: Acquire an image of the empty tray to be inspected, captured by the image acquisition device; The image of the empty tray to be detected is preprocessed to obtain preprocessed image data; The trained improved YOLOv8n model was used to perform target detection on the preprocessed image data, and the detection results and their confidence scores were obtained. The confidence level is compared with a pre-set confidence threshold to obtain the comparison result. If the comparison result is that the confidence level is higher than the confidence threshold, it is determined that there is something left on the tray. If the comparison result is that the confidence level is lower than the confidence threshold, it is determined that the tray is empty.

[0006] Furthermore, the method also includes: when it is determined that there is something left on the tray, sending an alarm signal to the alarm device and sending a control command to the IO module to control the tray to stop flowing into the next process.

[0007] Furthermore, the specific method for preprocessing the image of the empty tray to be detected includes: The empty tray image to be tested was processed sequentially using Gaussian filtering, contrast enhancement, and geometric correction.

[0008] Furthermore, the improved YOLOv8n model includes: a backbone network, a feature fusion neck, and a detection head. The backbone network adopts the EfficientRep Backbone and generates feature maps of different scales through cross-stage local connections. The backbone network generates feature maps of different scales through multi-level downsampling and adopts the SiLU activation function. The feature fusion neck adopts a Rep-PAN structure. High-level low-resolution features are upsampled and concatenated with mid-level semantic features to obtain fused features. The fused features are then downsampled and fused with low-level high-resolution features to obtain a fused multi-scale feature map. The detection head adopts a decoupled head structure, which separates the classification task from the bounding box regression task. The detection head receives the fused multi-scale feature map, and outputs three prediction branches at each scale to predict the target class probability, the bounding box center coordinates and width and height, and the object confidence, respectively. The output layer uses an anchor-free mechanism.

[0009] Furthermore, the method also includes training an improved YOLOv8n model, specifically including: The data samples include empty tray samples and samples with residue. Residues that are visible and have an area of ​​50 pixels or more are labeled, and data augmentation strategies are used to augment the data samples. Load the YOLOv8n weights pre-trained on the COCO dataset, perform transfer learning, use the SGD optimizer and set momentum and weight decay, define the dynamic rules for parameter updates during training, train according to the set number of training epochs, and terminate when there is no improvement after the set number of epochs, to obtain the trained improved YOLOv8n model.

[0010] Furthermore, after obtaining the trained improved YOLOv8n model, the model is converted into ONNX form and then input into the TensorRT framework for quantization and graph optimization processing.

[0011] Furthermore, the specific methods for using data augmentation strategies to augment data samples include: Geometric enhancement was performed using random rotation to simulate tray tilt and camera viewpoint shift. HSV color space perturbation is used to enhance the breadth of light to address uneven lighting or LED flicker in the workshop; The labeled remnants were randomly pasted onto images of empty trays of different types for composite enhancement, thereby expanding the positive samples. Four different sample images were stitched together using a frequency domain hybrid enhancement method, and different tray backgrounds were merged.

[0012] Secondly, another embodiment of the present invention provides a deep learning-based empty pallet residue detection system, comprising: Image acquisition device, used to acquire images of an empty tray to be inspected; The main control computer is used to acquire images of empty trays to be detected from the image acquisition device, preprocess the images of empty trays to be detected to obtain preprocessed image data, use a trained improved YOLOv8n model to perform target detection on the preprocessed image data, obtain detection results and their confidence levels, compare the confidence levels with a preset confidence threshold, obtain comparison results, if the comparison results show that the confidence level is higher than the confidence threshold, it is determined that there are leftover items on the tray, if the comparison results show that the confidence level is lower than the confidence threshold, it is determined that the tray is empty, and if it is determined that there are leftover items, a control command is sent to the actuator. The actuator is used to receive and execute control commands sent by the main control computer.

[0013] Thirdly, another embodiment of the present invention provides a smart terminal, including a processor, an input device, an output device, and a memory, wherein the processor, the input device, the output device, and the memory are interconnected, the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute the method described in the above embodiments.

[0014] Fourthly, another embodiment of the present invention provides a computer-readable storage medium storing a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the methods described in the above embodiments.

[0015] The beneficial effects of this invention are: This invention provides a method, system, terminal, and medium for detecting leftover items on empty pallets based on deep learning. It employs an improved YOLOv8n model to perform target detection on acquired images of empty pallets, outputting bounding boxes of medicine bags and their confidence scores. If the confidence score of any detection box is greater than or equal to a confidence threshold, the pallet is determined to contain leftover items. This method overcomes interference from changes in lighting, pallet differences, and the diversity of leftover item materials, achieving high accuracy and robustness in real-time online detection. It also possesses good scalability and adaptability, making it suitable for unmanned quality inspection needs in various production line environments. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be considered as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort. In the drawings: Figure 1 A flowchart of a deep learning-based method for detecting empty pallet residues provided in the first embodiment of the present invention; Figure 2 The diagram below shows a structural block diagram of an empty pallet residue detection system based on deep learning, which is provided as another embodiment of the present invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of this invention are only for explaining this invention and are not intended to limit this invention.

[0018] like Figure 1 As shown, the first embodiment of the present invention provides a method for detecting empty pallet residue based on deep learning, which includes the following steps: Acquire an image of the empty tray to be inspected, captured by the image acquisition device; The image of the empty tray to be detected is preprocessed to obtain preprocessed image data; The trained improved YOLOv8n model was used to perform target detection on the preprocessed image data, and the detection results and their confidence scores were obtained. The confidence level is compared with a pre-set confidence threshold to obtain the comparison result. If the comparison result is that the confidence level is higher than the confidence threshold, it is determined that there is something left on the tray. If the comparison result is that the confidence level is lower than the confidence threshold, it is determined that the tray is empty. If it is determined that there is something left on the tray, an alarm signal is sent to the alarm device, and a control command is sent to the IO module to stop the tray from flowing into the next process.

[0019] In this embodiment, the image acquisition device includes one or more high-definition cameras, positioned above the end of the tray return line. The high-definition cameras acquire images of empty trays to be inspected, with the remaining material being medicine bags. Under normal production conditions, a large number of sample images of trays containing empty trays and trays with remaining medicine bags are acquired (it is recommended to have no less than 5,000 images for each category). Annotation tools (such as LabelImg) are used to mark the bounding boxes of the medicine bags in the images, forming a training dataset. Simultaneously, metadata such as tray model, medicine bag type, and lighting conditions are recorded for subsequent model optimization.

[0020] The specific methods for preprocessing the empty tray image to be detected include: sequentially applying Gaussian filtering, contrast enhancement, and geometric correction to the empty tray image. The image is then uniformly scaled to the input size (e.g., 640×640), and the pixel values ​​are normalized to the [0,1] range.

[0021] A lightweight deep learning object detection model (YOLOv8n) was specifically applied to the task of identifying abandoned medicine bags on empty pallets, solving the problem of inaccurate detection of small and low-contrast targets in complex backgrounds by traditional methods. Custom optimizations were made based on the lightweight YOLOv8n architecture to balance detection accuracy and inference speed, meeting the real-time requirements of industrial environments. An improved YOLOv8n model was constructed, comprising a backbone network, a feature fusion neck, and a detection head. The backbone network uses the EfficientRep Backbone, reducing computational redundancy and improving feature extraction efficiency through cross-stage local connections. The input image size was uniformly 640×640×3. The backbone network generated feature maps of different scales, such as 80×80, 40×40, and 20×20, through multi-level downsampling (stride=2), corresponding to the detection capabilities of small, medium, and large targets, respectively. 80×80: Largest size, smallest receptive field, contains more refined spatial information, suitable for detecting small targets (such as small medicine bags); 40×40: Moderate size and receptive field, suitable for detecting medium-sized targets; 20×20: Smallest size, largest receptive field, contains more abstract semantic information, suitable for detecting large targets (such as large medicine bags or stacked medicine bags). After each convolutional layer of the backbone network, an activation function (such as SiLU or ReLU) is usually applied to perform a non-linear transformation on the features. In this embodiment, the SiLU (Sigmoid-weighted Linear Unit) activation function is used instead of the traditional ReLU, enabling it to learn complex feature patterns and improve non-linear expressive power and gradient stability.

[0022] The feature fusion neck section employs a Rep-PAN structure. The core idea of ​​Rep-PAN is to introduce reparameterized convolutions on top of PANet to improve inference speed without sacrificing accuracy. High-level low-resolution features are upsampled and concatenated with mid-level semantic features to obtain fused features. These fused features are then downsampled and fused with low-level high-resolution features to obtain a fused multi-scale feature map.

[0023] The detection head adopts a decoupled head structure, which separates the classification task from the bounding box regression task. The detection head receives the fused multi-scale feature map, and outputs three prediction branches at each scale, which predict the object class probability, bounding box center coordinates and width and height, and object confidence, respectively. The output layer uses an anchor-free mechanism to generate the highest priority bounding box size based on the training data clustering.

[0024] The training of the improved YOLOv8n model was completed offline. Training the improved YOLOv8n model specifically included: Data Preparation and Augmentation: A total of 12,000 high-resolution images were collected, including 6,200 empty pallet samples and 5,800 pallet samples with leftover medicine bags. These images covered various pallet models (e.g., Euro pallets, plastic crates), medicine bag colors (e.g., white, blue, transparent), lighting conditions (e.g., sunlight, LED, shadow), and placement postures (e.g., flat, tilted, stacked). Labeling Standards: Only visible medicine bags with an area of ​​50 pixels or more were labeled, ignoring severely occluded or truncated targets. Data augmentation strategies were employed to augment the data samples.

[0025] Specific methods for data augmentation strategies to enhance data samples include: Geometric enhancement was performed using random rotation (±15°) to simulate tray tilt and camera viewpoint shift; HSV color space perturbation (H±0.02, S±0.5, V±0.3) is used to enhance the breadth of light to address uneven illumination or LED flicker in the workshop; The labeled items (including transparent, semi-transparent, and reflective materials) were randomly pasted onto images of different types of empty trays for composite enhancement, expanding the positive samples, especially for rare medicine bag types or placement postures; A frequency domain hybrid enhancement method is used to stitch together four different sample images and integrate different tray backgrounds to improve the robustness of the model to dense and occluded scenes and prevent overfitting to a single production line layout.

[0026] Model initialization and training configuration: Pre-trained weights: YOLOv8n weights pre-trained on the COCO dataset are loaded for transfer learning. Fine-tuning stage: The first 3 stages of the EfficientRep Backbone are frozen, and only the feature fusion neck and detection head are fine-tuned to prevent overfitting on small datasets; Class balance sampling is introduced: Drug bag samples are grouped by type (transparent / opaque), size, and tray material to ensure that each batch of samples is balanced; Domain adaptation technique: The mAP difference between different production lines (e.g., workshop A vs. workshop B) is monitored on the validation set; If the performance of a certain production line drops by more than 5%, 100-200 new images of that production line are collected for incremental fine-tuning without retraining the entire model.

[0027] The SGD optimizer is employed with momentum and weight decay set to 0.937 and 0.0005 respectively. Momentum accelerates the gradient descent process, while weight decay suppresses overfitting, providing a computational engine for iterative parameter optimization during training. Dynamic rules for parameter updates during training are defined. Learning rate scheduling: The initial learning rate is 0.01, employing a cosine annealing strategy, smoothly decaying to 1e-5 with each training epoch. A higher learning rate is used for rapid parameter updates in the early stages, while a lower learning rate is used for fine-tuning in later stages. This avoids model oscillations around the optimal solution and is a key strategy to ensure high-quality convergence. Batch size is specified as 32 images per epoch, balancing training efficiency and memory usage. A reasonable batch size improves GPU utilization while ensuring gradient estimation stability, avoiding excessive gradient fluctuations caused by excessively small batches. Training proceeds according to the set number of epochs, terminating when no improvement is achieved, resulting in a well-trained improved YOLOv8n model. The training rounds are set to 300 rounds. An early stop mechanism monitors the validation set mAP@0.5 (the core evaluation metric for drug bag detection accuracy). If there is no improvement for 20 consecutive rounds, the training is terminated. This setting prevents both undertraining (e.g., insufficient rounds leading to suboptimal accuracy) and overtraining (e.g., too many rounds leading to overfitting of the validation set). Loss function configuration: For the three core tasks of object detection—boundary box regression, category classification, and object confidence assessment—adaptive loss functions are specified for each: Boundary box regression (CIoU Loss): This addresses the problem of traditional IoU Loss failing to distinguish between overlapping bounding boxes with significant differences in position and shape, improving the accuracy of medicine bag localization; Classification loss uses Binary Cross-Entropy Loss: Adapted to the binary classification scenario of medicine bags / non-medicine bags, accurately calculating the category prediction error; Confidence loss uses FocalLoss (α=0.25, γ=2.0): Alleviating the imbalance of positive and negative samples caused by a large number of empty tray areas and a small number of medicine bag areas, preventing the model from biased in predicting non-medicine bags, and improving the detection rate of leftover medicine bags. The dataset is divided into training, validation, and test sets in an 8:1:1 ratio. Core evaluation metrics include mAP@0.5 (mean Average Precision at IoU=0.5): measuring detection accuracy; and Recall@0.5: measuring the detection rate of leftover medicine bags.

[0028] Model Export and Deployment Optimization: After obtaining the trained improved YOLOv8n model, the lightweight detection model is converted into ONNX form and then input into the TensorRT framework for inference acceleration. This includes tensor fusion of network layers and low-precision / integer quantization. Tensor fusion of network layers includes: ① Horizontal merging, which combines Convolution layers, Bias layers, and ReLU layers with the same parameters in the model into a CBR layer; ② Vertical merging, which combines layers with the same network structure but different parameters into a CBR layer. During inference acceleration, FP16 and INT8 precision tensors are used to replace FP32 tensors.

[0029] The residues in the embodiments can be used not only for medicine bags, but also for other small medical consumables, such as syringes, blood collection tubes, electronic components, capsule bottles, etc. Residue detection can be achieved simply by changing the training data.

[0030] This invention provides a deep learning-based method for detecting leftover items on empty pallets. It employs an improved YOLOv8n model to perform target detection on acquired images of empty pallets, outputting bounding boxes of medicine bags and their confidence scores. If the confidence score of any detection box is greater than or equal to a confidence threshold, the pallet is determined to contain leftover items. This method overcomes interference from variations in lighting, pallet differences, and diverse medicine bag materials, achieving high accuracy and robustness in real-time online detection. It also possesses good scalability and adaptability, making it suitable for unmanned quality inspection needs in various production line environments.

[0031] like Figure 2 As shown, another embodiment of the present invention provides a deep learning-based empty pallet residue detection system, comprising: Image acquisition device, used to acquire images of an empty tray to be inspected; The main control computer is used to acquire images of empty trays to be detected from the image acquisition device, preprocess the images of empty trays to be detected to obtain preprocessed image data, use a trained improved YOLOv8n model to perform target detection on the preprocessed image data, obtain detection results and their confidence levels, compare the confidence levels with a preset confidence threshold, obtain comparison results, if the comparison results show that the confidence level is higher than the confidence threshold, it is determined that there are leftover items on the tray, if the comparison results show that the confidence level is lower than the confidence threshold, it is determined that the tray is empty, and if it is determined that there are leftover items, a control command is sent to the actuator. The actuator is used to receive and execute control commands sent by the main control computer.

[0032] The image acquisition device includes one or more high-definition cameras, positioned above the end of the pallet return line. The cameras are connected to the main control computer via a network port. The main control computer is equipped with a GPU accelerator card (such as an NVIDIA Jetson AGXXavier or RTX 3060 or higher), runs the deep learning-based empty pallet residue detection method provided in the first embodiment, and outputs the detection results. The actuator communicates with the production line control system via an I / O module. When residue is detected, an alarm signal is issued, awaiting manual intervention. Manual review and recovery: If the operator confirms a false alarm, such as a false detection due to pallet label reflection, they can release the pallet by clicking on the interface; the system automatically records false alarm samples and adds them to the difficult sample library for subsequent model iterations.

[0033] The third embodiment of the present invention provides a smart terminal, which includes a processor, an input device, an output device, and a memory. The processor, input device, output device, and memory are interconnected. The memory is used to store a computer program, which includes program instructions. The processor is configured to call the program instructions to execute the deep learning-based empty pallet residue detection method described in the first embodiment above.

[0034] It should be understood that, in the embodiments of the present invention, the processor may be a Central Processing Unit (CPU), but it may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0035] Input devices may include touchpads, microphones, etc., and output devices may include displays (LCDs, etc.), speakers, etc.

[0036] The memory may include read-only memory and random access memory, and provides instructions and data to the processor. A portion of the memory may also include non-volatile random access memory. For example, the memory may also store information about the device type.

[0037] In specific implementations, the processor, input device, and output device described in the embodiments of the present invention can execute the implementation methods described in the method embodiments of the present invention, or they can execute the implementation methods described in the system embodiments of the present invention, which will not be repeated here.

[0038] The present invention also provides an embodiment of a computer-readable storage medium storing a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the deep learning-based method for detecting empty pallet residues described in the above embodiments.

[0039] The computer-readable storage medium can be an internal storage unit of the terminal described in the foregoing embodiments, such as the terminal's hard drive or memory. The computer-readable storage medium can also be an external storage device of the terminal, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the terminal. Furthermore, the computer-readable storage medium can include both internal storage units and external storage devices of the terminal. The computer-readable storage medium is used to store the computer program and other programs and data required by the terminal. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0040] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0041] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the terminals and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0042] In the several embodiments provided in this application, it should be understood that the disclosed terminals and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices or units, or may be electrical, mechanical or other forms of connection.

[0043] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be covered within the scope of the claims and specification of the present invention.

Claims

1. A method for detecting items left on empty pallets based on deep learning, characterized in that, include: Acquire an image of the empty tray to be inspected, captured by the image acquisition device; The image of the empty tray to be detected is preprocessed to obtain preprocessed image data; The trained improved YOLOv8n model was used to perform target detection on the preprocessed image data, and the detection results and their confidence scores were obtained. The confidence level is compared with a pre-set confidence threshold to obtain the comparison result. If the comparison result is that the confidence level is higher than the confidence threshold, it is determined that there is something left on the tray. If the comparison result is that the confidence level is lower than the confidence threshold, it is determined that the tray is empty.

2. The deep learning-based method for detecting empty pallet residue as described in claim 1, characterized in that, The method further includes: when it is determined that there is something left on the tray, sending an alarm signal to the alarm device and sending a control command to the IO module to control the tray to stop flowing into the next process.

3. The method for detecting empty pallet residue based on deep learning as described in claim 1, characterized in that, The specific method for preprocessing the image of the empty tray to be detected includes: The empty tray image to be tested was processed sequentially using Gaussian filtering, contrast enhancement, and geometric correction.

4. The deep learning-based method for detecting empty pallet residue as described in claim 1, characterized in that, The improved YOLOv8n model includes: a backbone network, a feature fusion neck, and a detection head. The backbone network uses EfficientRepBackbone and generates feature maps of different scales through cross-stage local connections. The backbone network uses SiLU activation function to generate feature maps of different scales through multi-level downsampling. The feature fusion neck adopts a Rep-PAN structure. High-level low-resolution features are upsampled and concatenated with mid-level semantic features to obtain fused features. The fused features are then downsampled and fused with low-level high-resolution features to obtain a fused multi-scale feature map. The detection head adopts a decoupled head structure, which separates the classification task from the bounding box regression task. The detection head receives the fused multi-scale feature map, and outputs three prediction branches at each scale to predict the target class probability, the bounding box center coordinates and width and height, and the object confidence, respectively. The output layer uses an anchor-free mechanism.

5. The deep learning-based method for detecting empty pallet residue as described in claim 4, characterized in that, The method also includes training an improved YOLOv8n model, specifically including: The data samples include empty tray samples and samples with residue. Residues that are visible and have an area of ​​50 pixels or more are labeled, and data augmentation strategies are used to augment the data samples. Load the YOLOv8n weights pre-trained on the COCO dataset, perform transfer learning, use the SGD optimizer and set momentum and weight decay, define the dynamic rules for parameter updates during training, train according to the set number of training epochs, and terminate when there is no improvement after the set number of epochs, to obtain the trained improved YOLOv8n model.

6. The deep learning-based method for detecting empty pallet residue as described in claim 5, characterized in that, After obtaining the trained improved YOLOv8n model, the model is converted into ONNX form and then input into the TensorRT framework for quantization and graph optimization.

7. The deep learning-based method for detecting empty pallet residue as described in claim 5, characterized in that, The specific methods for data augmentation of data samples using data augmentation strategies include: Geometric enhancement was performed using random rotation to simulate tray tilt and camera viewpoint shift. HSV color space perturbation is used to enhance the breadth of light to address uneven lighting or LED flicker in the workshop; The labeled remnants were randomly pasted onto images of empty trays of different types for composite enhancement, thereby expanding the positive samples. Four different sample images were stitched together using a frequency domain hybrid enhancement method, and different tray backgrounds were merged.

8. A deep learning-based empty pallet residue detection system, characterized in that, include: Image acquisition device, used to acquire images of an empty tray to be inspected; The main control computer is used to acquire images of empty trays to be detected from the image acquisition device, preprocess the images of empty trays to be detected to obtain preprocessed image data, use a trained improved YOLOv8n model to perform target detection on the preprocessed image data, obtain detection results and their confidence levels, compare the confidence levels with a preset confidence threshold, obtain comparison results, if the comparison results show that the confidence level is higher than the confidence threshold, it is determined that there are left items on the tray, if the comparison results show that the confidence level is lower than the confidence threshold, it is determined that the tray is empty, and if it is determined that there are left items on the tray, a control command is sent to the actuator. The actuator is used to receive and execute control commands sent by the main control computer.

9. A smart terminal, comprising a processor, an input device, an output device, and a memory, wherein the processor, input device, output device, and memory are interconnected, and the memory is used to store a computer program, the computer program comprising program instructions, characterized in that, The processor is configured to invoke the program instructions to execute the computer program and implement the method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer storage medium stores a computer program, the computer program including program instructions, which, when executed by a processor, cause the processor to perform the method as described in any one of claims 1 to 7.

Images