Container damage detection method based on lightweight U-Net and visual Transform
By improving the container damage detection method using lightweight U-Net and visual Transformer, the problems of detection efficiency and accuracy in complex port environments are solved. Real-time and efficient detection is achieved on edge devices, reducing false alarm rate and manual re-inspection rate, making it suitable for resource-constrained deployments.
Patent Information
- Application Number
- CN202511708337.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-20
- Publication Date
- 2026-02-13
AI Technical Summary
Existing container damage detection technologies are inefficient, inaccurate, and difficult to deploy in real time in complex port environments. They are prone to false detections and missed detections, especially under complex lighting conditions and in detecting minute damage. Furthermore, high-precision models have high computational complexity and are difficult to implement on edge devices.
An improved lightweight U-Net is used for box segmentation and background stripping. It is combined with the adaptive dual-threshold damage classification of the visual Transformer. The number of model parameters and computation is reduced by using a hybrid convolution strategy and CBAM attention mechanism. An adaptive dual-threshold decision mechanism is introduced to improve detection accuracy.
It enables real-time and efficient container damage detection on edge devices, significantly reducing false alarm rates and manual re-inspection rates, and reducing reliance on large-scale labeled data, making it suitable for resource-constrained deployment environments.
Smart Images
Figure CN121527520A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of container inspection technology, specifically to a container damage detection method based on lightweight U-Net and visual Transformer. Background Technology
[0002] As the core carrier of the global logistics system, containers handle over 90% of cargo transportation. Their structural integrity directly impacts cargo safety, transportation efficiency, and port operating costs. Currently, container damage detection primarily relies on manual visual inspection, a method inherently flawed by inefficiency, high costs, and strong subjectivity. Especially under complex lighting conditions, variable weather, and in scenarios with minor defects (such as scratches and rust), the accuracy and stability of manual inspection are difficult to guarantee, failing to meet the high-throughput operational demands of modern ports.
[0003] In recent years, automated inspection technologies based on machine vision have offered a possibility for solving the aforementioned problems. Researchers have attempted to apply object detection algorithms (such as the YOLO series and Faster R-CNN) to this field. However, these methods face severe challenges in real-world port operation environments: firstly, cluttered yard backgrounds significantly increase the risk of false detections; secondly, for minute damaged targets with sparse features and low contrast, there is a high rate of missed detections or false positives; thirdly, high-precision models typically have high computational complexity, making it difficult to implement real-time inference on edge devices with limited computing power, thus restricting their large-scale deployment. Furthermore, deep learning models rely on large-scale, accurately labeled data for training, while the collection of container damage samples and the labeling of various types of minute defects are costly, further hindering the practical application of the technology.
[0004] Therefore, existing technologies still lack an automated container damage detection solution that can effectively balance detection accuracy, inference speed, environmental robustness, and deployment cost, and innovative technological breakthroughs are urgently needed. Summary of the Invention
[0005] To address the shortcomings of existing technologies, this invention provides a container damage detection method based on lightweight U-Net and visual Transformer, which solves the technical problems of low efficiency, poor accuracy, and difficulty in real-time deployment of automated container damage detection in complex port environments.
[0006] To achieve the above objectives, the present invention adopts the following technical solution:
[0007] A container damage detection method based on lightweight U-Net and visual Transformer includes the following steps:
[0008] S1. Container image input and preprocessing:
[0009] The video stream is obtained from the 4K super-definition monitoring camera deployed at the port gate, and the improved inter-frame difference method is used for key frame extraction to obtain clear images containing the complete container body;
[0010] S2, box face segmentation and background stripping based on improved lightweight U-Net:
[0011] The image obtained in S1 is input into an improved lightweight U-Net model to accurately segment the container face area and achieve background stripping;
[0012] S3, adaptive double-threshold damage classification based on visual Transformer:
[0013] The pure box face image segmented in S2 is input into a visual Transformer model for binary classification of whether it is damaged, and an adaptive double-threshold mechanism is introduced;
[0014] Preferably, the S2 step comprises the following sub-steps:
[0015] S2.1: Main network design using hybrid convolution strategy: In Block1 and Block2 of the encoder, and Block1 and Block2 of the decoder, use depth separable convolution instead of standard convolution; in Block3 to Block5 of the encoder and Block3 and Block4 of the decoder, keep the standard convolution;
[0016] S2.2: Integration of CBAM attention mechanism in skip connection: In the skip connection of U-Net, introduce a convolution block attention module;
[0017] S2.3: Model optimization using weighted cross-entropy loss function: Use a weighted cross-entropy loss function to solve the class imbalance problem, which is defined as follows:
[0018] ;
[0019] wherein, represents the total number of pixels in the image; is the true label of the ith pixel, when represents the box face, 0 represents the background; is the probability that the ith pixel is predicted as a box face; is the weight coefficient of the background pixel, according to the ratio of container face pixels to background pixels in the training set samples, which is about 1:3.4, according to the class inverse frequency weighting principle, set =0.294.
[0020] Preferably, the S3 step comprises the following sub-steps:
[0021] S3.1: Image patching and serialization: The input segmented container image is divided into 16x16 non-overlapping patches, each patch is converted into a 256-dimensional embedding vector through linear projection, and a learnable position encoding is added to convert the two-dimensional image into a one-dimensional sequence Z;
[0022] S3.2: Feature encoding: input sequence Z into an encoder stacked by 12 layers of Transformer modules. Each layer contains a multi-head self-attention module and a feedforward network;
[0023] S3.3: Double threshold decision and confidence calibration: the model output layer adopts Softmax to generate damage probability P, and introduces an adaptive double threshold decision mechanism:
[0024] Compare the probability P with the classification threshold , preliminarily determine "damage" or "normal";
[0025] Calculate the standard deviation of the prediction probability of the current sample obtained by multiple random forward propagation of the model , and compare it with the confidence threshold , finally decide whether to adopt the automatic judgment result or trigger manual review.
[0026] Preferably, it further comprises S4, detection result output
[0027] According to the determination result of S3.3, the system outputs the final "damage", "normal" or "manual review" conclusion.
[0028] The application provides a container damage detection method based on a lightweight U-Net and a visual Transformer. At least the following beneficial effects are included:
[0029] 1. The improved lightweight U-Net model is used in the application, which integrates depth separable convolution and attention mechanism in the model. This design significantly reduces the parameter quantity and calculation quantity of the model while maintaining high segmentation accuracy and achieving real-time inference speed, effectively meeting the needs of high-speed detection operations in ports.
[0030] 2. The application adopts a two-stage detection framework. The first stage effectively eliminates complex environmental interference through precise background stripping; the second stage uses the global modeling capability of the visual transformer to classify damage, and combines an adaptive double threshold decision mechanism. This mechanism can dynamically adjust the classification strategy, improve the overall classification accuracy, significantly reduce the false positive rate and manual review rate, and realize an efficient intelligent detection process of "machine leading and manual assistance".
[0031] 3、The model of the application has the characteristics of significant light weight, and the parameter quantity and computational complexity are much lower than many existing mainstream models, and is especially suitable for deployment on resource-limited edge computing devices. At the same time, the macroscopic binary classification strategy adopted by the method effectively reduces the dependence on large-scale fine annotation data, solves the industry problem of high data annotation cost, and creates favorable conditions for the large-scale practical application of technology. BRIEF DESCRIPTION OF DRAWINGS
[0032] Figure 1 is the overall flowchart of the embodiment of the application;
[0033] Figure 2 is the lightweight U-Net model structure diagram in the embodiment of the application;
[0034] Figure 3 is the depth separable convolution structure diagram in the embodiment of the application;
[0035] Figure 4 is the convolution block attention module (CBAM) structure diagram in the embodiment of the application;
[0036] Figure 5 is the self-adaptive double threshold determination flowchart in the embodiment of the application. DETAILED DESCRIPTION
[0037] The technical solutions of the application will be described in detail below with reference to the drawings of the application. Obviously, the described embodiments are only part of the embodiments of the application, not all embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the application.
[0038] In order for those skilled in the art to understand and implement the application, the specific embodiments of the application are described in detail below in the form of steps in combination with the drawings and experimental data. All experiments were carried out under a unified software and hardware environment: the computer was configured with a CPU frequency of 4.89GHz, a memory of 16GB, and a GPU of NVIDIA GeForce RTX 2080Ti; the operating system was Windows 11, and the Python 3.8 programming language and PyTorch deep learning framework were used. Specifically, the following steps are included:
[0039] S1, data set construction and preprocessing
[0040] S1.1, data acquisition and annotation
[0041] The 4K super-definition monitoring camera deployed at the port gate obtains the actual operation video stream. The improved inter-frame difference method is used for key frame extraction: the video stream is grayed, the absolute difference matrix of adjacent frames is calculated, the local area difference intensity is analyzed by setting a sliding window, and the clear frame of the complete container entering the monitoring field is accurately captured by setting the difference intensity of 80% peak value as the trigger threshold. Finally, a data set containing 1400 high-quality images is constructed, covering five views of the front, rear, left, right and top of the container.
[0042] The data set covers seven common damage types (hole, damage, rust, scratch, surface deformation, beam deformation, and corner column deformation), and uses an efficient binary classification labeling strategy (damage / normal). The damage sample accounts for 38.2% of the data set, and the normal sample accounts for 61.8%. The stratified random sampling method is used to divide the data set into a training set (1120 images) and a validation set (280 images) in an 8:2 ratio, ensuring consistent data distribution.
[0043] S1.2, image enhancement
[0044] The following data enhancement operations are applied to the training set images to improve the model's generalization ability:
[0045] Random horizontal and vertical flipping;
[0046] Random rotation within ±15 degrees;
[0047] Translation within ±5% of image width and height;
[0048] ±10% brightness and contrast adjustment;
[0049] All transformations are implemented using Pytorch's built-in functions, which expand the training samples while maintaining the integrity of the original damage features.
[0050] S2, construction and training of lightweight U-Net segmentation model
[0051] S2.1, model architecture construction
[0052] An improved lightweight U-Net model is constructed as shown in the accompanying Figure 2 The core innovations include:
[0053] Mixed convolution strategy: In Block1 and Block2 of the Encoder and Block1 and Block2 of the Decoder, deep separable convolution (DSC, structure as shown in the accompanying Figure 3) instead of standard convolution to significantly reduce the number of parameters and calculations. In Block3 to Block5 of the encoder and Block3 and Block4 of the decoder, standard convolution is retained to ensure effective extraction of deep semantic features.
[0054] CBAM attention enhanced skip connection: in the skip connection of U-Net, the convolution block attention module (CBAM, structure as shown in the attached Figure 4 ), through its channel and spatial dual-channel attention sub-module to strengthen the target feature extraction, improve the segmentation robustness of the model in complex environment.
[0055] S2.2, loss function and optimization
[0056] The last layer of the model uses a 1x1 convolution kernel for pixel-level binary classification (background or box surface). To solve the class imbalance problem of box surface and background pixels (ratio about 1:3.4), a weighted cross-entropy loss function is used, which is defined as follows:
[0057] ;
[0058] where, represents the total number of pixels in the image; is the true label of the i-th pixel, when represents the box surface, 0 represents the background; is the probability of the i-th pixel being predicted as a box surface; is the weight coefficient of the background pixel, according to the ratio of box surface pixels to background pixels in the training set samples, which is about 1:3.4, according to the class inverse frequency weighting principle, set =0.294.
[0059] S2.3, model training
[0060] Using Adam optimizer, the initial learning rate is set to 1e-4, and the batch size is 16. Train the model on the constructed dataset until the loss function converges.
[0061] S3, construction and training of visual Transformer classification model
[0062] S3.1, data preprocessing
[0063] Use the trained lightweight U-Net model to segment and background stripping of all training set and validation set images, generate pure box surface images, and unify the scaling to 512x512 pixels as the input of the ViT classification model.
[0064] S3.2, model architecture construction and training
[0065] Constructing Vision Transformer (ViT) model:
[0066] Image patching and serialization: The input box surface image is divided into 16x16 non-overlapping image patches, each of which is converted into a 256-dimensional embedding vector through linear projection and added with learnable position encoding , forming a one-dimensional sequence Z:
[0067] ;
[0068] wherein, represents the image patch after adding the position embedding vector, N = 1024 is the sequence length; an additional learnable classification token vector is added at the front of the sequence.
[0069] Feature encoding: the sequence Z is input into an encoder stacked by 12 layers of Transformer modules. Each layer contains multi-head self-attention and feed-forward network (FFN). MSA is implemented through scaled dot-product attention:
[0070] ;
[0071] wherein, is the feature dimension; in the present application, 8 heads are used for parallel attention calculation; the Adam optimizer is used for training on the box surface image binary classification dataset.
[0072] S4, implementation of adaptive double-threshold decision mechanism
[0073] In the inference stage, the ViT model outputs the damage probability , and then the following determination process (as shown in the accompanying Figure 5 ) is performed:
[0074] S4.1, preliminary classification: compare the probability P with the classification threshold , if , the preliminary determination is "damage"; otherwise, the preliminary determination is "normal"; the classification threshold is determined by drawing the PR curve on the validation set and selecting the threshold point that maximizes the F1 score.
[0075] S4.2, confidence evaluation and final decision: calculate the standard deviation of the prediction probability obtained by multiple random forward propagation (with Dropout enabled) of the model on the current sample ; compare with the preset confidence threshold (set to 0.1):
[0076] if , indicating that the model prediction confidence is high, and directly adopting the preliminary classification result;
[0077] If , indicating that the model prediction uncertainty is high, the sample is marked as "manual review".
[0078] S5, experimental results and performance analysis
[0079] S5.1, lightweight U-Net ablation experiment
[0080] To verify the mixed convolution strategy, a symmetric replacement experiment is performed under the same training conditions, and the results are as follows:
[0081] Table 1: Ablation experiment results of mixed convolution model
[0082] Configuration mIoU (%) Parameter quantity (M) Inference speed (FPS) Gradient update time-consuming (s / epoch) Benchmark model 99.06 28.24 32 58 1st level replacement 98.88 24.13 39 49 2nd level replacement 98.51 18.75 50 37 3rd level replacement 97.12 13.42 61 28 4th level replacement 94.37 9.06 70 21 All replacement 88.95 5.31 82 15
[0083] The results in Table 1 show that the mixed convolution strategy adopted by the present application has a significant effect on model lightweight. With the increase of the replacement level of deep separable convolution, the parameter quantity and computational quantity of the model are significantly reduced, and the inference speed is simultaneously improved. In particular, when the first two layers (2-level) of the encoder and the decoder are replaced, the model reduces the parameter quantity by 33.6%, improves the inference speed by 56.3%, and only decreases the segmentation accuracy (mIoU) by 0.55 percentage points, achieving the best balance between accuracy and efficiency, and proving the effectiveness of the mixed strategy.
[0084] To verify the contribution of each improved component, the following 5 configurations are compared under the same training set and test environment, and the complete model is the final configuration adopted by the present application:
[0085] Table 2: Comparison of network performance of lightweight U-Net model improvement ablation experiment
[0086] Configuration Parameter quantity (M) mIoU (%) Inference speed (FPS) Benchmark model (standard U-Net) 28.24 99.06 32 + depth separable convolution 20.21 98.51 48 + CBAM 28.83 99.32 30 + weighted cross-entropy loss 28.24 99.18 32 Complete model (the application) 20.79 99.03 45
[0087] The results in Table 2 clearly show the role of each improved component. Deep separable convolution is the core of model lightweight, which significantly improves the speed; CBAM attention mechanism and weighted cross-entropy loss improve the feature extraction ability and robustness to class imbalance of the model, respectively. Finally, the complete model (Full Model) of the present application successfully integrates these improvements, reduces the parameter quantity by 26.4% while maintaining the top segmentation accuracy of 99.03%, and improves the inference speed to 45 FPS, achieving the perfect unity of accuracy, robustness and efficiency.
[0088] S5.2, ablation experiment of double threshold determination mechanism
[0089] Compare the performance of double threshold and single threshold strategy:
[0090] Table 3: Comparison of classification strategy performance
[0091] Model type Precision Recall F1 score False positive rate Missed detection rate Manual review rate Baseline model (double threshold) (the application) 95.0% 94.5% 94.7% 5.0% 5.5% 10.0% Ablation model (single threshold) 92.0% 93.0% 92.5% 8.0% 7.0% 15.0%
[0092] The results in Table 3 fully demonstrate the great advantages of the adaptive double-threshold decision mechanism proposed in the present application. Compared with the single-threshold method, the double-threshold mechanism has significant improvements in key indicators such as precision (3.0% improvement), F1 score (2.2% improvement), and false positive rate (3.0% reduction). More importantly, this mechanism successfully reduces the manual review rate from 15.0% to 10.0%, which effectively improves the automation level and actual application efficiency of the entire system while ensuring high detection accuracy.
[0093] S5.3, Performance comparison with prior art
[0094] Comparison of segmentation model performance:
[0095] Table 4: Comparison of segmentation model performance
[0096] Model Parameter quantity (M) Computing amount (GFLOPs) Inference speed (FPS) Memory occupation (MB) mIoU (%) MobileNetV3-Seg 22.83 6.82 45 820 98.1 Fast-SCNN 1.14 2.45 62 450 96.78 DeepLabV3 5.79 9.63 38 680 98.4 U-Net (original) 28.24 12.34 32 1080 99.06 Lightweight U-Net (the application) 20.21 5.75 48 760 99.03
[0097] The comparison results of the segmentation models in Table 4 show that the lightweight U-Net proposed in the present application has obvious advantages in overall performance. Compared with the original U-Net, this model achieves almost lossless segmentation accuracy (99.03% vs 99.06%) while reducing the parameter amount by 28.4% and the computational amount by 53.4%, and the inference speed is improved by 50%. Compared with other lightweight models, this model maintains the highest accuracy while achieving the best balance in parameter amount, computational efficiency, and memory occupation, proving the superiority of its design.
[0098] Comparison of overall classification detection performance:
[0099] Table 5: Comparison of classification model performance
[0100] Model type Model Precision Recall F1 score False positive rate Missed detection rate Target detection YOLO v5 46.1% 60.3% 52.3% - - Target detection YOLO v7 58.3% 70.0% 63.6% - - Classification Vggnet 88.9% 80.0% 84.2% 11.1% 20.0% Classification Resnet 85.7% 92.3% 88.9% 14.3% 7.7% Classification Vit 92.2% 90.6% 90.1% 7.8% 9.4% Segmentation + classification (the application) Lightweight U-Net + double threshold Vit (the application) 95.0% 94.5% 94.7% 5.0% 5.5%
[0101] The comprehensive comparison results in Table 5 indisputably prove the excellence of the two-stage framework proposed in the present application. The final F1 score of the present application reaches 94.7%, which is significantly higher than all comparison models, including mainstream YOLO series target detection models and advanced classification networks. At the same time, the present application achieves the lowest false positive rate (5.0%) and missed detection rate (5.5%), which proves its high reliability in actual port complex environments. Finally, the framework of the present application realizes the design goal of "machine dominance and human assistance" with a manual review rate of 10.0%, setting a new benchmark in detection accuracy, robustness, and automation level
[0102] S6, Conclusion
[0103] Through the systematic implementation steps of S1 to S5 and the comprehensive experimental verification, including ablation experiments and comparison with prior art, it is fully shown that the "container damage detection method based on lightweight U-Net and visual Transformer" provided by the present application is superior to the existing mainstream method in terms of segmentation accuracy (mIoU 99.03%), classification accuracy (F1 score 94.7%), reasoning speed (48 FPS) and system automation degree (manual review rate 10.0%). The method successfully balances the accuracy, speed and deployability of edge devices, providing a powerful "machine-led, human-assisted" solution for automated port inspection, has significant industrial application value, and meets the requirements of the Patent Law on novelty, inventiveness and practicality.
[0104] Although embodiments of the present application have been shown and described, it will be understood by those having ordinary skill in the art that various changes, modifications, substitutions and alterations can be made therein without departing from the principles and spirit of the present application, the scope of which is defined by the appended claims and their equivalents.
Claims
1. A container damage detection method based on lightweight U-Net and visual Transformer, characterized in that, The method comprises the following steps: S1, container image input and preprocessing: obtaining a video stream from a monitoring camera deployed at a port gate, and extracting key frames by using an inter-frame difference method to obtain a clear image containing a complete container body; S2, box face segmentation and background stripping based on an improved lightweight U-Net: inputting the image obtained in S1 into an improved lightweight U-Net model to accurately segment the container box face region and realize background stripping; S3, adaptive double-threshold damage classification based on a visual Transformer: inputting the box face image segmented in S2 into a visual Transformer model to perform binary classification of whether there is damage, and introducing an adaptive double-threshold mechanism; S4, detection result output: outputting a final "damage", "normal" or "manual review required" conclusion according to the determination result of S3.
2. The container damage detection method based on lightweight U-Net and visual Transformer according to claim 1, characterized in that, In the S1 step, the inter-frame difference method is an improved inter-frame difference method, specifically including: performing gray scale processing on the video stream, calculating the absolute difference matrix of adjacent frames, analyzing the local area difference intensity by setting a sliding window, and taking the difference intensity of 80% of the peak value as the trigger threshold to accurately capture the clear frame of the complete container entering the monitoring field.
3. The container damage detection method based on lightweight U-Net and visual Transformer according to claim 1, characterized in that, In the S2 step, the improved lightweight U-Net model adopts a hybrid convolution strategy, specifically including: using a depth separable convolution instead of a standard convolution in Block1 and Block2 of the encoder and Block1 and Block2 of the decoder, and retaining the standard convolution in Block3 to Block5 of the encoder and Block3 and Block4 of the decoder.
4. The container damage detection method based on lightweight U-Net and visual Transformer according to claim 1, characterized in that, In the S2 step, the improved lightweight U-Net model integrates a convolution block attention module in the skip connection to strengthen target feature extraction through a channel and spatial dual-path attention mechanism.
5. The container damage detection method based on lightweight U-Net and visual Transformer according to claim 1, characterized in that, In the S2 step, the improved lightweight U-Net model uses a weighted cross-entropy loss function for model optimization, wherein the weight coefficient of the background pixels is set according to the ratio of the box face pixels to the background pixels in the training set samples, and the ratio is about 1:3.
4. According to the category inverse frequency weighting principle, the weight coefficient of the background pixels is set to 0.
294.
6. The container damage detection method based on lightweight U-Net and visual Transformer according to claim 1, characterized in that, In the S3 step, the visual Transformer model performs image blocking and serialization processing, specifically including: dividing the input segmented box face image into 16x16 non-overlapping blocks, converting each block into a 256-dimensional embedding vector through linear projection, and adding learnable position encoding to convert the two-dimensional image into a one-dimensional sequence.
7. The container damage detection method based on lightweight U-Net and visual Transformer according to claim 1, characterized in that, In the S3 step, the feature encoding part of the visual Transformer model is stacked by a 12-layer Transformer module, each layer containing a multi-head self-attention module and a feedforward network.
8. The container damage detection method based on lightweight U-Net and visual Transformer according to claim 1, characterized in that, The adaptive double-threshold mechanism in the S3 step includes: S3.1, preliminary classification: comparing the damage probability P output by the model with the classification threshold, if the damage probability P is greater than or equal to the classification threshold, it is preliminarily determined as "damage", otherwise it is preliminarily determined as "normal"; S3.2, confidence evaluation and final decision: the standard deviation of the prediction probability obtained by multiple random forward propagation of the current sample by the calculation model is calculated, and the standard deviation is compared with the preset confidence threshold value. If the standard deviation is less than or equal to the confidence threshold value, the preliminary classification result is adopted, otherwise the sample is marked as "manual review is required".
9. The container damage detection method based on lightweight U-Net and visual Transformer according to claim 1, characterized in that, The classification threshold is determined by drawing a PR curve on a validation set and selecting a threshold point that maximizes the F1 score.
10. The container damage detection method based on lightweight U-Net and visual Transformer according to claim 1, characterized in that, In the S4 step, the system automatically outputs the "damage", "normal" or "manual review is required" conclusion according to the determination result of S3, realizing an intelligent detection process dominated by machines and assisted by humans.