A method and system for detecting container packing photos based on computer vision

Abstract

This invention relates to a computer vision-based method and system for detecting container packing photos. Based on YOLO and CRNN networks, target detection models and container number recognition models are constructed for container opening / closing status, container number location, and cargo packaging, respectively. The interface receives packing photo addresses and container numbers from the business system. After successfully loading the corresponding images, the optimized target detection and container number recognition models comprehensively and accurately obtain the specific content of the container type and number. Finally, the information identified by each model is organized, and cross-validation is performed based on the identified container number to find the corresponding information in the business system, determining the compliance of the uploaded photos. This completes the intelligent detection of packing photos, significantly reducing the workload of customer service personnel while effectively identifying non-compliant or problematic photos. It reduces the risks and costs caused by incorrect or missed detection of non-compliant photos, and can greatly improve the efficiency and accuracy of packing photo detection.

Description

Technical Field

[0001] This invention relates to the field of container loading photo inspection technology, and in particular to a method and system for container loading photo inspection based on computer vision. Background Technology

[0002] In container shipping, customers are required to promptly upload photos of their cargo to the shipping company after loading. This serves several purposes: proving the cargo is intact and identifying the type of goods; preventing the concealment or misdeclaration of dangerous goods, ensuring the safety of cargo and the vessel; preventing customers from using low-priced cargo to secure contracts, thus protecting the company's route profitability; allowing the shipping company to arrange loading plans based on the cargo situation, improving transportation efficiency and safety; and serving as important archival documents, quality monitoring records, and for future reference and retrieval. Photos must clearly show container doors fully open, partially open, and fully closed. Image quality is paramount; container numbers must be clearly visible and match the order numbers. Even in partially open and fully open photos, the cargo must be clearly visible. However, in practice, the quality of uploaded loading photos varies greatly, with many failing to meet requirements. Examples include photos completely unrelated to loading, low-resolution photos, photos taken at incorrect angles, and photos with inconsistent container numbers. This significantly impacts subsequent business processes and progress.

[0003] However, due to the large number of photos, the traditional method of verifying packing photos can only rely on customer service staff to manually check them. This process consumes a lot of manpower, is slow, and has a low rate of manual checks, making it impossible to achieve full coverage. This poses risks of misidentifying or missing non-compliant photos and subsequent business disputes, resulting in a low accuracy rate.

[0004] Currently, some container number identification methods have gradually emerged, but existing solutions for detecting containers are relatively singular in their approach. They can only identify the container number and do not pay much attention to the detection of goods and packaging types inside the container, thus failing to achieve comprehensive detection.

[0005] Therefore, there is an urgent need for a more efficient, accurate, and comprehensive method to optimize the traditional manual inspection of packing photos, realize the automated identification and verification of packing photos, replace manual verification work to the greatest extent, greatly reduce the workload of customer service, and reduce the risks and costs caused by incorrect or missed detection of non-compliant photos, thereby improving the overall competitiveness and efficiency of enterprises. Summary of the Invention

[0006] This invention addresses the problems of traditional manual inspection of container packing photos, which is labor-intensive and prone to errors and omissions in detecting non-compliant photos, as well as the limitations of current container number recognition methods which only identify the container number and do not provide comprehensive detection. It proposes a computer vision-based method for container packing photo detection. Based on YOLO and CRNN networks, it constructs target detection models and container number recognition models for container opening / closing status, container number location, and cargo packaging. The interface receives the container packing photo address and container number from the business system. After successfully loading the corresponding image, it comprehensively and accurately obtains the specific content of the container type and number through optimized target detection and container number recognition models. Finally, the information identified by each model is organized, and the corresponding information in the business system is cross-validated based on the identified container number to determine the compliance of the uploaded photo. This significantly reduces the workload of customer service personnel while effectively identifying non-compliant or problematic photos, reducing the risks and costs caused by errors or omissions in detecting non-compliant photos, and greatly improving the efficiency and accuracy of container packing photo detection. This invention also relates to a computer vision-based container packing photo detection system.

[0007] The technical solution of the present invention is as follows:

[0008] A method for detecting container packing photos based on computer vision, characterized by the following steps:

[0009] The dataset acquisition and processing steps involve obtaining a large number of container loading photos from the business system server. The container loading photos include fully open, half open, and fully closed loading situations. The container loading photos are then filtered according to preset filtering conditions to obtain the filtered dataset.

[0010] The object detection model generation steps involve generating several object detection models based on the YOLO network. These models include a container opening / closing status detection model, a cargo location detection model, and a cargo packaging type detection model. A labeling tool is used to label the selected dataset for container opening / closing status, cargo location, and cargo packaging type. The labeled dataset is then divided into an object detection training set, an object detection validation set, and an object detection test set according to a preset ratio. The divided training and validation sets are input into each object detection model for training. After a preset number of training rounds, the corresponding object detection model is automatically validated to evaluate its performance. Finally, the divided test sets are input into each trained and validated object detection model for testing, thereby generating optimized object detection models.

[0011] The container number recognition model generation steps are as follows: a container number recognition model is generated based on a CRNN network, and the selected dataset is labeled with container numbers using a labeling tool. The labeled dataset is then divided into a container number recognition training set, a container number recognition verification set, and a container number recognition test set according to a preset ratio. The segmented container number recognition training set and container number recognition verification set are input into the container number recognition model for training. After a preset number of training rounds, the current container number recognition model is automatically verified to evaluate its performance. Finally, the segmented container number recognition test set is input into the trained and verified container number recognition model for testing to generate an optimized container number recognition model.

[0012] The container packing photo detection and processing steps involve receiving several container packing photos uploaded by the user via an API interface. An optimized container opening / closing status detection model is used to detect the container's opening / closing status. If a container is detected as fully open or partially open, an optimized cargo location detection model is used to detect the cargo location, and an optimized cargo packaging type detection model is used to detect the cargo packaging type. The detection results for cargo location and cargo packaging type are then output. If a container is detected as fully closed or partially open, an optimized container number recognition model is used to identify the container number, and the identification result is output. Finally, the cargo location and cargo packaging type detection results are cross-validated with the container number recognition results to complete the intelligent detection of the container packing photos.

[0013] Preferably, in the target detection model generation step, the target detection model generated based on the YOLO network further includes a seal number location detection model and a box number location detection model. The filtered dataset is also labeled with seal number location and box number location using a labeling tool. The labeled dataset is then divided into a target detection training set, a target detection validation set, and a target detection test set according to a preset ratio. Each target detection model is then trained, validated, and tested to generate optimized target detection models.

[0014] In the container photo detection and processing step, when the container is detected to be in an open or closed state, the position of the seal number is detected by an optimized seal number position detection model, and the seal number position detection result is output. When the container is detected to be in an open or closed state and is either fully closed or half open, the container number position is first detected by an optimized container number position detection model, and then the container number is identified by an optimized container number recognition model, and the container number position detection result and container number recognition result are output. Then, the detection results of cargo position, cargo packaging type, seal number position, and container number position are cross-validated with the container number recognition result to complete the intelligent detection of the container photo.

[0015] Preferably, in the target detection model generation step, the target detection model generated based on the YOLO network also includes a model for recognizing each component of the container number. Furthermore, the selected dataset is labeled with each component of the container number using a labeling tool. The labeled dataset is then divided into a target detection training set, a target detection validation set, and a target detection test set according to a preset ratio. Finally, the model for recognizing each component of the container number is trained, validated, and tested to generate an optimized model for recognizing each component of the container number.

[0016] In the container photo detection and processing step, when the container is detected to be in an open or closed state and is either fully closed or partially open, the container number position is first detected by an optimized container number position detection model, then the container number components are identified by an optimized container number component identification model, and finally the container number is identified by an optimized container number identification model. The container number position detection result, the container number component identification result, and the container number identification result are then output. Furthermore, the cargo position, cargo packaging type, seal number position, and container number position detection results, container number component identification results, and container number identification results are cross-validated to complete the intelligent detection of the container photo.

[0017] Preferably, in the container packing photo detection and processing step, after receiving several container packing photos uploaded by the user through the API interface, the container position is first detected in the photos, and when the container is detected, the container opening and closing status is detected by an optimized container opening and closing status detection model.

[0018] Preferably, in the container photo detection and processing step, when a container is detected, the orientation of the container photo is automatically detected. Based on the detected orientation of the container photo, the container photo is automatically rotated at 0 degrees, 90 degrees, 180 degrees or 270 degrees using a rotating image orientation tool. After rotation, the container opening and closing status is detected by an optimized container opening and closing status detection model.

[0019] Preferably, in the dataset acquisition and processing step, the filtered dataset is further subjected to image grayscale conversion, binarization, balancing of the number of samples in each category, image size adjustment, normalization and tensor quantization to obtain the filtered and preprocessed dataset; the subsequent target detection model generation step and box number recognition model generation step are performed by labeling the filtered and preprocessed dataset with a labeling tool, and the corresponding training set, validation set and test set are split in a ratio of 3:1:1.

[0020] A computer vision-based container packing photo detection system is characterized by comprising a dataset acquisition and processing module, a target detection model generation module, a container number recognition model generation module, and a packing photo detection and processing module. The dataset acquisition and processing module is connected to both the target detection model generation module and the container number recognition model generation module, and both the target detection model generation module and the container number recognition model generation module are connected to the packing photo detection and processing module.

[0021] The dataset acquisition and processing module acquires a massive amount of container loading photos from the business system server. The container loading photos include fully open, half open, and fully closed loading situations. The module then filters the container loading photos according to preset filtering conditions to obtain the filtered dataset.

[0022] The target detection model generation module generates several target detection models based on the YOLO network. These models include a container opening / closing status detection model, a cargo location detection model, and a cargo packaging type detection model. A labeling tool is used to label the selected dataset for container opening / closing status, cargo location, and cargo packaging type. The labeled dataset is then divided into a target detection training set, a target detection validation set, and a target detection test set according to a preset ratio. The divided training and validation sets are input into each target detection model for training. After a preset number of training rounds, the module automatically validates the obtained target detection model to evaluate its performance. Finally, the divided test sets are input into each trained and validated target detection model for testing, generating optimized target detection models.

[0023] The container number recognition model generation module generates a container number recognition model based on a CRNN network, and uses a labeling tool to label the selected dataset with container numbers. Then, the labeled dataset is divided into a container number recognition training set, a container number recognition verification set, and a container number recognition test set according to a preset ratio. The segmented container number recognition training set and container number recognition verification set are input into the container number recognition model for training. After a preset number of training rounds, the module automatically verifies the current container number recognition model to evaluate its performance. Finally, the segmented container number recognition test set is input into the trained and verified container number recognition model for testing to generate an optimized container number recognition model.

[0024] The container packing photo detection and processing module receives several container packing photos uploaded by the user via an API interface. It detects the container's opening and closing status using an optimized container opening and closing status detection model. When a container is detected to be in an open or closed state, and is either fully open or partially open, it detects the cargo location using an optimized cargo location detection model and the cargo packaging type using an optimized cargo packaging type detection model, outputting the cargo location and packaging type detection results. When a container is detected to be in an open or closed state, and is either fully closed or partially open, it identifies the container number using an optimized container number recognition model and outputs the container number recognition result. Finally, it cross-validates the cargo location and packaging type detection results with the container number recognition results to complete the intelligent detection of the container packing photos.

[0025] Preferably, in the target detection model generation module, the target detection model generated based on the YOLO network further includes a seal number location detection model and a box number location detection model. The filtered dataset is also labeled with seal number location and box number location using a labeling tool. The labeled dataset is then divided into a target detection training set, a target detection validation set, and a target detection test set according to a preset ratio. Each target detection model is then trained, validated, and tested to generate optimized target detection models.

[0026] In the container photo detection and processing module, when the container is detected to be in an open or closed state, the position of the seal number is detected by an optimized seal number position detection model, and the seal number position detection result is output. When the container is detected to be in an open or closed state and is either fully closed or half open, the container number position is first detected by an optimized container number position detection model, and then the container number is identified by an optimized container number recognition model, and the container number position detection result and container number recognition result are output. Then, the detection results of cargo position, cargo packaging type, seal number position, and container number position are cross-validated with the container number recognition result to complete the intelligent detection of the container photo.

[0027] Preferably, in the target detection model generation module, the target detection model generated based on the YOLO network also includes a model for recognizing each component of the container number. Furthermore, the selected dataset is labeled with each component of the container number using a labeling tool. The labeled dataset is then divided into a target detection training set, a target detection validation set, and a target detection test set according to a preset ratio. Finally, the model for recognizing each component of the container number is trained, validated, and tested to generate an optimized model for recognizing each component of the container number.

[0028] In the container photo detection and processing module, when a container is detected to be in an open or closed state, and is either fully closed or partially open, the module first detects the container number position using an optimized container number position detection model, then identifies each component of the container number using an optimized container number component recognition model, and finally identifies the container number using an optimized container number recognition model. The module then outputs the container number position detection result, the container number component recognition result, and the container number recognition result. Furthermore, the module cross-validates the cargo position, cargo packaging type, seal number position, container number position detection result, container number component recognition result, and container number recognition result to complete the intelligent detection of the container photo.

[0029] Preferably, in the container packing photo detection and processing module, after receiving several container packing photos uploaded by the user via the API interface, the module first performs container position detection on the photos. When a container is detected, it also automatically performs container photo orientation detection. Based on the detected container photo orientation, the module uses an image rotation tool to automatically rotate the container photo in 0, 90, 180, or 270 degree directions. After rotation, the module then uses an optimized container opening and closing status detection model to detect the container opening and closing status.

[0030] The technical effects of this invention are as follows:

[0031] This invention relates to a computer vision-based method for detecting container packing photos. After acquiring a massive amount of container packing photos, several target detection models are generated based on a YOLO network, including a container opening / closing status detection model, a cargo location detection model, and a cargo packaging type detection model. A container number recognition model is also generated based on a CRNN network. The dataset is labeled, proportionally segmented, and the models are trained, validated, and tested to generate optimized target detection models and container number recognition models. Then, several container packing photos uploaded by users are received via an API interface. The optimized container opening / closing status detection model detects the container's opening / closing status. When a container is detected to be in an open / closed state and is either fully open or partially open, the optimized cargo location detection model detects the cargo location, and the optimized cargo packaging type detection model detects the cargo packaging type. The detection results for cargo location and cargo packaging type are output. When a container is detected to be in an open / closed state and is either fully closed or partially open... The invention identifies container numbers and outputs the identification results through an optimized container number recognition model. Using a deep learning model based on YOLO and CRNN networks, it can accurately identify the container opening / closing status, cargo location, cargo packaging type, and container number in container packing photos. In other words, it can further identify the container opening / closing status, cargo location, cargo packaging type, container orientation, seal number location, and container type code in container packing photos, building upon accurate container number identification. It integrates computer vision technologies such as image processing, machine learning, feature extraction, and object detection and recognition. By cross-validating the identified container number with the corresponding information (cargo location and cargo packaging type detection results), it achieves intelligent detection of packing photos. This significantly reduces the workload of shipping company customer service, enabling full coverage verification of packing photos and reducing the risks and costs associated with incorrect or missed detections of non-compliant photos. It can greatly improve the efficiency and accuracy of packing photo detection.

[0032] This invention presents a computer vision-based method for detecting container packing photos. It employs a deep learning model for rapid, accurate, and comprehensive detection of packing photos, replacing the traditional, tedious, and inefficient manual sampling. Existing solutions only detect the container's orientation and angle, and can only identify the container number. This invention, however, not only accurately identifies the container number but also further identifies whether the cargo and packaging types match the information stored in the business system for the same container number. Through an API service interface, it can meet the reuse needs of various shipowner systems, exhibiting good robustness and scalability. This invention optimizes the traditional manual sampling method for detecting packing photos through a more efficient, accurate, and comprehensive approach, achieving automated identification and verification of packing photos. It largely replaces manual verification work; only suspected problematic photos are screened after cross-validation and submitted to customer service personnel for further manual review. Normal, qualified photos that pass cross-validation do not require manual intervention, significantly reducing customer service workload and minimizing the risks and costs associated with incorrect or missed detections of non-compliant photos, thereby improving the overall competitiveness and efficiency of the enterprise.

[0033] This invention also relates to a computer vision-based container packing photo detection system. This system corresponds to the aforementioned computer vision-based container packing photo detection method and can be understood as a system that implements the aforementioned computer vision-based container packing photo detection method. It includes a dataset acquisition and processing module, an object detection model generation module, a container number recognition model generation module, and a packing photo detection and processing module. These modules work collaboratively, using deep learning models based on YOLO and CRNN networks to construct object detection models and container number recognition models for container opening / closing status, container number location, and cargo packaging. The interface receives data such as the packing photo address and container number from the business system, and after successfully loading the corresponding image... The system utilizes target detection models based on container opening / closing status, container number location, and cargo packaging. Images with container numbers are further processed using a text recognition algorithm within the container number recognition model. This comprehensive and accurate acquisition of container type and number information is achieved. The information from each model is then compiled and cross-validated against the corresponding information in the business system to determine the compliance of uploaded photos. Problematic data is then identified and submitted to customer service for further verification. This significantly reduces the workload of customer service personnel while effectively identifying non-compliant or problematic photos. It minimizes the risks and costs associated with incorrect or missed detections of non-compliant photos, substantially improving the efficiency and accuracy of container photo detection and demonstrating promising engineering application prospects. Attached Figure Description

[0034] Figure 1 This is a flowchart of the container packing photo detection method based on computer vision according to the present invention.

[0035] Figure 2A schematic diagram illustrating the steps involved in generating the target detection model.

[0036] Figure 3 A schematic diagram illustrating the steps involved in generating the container number recognition model.

[0037] Figure 4 The preferred flowchart for the packing photo inspection and processing steps.

[0038] Figure 5 This is a structural block diagram of the container packing photo detection system based on computer vision according to the present invention. Detailed Implementation

[0039] The present invention will now be described with reference to the accompanying drawings.

[0040] This invention relates to a computer vision-based method for detecting container packing photos, the process of which is as follows: Figure 1 As shown, it includes the following steps:

[0041] S1: Dataset acquisition and processing steps: Obtain a large number of container loading photos from the business system server. The container loading photos include fully open, half open (i.e., half closed) and fully closed loading situations. The container loading photos are filtered according to preset filtering conditions to obtain the filtered dataset.

[0042] Preferably, this step may include collecting packing photos, screening packing photos, and image preprocessing.

[0043] S11: Container packing photo collection: Obtain a large number of container packing photos from the business system server (i.e., obtain historical data stored in the business system server), including packing situations such as fully open, half open, and fully closed.

[0044] S12: Packing Photo Filtering: Filter container packing photos according to preset filtering conditions to remove invalid photos such as those without containers or duplicate photos. The filtered photos are used as training, validation and test datasets.

[0045] S13: Image preprocessing: Preferably, the filtered dataset is subjected to image grayscale conversion, binarization, balancing of the number of samples in each category, image size adjustment, normalization and tensor quantization to reduce interference from factors such as illumination and blurring, and improve the recognition effect after training, so as to obtain the filtered and preprocessed dataset; through a series of processing steps, the original data is transformed into a form suitable for model training.

[0046] S2: Object Detection Model Generation Step. Several object detection models are generated based on the YOLO network. These models include a container opening / closing status detection model, a cargo location detection model, and a cargo packaging type detection model. A labeling tool is used to label the selected dataset for container opening / closing status, cargo location, and cargo packaging type. The labeled dataset is then divided into an object detection training set, an object detection validation set, and an object detection test set according to a preset ratio. The divided training and validation sets are input into each object detection model for training. After a preset number of training rounds, the corresponding object detection model is automatically validated to evaluate its performance. The divided test sets are then input into each trained and validated object detection model for testing, generating optimized object detection models. This step involves labeling the selected and preprocessed dataset and inputting it into each object detection network model for training and testing. Specifically, as shown... Figure 2 The schematic diagram is shown.

[0047] S21: Initial target detection model generation: Several target detection models are generated based on the YOLO network. The target detection models include a container opening / closing status detection model, a seal number location detection model, a container number location detection model, a container number component identification model, a cargo location detection model, and a cargo packaging type detection model.

[0048] S22: Object Detection Dataset Labeling: The dataset filtered and preprocessed in S13 is labeled using a labeling tool (such as the IabelImg labeling tool). Preferably, the following are labeled separately: container opening / closing status (door opening / closing status), seal number location, container number location, components of the container number, cargo location, and cargo packaging type (or special packaging type). Among them, the labeled components of the container number include the container owner code, container registration code, verification number, and container type code. The IabelImg labeling tool can directly generate a label format suitable for training the object detection model.

[0049] S23: Object detection dataset segmentation: The labeled dataset in S22 is segmented into an object detection training set, an object detection validation set, and an object detection test set according to a preset ratio; preferably, the segmentation ratio of the corresponding training set, validation set, and test set can be set to 3:1:1, and it is necessary to ensure that there is no overlap between the datasets, so as to ensure the accuracy and reliability of the training data;

[0050] S24: Object detection model training and validation: Input the object detection training set and object detection validation set segmented in S23 into each object detection model network for training, and after a specified number of training rounds, the network automatically validates the corresponding object detection model obtained at the current time, evaluates the performance of the corresponding object detection model at the current time, and formulates subsequent training strategies in order to obtain the current optimal performance model;

[0051] S25: Object detection model testing: Input the object detection test set segmented in S23 into each object detection model trained and validated in S24 for testing. If the performance is not good, repeat S24 for iterative training to optimize the model, and finally generate optimized object detection models.

[0052] It's worth noting that the S2 object detection model generation process uses a YOLO network-based model, which is an end-to-end trained model capable of fast and accurate object detection. The detection process can be viewed as a black box; one only needs to focus on the image input to the model and the model's detection results.

[0053] To improve the overall recognition performance of photos of containers, each different detection target needs to be assigned a separate detection model. If multiple detection targets are mixed in a single model, it can easily lead to misidentification between targets, and it's difficult to determine the number of bounding boxes for each category. If the photo contains multiple shipping containers, unnecessary containers may be identified, interfering with the overall results and resulting in a low recognition rate. Furthermore, training each detection target separately offers better scalability; if new detection targets need to be added later, a new detection model can be trained separately.

[0054] S3: The container number recognition model generation step involves generating a container number recognition model based on a CRNN network. The selected dataset is labeled with container numbers using a labeling tool. The labeled dataset is then divided into a training set, a validation set, and a test set according to a preset ratio. The divided training and validation sets are input into the container number recognition model for training. After a preset number of training rounds, the model is automatically validated to evaluate its performance. Finally, the divided test set is input into the trained and validated model for testing, generating an optimized container number recognition model. This step involves labeling the selected and preprocessed dataset and inputting it into the container number recognition model for training and testing. Specifically, as shown... Figure 3 The schematic diagram is shown.

[0055] S31: Initial Container Number Recognition Model Generation: A container number recognition model is generated based on a CRNN network. This model is a text recognition network model. Preferably, a text recognition model suitable for container number recognition is trained using transfer learning based on the CRNN network. The CRNN network, or Convolutional Recurrent Neural Network, is an end-to-end training model, mainly composed of CNN, RNN, and CTC. The CNN convolutional layer is mainly used for image processing, extracting feature sequences from a given image. The RNN recurrent layer is mainly used for natural language processing, making predictions based on the feature sequences generated by the CNN convolutional layer. CTC can organically combine the two, outputting the predicted character sequences from the RNN recurrent layer as labels, thereby achieving the recognition of variable-length container number text.

[0056] S32: Container number recognition dataset generation: Using the optimized container number location detection model obtained in S25, extract the dataset images after filtering and preprocessing in S13 to obtain images of each component of the container number area (container owner code, container registration code, verification number and container type code);

[0057] S33: Labeling and Segmentation of Container Number Dataset: Label the dataset generated in S32 and segment it into a container number recognition training set, a container number recognition verification set, and a container number recognition test set according to a preset ratio; preferably, the segmentation ratio of the corresponding training set, verification set, and test set can be set to 3:1:1;

[0058] S34: Container number recognition model training: Input the container number recognition training set and container number recognition validation set segmented in S33 into the container number recognition model network for training. After a specified number of training rounds, the network automatically validates the current container number recognition model, evaluates the performance of the current container number recognition model, and formulates subsequent training strategies to obtain the current optimal performance model.

[0059] S35: Container number recognition model test: Input the container number recognition test set segmented in S33 into the container number recognition model trained in S34 for testing. If the performance is not good, repeat S34 for iterative training to optimize the model and finally generate an optimized container number recognition model.

[0060] S4: Packing photo detection and processing steps. The API interface receives several container packing photos uploaded by the user. An optimized container opening and closing status detection model is used to detect the container's opening and closing status. When a container is detected to be in an open or closed state and is either fully open or partially open, an optimized cargo location detection model is used to detect the cargo location, and an optimized cargo packaging type detection model is used to detect the cargo packaging type. The cargo location and cargo packaging type detection results are then output. When a container is detected to be in an open or closed state and is either fully closed or partially open, an optimized container number recognition model is used to identify the container number and the container number recognition result is output. Finally, the cargo location and cargo packaging type detection results are cross-validated with the container number recognition results to complete the intelligent detection of the packing photos. Specifically, this step can be understood as the deployment and application of the packing photo detection model. By writing detection code and generating an API interface, the optimized target detection model and container number recognition model generated in S2 and S3 are integrated, and the interface is deployed to the server. This allows the business system to input packing photos, and the interface returns information such as the container location (i.e., container position), open / closed status, seal number location, container number location, cargo location, cargo packaging type, container number, and container type code. The business system then performs further verification processing based on the returned detection results. Figure 4 The preferred flowchart is shown below.

[0061] exist Figure 4 In the process, the interface receives data such as the container photo address and container number from the business system. After successfully loading the corresponding image, it uses target detection models for container opening / closing status, container number location, and cargo packaging. Images with container numbers are further processed using a container number recognition model based on text recognition algorithms to comprehensively and accurately obtain the specific information about the container type and number. Finally, the information recognized by each model is organized, and cross-validation is performed based on the corresponding information in the business system to determine the compliance of the uploaded photos. Problematic data is then filtered and submitted to customer service personnel for further verification, significantly reducing the workload of customer service staff while effectively identifying non-compliant or problematic photos. Specifically,

[0062] S41: Code writing and interface generation: Using development tools, write code according to the established packing photo detection process and generate an API service interface to receive parameters from the business system; preferably, Python can be used for development, the development tool is PyCharm, and the API service interface transmission format is JSON;

[0063] S42: Interface Deployment and Detection Result Generation: Deploy the interface code written in S41 to the server. The business system passes the parameters required by each detection model through the interface. After receiving several container loading photos uploaded by the user via the API interface, the container position can be detected first, i.e., the presence or absence of container photos can be determined. The detection ends when no container photo is found (no container in the photo). When a container is detected, the container opening and closing status detection model is used to detect the container opening and closing status. When a container is detected to be in an open or closed state and is in the fully closed type, the container number position detection model is used to detect the container number position. The system first identifies the container number using an optimized model for recognizing each component, then identifies the container number itself using an optimized model for recognizing the container number, and outputs the container number location detection result, the container number component recognition result, and the container number recognition result. This is then returned to the business system via an interface. When a container is detected to be in an open or closed state and is in a fully open or partially open type (i.e., not fully closed), the system detects the cargo location using an optimized cargo location detection model and the cargo packaging type using an optimized cargo packaging type detection model, outputting the cargo location and cargo packaging type detection results. These results are then returned to the business system via an interface, or further... Figure 4 As shown, in the fully open type, the results are returned to the business system through the interface. In the half-open type, the box number position is detected by the optimized box number position detection model, the box number components are identified by the optimized box number component identification model, and the box number is identified by the optimized box number identification model. The box number position detection result, the box number component identification result, and the box number identification result are output and then returned to the business system through the interface.

[0064] When the container is detected to be in an open or closed state, the position of the seal number is detected by an optimized seal number position detection model, and the seal number position detection result is output. The detection result is returned to the business system through the interface. The detection ends when no seal number position is detected.

[0065] S43: Business System Cross-Verification: The business system cross-verifies the various detection results generated in S42 (container number location detection results, container number component identification results, container number identification results, seal number location detection results, cargo location and cargo packaging type detection results) with the container number and the business information stored in the system. It checks whether the container is a compliant container or seal number, whether all three container door opening / closing states (fully open, half open, and fully closed) are present, whether there is cargo, and whether the packaging type, container number, and container type code are consistent with those stored in the business system. Qualified photos that cross-verify do not require manual intervention, completing the intelligent detection of packing photos. Only the results of inconsistent cross-verification are provided to customer service for further processing.

[0066] This invention optimizes the traditional manual inspection of container packing photos by using a more efficient, accurate, and comprehensive method. It achieves automated identification and verification of container packing photos, replacing manual verification work to the greatest extent. After cross-verification, suspected problematic photos are screened out and handed over to customer service personnel for further manual review. Container packing photos that cross-verify are considered normal and qualified photos and do not require manual intervention. This greatly reduces the workload of customer service and reduces the risks and costs caused by incorrect or missed detection of non-compliant photos, thereby improving the overall competitiveness and efficiency of enterprises.

[0067] Furthermore, in the S4 container photo detection and processing step, in S42, when a container is detected, the orientation of the container photo can be automatically detected. Based on the detected orientation of the container photo, the container photo is automatically rotated at 0 degrees, 90 degrees, 180 degrees, or 270 degrees using an image rotation tool. After rotation, the container opening and closing status is detected by an optimized container opening and closing status detection model, thereby further improving the efficiency of intelligent detection of container photos.

[0068] This invention also relates to a computer vision-based container packing photo detection system, which corresponds to the aforementioned computer vision-based container packing photo detection method. This system can be understood as a system that implements the aforementioned computer vision-based container packing photo detection method. Figure 5 As shown, the system includes a dataset acquisition and processing module, an object detection model generation module, a container number recognition model generation module, and a container photo detection and processing module. The dataset acquisition and processing module is connected to both the object detection model generation module and the container number recognition model generation module. The object detection model generation module is connected to the container number recognition model generation module, and both the object detection model generation module and the container number recognition model generation module are connected to the container photo detection and processing module.

[0069] The dataset acquisition and processing module acquires a massive amount of container loading photos from the business system server. The container loading photos include fully open, half open, and fully closed loading situations. The module then filters the container loading photos according to preset filtering conditions to obtain the filtered dataset.

[0070] The target detection model generation module generates several target detection models based on the YOLO network. These models include a container opening / closing status detection model, a cargo location detection model, and a cargo packaging type detection model. A labeling tool is used to label the selected dataset for container opening / closing status, cargo location, and cargo packaging type. The labeled dataset is then divided into a target detection training set, a target detection validation set, and a target detection test set according to a preset ratio. The divided training and validation sets are input into each target detection model for training. After a preset number of training rounds, the module automatically validates the obtained target detection model to evaluate its performance. Finally, the divided test sets are input into each trained and validated target detection model for testing, generating optimized target detection models.

[0071] The container number recognition model generation module generates a container number recognition model based on a CRNN network, and uses a labeling tool to label the selected dataset with container numbers. Then, the labeled dataset is divided into a container number recognition training set, a container number recognition verification set, and a container number recognition test set according to a preset ratio. The segmented container number recognition training set and container number recognition verification set are input into the container number recognition model for training. After a preset number of training rounds, the module automatically verifies the current container number recognition model to evaluate its performance. Finally, the segmented container number recognition test set is input into the trained and verified container number recognition model for testing to generate an optimized container number recognition model.

[0072] The container packing photo detection and processing module receives several container packing photos uploaded by the user via an API interface. It detects the container's opening and closing status using an optimized container opening and closing status detection model. When a container is detected to be in an open or closed state, and is either fully open or partially open, it detects the cargo location using an optimized cargo location detection model and the cargo packaging type using an optimized cargo packaging type detection model, outputting the cargo location and packaging type detection results. When a container is detected to be in an open or closed state, and is either fully closed or partially open, it identifies the container number using an optimized container number recognition model and outputs the container number recognition result. Finally, it cross-validates the cargo location and packaging type detection results with the container number recognition results to complete the intelligent detection of the container packing photos.

[0073] Furthermore, in the target detection model generation module, the target detection models generated based on the YOLO network also include a seal number location detection model and a box number location detection model. The filtered dataset is also labeled with seal number location and box number location using a labeling tool. The labeled dataset is then divided into a target detection training set, a target detection validation set, and a target detection test set according to a preset ratio. Each target detection model is then trained, validated, and tested to generate optimized target detection models.

[0074] In the container photo detection and processing module, when the container is detected to be in an open or closed state, the position of the seal number is detected by an optimized seal number position detection model, and the seal number position detection result is output. When the container is detected to be in an open or closed state and is either fully closed or half open, the container number position is first detected by an optimized container number position detection model, and then the container number is identified by an optimized container number recognition model, and the container number position detection result and container number recognition result are output. Then, the detection results of cargo position, cargo packaging type, seal number position, and container number position are cross-validated with the container number recognition result to complete the intelligent detection of the container photo.

[0075] Furthermore, in the target detection model generation module, the target detection model generated based on the YOLO network also includes a model for recognizing each component of the container number. The selected dataset is labeled with each component of the container number using a labeling tool. The labeled dataset is then divided into a target detection training set, a target detection validation set, and a target detection test set according to a preset ratio. The model for recognizing each component of the container number is then trained, validated, and tested to generate an optimized model for recognizing each component of the container number.

[0076] In the container photo detection and processing module, when a container is detected to be in an open or closed state, and is either fully closed or partially open, the module first detects the container number position using an optimized container number position detection model, then identifies each component of the container number using an optimized container number component recognition model, and finally identifies the container number using an optimized container number recognition model. The module then outputs the container number position detection result, the container number component recognition result, and the container number recognition result. Furthermore, the module cross-validates the cargo position, cargo packaging type, seal number position, container number position detection result, container number component recognition result, and container number recognition result to complete the intelligent detection of the container photo.

[0077] Furthermore, in the container packing photo detection and processing module, after receiving several container packing photos uploaded by the user via the API interface, the module first performs container position detection on the photos. When a container is detected, it also automatically performs container photo orientation detection. Based on the detected container photo orientation, the module uses an image rotation tool to automatically rotate the container photo in 0, 90, 180, or 270 degree directions. After rotation, the module then uses an optimized container opening and closing status detection model to detect the container opening and closing status.

[0078] It should be noted that the specific embodiments described above enable those skilled in the art to more fully understand the present invention, but do not limit the present invention in any way. Therefore, although the present invention has been described in detail with reference to the accompanying drawings and embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the present invention. In short, all technical solutions and improvements that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the present invention patent.

Claims

1. A method for detecting container packing photos based on computer vision, characterized in that, Includes the following steps: The dataset acquisition and processing steps involve obtaining a large number of container loading photos from the business system server. The container loading photos include fully open, half open, and fully closed loading situations. The container loading photos are then filtered according to preset filtering conditions to obtain the filtered dataset. The object detection model generation steps involve generating several object detection models based on the YOLO network. These models include a container opening / closing status detection model, a cargo location detection model, and a cargo packaging type detection model. A labeling tool is used to label the selected dataset for container opening / closing status, cargo location, and cargo packaging type. The labeled dataset is then divided into an object detection training set, an object detection validation set, and an object detection test set according to a preset ratio. The divided training and validation sets are input into each object detection model for training. After a preset number of training rounds, the corresponding object detection model is automatically validated to evaluate its performance. Finally, the divided test sets are input into each trained and validated object detection model for testing, thereby generating optimized object detection models. The container number recognition model generation steps are as follows: a container number recognition model is generated based on a CRNN network, and the selected dataset is labeled with container numbers using a labeling tool. The labeled dataset is then divided into a container number recognition training set, a container number recognition verification set, and a container number recognition test set according to a preset ratio. The segmented container number recognition training set and container number recognition verification set are input into the container number recognition model for training. After a preset number of training rounds, the current container number recognition model is automatically verified to evaluate its performance. Finally, the segmented container number recognition test set is input into the trained and verified container number recognition model for testing to generate an optimized container number recognition model. The container packing photo detection and processing steps involve receiving several container packing photos uploaded by the user via an API interface. An optimized container opening / closing status detection model is used to detect the container's opening / closing status. If a container is detected as fully open or partially open, an optimized cargo location detection model is used to detect the cargo location, and an optimized cargo packaging type detection model is used to detect the cargo packaging type. The detection results for cargo location and cargo packaging type are then output. If a container is detected as fully closed or partially open, an optimized container number recognition model is used to identify the container number, and the identification result is output. Finally, the cargo location and cargo packaging type detection results are cross-validated with the container number recognition results to complete the intelligent detection of the container packing photos.

2. The container packing photo detection method based on computer vision according to claim 1, characterized in that, In the target detection model generation step, the target detection model generated based on the YOLO network also includes a seal number location detection model and a box number location detection model. The filtered dataset is further labeled with seal number location and box number location using a labeling tool. The labeled dataset is then divided into a target detection training set, a target detection validation set, and a target detection test set according to a preset ratio. Each target detection model is then trained, validated, and tested to generate optimized target detection models. In the container photo detection and processing step, when the container is detected to be in an open or closed state, the position of the seal number is detected by an optimized seal number position detection model, and the seal number position detection result is output. When the container is detected to be in an open or closed state and is either fully closed or half open, the container number position is first detected by an optimized container number position detection model, and then the container number is identified by an optimized container number recognition model, and the container number position detection result and container number recognition result are output. Then, the detection results of cargo position, cargo packaging type, seal number position, and container number position are cross-validated with the container number recognition result to complete the intelligent detection of the container photo.

3. The container packing photo detection method based on computer vision according to claim 2, characterized in that, In the target detection model generation step, the target detection model generated based on the YOLO network also includes a model for recognizing each component of the container number. The selected dataset is labeled with each component of the container number using a labeling tool. The labeled dataset is then divided into a target detection training set, a target detection validation set, and a target detection test set according to a preset ratio. The model for recognizing each component of the container number is then trained, validated, and tested to generate an optimized model for recognizing each component of the container number. In the container photo detection and processing step, when the container is detected to be in an open or closed state and is either fully closed or partially open, the container number position is first detected by an optimized container number position detection model, then the container number components are identified by an optimized container number component identification model, and finally the container number is identified by an optimized container number identification model. The container number position detection result, the container number component identification result, and the container number identification result are then output. Furthermore, the cargo position, cargo packaging type, seal number position, and container number position detection results, container number component identification results, and container number identification results are cross-validated to complete the intelligent detection of the container photo.

4. The container packing photo detection method based on computer vision according to any one of claims 1 to 3, characterized in that, In the container packing photo detection and processing step, after receiving several container packing photos uploaded by the user through the API interface, the container position is first detected in the photos. When a container is detected, the container opening and closing status is detected through an optimized container opening and closing status detection model.

5. The container packing photo detection method based on computer vision according to claim 4, characterized in that, In the container photo detection and processing step, when a container is detected, the orientation of the container photo is automatically detected. Based on the detected orientation of the container photo, the container photo is automatically rotated at 0 degrees, 90 degrees, 180 degrees or 270 degrees using an image rotation tool. After rotation, the container opening and closing status is detected by an optimized container opening and closing status detection model.

6. The container packing photo detection method based on computer vision according to claim 4, characterized in that, In the dataset acquisition and processing step, the filtered dataset is further subjected to image grayscale conversion, binarization, balancing of the number of samples in each category, image size adjustment, normalization and tensor quantization to obtain the filtered and preprocessed dataset. The subsequent target detection model generation step and box number recognition model generation step are to label the filtered and preprocessed dataset with a labeling tool, and the corresponding training set, validation set and test set are split in a ratio of 3:1:

1.

7. A computer vision-based container packing photo detection system, characterized in that, The system includes a dataset acquisition and processing module, an object detection model generation module, a container number recognition model generation module, and a container photo detection and processing module. The dataset acquisition and processing module is connected to both the object detection model generation module and the container number recognition model generation module. Both the object detection model generation module and the container number recognition model generation module are connected to the container photo detection and processing module. The dataset acquisition and processing module acquires a large number of container loading photos from the business system server. The container loading photos include fully open, half open and fully closed loading situations. The module then filters the container loading photos according to preset filtering conditions to obtain the filtered dataset. The target detection model generation module generates several target detection models based on the YOLO network. These models include a container opening / closing status detection model, a cargo location detection model, and a cargo packaging type detection model. A labeling tool is used to label the selected dataset for container opening / closing status, cargo location, and cargo packaging type. The labeled dataset is then divided into a target detection training set, a target detection validation set, and a target detection test set according to a preset ratio. The divided training and validation sets are input into each target detection model for training. After a preset number of training rounds, the module automatically validates the obtained target detection model to evaluate its performance. Finally, the divided test sets are input into each trained and validated target detection model for testing, generating optimized target detection models. The container number recognition model generation module generates a container number recognition model based on a CRNN network, and uses a labeling tool to label the selected dataset with container numbers. Then, the labeled dataset is divided into a container number recognition training set, a container number recognition verification set, and a container number recognition test set according to a preset ratio. The segmented container number recognition training set and container number recognition verification set are input into the container number recognition model for training. After a preset number of training rounds, the module automatically verifies the current container number recognition model to evaluate its performance. Finally, the segmented container number recognition test set is input into the trained and verified container number recognition model for testing to generate an optimized container number recognition model. The container packing photo detection and processing module receives several container packing photos uploaded by the user via an API interface. It detects the container's opening and closing status using an optimized container opening and closing status detection model. When a container is detected to be in an open or closed state, and is either fully open or partially open, it detects the cargo location using an optimized cargo location detection model and the cargo packaging type using an optimized cargo packaging type detection model, outputting the cargo location and packaging type detection results. When a container is detected to be in an open or closed state, and is either fully closed or partially open, it identifies the container number using an optimized container number recognition model and outputs the container number recognition result. Finally, it cross-validates the cargo location and packaging type detection results with the container number recognition results to complete the intelligent detection of the container packing photos.

8. The container packing photo detection system based on computer vision according to claim 7, characterized in that, In the target detection model generation module, the target detection models generated based on the YOLO network also include a seal number location detection model and a box number location detection model. The filtered dataset is further labeled with seal number location and box number location using a labeling tool. The labeled dataset is then divided into a target detection training set, a target detection validation set, and a target detection test set according to a preset ratio. Each target detection model is then trained, validated, and tested to generate optimized target detection models. In the container photo detection and processing module, when the container is detected to be in an open or closed state, the position of the seal number is detected by an optimized seal number position detection model, and the seal number position detection result is output. When the container is detected to be in an open or closed state and is either fully closed or half open, the container number position is first detected by an optimized container number position detection model, and then the container number is identified by an optimized container number recognition model, and the container number position detection result and container number recognition result are output. Then, the detection results of cargo position, cargo packaging type, seal number position, and container number position are cross-validated with the container number recognition result to complete the intelligent detection of the container photo.

9. The container packing photo detection system based on computer vision according to claim 8, characterized in that, In the target detection model generation module, the target detection model generated based on the YOLO network also includes a model for recognizing each component of the container number. The selected dataset is labeled with each component of the container number using a labeling tool. The labeled dataset is then divided into a target detection training set, a target detection validation set, and a target detection test set according to a preset ratio. The model for recognizing each component of the container number is then trained, validated, and tested to generate an optimized model for recognizing each component of the container number. In the container photo detection and processing module, when a container is detected to be in an open or closed state, and is either fully closed or partially open, the module first detects the container number position using an optimized container number position detection model, then identifies each component of the container number using an optimized container number component recognition model, and finally identifies the container number using an optimized container number recognition model. The module then outputs the container number position detection result, the container number component recognition result, and the container number recognition result. Furthermore, the module cross-validates the cargo position, cargo packaging type, seal number position, container number position detection result, container number component recognition result, and container number recognition result to complete the intelligent detection of the container photo.

10. The container packing photo detection system based on computer vision according to any one of claims 7 to 9, characterized in that, In the container packing photo detection and processing module, after receiving several container packing photos uploaded by the user via the API interface, the module first performs container position detection on the photos. When a container is detected, it also automatically performs container photo orientation detection. Based on the detected container photo orientation, the module uses an image rotation tool to automatically rotate the container photo in 0, 90, 180, or 270 degree directions. After rotation, the module then uses an optimized container opening and closing status detection model to detect the container opening and closing status.

Images