Airport luggage damage detecting and recording equipment based on machine vision recognition

Abstract

The invention relates to the technical field of luggage nondestructive testing, and discloses airport luggage damage detecting and recording equipment based on machine vision recognition, which comprises the following modules: an image acquisition module for acquiring image data and related information of luggage; the preprocessing module is connected with the image acquisition module and is used for preprocessing the acquired image data; the feature extraction module is connected with the preprocessing module and used for extracting preprocessed image features; the multi-model fusion module is connected with the feature extraction module and is used for analyzing and fusing the extracted features; and the multi-layer structure analysis module is connected with the multi-model fusion module and is used for analyzing the multi-layer structure of the luggage. Through multi-module cooperation, technologies such as self-supervised learning, multi-scale feature extraction, YOLO and U-Net algorithms, reinforcement learning and the like are creatively fused, background interference is accurately removed, damage features are highlighted, the system adapts to different environments, and the precision and the intelligent level of airport luggage damage detection are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of luggage damage detection, and in particular to an airport luggage damage detection and recording device based on machine vision recognition. Background Art

[0002] Baggage damage detection is a critical task in airport baggage handling. Traditional baggage damage detection relies primarily on manual visual inspection, which is not only inefficient but also prone to missed and false detections due to human error, making it difficult to meet the needs of large-scale baggage handling. With the development of computer vision technology, automatic detection methods based on image recognition are gradually gaining popularity. However, existing technologies still have many shortcomings. First, the airport environment is complex and ever-changing, and baggage has a variety of appearances, including different materials, colors, shapes, and sizes. Existing technologies cannot effectively remove background interference and accurately identify baggage damage features. For example, in dimly lit environments or when baggage is stacked, damage features are easily obscured, resulting in limited detection accuracy. Second, when detecting internal damage in multi-layered baggage, traditional methods cannot fully assess the damage level.

[0003] Based on this, those skilled in the art have proposed an airport luggage damage detection and recording device based on machine vision recognition to solve the above problems. Summary of the Invention

[0004] In view of the shortcomings of the existing technology, the present invention provides an airport luggage damage detection and recording device based on machine vision recognition, which solves the problems raised in the above background technology.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solutions: an airport baggage damage detection system based on machine vision recognition, including the following modules:

[0006] Image acquisition module, used to obtain image data and related information of luggage;

[0007] A preprocessing module is connected to the image acquisition module and performs preprocessing operations on the acquired image data;

[0008] A feature extraction module, connected to the preprocessing module, is used to extract features of the preprocessed image;

[0009] The multi-model fusion module is connected to the feature extraction module and is used to analyze and fuse the extracted features;

[0010] The multi-layer structure analysis module is connected to the multi-model fusion module and is used to analyze the multi-layer structure of luggage;

[0011] The adaptive optimization module is connected to the multi-model fusion module and the multi-layer structure analysis module to perform adaptive optimization on the system;

[0012] The detection data security recording module is connected to the multi-model fusion module and the multi-layer structure analysis module to record the detection data;

[0013] The output module is connected to the multi-model fusion module, the multi-layer structure analysis module and the detection data security recording module, and is used to output the detection results.

[0014] Preferably, the pre-processing module includes a self-supervised learning unit and an image enhancement unit, wherein the self-supervised learning unit is used to segment the luggage body from the background, and the image enhancement unit is used to perform contrast enhancement on the segmented luggage image.

[0015] Preferably, the feature extraction module adopts a multi-scale feature extraction network to obtain local detail features and overall shape features of the damaged luggage, and highlights important features related to the damage through an attention mechanism.

[0016] Preferably, the multi-model fusion module integrates the YOLO series target detection algorithm and the U-Net image segmentation algorithm. The YOLO algorithm first performs preliminary positioning and classification of the luggage, and then the U-Net algorithm performs accurate segmentation of the determined area.

[0017] Preferably, the multi-layer structure analysis module introduces 3D point cloud data acquired by a depth camera, constructs a three-dimensional model of the luggage through a 3D reconstruction algorithm, and performs layer-by-layer slice analysis on the multi-layer structure luggage. The adaptive optimization module adopts a reinforcement learning algorithm, uses model parameter adjustment as an intelligent agent action, uses detection accuracy as a reward function, and learns the optimal model parameter adjustment strategy through interaction with different airport environments.

[0018] Preferably, the detection data security recording module adopts blockchain technology to encrypt the detection results, detection time, detection equipment information and other data and store them in the blockchain to form an unalterable record. The output module is used to generate a detection report containing the damage location, damage degree, basic luggage information and corresponding flight information, and transmit the detection results to the airport's baggage management system.

[0019] Preferably, it further includes a feedback unit connected to the adaptive optimization module, which is used to correct information based on the detection results and manual work.

[0020] The airport luggage damage detection and recording equipment based on machine vision recognition includes a mounting frame and two support columns. The two support columns are respectively located on both sides of the mounting frame. A support frame is installed on the top of the mounting frame. A searchlight source, a depth camera and a storage device are installed on the outside of the support frame from left to right in sequence. A controller is installed on the outside of the depth camera.

[0021] Preferably, a collecting box is fixedly connected to one side of the top of the support column, a dust suction pump is installed on the outside of the collecting box, a connecting pipe is connected to the outside of the collecting box, a cleaning pipe is installed at the end of the connecting pipe, a robotic arm is installed on the top of the support column, and an electromagnetic suction cup is installed at the output end of the robotic arm.

[0022] Preferably, a conveyor belt is installed on the top of the mounting frame, and the conveyor belt is used to transport airport luggage.

[0023] The present invention provides an airport luggage damage detection and recording device based on machine vision recognition. It has the following beneficial effects:

[0024] 1. This invention leverages multi-module collaboration and innovatively integrates self-supervised learning, multi-scale feature extraction, YOLO and U-Net algorithms, and reinforcement learning technologies to precisely remove background interference, highlight damage features, and adapt to diverse environments. This significantly improves the accuracy and intelligence of airport baggage damage detection. Furthermore, by combining depth cameras, 3D reconstruction, and blockchain technology, it efficiently analyzes damage within multi-layered baggage while ensuring data security, providing airports with a highly efficient and accurate detection solution.

[0025] 2. This invention innovatively integrates multiple modules and achieves rapid and accurate detection and efficient data processing through technologies such as high-definition imaging, self-supervised learning preprocessing, multi-scale feature extraction and attention mechanism, YOLO and U-Net fusion, reinforcement learning adaptive optimization, and blockchain data management. This significantly improves the efficiency and intelligence level of airport baggage damage detection, and effectively promotes the upgrade of airport baggage handling processes.

[0026] 3. This invention can automatically flip luggage at airports, ensuring that all sides are fully inspected. This effectively improves the comprehensiveness and accuracy of damage detection and avoids missed inspections due to luggage placement angles. It also provides dustproof treatment on the exterior of the depth camera, effectively protecting the camera lens and preventing dust accumulation from affecting image quality, thereby ensuring the reliability of inspection data. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A perspective view of the present invention;

[0028] Figure 2 Schematic diagram of the searchlight source structure of the present invention;

[0029] Figure 3 This is a schematic diagram of the connecting pipe structure of the present invention;

[0030] Figure 4 This is a flow chart of the overall architecture of the airport baggage damage detection system of the present invention;

[0031] Figure 5This is the model fusion module and feature extraction flow chart of the present invention.

[0032] Among them, 1. Mounting frame; 2. Support frame; 3. Conveyor belt; 4. Support column; 5. Depth camera; 6. Controller; 7. Searchlight source; 8. Storage device; 9. Robotic arm; 10. Collection box; 11. Vacuum pump; 12. Connecting pipe. DETAILED DESCRIPTION

[0033] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the present specification. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0034] Please see the attached Figure 1 -Attached Figure 5 , the embodiment of the present invention provides 1. an airport baggage damage detection system based on machine vision recognition, comprising the following modules:

[0035] Image acquisition module, used to obtain image data and related information of luggage;

[0036] The preprocessing module is connected to the image acquisition module and performs preprocessing operations on the acquired image data. The preprocessing module includes a self-supervised learning unit and an image enhancement unit. The self-supervised learning unit is used to segment the luggage body from the background, and the image enhancement unit is used to enhance the contrast of the segmented luggage image.

[0037] Specifically, the image acquisition module is the fundamental data collection unit of the entire baggage damage detection system. It is equipped with a depth camera 5 and a searchlight 7. The depth camera 5 measures the distances to different points on the baggage surface by emitting and receiving infrared light, generating images containing depth information. These images not only reveal the baggage's external contours but also provide details about the surface's subtle structure, such as the depth of scratches or dents. The searchlight 7 ensures that the depth camera 5 can capture high-quality images even in low-light or complex airport environments.

[0038] The preprocessing module is closely linked to the image acquisition module. Its primary function is to optimize image data in preparation for subsequent feature extraction and analysis. The preprocessing module consists of two key units: a self-supervised learning unit and an image enhancement unit. The self-supervised learning unit utilizes a deep learning model to accurately segment the luggage from the background by analyzing image features such as texture, edges, and shape. This module's advantage lies in the fact that it does not require a large amount of labeled data, reducing the cost and time of data preparation. The image enhancement unit performs contrast enhancement on the segmented luggage image. By adjusting the image's brightness and contrast, it makes surface damage features (such as scratches and cracks) more visible, thereby improving the accuracy of subsequent inspections.

[0039] The feature extraction module is connected to the preprocessing module and is used to extract features from the preprocessed image. The feature extraction module uses a multi-scale feature extraction network to obtain local detail features and overall shape features of damaged luggage, and uses an attention mechanism to highlight important features related to the damage.

[0040] Specifically, the multi-scale feature extraction network analyzes image data at multiple scales by using convolution kernels of different sizes. This allows the network to simultaneously capture subtle damage features on the luggage surface (such as small scratches, cracks, and other local details) and overall shape features (such as the luggage's outline and structure). The application of the attention mechanism enables the system to better process images in complex backgrounds, reduce interference from irrelevant information, and further improve the performance of damage detection. Through the combination of the multi-scale feature extraction network and the attention mechanism, the feature extraction module can comprehensively and accurately extract feature information of damaged luggage, providing high-quality feature data for subsequent multi-model fusion analysis, ensuring the efficient operation of the entire detection system.

[0041] The multi-model fusion module is connected to the feature extraction module and is used to analyze and fuse the extracted features. The multi-model fusion module integrates the YOLO series target detection algorithm and the U-Net image segmentation algorithm. The YOLO algorithm first performs preliminary positioning and classification of luggage, and then the U-Net algorithm accurately segments the determined area.

[0042] Specifically, the YOLO series of object detection algorithms treats object detection as a regression problem, dividing the image into multiple grids and predicting bounding boxes and probabilities for each grid. The bounding boxes are weighted by the predicted probabilities, and high-scoring detections are displayed through thresholding. The convolution operation is the core of the YOLO series of object detection algorithms, which is used to extract image features and is defined as:

[0043] y(x,y)=(f\ * g)(x,y)=∑ m ∑ n f(m,n)g(xm,yn);

[0044] Among them, f(m,n) represents the pixel value of the input image at position (m,n), g(xm,yn) represents the position of the filter at (xm,yn), y(x,y) represents the value of the feature map obtained after the convolution operation at position (x,y), ∑ m ∑ n Represents the summation operation of all positions of the input image and convolution kernel.

[0045] The U-Net architecture consists of an encoder and a decoder. The encoder extracts image features and reduces spatial resolution, while the decoder restores the feature maps to the original input image size through upsampling. The output of each convolutional layer in the encoder is connected to the input of the corresponding layer in the decoder, preserving low-level details during the upsampling process.

[0046] The multi-model fusion module leverages the strengths of object detection and image segmentation by integrating the YOLO and U-Net algorithms. The YOLO algorithm initially locates and classifies the luggage, identifying possible damaged areas. The U-Net algorithm then accurately segments these areas, outputting detailed boundary and shape information for the damaged areas. This fusion approach ensures detection accuracy while improving efficiency.

[0047] The multi-layer structure analysis module, connected to the multi-model fusion module, is used to analyze the multi-layer structure of luggage. The multi-layer structure analysis module introduces 3D point cloud data obtained by the depth camera, constructs a three-dimensional model of the luggage through a 3D reconstruction algorithm, and performs layer-by-layer slice analysis on the multi-layer structure luggage. The adaptive optimization module uses a reinforcement learning algorithm, adjusts model parameters as the action of the intelligent agent, and uses detection accuracy as the reward function. It learns the optimal model parameter adjustment strategy through interaction with different airport environments.

[0048] Specifically, the Multi-Layer Structure Analysis Module, connected to the Multi-Model Fusion Module, is responsible for in-depth analysis of the multi-layered structure of luggage. It uses a depth camera to acquire 3D point cloud data and constructs a three-dimensional model of the luggage using a 3D reconstruction algorithm. This enables layer-by-layer analysis and accurately determines the internal damage of multi-layered luggage. The Adaptive Optimization Module utilizes a reinforcement learning algorithm, using model parameter adjustments as agent actions and detection accuracy as a reward function. The module learns the optimal model parameter adjustment strategy through interaction with different airport environments. This design improves the comprehensiveness and accuracy of detection while enhancing the system's adaptability to complex environments.

[0049] The adaptive optimization module is connected to the multi-model fusion module and the multi-layer structure analysis module to perform adaptive optimization on the system;

[0050] The detection data security recording module is connected to the multi-model fusion module and the multi-layer structure analysis module to record the detection data. The detection data security recording module uses blockchain technology to encrypt the detection results, detection time, detection equipment information and other data and store them in the blockchain to form an unalterable record. The output module is used to generate a detection report containing the damage location, damage degree, basic luggage information and corresponding flight information, and transmit the detection results to the airport's baggage management system.

[0051] Specifically, the test data security recording module is connected to the multi-model fusion module and the multi-layer structure analysis module. Its primary function is to record various data generated during the testing process. Specifically, this module uses blockchain technology to encrypt data such as test results, test time, and test equipment information before storing it on the blockchain. The characteristics of blockchain ensure that these records cannot be tampered with once generated, thus ensuring the authenticity and reliability of the test data and providing reliable data support for subsequent baggage compensation and other processes.

[0052] The output module is responsible for integrating and outputting the relevant information obtained from the inspection. It generates a detailed inspection report containing the damage location, extent, basic baggage information, and corresponding flight information. These reports help airport staff quickly and accurately understand the damage status of the baggage, facilitating timely action. The output module also transmits the inspection results to the airport's baggage management system, allowing the baggage inspection data to be shared within the airport's internal information network, facilitating timely access and processing by relevant departments, thereby improving the efficiency and management of the entire baggage handling process.

[0053] The output module is connected to the multi-model fusion module, the multi-layer structure analysis module and the detection data security recording module, and is used to output the detection results.

[0054] It also includes a feedback unit connected to the adaptive optimization module, which is used to correct information based on the detection results and manual work.

[0055] Specifically, the output module connects to the multi-model fusion module, the multi-layer structure analysis module, and the detection data security recording module to receive and integrate data, ultimately presenting comprehensive detection results through a user interface or report format. The feedback unit connects to the adaptive optimization module, transmitting detection results and manual correction information to it to adjust and optimize system parameters and model structure, thereby improving detection accuracy and efficiency. This design ensures that the system not only outputs accurate detection results but also has the ability to self-optimize to adapt to the needs of baggage damage detection in different environments and conditions.

[0056] The airport baggage damage detection and recording device based on machine vision recognition includes a mounting frame 1 and two support columns 4, one located on either side of the mounting frame 1. A support frame 2 is mounted on the top of the mounting frame 1. A searchlight 7, a depth camera 5, and a storage device 8 are mounted on the outside of the support frame 2, from left to right. A controller 6 is mounted on the outside of the depth camera 5. A collection box 10 is fixedly connected to one side of the top of the support column 4. A dust pump 11 is mounted on the outside of the collection box 10. A connecting pipe 12 is connected to the outside of the collection box 10. A cleaning pipe is mounted at the end of the connecting pipe 12. A robotic arm 9 is mounted on the top of the support column 4. An electromagnetic suction cup is mounted on the output end of the robotic arm 9. A conveyor belt 3 is mounted on the top of the mounting frame 1. The conveyor belt 3 is used to transport airport baggage.

[0057] Specifically, the function of the searchlight 7 is to provide sufficient illumination for image acquisition by the depth camera 5. The searchlight 7 can emit uniform light, ensuring clear images of the luggage in different ambient lighting conditions. The depth camera 5 is used to capture two-dimensional images and depth information of the luggage and is key to achieving machine vision recognition. The controller 6 is responsible for controlling the operation of the depth camera 5, including starting and stopping image acquisition and adjusting parameters. The storage device 8 is used to store the image data captured by the depth camera 5. The storage device 8 has a large-capacity storage capacity to cope with the large amount of data generated during the airport luggage inspection process.

[0058] The collection box 10 is used to collect dust or other impurities to keep the device clean. A dust suction pump 11 is installed outside the collection box 10. The dust suction pump 11 is connected to a cleaning pipe through a connecting pipe 12. The cleaning pipe is used to remove dust from the camera lens to ensure imaging quality.

[0059] The robotic arm 9 automatically adjusts its position, grasps the luggage using an electromagnetic suction cup, and flips it, ensuring that all sides of the luggage are visible to the depth camera 5, thus preventing any missed inspections due to luggage placement angles. The conveyor belt 3 transports airport luggage to the inspection area. Made of wear-resistant material, the conveyor belt 3 operates stably over extended periods, ensuring smooth movement of luggage during the inspection process.

[0060] Working principle: When using this device, it includes the following detailed operating principles:

[0061] Airport luggage is placed on conveyor belt 3, which smoothly transports it to the inspection area. Once the luggage enters the inspection area, a depth camera 5, under the control of a controller 6, captures multi-angle and multi-directional images of the luggage, acquiring two-dimensional image data and depth information, ensuring that the luggage's appearance is fully captured from different perspectives. Simultaneously, a searchlight 7 provides sufficient illumination for image capture, ensuring image quality.

[0062] The collected image data is transmitted to the pre-processing module. The self-supervised learning unit within this module first analyzes the image, accurately segmenting the luggage from the complex background and removing background interference. The image enhancement unit then performs contrast enhancement on the segmented luggage image, highlighting surface damage and making subsequent feature extraction more accurate and efficient.

[0063] The preprocessed image data is then transferred to the feature extraction module, which uses a multi-scale feature extraction network to extract both local details and overall shape features of the damaged luggage from the image. This multi-scale analysis captures damage features of varying sizes and shapes, such as scratches, dents, and cracks. Furthermore, an attention mechanism plays a role in the feature extraction process, automatically focusing on important features related to the damage, further improving the relevance and effectiveness of feature representation.

[0064] The extracted feature data is transmitted to the multi-model fusion module, which combines the YOLO family of object detection algorithms with the U-Net image segmentation algorithm. The YOLO algorithm is used to initially locate and classify the luggage, quickly determining its approximate location and the extent of any damaged areas. This reduces the detection range quickly and improves detection efficiency. The U-Net algorithm is then used to precisely segment the areas identified by the YOLO algorithm, demarcating the boundaries between damaged and normal areas at the pixel level and accurately locating the specific location and shape of the damage.

[0065] The 3D point cloud data captured by the depth camera 5 is transmitted to the multi-layer structure analysis module, which uses a 3D reconstruction algorithm to construct a 3D model of the luggage from the 3D point cloud data. Based on this 3D model, the multi-layered luggage is sliced ​​and analyzed layer by layer, enabling in-depth inspection of internal damage and achieving comprehensive, no-blind-angle inspection of the luggage. Simultaneously, the adaptive optimization module utilizes a reinforcement learning algorithm, treating model parameter adjustments as agent actions, with detection accuracy as the reward function. As the device is deployed in different airport environments, the agent continuously interacts with the environment to learn and automatically adjust model parameters to adapt to the complex airport environments and baggage types, ensuring optimal performance of the detection system.

[0066] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. The airport baggage damage detection system based on machine vision recognition is characterized by: Includes the following modules: Image acquisition module, used to obtain image data and related information of luggage; A preprocessing module is connected to the image acquisition module and performs preprocessing operations on the acquired image data; A feature extraction module, connected to the preprocessing module, is used to extract features of the preprocessed image; The multi-model fusion module is connected to the feature extraction module and is used to analyze and fuse the extracted features; The multi-layer structure analysis module is connected to the multi-model fusion module and is used to analyze the multi-layer structure of luggage; The adaptive optimization module is connected to the multi-model fusion module and the multi-layer structure analysis module to perform adaptive optimization on the system; The detection data security recording module is connected to the multi-model fusion module and the multi-layer structure analysis module to record the detection data; The output module is connected to the multi-model fusion module, the multi-layer structure analysis module and the detection data security recording module, and is used to output the detection results.

2. The airport baggage damage detection system based on machine vision recognition according to claim 1 is characterized in that: The pre-processing module includes a self-supervised learning unit and an image enhancement unit. The self-supervised learning unit is used to segment the luggage body from the background, and the image enhancement unit is used to enhance the contrast of the segmented luggage image.

3. The airport baggage damage detection system based on machine vision recognition according to claim 1 is characterized in that: The feature extraction module uses a multi-scale feature extraction network to obtain local detail features and overall shape features of damaged luggage, and highlights important features related to the damage through an attention mechanism.

4. The airport baggage damage detection system based on machine vision recognition according to claim 1 is characterized in that: The multi-model fusion module integrates the YOLO series target detection algorithm and the U-Net image segmentation algorithm. The YOLO algorithm first performs preliminary positioning and classification of the luggage, and then the U-Net algorithm accurately segments the determined area.

5. The airport baggage damage detection system based on machine vision recognition according to claim 1 is characterized in that: The multi-layer structure analysis module introduces 3D point cloud data acquired by a depth camera, constructs a three-dimensional model of the luggage through a 3D reconstruction algorithm, and performs layer-by-layer slice analysis on the multi-layer structured luggage. The adaptive optimization module uses a reinforcement learning algorithm, adjusts model parameters as the agent action, uses detection accuracy as the reward function, and learns the optimal model parameter adjustment strategy through interaction with different airport environments.

6. The airport baggage damage detection system based on machine vision recognition according to claim 1 is characterized in that: The inspection data security recording module uses blockchain technology to encrypt data such as inspection results, inspection time, and inspection equipment information and store them in the blockchain to form an unalterable record. The output module is used to generate an inspection report containing the damage location, damage extent, basic luggage information, and corresponding flight information, and transmit the inspection results to the airport's baggage management system.

7. The airport baggage damage detection system based on machine vision recognition according to claim 1 is characterized in that: It also includes a feedback unit connected to the adaptive optimization module, which is used to correct information based on the detection results and manual work.

8. Airport luggage damage detection and recording equipment based on machine vision recognition, according to the airport luggage damage detection system based on machine vision recognition according to any one of claims 1 to 7, characterized in that: The invention comprises a mounting frame (1) and two supporting columns (4), wherein the two supporting columns (4) are respectively located on both sides of the mounting frame (1); a supporting frame (2) is installed on the top of the mounting frame (1); a searchlight source (7), a depth camera (5) and a storage device (8) are installed on the outer side of the supporting frame (2) in sequence from left to right; and a controller (6) is installed on the outside of the depth camera (5).

9. The airport luggage damage detection and recording device based on machine vision recognition according to claim 8 is characterized in that: A collecting box (10) is fixedly connected to one side of the top of the support column (4), a dust suction pump (11) is installed on the outside of the collecting box (10), a connecting pipe (12) is connected to the outside of the collecting box (10), a cleaning pipe is installed at the end of the connecting pipe (12), a mechanical arm (9) is installed on the top of the support column (4), and an electromagnetic suction cup is installed at the output end of the mechanical arm (9).

10. The airport luggage damage detection and recording device based on machine vision recognition according to claim 8, characterized in that: A conveyor belt (3) is installed on the top of the mounting frame (1), and the conveyor belt (3) is used to transport airport luggage.

Images