An intelligent recognition method for airport visual images

Abstract

The present invention relates to the technical field of baggage identification, and provides an airport visual image intelligent identification method, comprising: step 1, collecting data through a horizontal camera and a vertical camera, and making a model training data set and a production environment data set; step 2, passing the data set collected by the horizontal camera into the target detection training network, and passing the data set collected by the vertical camera into the semantic segmentation training network, respectively obtaining the target detection network of the target category and number, as well as the target detection model and the semantic segmentation model results; step 3, passing the production environment data set into the image preprocessing model, and then passing the processed image into the target detection model and the semantic segmentation model, and finally obtaining the category, quantity, stacking, and whether the baggage consignment object contains a tray. The present invention can effectively improve the efficiency of baggage consignment and reduce sorting errors caused by manual fatigue.

Description

Technical Field

[0001] The present invention relates to the technical field of baggage identification, and in particular to an intelligent identification method for airport visual images. Background Art

[0002] Intelligent baggage identification is an important part of passenger service. It is related to the overall efficiency of the airport operation and affects the travel experience of passengers. Improving the quality of baggage services throughout the entire process is of great significance to improving the travel experience of passengers, ensuring flight safety and improving passenger transportation efficiency.

[0003] The entire baggage process includes check-in (handover), security check, sorting, loading, installation, and arrival (returning passengers). It involves the airline collecting baggage from passengers or other agents, loading baggage into aircraft, delivering baggage to airlines, ground handling delivery, and baggage arrival collection and transportation equipment. It affects the operating efficiency of airlines and ground handling services, and the passenger journey experience.

[0004] In the original baggage handling process, staff members need to manually check the flight number and destination station on the baggage tag, sort the baggage according to the corresponding number, manually check the special-shaped baggage, baggage with fragile marks and damaged baggage, and take close-up, mid-range, long-range and close-up images. The efficiency of shooting and recognition is highly related to the staff's proficiency in the equipment. This has a great impact on the operating efficiency of the entire process, and it is easy to cause errors in sorting and checking due to manual fatigue, resulting in unnecessary economic losses.

[0005] To solve the relevant problems, in recent years, airports and other civil aviation-related institutions have changed the traditional manual baggage inspection method to an intelligent baggage identification method. Intelligent baggage identification is a new type of service. With the development and combination of intelligent systems and artificial intelligence algorithms, the traditional baggage process is improved, and advanced technologies and equipment with algorithms are introduced into the traditional methods to realize the automation and intelligence of the entire process of passenger baggage. For example, in the direct flight link, the traditional check-in service requires manual intervention to record and judge the color, size, tray placement, etc. of the backpack. At present, passengers only need to scan the corresponding barcode or identity binding information on the self-service check-in device, and the system will automatically identify the relevant attributes of the baggage and complete the registration. In this process, only a small amount of manual judgment is required for special situations, which saves a lot of manpower and material resources. It improves the efficiency of baggage check-in, and at the same time avoids the loss and information binding errors caused by manual fatigue to a certain extent.

[0006] In the check-in (transportation) link, the existing self-service intelligent check-in and consignment systems usually use computer vision technology to identify the color and category of baggage images in order to improve the efficiency of baggage handling and reduce flight delays caused by baggage handling errors. Such algorithms can reduce the sorting error rate. However, the existing system is still insufficient in judging the stacking of backpacks and special-shaped bags, and there are differences between different airports, which makes data collection and interaction between different airports difficult. Summary of the invention

[0007] The present invention provides an airport visual image intelligent recognition method, which can effectively improve the efficiency of baggage check-in and reduce sorting errors caused by manual fatigue.

[0008] According to the present invention, a method for intelligently identifying airport visual images comprises the following steps: Step 1: Collect data through the horizontal camera and the vertical camera, and create a model training data set and a production environment data set; the data collected by the horizontal camera is used to determine the number and category of the checked baggage objects, and whether they contain pallets. The data sets of the horizontal camera and the vertical camera are combined to determine whether the objects are stacked; Step 2: The dataset collected by the horizontal camera is passed into the target detection training network, and the dataset collected by the vertical camera is passed into the semantic segmentation training network, to obtain the target detection network of target category and number, as well as the target detection model and semantic segmentation model results respectively; Step 3: The production environment dataset is passed into the image preprocessing model, and then the processed images are passed into the target detection model and the semantic segmentation model to finally obtain the category, quantity, stacking, and whether the baggage check-in object contains a pallet.

[0009] As a preferred method, the target detection model uses the YOLOv5 network for the type of checked baggage and whether it contains a tray. The YOLOv5 network architecture includes the backbone network Backbone, the neck network Neck, and the head network Head; The backbone network Backbone uses CSPDarknet53 as the basis for extracting image features. CSPDarknet53 gradually extracts features through a series of convolutional layers and residual blocks, in which the CSP structure is used to improve the representation ability of features. The backbone network gradually reduces the size of the feature map through multiple stages to detect objects at different scales. The neck network Neck is located after the backbone network and is used to further process feature maps and enhance representation capabilities. The neck network uses a series of convolution and upsampling layers to map the feature maps extracted by the backbone network to feature maps of different scales. These feature maps of different scales correspond to objects of different sizes. By gradually reducing the resolution of the feature map, the neck network can capture the contextual information of objects of different scales. The head network Head is responsible for generating the output of target detection; the head network operates on the feature map of each scale and generates bounding boxes and category predictions; the bounding box prediction uses the idea of ​​Anchor Boxes to represent the location and size of the target by predicting the center coordinates, width, height and confidence of the bounding box; the category prediction uses the softmax function to output the probabilities of different categories; The head network of YOLOv5 uses the SPP module to increase the receptive field and the PANet module to fuse features of different scales.

[0010] As a preference, YOLOv5 introduces adaptive training data augmentation, training resolutions of different sizes, multi-scale reasoning, and enhanced data augmentation strategies to further improve detection performance and robustness.

[0011] As a preferred method, BIFPN is embedded into the Yolov5 model to improve the performance of Yolov5 in dealing with the complex baggage stacking situation in airport security inspection scenarios; In the bottom-to-top stage of BIFPN, an upsampling mechanism is introduced to transfer the underlying features to the top layer step by step. Specifically, for adjacent feature maps, BIFPN combines low-level features with high-level features through upsampling operations to achieve upward transmission of information; in the top-to-bottom stage, BIFPN introduces a downsampling mechanism to transfer high-level features to low-level features; BIFPN realizes bidirectional information flow, and the bottom-to-top and top-to-bottom information transfer complement each other, establishing a close connection between multi-scale features.

[0012] Preferably, the ghost-conv module is used to reduce YOLOv5 parameters. Ghost-conv is a convolution operation in a convolutional neural network, which aims to significantly reduce the number of parameters and computational complexity while maintaining model performance.

[0013] Preferably, the matching strategy of YOLOv5 is responsible for effectively matching the predicted anchor points with the actual target bounding boxes.

[0014] As a preferred method, Soft NMS is introduced to solve the problem of baggage stacking. Soft NMS introduces a decay function to gradually reduce the scores of adjacent bounding boxes instead of directly removing these boxes; the formula for the Soft-NMS score reset function is as follows: ; in represents the adjusted score of the i-th detection box, represents the initial score of the i-th detection box, IOU represents the intersection-over-union ratio, M represents the detection box with the highest score, represents the i-th detection box, is the standard deviation; Soft NMS solves the shortcomings in handling baggage stacking and small target detection through a flexible suppression mechanism.

[0015] Preferably, in YOLOv5, the loss function includes: category loss, object confidence loss, and bounding box regression loss; use and structure , the formula is as follows: ; is the loss value, r is the adjustment factor used for dynamic adjustment , It can reflect the ratio of the current IOU loss of the model to the average IOU loss. and is a hyperparameter; These three loss parts are the loss functions of YOLOv5, which enable the model to focus on classification accuracy, object existence judgment, and precise positioning of bounding boxes during training.

[0016] The beneficial effects of the present invention are as follows: The present invention develops a vision-based intelligent airport baggage consignment method based on YOLOv5 classification, detection and semantic segmentation algorithms, combines a customized industrial computer to collect checked baggage data, and implements methods such as baggage type detection, stacking determination and special-shaped bag sorting. A software-hardware-combined airport baggage intelligent consignment system is implemented. The present invention can effectively improve baggage consignment efficiency and reduce sorting errors caused by manual fatigue. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 The figure is a flow chart of a method for intelligent recognition of airport visual images in an embodiment. DETAILED DESCRIPTION

[0018] In order to further understand the content of the present invention, the present invention is described in detail in conjunction with the accompanying drawings and embodiments. It should be understood that the embodiments are only for explaining the present invention and are not intended to limit it.

[0019] Example

[0020] like Figure 1 As shown, this embodiment provides an airport visual image intelligent recognition method, which is characterized by comprising the following steps: Step 1: Collect data through the horizontal camera and the vertical camera, and create a model training data set and a production environment data set. The main difference between the model training data set and the production environment data set is whether the environment is controllable, that is, whether there are influences such as lighting and human factors; the data collected by the horizontal camera is used to determine the number and category of baggage check-in objects, and whether they contain pallets. The data sets of the horizontal camera and the vertical camera are combined to determine whether objects are stacked.

[0021] To ensure the consistency between the production environment and the training / testing environment. When collecting training / testing data sets, a specific intelligent automatic baggage check-in machine that is consistent with the production environment equipment is selected for collection. In this case, interference from related factors such as human body exposure to the camera equipment screen and uneven light will still occur in the production environment. In order to suppress the occurrence of related situations, this embodiment divides the work area, and adjusts and improves the algorithm and equipment for abnormal light and reflection conditions in the environment. This ensures that the picture to be detected is always in the same detection range and the overall picture effect is consistent.

[0022] Due to the differences in lighting and color between the formal working environment and the image acquisition environment, such as visible light in the environment and reflection of objects due to sunlight, equalization is used to evenly distribute the pixels in the image in grayscale, and to enhance the image contrast so that the details in the image are clearer and more visible. Especially in images with uneven lighting, equalization can effectively improve the visual effect of the image.

[0023] Step 2: The dataset collected by the horizontal camera is passed into the target detection training network, and the dataset collected by the vertical camera is passed into the semantic segmentation training network, to obtain the target detection network of target category and number, as well as the target detection model and semantic segmentation model results respectively; Step 3: The production environment dataset is passed into the image preprocessing model, and then the processed images are passed into the target detection model and the semantic segmentation model to finally obtain the category, quantity, stacking, and whether the baggage check-in object contains a pallet.

[0024] According to the type of checked baggage and whether it contains a tray, the target detection model uses the YOLOv5 network. YOLOv5 is a deep learning network architecture for target detection. Its network architecture adopts a single-stage target detection method, that is, the object detection task is solved as a regression problem; the YOLOv5 network architecture includes the backbone network Backbone, the neck network Neck and the head network Head.

[0025] The backbone network Backbone uses CSPDarknet53 as the basis for extracting image features; CSPDarknet53 gradually extracts features through a series of convolutional layers and residual blocks, where the CSP structure is used to improve the representation ability of features; the backbone network gradually reduces the size of the feature map through multiple stages to detect targets at different scales.

[0026] The neck network Neck is located after the backbone network and is used to further process feature maps and enhance representation capabilities. The neck network uses a series of convolution and upsampling layers to map the feature maps extracted by the backbone network to feature maps of different scales. These feature maps of different scales correspond to objects of different sizes. By gradually reducing the resolution of the feature map, the neck network can capture the contextual information of objects of different scales. The head network Head is responsible for generating the output of target detection; the head network operates on the feature map of each scale and generates bounding boxes and category predictions; the bounding box prediction uses the idea of ​​Anchor Boxes to represent the location and size of the target by predicting the center coordinates, width, height and confidence of the bounding box; the category prediction uses the softmax function to output the probabilities of different categories; The head network of YOLOv5 uses the SPP module to increase the receptive field and the PANet module to fuse features of different scales.

[0027] YOLOv5 introduces adaptive training data augmentation, training resolutions of different sizes, multi-scale reasoning, and enhanced data augmentation strategies to further improve detection performance and robustness.

[0028] The BIFPN is embedded into the Yolov5 model to improve the performance of Yolov5 in dealing with the complex baggage stacking situation in airport security inspection scenarios.

[0029] BIFPN (Bi-directional Feature Pyramid Network) is a feature pyramid network for target detection. It aims to solve the limitation of the singleness of traditional feature pyramid network in information transmission. Its design is inspired by the complexity of targets existing at different scales and levels, requiring the network to capture these features comprehensively and flexibly. In the bottom-top stage of BIFPN, the bottom-level features are transmitted to the top level step by step by introducing an upsampling mechanism. This process aims to enrich the semantic information of high-level features so that the network can better understand the semantics and structure of the target. Specifically, for adjacent feature maps, BIFPN combines low-level features with high-level features through upsampling operations to achieve uplink transmission of information; in the top-bottom stage, BIFPN introduces a downsampling mechanism to transmit high-level features to low-level features; this helps to improve the spatial information of low-level features and enables the network to perceive the location and shape of the target more sensitively. By downsampling adjacent feature maps, BIFPN achieves excellent results in downlink transmission of information.

[0030] BIFPN realizes bidirectional information flow, and the bottom-top and top-bottom information transfer complement each other, so that multi-scale features are closely connected. This means that the network can more comprehensively understand the semantic information of objects of different scales, thereby improving the performance of target detection. By embedding BIFPN into the Yolov5 model, we aim to improve the performance of Yolov5 in dealing with complex situations such as luggage stacking in airport security scenes. Through the introduction of BIFPN, we expect to improve the model's detection accuracy for small targets and targets of different scales, and verify the effectiveness of this improvement in experiments. This design aims to make Yolov5 more adaptable to the needs of specific scenarios and improve the robustness and accuracy of target detection tasks.

[0031] In the conventional convolution process, in addition to the large number of model parameters and computational complexity, the redundancy of the feature maps generated is also very high. This means that although the model consumes more computing resources, the improvement in accuracy has little effect. In the environment of airport baggage security, the accuracy of the algorithm is as important as the computing speed. To this end, this embodiment uses a ghost-conv module to reduce model parameters. The ghost-conv module is used to reduce YOLOv5 parameters. Ghost-conv is a convolution operation in a convolutional neural network, which aims to significantly reduce the number of parameters and computational complexity while maintaining model performance. The goal of introducing Ghost Convolution is to achieve a lighter-weight deep neural network in a resource-constrained environment, such as mobile devices or embedded systems.

[0032] The core idea of ​​Ghost Convolution is to replace all filters in standard convolution by introducing "ghost filters". Ghost filters are a set of smaller filters, whose number is much less than the number of filters in the original convolution. By using these fewer ghost filters, Ghost Convolution effectively reduces the number of parameters, thereby reducing computational costs. This approach not only makes the network more efficient, but also alleviates the pressure on hardware resources to a certain extent, making the model easier to deploy in resource-constrained environments.

[0033] The design of Ghost Convolution is inspired by the observation that in deep neural networks, many filters have similar weights at adjacent positions. By taking advantage of this similarity, Ghost Convolution designs some filters as "ghost filters" so that they can be shared by multiple positions, reducing parameter redundancy. This parameter sharing strategy not only reduces the storage space of the model, but also improves computational efficiency.

[0034] YOLOv5's matching strategy is one of its core components and plays a decisive role in the performance of the model in practical applications. When dealing with object detection tasks, YOLOv5's matching strategy is responsible for effectively matching the predicted anchor points with the actual object bounding boxes to ensure accuracy and efficiency during the training process. This matching strategy of YOLOv5 not only improves the model's detection capabilities for various objects, especially in complex scenes, but also optimizes the training process and reduces the risk of false matches. With this fine-tuned matching strategy, YOLOv5 is able to effectively handle a variety of object detection tasks, thereby demonstrating better performance and robustness in practical applications.

[0035] To address the problem of baggage stacking, Soft NMS is introduced, which is an improved solution based on a flexible suppression mechanism.

[0036] Soft NMS introduces a decay function to gradually reduce the scores of adjacent bounding boxes instead of directly removing these boxes; the formula for the Soft-NMS score reset function is as follows: ; in represents the adjusted score of the i-th detection box, represents the initial score of the i-th detection box, IOU represents the intersection-over-union ratio, M represents the detection box with the highest score, represents the i-th detection box, is the standard deviation. In this formula, by introducing the exponential term, Soft NMS no longer directly sets the score to zero like traditional NMS, but gradually reduces the scores of adjacent bounding boxes. This allows potential real targets to be retained even if there is a certain overlap, improving adaptability to baggage stacking scenarios. The advantage of Soft NMS lies not only in its flexible suppression mechanism, but also in its increased sensitivity to small targets. Through the attenuation function, Soft NMS is more flexible in handling small targets and can retain the detection results of small-sized targets, which is better than traditional NMS in this regard.

[0037] Soft NMS solves the problems of baggage stacking and small target detection through a flexible suppression mechanism. This optimization provides a more accurate and reliable solution for target detection models in actual scenarios such as airport security checks.

[0038] The loss function of YOLOv5 is a key part of its training process, which directly affects the effect of the model learning target detection tasks. In YOLOv5, the loss functions include: category loss, object confidence loss, and bounding box regression loss; use and structure , the formula is as follows: ; is the loss value, r is the adjustment factor used for dynamic adjustment , It can reflect the ratio of the current IOU loss of the model to the average IOU loss. and is a hyperparameter; These three loss parts are the loss function of YOLOv5, which enables the model to focus on classification accuracy, object existence judgment, and precise positioning of bounding boxes during training. This comprehensive loss function design is one of the key factors that enable YOLOv5 to perform well in various challenging scenarios. Especially when dealing with scenes with complex backgrounds and diverse targets, this loss function can effectively guide model optimization and ensure high performance in all aspects of target detection. In addition, the loss function of YOLOv5 also has a certain degree of flexibility, allowing it to be adjusted according to different application requirements, thereby further improving the performance of the model on specific tasks. This flexible and efficient loss function design is one of the important reasons why YOLOv5 is so popular in the field of target detection.

[0039] The present invention and its embodiments are described schematically above, and the description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. Therefore, if a person skilled in the art is inspired by it and designs a structural method and an embodiment similar to the technical solution without creativity without departing from the purpose of the invention, they shall all fall within the protection scope of the present invention.

Claims

1. An intelligent airport visual image recognition method, characterized by: The following steps are involved: Step 1: Collect data through the horizontal camera and the vertical camera, and create a model training data set and a production environment data set; The data collected by the horizontal camera is used to determine the number and type of baggage items, and whether they contain trays. The data sets of the horizontal camera and the vertical camera are combined to determine whether the objects are stacked. Step 2: The dataset collected by the horizontal camera is passed into the target detection training network, and the dataset collected by the vertical camera is passed into the semantic segmentation training network, to obtain the target detection network of target category and number, as well as the target detection model and semantic segmentation model results respectively; Step 3: The production environment dataset is passed into the image preprocessing model, and then the processed images are passed into the target detection model and the semantic segmentation model to finally obtain the category, quantity, stacking, and whether the baggage check-in object contains a pallet.

2. The method for intelligent airport visual image recognition according to claim 1, characterized in that: According to the type of checked baggage and whether it contains a tray, the target detection model uses the YOLOv5 network. The YOLOv5 network architecture includes the backbone network Backbone, the neck network Neck, and the head network Head; The backbone network Backbone uses CSPDarknet53 as the basis for extracting image features. CSPDarknet53 gradually extracts features through a series of convolutional layers and residual blocks, in which the CSP structure is used to improve the representation ability of features. The backbone network gradually reduces the size of the feature map through multiple stages to detect objects at different scales. The neck network Neck is located after the backbone network and is used to further process feature maps and enhance representation capabilities. The neck network uses a series of convolution and upsampling layers to map the feature maps extracted by the backbone network to feature maps of different scales. These feature maps of different scales correspond to objects of different sizes. By gradually reducing the resolution of the feature map, the neck network can capture the contextual information of objects of different scales. The head network Head is responsible for generating the output of target detection; The head network operates on the feature maps of each scale and generates bounding boxes and category predictions; the bounding box prediction uses the idea of ​​Anchor Boxes to represent the location and size of the target by predicting the center coordinates, width, height and confidence of the bounding box; the category prediction uses the softmax function to output the probabilities of different categories; The head network of YOLOv5 uses the SPP module to increase the receptive field and the PANet module to fuse features of different scales.

3. The method for intelligent airport visual image recognition according to claim 2, characterized in that: YOLOv5 introduces adaptive training data augmentation, training resolutions of different sizes, multi-scale reasoning, and enhanced data augmentation strategies to further improve detection performance and robustness.

4. The method for intelligent airport visual image recognition according to claim 3 is characterized by: Embed BIFPN into the Yolov5 model to improve Yolov5's performance in dealing with the complex baggage stacking situation in airport security inspection scenarios; In the bottom-to-top stage of BIFPN, an upsampling mechanism is introduced to transfer the bottom-level features to the top-level step by step. Specifically, for adjacent feature maps, BIFPN combines low-level features with high-level features through upsampling operations to achieve the uplink transmission of information; in the top-to-bottom stage, BIFPN introduces a downsampling mechanism to transfer high-level features to low-level features; FPN realizes bidirectional information flow, and the bottom-top and top-bottom information transfer complement each other, establishing a close connection between multi-scale features.

5. The method for intelligent airport visual image recognition according to claim 4 is characterized in that: The ghost-conv module is used to reduce YOLOv5 parameters. Ghost-conv is a convolution operation in convolutional neural networks, which aims to significantly reduce the number of parameters and computational complexity while maintaining model performance.

6. The method for intelligent airport visual image recognition according to claim 5, characterized in that: The matching strategy of YOLOv5 is responsible for effectively matching the predicted anchor points with the actual target bounding boxes.

7. The method for intelligent airport visual image recognition according to claim 6, characterized in that: To solve the problem of baggage stacking, Soft NMS was introduced; Soft NMS introduces a decay function to gradually reduce the scores of adjacent bounding boxes instead of directly removing these boxes; the formula for the Soft-NMS score reset function is as follows: ； in represents the adjusted score of the i-th detection box, represents the initial score of the i-th detection box, IOU represents the intersection-over-union ratio, M represents the detection box with the highest score, represents the i-th detection box, is the standard deviation; Soft NMS solves the shortcomings in handling baggage stacking and small target detection through a flexible suppression mechanism.

8. The method for intelligent airport visual image recognition according to claim 7, characterized in that: In YOLOv5, the loss functions include: category loss, object confidence loss, and bounding box regression loss; use and structure , the formula is as follows: ； is the loss value, r is the adjustment factor used for dynamic adjustment , It can reflect the ratio of the current IOU loss of the model to the average IOU loss. and is a hyperparameter; These three loss parts are the loss functions of YOLOv5, which enable the model to focus on classification accuracy, object existence judgment, and precise positioning of bounding boxes during training.

Images