Luggage falling identification system and method for independent carrying unit of luggage handling system

Abstract

The invention discloses a luggage falling identification system and method for an independent carrying unit of a luggage processing system, and the system is characterized in that the system comprises an image collection unit, an edge calculation unit, an information processing unit, and an alarm prompt unit; the image acquisition unit is used for acquiring image data of a luggage loading node and a plurality of nodes in a luggage conveying process; the edge calculation unit comprises a preprocessing module, a luggage feature recognition module, a character recognition module and a network module; the luggage feature recognition module comprises an input layer, a luggage feature enhancement layer, a backbone network, a feature fusion and attention mechanism layer and a luggage feature prediction head which are connected in sequence; the information processing unit is used for receiving the feature data sent by the edge device, performing deep analysis and comparison, and detecting the luggage falling condition; and the alarm prompt unit is used for detecting the falling of the luggage and immediately notifying related personnel when the luggage falls. According to the invention, through cooperative work of the edge calculation and the background server, a luggage falling event is accurately identified, and an alarm is triggered in time, so that the security and efficiency of the airport luggage processing system are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of airport baggage handling systems, and in particular to a baggage drop recognition system for an independent carrier unit in an airport baggage handling system, and a recognition method involved in realizing the system. Background Art

[0002] In the busy airport baggage handling system, independent carriers are the central link of baggage flow. The stable operation of independent carriers after loading baggage is the key to ensuring efficient baggage sorting and timely loading. However, faced with the rapid switching of merging and diversion areas, the centrifugal force challenges of curves, and the instability caused by slope changes, coupled with the inherent characteristics of baggage such as different shapes and different center of gravity positions, baggage is very likely to slip or fall during transportation. If these slipped or fallen baggage are not discovered in time, it will cause baggage to be stranded, which will have a serious impact on the airport's operating efficiency and will also greatly reduce passenger satisfaction.

[0003] The existing solution for handling baggage drop identification and alarm is to set a photoelectric detection switch on the chute downstream of the tipping sorting. If no baggage signal is detected after the sorting mechanism is tipped, it is judged that the baggage has fallen during the transportation process. Since the drop position is uncertain, the staff needs to search along the conveying equipment, which is not only time-consuming and laborious, but also often unable to handle the dropped baggage in time. Summary of the invention

[0004] The purpose of the present invention is to provide a baggage drop identification system and method for an independent carrying unit of a baggage handling system in response to the above-mentioned problems. The system realizes real-time monitoring and data analysis of the baggage handling process through the collaborative work of edge computing and a background server, and can accurately identify baggage drop events and trigger an alarm in time, thereby improving the safety and efficiency of the airport baggage handling system.

[0005] The technical solution of the present invention is as follows:

[0006] The invention discloses a baggage drop recognition system for an independent carrying unit of a baggage handling system, comprising an image acquisition unit, an edge computing unit, an information processing unit and an alarm prompting unit.

[0007] The image acquisition unit is used to collect image data of a baggage loading node and several nodes in the baggage conveying process.

[0008] The edge computing unit includes a preprocessing module, a baggage feature recognition module, a character recognition module and a network module, which are used to preprocess, extract features, and perform baggage feature recognition and number recognition on the image data collected by the image acquisition unit, and transmit the baggage feature information and independent transport unit number information after recognition to the information processing unit, so as to provide basic data for in-depth analysis, tracking management and drop detection.

[0009] Among them, the baggage feature recognition module includes an input layer, a baggage feature enhancement layer, a backbone network, a feature fusion and attention mechanism layer, and a baggage feature prediction head which are connected in sequence.

[0010] The input layer is used to receive the preprocessed baggage image data.

[0011] The baggage feature enhancement layer is used to splice shallow and deep feature maps to enhance feature representation capabilities.

[0012] The backbone network uses the TPH YOLOv8 network to extract the underlying features of the luggage. TPH YOLOv8 introduces the Transformer mechanism based on YOLOv8, which enhances the model's ability to detect small objects and objects in complex backgrounds.

[0013] The feature fusion and attention mechanism layer integrates the CBAM attention mechanism, dynamically adjusts the weight of the feature map, and combines the Bi-FPN feature fusion structure to achieve accurate fusion of multi-scale features;

[0014] The baggage feature prediction head can accurately identify and locate baggage features in complex backgrounds;

[0015] The information processing unit includes a target tracking module, a feature information comparison module and a data storage module.

[0016] The target tracking module is used to track and mark the identified numbers, obtain the number information of consecutive frames, and support baggage tracking and management.

[0017] The feature information comparison module is used to receive the feature data sent by the edge device, perform in-depth analysis and comparison, and determine the luggage drop situation.

[0018] The data storage module is used to store structured data and unstructured data, where structured data includes baggage feature data, independent transport unit number, and detection time; unstructured data includes video clips and baggage images.

[0019] The alarm prompt unit is used to immediately notify relevant personnel when luggage is detected to have fallen.

[0020] Furthermore, the image acquisition unit includes a high-definition camera, which is communicatively connected to the edge computing unit, and the high-definition camera is arranged at a baggage loading node and several baggage conveying nodes.

[0021] Furthermore, the system also includes a target tracking module for tracking and marking the identified numbers, obtaining the number information of consecutive frames, and supporting baggage tracking and management.

[0022] The present invention also discloses a baggage drop recognition method for an independent carrying unit of a baggage handling system, comprising the following steps:

[0023] Step 1: Collect image data of baggage loading nodes and several nodes during the transportation process;

[0024] Step 2: Preprocess the image data, extract features, identify numbers and baggage features, and send the results to the information processing unit;

[0025] Step 3: Target tracking and in-depth analysis of the received feature data are performed to determine the luggage drop situation. Step 4: When luggage drop is detected, the alarm prompt unit notifies relevant personnel.

[0026] Furthermore, the pre-processing in step 2 specifically includes the following steps:

[0027] S1, image data preprocessing;

[0028] As the starting point of the recognition process, high-definition baggage image data is first received and a series of preprocessing operations are performed to optimize the image quality. These operations include: normalization processing: unifying the data format and range to ensure the stability of the model input; grayscale processing: reducing the computational complexity and improving the processing speed; filtering processing: removing noise and improving image quality; binarization processing: enhancing image contrast and highlighting key features; top-hat transformation processing: further highlighting the detailed features in the image, laying the foundation for subsequent processing.

[0029] S2. Establishing a baggage feature recognition model;

[0030] S3, establishing a number recognition model;

[0031] S4, character recognition and information fusion;

[0032] S5. Output and storage.

[0033] Furthermore, in the baggage feature recognition of step 2, the Transformer mechanism is introduced on the basis of YOLOv8, and the position attention mechanism is integrated into the feature extraction architecture of YOLOv8. The weight distribution of the position attention is calculated according to the following formula:

[0034]

[0035] Among them, f i is the feature vector of the th position of the feature map; W b , b p is the weight matrix and bias term in the attention mechanism; α i is the attention weight of the i-th position in the feature map.

[0036] Furthermore, when identifying baggage features, the detection result is screened and confirmed based on the effective position area that the baggage may occupy on the independent carrier unit; the mathematical expression of the effective detection area ROI on the independent carrier unit ULD is as follows:

[0037] ROI={(x,y)|x min ≤x≤x max ,y min ≤y≤y max}

[0038] Among them, x min , x max is the horizontal boundary of the effective area of ​​the loading unit; min ,y max is the vertical boundary of the effective area of ​​the loading unit.

[0039] Furthermore, when the baggage is successfully detected, the center position of the baggage is accurately corrected with the effective area of ​​the independent carrier unit ULD. The mathematical expression of the deviation degree of the center position of the baggage is as follows:

[0040]

[0041] Among them, Δ d is the Euclidean distance between the center of the baggage detection frame and the center of ROI; (x center ,y center ) is the coordinate of the center point of the detection frame; (x roi_center ,y roi_center ) are the coordinates of the ROI center point.

[0042] Furthermore, a position-constrained regularization component is incorporated into the loss function of YOLOv8. The mathematical expression of the position-constrained regularization term is as follows:

[0043]

[0044] Among them, L position is the position constraint regularization term; N is the total number of detection boxes; D threshold is the distance threshold, which limits the position of the effective detection box; λ is the weight of the regularization term, which controls its impact on the total loss.

[0045] Furthermore, in step 2, the luggage loading image data of the continuous frames are subjected to number detection and target tracking, including: S10: the luggage loading image data of the current frame is evenly grid-divided, and a prediction frame of each grid is obtained; the prediction frame contains the number information of the ULD;

[0046] S11: Obtain the offset between the prediction box and the prior box of the current grid, and adjust the position and size of the prediction box;

[0047] S12: obtaining an initial prediction frame of the current grid according to the offset, repeating the above steps, traversing all grids, and obtaining initial prediction frames of all grids of the luggage loading image data of the current frame;

[0048] S13: Perform non-maximum suppression (NMS) screening on all initial prediction boxes according to the preset intersection-over-union (IoU) threshold and confidence threshold, remove redundant and overlapping prediction boxes, and obtain the final prediction box of the current frame;

[0049] S14: intercepting the baggage loading image area in each final prediction frame to obtain the numbered image data of each ULD in the current frame; S15: replacing the baggage loading image data of the next frame for number detection, and using the target tracking algorithm to track and mark the same ULD in the final prediction frame;

[0050] S16: Repeat the number detection and target tracking steps to obtain character image data of each ULD continuous frame.

[0051] Furthermore, in step 3, by constructing a deep relationship model between image features and luggage drop status, a quick and accurate judgment of whether the luggage has dropped is achieved. The hidden layer of the deep relationship model uses Gaussian function for feature mapping, and the output layer uses sparse cross entropy mapping. The luggage drop judgment not only combines trajectory prediction, but also considers its position on the ULD. Once the predicted position exceeds the effective area of ​​the ULD, the drop alarm is directly triggered. The mathematical expression of the drop risk score is as follows:

[0052] S = fuzzy(v tray ,I,F luggage ,B score )

[0053] Where S is the drop risk score; v tray is the ULD speed; I is the light intensity; F luggage B is the baggage characteristics, including area and shape; score is the saliency score of the baggage region after background suppression.

[0054] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0055] 1. The present invention realizes real-time monitoring and alarm of luggage dropping through the collaborative work of edge computing and background servers, thereby improving the efficiency and safety of luggage handling; and reduces the amount of data transmission and processing delay through optimization measures such as asynchronous transmission and feature extraction.

[0056] 2. The present invention improves the accuracy of baggage feature recognition and independent transport unit number recognition through incremental learning and self-optimization of the deep learning model.

[0057] 3. The present invention adopts TPH YOLOv8 (Transformer Prediction Head YOLOv8). Based on YOLOv8, the Transformer mechanism is introduced to enhance the model's detection ability for small targets and targets in complex backgrounds.

[0058] 4. When identifying baggage features, the present invention screens and confirms the detection results based on the valid position area that the baggage may occupy on the ULD, which can reduce the error detection rate of the ULD edge or non-baggage area.

[0059] 5. After the luggage is successfully detected, the present invention performs a key step, that is, accurately calibrating the center position of the luggage with the effective area of ​​the ULD, which can improve the accuracy of luggage detection and provide more reliable location information for the subsequent luggage handling process.

[0060] 6. The present invention integrates the position attention mechanism into the feature extraction architecture of YOLOv8, and significantly enhances the model's recognition sensitivity to luggage on ULD by implementing a dynamic weighting strategy for the region of interest (ROI) where the luggage is located. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The present invention will now be described by way of example with reference to the accompanying drawings, in which:

[0062] Figure 1 The present invention is a schematic structural diagram of a baggage drop identification system for an independent carrying unit of a baggage handling system.

[0063] Figure 2 It is a schematic diagram of the luggage loading position layout in the present invention.

[0064] Figure 3 The present invention is a flow chart of a method for identifying baggage drop of an independent carrying unit of a baggage handling system.

[0065] Reference numerals: 100 - belt conveyor equipment, 200 - independent transport unit, 201 - luggage, 202 - number of independent luggage transport unit, 300 - image acquisition unit. DETAILED DESCRIPTION

[0066] All features disclosed in this specification, or steps in all methods or processes disclosed, except mutually exclusive features and / or steps, can be combined in any manner.

[0067] Any feature disclosed in this specification (including any additional claims and abstract), unless otherwise stated, may be replaced by other equivalent or alternative features having similar purposes. That is, unless otherwise stated, each feature is only an example of a series of equivalent or similar features.

[0068] The features and performance of the present invention are further described in detail below in conjunction with the embodiments.

[0069] like Figure 1 As shown, the present invention discloses a baggage drop recognition system for an independent carrying unit of a baggage handling system, comprising an image acquisition unit, an edge computing unit, an information processing unit and an alarm prompt unit.

[0070] The image acquisition unit is used to collect image data of a baggage loading node and several nodes in the baggage conveying process.

[0071] The edge computing unit includes a preprocessing module, a baggage feature recognition module, a character recognition module and a network module, which are used to preprocess, extract features, and perform baggage feature and number recognition on the image data collected by the image acquisition unit, and transmit the baggage feature information and independent transport unit number information after recognition to the information processing unit, providing basic data support for in-depth analysis, tracking management and drop detection.

[0072] Among them, the baggage feature recognition module includes an input layer, a baggage feature enhancement layer, a backbone network, a feature fusion and attention mechanism layer, and a baggage feature prediction head which are connected in sequence.

[0073] The input layer is used to receive the preprocessed baggage image data.

[0074] The baggage feature enhancement layer is used to splice shallow and deep feature maps to enhance feature representation capabilities.

[0075] The backbone network uses the TPH YOLOv8 network to extract the underlying features of the luggage. TPH YOLOv8 introduces the Transformer mechanism based on YOLOv8, which enhances the model's ability to detect small objects and objects in complex backgrounds.

[0076] The feature fusion and attention mechanism layer integrates the CBAM attention mechanism, dynamically adjusts the weight of the feature map, and combines the Bi-FPN feature fusion structure to achieve accurate fusion of multi-scale features;

[0077] The baggage feature prediction head can accurately identify and locate baggage features in complex backgrounds;

[0078] The information processing unit includes a target tracking module, a feature information comparison module and a data storage module.

[0079] The target tracking module is used to track and mark the identified numbers, obtain the number information of consecutive frames, and support baggage tracking and management.

[0080] The feature information comparison module is used to receive the feature data sent by the edge device, conduct in-depth analysis and comparison, and determine the baggage drop situation;

[0081] The data storage module is used to store structured data and unstructured data, wherein: structured data includes baggage feature data, independent carrier unit number, and detection time; unstructured data includes video clips and baggage images. The alarm prompt unit immediately notifies relevant personnel when detecting baggage falling.

[0082] The image acquisition unit includes a high-definition camera, which is connected to the edge computing unit and is set at the baggage loading node and several nodes of baggage transportation. The core component of the image acquisition unit is a high-definition camera, which is installed in the monitoring area that the baggage system ULD must pass through to ensure all-round and high-quality monitoring of the ULD and its loaded baggage. The camera uses a high-resolution color CCD camera.

[0083] The image acquisition unit includes a plurality of image acquisition terminals, each of which corresponds to at least one camera, and the image acquisition terminal is used to acquire a corresponding image of at least one of the luggage loads.

[0084] In this embodiment, the preset format is the H.265 format, but the decoding tool is updated to use the more efficient FFmpeg 4.x version that supports more features. In order to reduce the size of the video and the bandwidth occupied during transmission, the video will be processed by advanced encoding compression technology, such as HEVC, to further improve the compression efficiency. In the H.265 format, encoding is achieved through intra-frame compression, inter-frame compression, and advanced motion estimation technology, where the I frame is a key frame, the P frame records the difference from the previous frame, and the B frame records the difference from the previous and next frames, and uses the efficient encoding structure of HEVC to reduce the amount of data. Only after decoding can each frame in the video data be restored to ensure data quality.

[0085] The edge computing unit includes a main control module, a preprocessing module, a network module, a baggage feature recognition module and a character recognition module. The main control module is electrically connected to the preprocessing module, the network module, the baggage feature recognition module and the character recognition module respectively, and the main control module is communicatively connected to the baggage camera unit, the network module is communicatively connected to the backend data center, the character recognition module is provided with a character recognition model, and the baggage feature recognition module is provided with a baggage feature recognition model.

[0086] The edge computing unit is deployed at key locations of baggage loading and transmission, and obtains the ULD number image and baggage image in real time through the video acquisition device. A lightweight target detection model (TPH YOLOv8) is used for real-time processing on the edge device to directly identify the ULD number at the edge and extract the baggage features (such as size, shape, color, stripes, coordinates, etc.). The results of edge computing include the ULD number and its corresponding baggage feature information, which are uploaded to the background server as an index for subsequent global monitoring and drop detection. Among them: The edge computing unit mainly implements the following functions:

[0087] The luggage loading image data of each ULD continuous frame is preprocessed, including normalization processing, grayscale processing, binarization processing and top-hat transformation processing, to obtain the preprocessed luggage loading image data of each ULD continuous frame.

[0088] The pre-processed baggage loading image data of each ULD continuous frame is input into the character recognition model based on deep learning for number recognition, and the number recognition result of each ULD is obtained.

[0089] The preprocessing steps in the preprocessing module include: frame interception of the original video data to obtain continuous frames of luggage loading image data; normalization of each frame of the image to ensure the consistency of the image data; grayscale processing to reduce the amount of calculation; binarization processing to enhance the image contrast; and finally top-hat transformation processing to further highlight the feature information in the image.

[0090] The information processing unit is set up in the background server to receive the feature data sent by the edge device, perform in-depth analysis and comparison, and detect luggage falling.

[0091] The information processing unit includes a target tracking module, a feature information comparison module and a data storage module.

[0092] The target tracking module is used to track and mark the identified numbers, obtain the number information of consecutive frames, and support baggage tracking and management.

[0093] The feature information comparison module is used to receive the feature data sent by the edge device, perform in-depth analysis and comparison, and determine the luggage drop situation.

[0094] The data storage module is used to store structured data and unstructured data, wherein: structured data includes baggage feature data, independent carrier unit number, and detection time; unstructured data includes video clips and baggage images. The alarm prompt unit immediately notifies relevant personnel when detecting baggage falling.

[0095] The alarm prompt unit includes an alarm communication module. Once luggage falls, it will immediately notify relevant personnel, including baggage handlers, security managers, etc., through information prompts, sound and light alarms, etc., so that quick measures can be taken to deal with it.

[0096] In another embodiment, a target tracking module is further included, which is used to track and mark the identified number and obtain the number information of consecutive frames to support baggage tracking and management. It also includes fusing the identified number information with the baggage features and storing them in a data storage module for subsequent query, tracking and management.

[0097] like Figure 2 As shown, first, after the luggage 201 is loaded into the independent carrier unit 200 through the belt conveyor 100, it is sent to the position of the tipping sorting mechanism through the conveying line. The image acquisition unit 300 deployed on the luggage loading and each conveying route transmits the video data to the edge computing unit, and the edge computing unit performs pre-processing operations such as frame extraction, graying, and denoising on the collected video data. Secondly, using deep learning and image processing technology, the size, color, shape, position and other features of the luggage, as well as the OCR recognition result of the luggage ULD number 202 are extracted. The extracted luggage features are preliminarily analyzed to determine whether there are abnormal situations such as luggage falling. The data after preliminary analysis is transmitted to the information processing unit for further processing through a low-latency network protocol (such as MQTT or WebSocket). Thirdly, the information processing unit uses a deep learning model to perform a deep analysis on the received data, and stores the analyzed data in a relational database (such as MySQL) and an unstructured database (such as MongoDB) for subsequent query and tracking. Finally, according to the results of the deep analysis, it is determined whether the alarm mechanism is triggered, and the alarm information is sent to the operation and maintenance personnel by email, SMS or push notification. In order to efficiently store and manage ULD numbers and their baggage feature data, the system adopts a combination of relational database and unstructured database. The identified ULD numbers, baggage features and other structured data are stored in the relational database, and the baggage images and other data are stored in the unstructured data database. The system also provides fast data query and retrieval functions so that operation and maintenance personnel can easily track and query baggage information.

[0098] like Figure 3As shown, the present invention also discloses a baggage drop identification method for an independent carrying unit of a baggage handling system, comprising the following steps:

[0099] Step 1: Collect image data of baggage loading nodes and several nodes during transportation;

[0100] Step 2: Preprocess the image data, extract features, identify numbers and baggage features, and send the results to the information processing unit.

[0101] Step 3: Perform target tracking and in-depth analysis on the received feature data to determine the luggage drop situation.

[0102] Step 4: When the luggage is detected to have fallen, relevant personnel are notified through the alarm prompt unit.

[0103] In the embodiment, step 2 specifically includes the following steps: S1, image data preprocessing

[0104] As the starting point of the recognition process, high-definition baggage image data is first received and a series of preprocessing operations are performed to optimize the image quality. These operations include: normalization processing: unifying the data format and range to ensure the stability of the model input; grayscale processing: reducing the computational complexity and improving the processing speed; filtering processing: removing noise and improving image quality; binarization processing: enhancing image contrast and highlighting key features; top-hat transformation processing: further highlighting the detailed features in the image, laying the foundation for subsequent processing.

[0105] S2. Establishing baggage feature recognition model

[0106] The baggage feature recognition model is built based on the TPH-YOLOv8 algorithm, including the sequentially connected input layer, baggage feature enhancement layer, backbone network (Backbone), feature fusion and attention mechanism layer. Each layer structure works together to achieve efficient and accurate recognition. First, the input layer receives the pre-processed high-definition baggage image data, providing a high-quality input basis for the entire model. The baggage feature enhancement layer is specially designed for the diversity and complexity of baggage. By intelligently splicing shallow and deep feature maps and combining refined detection strategies, the model's recognition ability for small-sized baggage and subtle features is significantly improved. The backbone network (Backbone) uses an optimized YOLOv8 network. This network not only maintains high efficiency, but also makes customized adjustments to baggage features, so that the underlying features of baggage can be extracted more accurately. The feature fusion and attention mechanism layer (Neck) further enhances the recognition ability of the model. This layer integrates the CBAM attention mechanism, which can dynamically adjust the weight of the feature map, so that the model pays more attention to baggage features. At the same time, combined with the Bi-FPN feature fusion structure, accurate fusion of multi-scale features is achieved, which not only improves the recognition accuracy of the model, but also enhances its robustness. In the output part of the model, the baggage feature prediction head (BFPH) can accurately identify and locate baggage features in complex backgrounds. In addition, as a specially designed functional module, the number recognition head can locate and recognize numbers based on the YOLO detection frame, providing key support information for number tracking and baggage management. In order to ensure that the model can accurately capture baggage features of various scales, a multi-scale prediction head structure is also specially designed, including a prediction head specifically for identifying tiny features of baggage.

[0107] S3. Establish ULD number recognition model

[0108] Apply OCR technology (such as Tesseract OCR library or custom deep learning model) to perform character recognition on the ULD number.

[0109] The acquired high-definition images are preprocessed, such as grayscale, denoising, binarization, tilt correction, and image enhancement, to improve the clarity and recognizability of characters; Tesseract OCR is configured, custom character sets and parameter settings are introduced, and the specific character set of ULD numbers is optimized; OCR recognition is performed to extract characters from the preprocessed images, and the accuracy of recognition is ensured through subsequent result processing and format verification.

[0110] S4. Information Fusion

[0111] The identified ULD number information is fused with the baggage characteristics (such as size, color, shape, location, etc.), and the ULD number is accurately associated with the baggage characteristics using spatial association and temporal association methods. After successful association, the data is fused into a complete record.

[0112] The association method includes but is not limited to nearest neighbor matching, Hungarian algorithm or multi-factor comprehensive matching algorithm

[0113] S5. Output and storage

[0114] Output the identified baggage feature information and number information, and store the integrated information in the data storage module for subsequent query, tracking and management.

[0115] During the data transmission process, asynchronous programming models such as Asyncio are used to realize asynchronous data transmission. The use of asynchronous transmission can improve the responsiveness and throughput of the system.

[0116] In step three, the information processing unit will compare the baggage feature information and number information provided by each edge computing unit. If it is found that the feature of a piece of baggage does not match its corresponding ULD number, or the baggage feature corresponding to a certain ULD number disappears (i.e., the baggage has fallen), the abnormal detection mechanism will be triggered. By further confirming the detected abnormal situation, the possibility of false alarm will be eliminated. When the baggage has fallen, the alarm mechanism will be triggered. According to the number, location, duration and other factors of the baggage falling, different alarm levels (such as low-level alarm, medium-level alarm, and high-level alarm) are defined, and the appropriate push method (such as SMS, email, push notification, etc.) is selected to send the alarm information to the operation and maintenance personnel in real time. At the same time, the alarm information is recorded in the log system for subsequent query.

[0117] In the embodiment, in order to avoid misjudging the background as luggage, the present invention adopts an adaptive background modeling algorithm, such as a mixture of Gaussian models (MOG), to accurately divide the foreground (luggage) and the background (ULD). The specific algorithm expression of the background modeling is as follows:

[0118]

[0119] Where M(x,y) represents the background probability distribution of the pixel (x,y); K represents the number of Gaussian components; ω i represents the weight of the i-th Gaussian component, satisfying represents the probability density function of the i-th Gaussian component; μ i represents the mean vector, which represents the expectation of the pixel value; Σ i is the covariance matrix, which represents the distribution range of pixel values. By continuously updating the background model, the changes in background illumination in the airport environment can be effectively dealt with.

[0120] When identifying baggage features, the detection results are screened and confirmed based on the valid position area that the baggage may occupy on the ULD. This method can reduce the false detection rate of the ULD edge or non-baggage area. Specifically, the mathematical expression of the valid detection area (ROI) on the ULD is as follows:

[0121] ROI={(x,y)|x min ≤x≤x max ,y min ≤y≤y max}

[0122] Among them, x min , x max is the horizontal boundary of the effective area of ​​the loading unit; min ,y max is the vertical boundary of the effective area of ​​the loading unit.

[0123] After the baggage is successfully detected, the present invention performs a key step, that is, accurately calibrating the center position of the baggage with the effective area of ​​the ULD. This method can improve the accuracy of baggage detection and provide more reliable position information for the subsequent baggage handling process. Specifically, the mathematical expression of the deviation degree of the center position of the baggage is as follows:

[0124]

[0125] Among them, Δ d : The Euclidean distance between the center of the baggage detection frame and the center of ROI; (x center ,y center ) is the coordinate of the center point of the detection frame; (x roi_center ,y roi_center ) are the coordinates of the center point of ROI;

[0126] In order to enhance the ability to accurately limit baggage detection to the ULD loading area, the present invention innovatively incorporates a position-constrained regularization component into the loss function of YOLOv8. This component effectively guides the model to generate more realistic prediction results by constraining the prediction boxes that exceed the effective boundary of the ULD. Specifically, the mathematical expression of the position-constrained regularization term is as follows:

[0127]

[0128] Among them, L position is the position constraint regularization term; N is the total number of detection boxes; D threshold is the distance threshold, which limits the position of the effective detection box; λ is the weight of the regularization term, which controls its impact on the total loss.

[0129] In order to optimize the model's ability to detect luggage within the effective area, the present invention integrates the position attention mechanism into the feature extraction architecture of YOLOv8. This mechanism significantly enhances the model's recognition sensitivity to luggage on ULD by implementing a dynamic weighting strategy for the region of interest (ROI) where the luggage is located. The weight distribution of the position attention is calculated according to the following formula:

[0130]

[0131] Where: f i is the feature vector of the feature map; W p , b p is the weight matrix and bias term in the attention mechanism;

[0132] α i is the attention weight of the i-th position in the feature map.

[0133] In an embodiment, inputting the luggage loading image data of consecutive frames into a character recognition model for number detection and target tracking includes the following steps:

[0134] Based on the number recognition model, the baggage loading image data of the current frame is evenly divided into grids, and the prediction box of each grid is obtained; the prediction box may contain the number information of the ULD.

[0135] Get the offset between the predicted box and the prior box of the current grid, which is used to adjust the position and size of the predicted box to more accurately locate the number.

[0136] The initial prediction frame of the current grid is obtained according to the offset, and the above steps are repeated to traverse all grids to obtain the initial prediction frames of all grids of the luggage loading image data of the current frame.

[0137] All initial prediction boxes are screened by non-maximum suppression (NMS) according to the preset intersection over union (IoU) threshold and confidence threshold to remove redundant and overlapping prediction boxes and obtain the final prediction box of the current frame.

[0138] The baggage loading image area in each final prediction frame is intercepted to obtain the numbered image data of the current frame of each ULD.

[0139] Replace the next frame of baggage loading image data for number detection, and use an appropriate target tracking algorithm (such as the SORT tracking algorithm) to track and mark the same ULD in the final prediction frame.

[0140] Repeat the number detection and target tracking steps to obtain the character image data of each ULD continuous frame.

[0141] On the edge computing device, optical character recognition (OCR) technology is used to perform character recognition on the character image data of each ULD continuous frame to extract accurate numbering information. OCR technology is based on the Tesseract OCR library or a custom deep learning model, which can accurately identify the character sequence in the numbered image and convert it into a readable digital number. The numbering information of each identified ULD is fused with the corresponding luggage features (such as size, color, shape, position, etc.) and stored in the database for subsequent query and tracking. At this point, the entire process of video data acquisition, frame capture, preprocessing, number recognition and information fusion of the ULD at the luggage loading position has been completed.

[0142] In some embodiments, the target edge computing terminal extracts a target area from at least one baggage loading area based on image features of each of the baggage loading areas, including: for each baggage loading area, binarizing the baggage loading area, generating a binarized image corresponding to the baggage loading area, determining a pixel point matrix corresponding to the binarized image, and determining a matrix similarity between the pixel point matrix corresponding to the binarized image and a preset pixel point matrix; and extracting the target area from at least one region of interest based on the matrix similarity.

[0143] Object detection incorporates deep learning technology, especially an object detection method that is optimized and trained for the specific features of luggage. This method can accurately identify whether the moving object in the image is luggage, and whether the luggage has fallen from the luggage ULD, thereby improving the accuracy and reliability of detection. In terms of image feature extraction, the present invention comprehensively considers multiple dimensions such as brightness, chromaticity, texture, and optical flow field motion features to ensure a comprehensive and detailed description of the luggage image, providing strong support for subsequent luggage drop detection. Finally, the luggage drop judgment adopts an advanced algorithm based on a convolutional neural network, and by constructing a deep relationship model between image features and the luggage drop state, it realizes a fast and accurate judgment of whether the luggage has fallen. Among them, the hidden layer uses a Gaussian function for feature mapping, and the output layer uses a sparse cross entropy mapping, which further improves the accuracy and robustness of the judgment. When it is determined that the luggage has fallen, the system can immediately trigger an alarm mechanism to notify relevant personnel to handle it in time.

[0144] Baggage drop determination not only combines trajectory prediction, but also considers its position on the ULD. Once the predicted position exceeds the effective area of ​​the ULD, the drop alarm is directly triggered. Specifically, the mathematical expression of the drop risk score is as follows:

[0145] S = fuzzy(v tray ,I,F luggage ,B score )

[0146] Where: S is the fall risk score; v tray is the ULD speed; I is the light intensity; F luggage For luggage characteristics,

[0147] Including area, shape, etc.; B score is the saliency score of the baggage region after background suppression.

[0148] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the aforementioned method for identifying dropped packages is implemented, comprising the following steps:

[0149] The image acquisition unit collects image data of the baggage loading node and several nodes in the transportation process. The deployment locations include but are not limited to turns, merges and divergences, and slope changes to ensure comprehensive coverage of the baggage transportation path.

[0150] The edge computing unit is used to pre-process the image data, extract features, identify numbers and baggage features. The implementation methods include but are not limited to the use of FPGA, GPU, CPU, etc.

[0151] The recognition results are sent to the information processing unit for in-depth analysis and comparison to detect luggage drops. The information processing unit further analyzes and compares the luggage features, transport unit numbers, and the relationships between them through deep learning and machine learning algorithms.

[0152] The data storage module is used to store baggage feature data, ULD number, detection time and other data. The implementation methods include but are not limited to using HDD, SSD, distributed storage system and the like.

[0153] When luggage falling is detected, relevant personnel are notified through the alarm prompt unit, and the implementation method is not limited to using sound alarms, flashing lights, text messages or email notifications, system interface prompts, etc.

[0154] The above-mentioned embodiments only express the specific implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the protection scope of the present application. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the technical solution concept of the present application, and these all belong to the protection scope of the present application.

Claims

1. A baggage drop identification system for an independent carrying unit of a baggage handling system, characterized in that: It includes an image acquisition unit, an edge computing unit, an information processing unit and an alarm prompt unit; The image acquisition unit is used to collect image data of a baggage loading node and several nodes in the baggage conveying process; The edge computing unit includes a preprocessing module, a baggage feature recognition module, a character recognition module and a network module, which are used to preprocess, extract features, recognize baggage features and identify numbers on the image data collected by the image acquisition unit; wherein the baggage feature recognition module includes an input layer, a baggage feature enhancement layer, a backbone network, a feature fusion and attention mechanism layer and a baggage feature prediction head connected in sequence; the input layer is used to receive the preprocessed baggage image data, and the baggage feature enhancement layer is used to splice the shallow and deep feature maps to enhance the feature representation capability; the backbone network uses the TPH YOLOv8 network to extract the underlying features of the baggage, and TPH YOLOv8 introduces the Transformer mechanism on the basis of YOLOv8; the feature fusion and attention mechanism layer integrates the CBAM attention mechanism, dynamically adjusts the weight of the feature map, and combines the Bi-FPN feature fusion structure to achieve accurate fusion of multi-scale features; the baggage feature prediction head realizes accurate recognition and positioning of baggage features in complex backgrounds; The information processing unit includes a target tracking module, a feature information comparison module and a data storage module; the target tracking module is used to track and mark the identified number, obtain the number information of the continuous frame, and support baggage tracking and management; the feature information comparison module is used to receive the feature data sent by the edge device, perform in-depth analysis and comparison, and determine the baggage drop situation; the data storage module is used to store structured data and unstructured data; The alarm prompt unit is used to immediately notify relevant personnel when luggage is detected to have fallen.

2. The baggage drop identification system for independent carrying units of a baggage handling system according to claim 1, characterized in that: The image acquisition unit includes a high-definition camera, which is communicatively connected to the edge computing unit and is arranged at a baggage loading node and several baggage conveying nodes; the structured data includes baggage feature data, an independent transport unit number, and a detection time; and the unstructured data includes video clips and baggage images.

3. A baggage handling system independent transport unit baggage drop identification method, characterized in that: The following steps are involved: Step 1: Collect image data of baggage loading nodes and several nodes during the transportation process; Step 2: Preprocess the image data, extract features, identify numbers and baggage features; Step 3: Perform target tracking and in-depth analysis on the received feature data to determine the baggage drop situation; Step 4: When the luggage is detected to have fallen, notify the relevant personnel.

4. The method for identifying baggage drop of an independent carrying unit of a baggage handling system according to claim 3, characterized in that: The step 2 specifically includes the following steps: S1. Image data preprocessing; preprocessing includes: normalization processing: unify data format and range to ensure the stability of model input; grayscale processing: reduce computational complexity and improve processing speed; filtering processing: remove noise and improve image quality; binarization processing: enhance image contrast and highlight key features; top-hat transformation processing: further highlight the detailed features in the image and lay the foundation for subsequent processing; S2. Establishing a baggage feature recognition model; S3, establishing a number recognition model; S4, character recognition and information fusion: using spatial association and temporal association methods to accurately associate the ULD number with baggage features; S5, output and storage, output the identified baggage feature information and number information, and store the fused information in the data storage module.

5. The method for identifying baggage drop of an independent carrying unit of a baggage handling system according to claim 3 or 4, characterized in that: In the baggage feature recognition of step 2, the Transformer mechanism is introduced on the basis of YOLOv8, and the position attention mechanism is integrated into the feature extraction architecture of YOLOv8. The weight distribution of the position attention is calculated according to the following formula: Among them, f i is the feature vector of the th position of the feature map; W p , b p is the weight matrix and bias term in the attention mechanism; α i is the attention weight of the i-th position in the feature map.

6. The method for identifying baggage drop of an independent carrying unit of a baggage handling system according to claim 3 or 4, characterized in that: When identifying baggage features, the detection results are screened and confirmed based on the effective position area that the baggage may occupy on the independent carrier unit; the mathematical expression of the effective detection area ROI on the independent carrier unit ULD is as follows: ROI={(x,y)∣x min ≤x≤x max ,and min ≤y≤y max } Among them, x min , x max is the horizontal boundary of the effective area of ​​the loading unit; y min ,y max is the vertical boundary of the effective area of ​​the loading unit.

7. The method for identifying dropped baggage in an independent carrying unit of a baggage handling system according to claim 6, characterized in that: When the baggage is successfully detected, the center position of the baggage is accurately corrected with the effective area of ​​the independent load unit ULD. The mathematical expression of the deviation degree of the center position of the baggage is as follows: Among them, Δ d is the Euclidean distance between the center of the baggage detection frame and the center of ROI; (x center ,y center ) is the coordinate of the center point of the detection frame; (x roi_center ,y roi_center ) are the coordinates of the ROI center point.

8. The method for identifying dropped baggage in an independent carrying unit of a baggage handling system according to claim 5, characterized in that: A position-constrained regularization component is incorporated into the loss function of YOLOv8. The mathematical expression of the position-constrained regularization term is as follows: Among them, L position is the position constraint regularization term; N is the total number of detection boxes; D threshold is the distance threshold, which limits the position of the effective detection box; λ is the weight of the regularization term, which controls its impact on the total loss.

9. The method for identifying dropped baggage in an independent carrying unit of a baggage handling system according to claim 3, characterized in that: In step 2, the luggage loading image data of consecutive frames are subjected to number detection and target tracking, including: S10: divide the baggage loading image data of the current frame into uniform grids, and obtain a prediction frame of each grid; the prediction frame contains the number information of the ULD; S11: Obtain the offset between the prediction box and the prior box of the current grid, and adjust the position and size of the prediction box; S12: obtaining an initial prediction frame of the current grid according to the offset, repeating the above steps, traversing all grids, and obtaining initial prediction frames of all grids of the luggage loading image data of the current frame; S13: Perform non-maximum suppression (NMS) screening on all initial prediction boxes according to the preset intersection-over-union (IoU) threshold and confidence threshold, remove redundant and overlapping prediction boxes, and obtain the final prediction box of the current frame; S14: intercepting the baggage loading image area in each final prediction frame to obtain the numbered image data of the current frame of each ULD; S15: replacing the baggage loading image data of the next frame for number detection, and using the target tracking algorithm to track and mark the same ULD in the final prediction frame; S16: Repeat the number detection and target tracking steps to obtain character image data of each ULD continuous frame.

10. The method for identifying dropped baggage of an independent carrying unit of a baggage handling system according to claim 3, characterized in that: In step 3, by constructing a deep relationship model between image features and luggage drop status, a quick and accurate judgment of whether the luggage has dropped is achieved. The hidden layer of the deep relationship model uses Gaussian function for feature mapping, and the output layer uses sparse cross entropy mapping. The luggage drop judgment not only combines trajectory prediction, but also considers its position on the ULD. Once the predicted position exceeds the effective area of ​​the ULD, the drop alarm is directly triggered. The mathematical expression of the drop risk score is as follows: S=fuzzy(v tray ,I,F luggage ,B score ) Where S is the drop risk score; v tray is the ULD speed; I is the light intensity; F luggage B is the baggage characteristics, including area and shape; score is the saliency score of the baggage region after background suppression.

Images