Luggage handle and RFID tag recognition method based on deep network vision, terminal device and storage medium

Abstract

The application is suitable for the field of image recognition, and discloses a luggage handle and RFID tag identification method based on deep network vision, a terminal device and a storage medium. The luggage handle and RFID tag identification method based on deep network vision comprises the following steps: controlling an industrial camera to collect image data of a luggage in an identification area, and pre-processing the image data to obtain a target image; inputting the target image into a pre-trained luggage identification algorithm model to obtain a prediction result containing a luggage handle state, an RFID tag state and a relative position of the luggage and the handle, wherein the luggage identification algorithm model is a model for identifying the luggage handle and the RFID tag, which is obtained based on deep learning; and controlling a mechanical arm to perform a clamping operation or issuing a warning signal according to the prediction result. The application significantly improves the reliability of a luggage processing system.

Description

Technical Field

[0001] This invention belongs to the field of image recognition, and particularly relates to a method, terminal device and storage medium for identifying luggage handles and RFID tags based on deep network vision. Background Technology

[0002] The efficiency of baggage handling systems has become a focus of attention. Baggage needs to be handled reliably.

[0003] Because RFID tags and luggage handles share certain visual similarities, traditional methods often misidentify RFID tags as luggage handles, leading to the robotic arm missing items that should be grabbed. Furthermore, the point cloud formed by RFID tags can interfere with the accuracy of luggage pose calculations, causing the robotic arm to misjudge the grab position and severely impacting the reliability of luggage handling. A new technological approach is needed to address these problems. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a method, terminal device and storage medium for identifying luggage handles and RFID tags based on deep network vision, which can solve the problem of low reliability in luggage handling systems in related technologies.

[0005] The first aspect of this invention provides a method for identifying luggage handles and RFID tags based on deep network vision, comprising:

[0006] The industrial camera is controlled to acquire image data of suitcases within the recognition area, and the image data is preprocessed to obtain the target image;

[0007] The target image is input into a pre-trained suitcase recognition algorithm model to obtain prediction results including the state of the suitcase handle, the state of the RFID tag, and the relative position of the suitcase and the handle. The suitcase recognition algorithm model is a model trained based on deep learning for recognizing the suitcase handle and RFID tag.

[0008] Based on the prediction results, control the robotic arm to perform a gripping operation or issue a warning signal.

[0009] Optionally, in a first implementation of the first aspect of the present invention, the preprocessing of the image data to obtain the target image includes:

[0010] The image data is cropped, retaining the suitcase portion of the image and removing the surrounding background. The cropped image is then scaled and filled to a preset size to complete the preprocessing of the image data and obtain the target image.

[0011] Optionally, in a second implementation of the first aspect of the present invention, the overall structure of the suitcase recognition algorithm model is an Encoder-Decoder format, including an encoder layer and a decoder layer; the encoder layer is used to encode the target image and extract coarse and fine features of the target image; the decoder layer is used to fuse the coarse and fine features and predict the semantic segmentation mask to generate the prediction result.

[0012] Optionally, in a third implementation of the first aspect of the present invention, the encoder layer consists of three attention modules, which process the input target image in a hierarchical manner, so that the resolution of the output feature map of each layer is gradually reduced to achieve multi-scale feature extraction; each attention module includes a self-attention mechanism and a feedforward neural network, and the self-attention mechanism uses localized window attention.

[0013] Optionally, in a fourth implementation of the first aspect of the present invention, the step of controlling the robotic arm to perform a gripping operation or issue a warning signal based on the prediction result includes:

[0014] If, based on the predicted status of the suitcase handle, the RFID tag, and the relative position of the suitcase and the handle, it is determined that the suitcase handle and the RFID tag are present, then, according to the relative position of the suitcase and the handle, the robotic arm is controlled to avoid the handle and perform the clamping operation on the suitcase.

[0015] Optionally, in a fifth implementation of the first aspect of the present invention, the step of controlling the robotic arm to perform a gripping operation or issue a warning signal based on the prediction result includes:

[0016] Obtain the RFID tag status from the prediction results;

[0017] If the RFID tag is not found, a warning signal for missing RFID tag will be issued, but the robotic arm will not be triggered to perform a gripping operation.

[0018] Optionally, in a sixth implementation of the first aspect of the present invention, the step of controlling the robotic arm to perform a gripping operation or issue a warning signal based on the prediction result includes:

[0019] If, based on the predicted status of the suitcase handle, the RFID tag, and the relative position of the suitcase and the handle, it is determined that the suitcase handle is present but the RFID tag is absent, then an RFID tag omission warning signal will be issued, and the robotic arm's gripping operation will not be triggered.

[0020] Optionally, in a seventh implementation of the first aspect of the present invention, the step of controlling the robotic arm to perform a gripping operation or issue a warning signal based on the prediction result includes:

[0021] If, based on the predicted status of the suitcase handle, the RFID tag, and the relative position of the suitcase and the handle, it is determined that the suitcase handle is present but the RFID tag is absent, then an RFID tag omission warning signal will be issued, and the robotic arm's gripping operation will not be triggered.

[0022] Secondly, embodiments of the present invention provide a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the above-described method for identifying a suitcase handle and RFID tag based on deep network vision.

[0023] Thirdly, embodiments of the present invention provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method for identifying a suitcase handle and RFID tag based on deep network vision.

[0024] Fourthly, embodiments of the present invention provide a computer program product that, when run on a terminal device, causes the terminal device to execute the aforementioned method for identifying luggage handles and RFID tags based on deep network vision.

[0025] The beneficial effects of this invention compared to existing technologies are as follows: By accurately identifying the status of the suitcase handle and its relative position to the suitcase using a deep learning model, damage to the suitcase caused by misjudging the gripping position of the robotic arm in traditional methods is avoided. This reduces operational errors, pauses, and delays during the baggage handling process, thereby improving baggage handling efficiency. Furthermore, by identifying the status of RFID tags using the model, the shortcomings of traditional methods in lacking an effective detection mechanism for missing RFID tags are compensated for, preventing the problem of untraceable baggage due to missing RFID tags. From the aspects of reducing damage, avoiding errors, and detecting tags and ensuring tracking, the reliability of the baggage handling system is significantly improved. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a schematic diagram of an embodiment of the luggage handle and RFID tag identification method based on deep network vision in this invention.

[0028] Figure 2This is a schematic diagram of a specific embodiment of step S101 of the method for identifying a suitcase handle and an RFID tag based on deep network vision in this invention.

[0029] Figure 3 This is a schematic diagram of the first specific embodiment of step S103 of the luggage handle and RFID tag identification method based on deep network vision in this invention.

[0030] Figure 4 This is a schematic diagram of the second specific embodiment of step S103 of the luggage handle and RFID tag identification method based on deep network vision in this invention.

[0031] Figure 5 This is a schematic diagram of one embodiment of the terminal device in this invention. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are protected by this invention.

[0033] It should be noted that the terms "comprising," "including," and "having," and any variations thereof, in the specification, claims, and accompanying drawings of this invention, are intended to cover non-exclusive inclusion. For example, a process, method, terminal, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices. In the claims, specification, and accompanying drawings of this invention, relational terms such as "first" and "second" are used merely to distinguish one entity / operation / object from another entity / operation / object, and do not necessarily require or imply any such immediate relationship or order between these entities / operations / objects.

[0034] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0035] The efficiency of baggage handling systems has become a focus of attention. Baggage needs to be processed quickly and accurately.

[0036] Traditional baggage handling methods are inefficient and error-prone, easily leading to damage to suitcases due to misjudging the gripping position by robotic arms. Even with machine vision technology, inaccurate identification and operational errors still occur when dealing with complex shapes such as extended suitcase handles. Furthermore, the lack of an effective detection mechanism for missing RFID tags often results in stalls, delays, damaged luggage, or untraceable items due to missing RFID tags. Clearly, traditional baggage handling systems have low reliability. A new technological approach is needed to solve these problems.

[0037] In view of this, embodiments of the present invention provide a method, terminal device, and storage medium for identifying luggage handles and RFID tags based on deep network vision. By using a deep learning model to accurately identify the state of the luggage handle and its relative position to the luggage, the invention avoids damage to the luggage caused by errors in the gripping position judgment of the robotic arm in traditional methods. This reduces operational errors, pauses, and delays during luggage handling, thus improving luggage handling efficiency. Furthermore, by identifying the state of the RFID tags through the model, the invention compensates for the lack of an effective detection mechanism for missing RFID tags in traditional methods, preventing the problem of luggage being untraceable due to missing RFID tags. From the aspects of reducing damage, avoiding errors, and detecting tags to ensure tracking, the invention significantly improves the reliability of the luggage handling system.

[0038] To illustrate the technical solution of the present invention, specific embodiments are described below.

[0039] Figure 1 This illustration shows a flowchart of a method for identifying luggage handles and RFID tags based on deep network vision, according to an embodiment of the present invention. This method can be applied to terminal devices, such as mobile phones, tablets, laptops, ultra-mobile personal computers (UMPCs), and netbooks.

[0040] Specifically, the above-mentioned method for identifying luggage handles and RFID tags based on deep network vision may include the following steps S101 to S103.

[0041] Step S101: Control the industrial camera to collect image data of the suitcases in the recognition area, and preprocess the image data to obtain the target image.

[0042] In an embodiment of the present invention, the terminal device initiates an image acquisition control process, sends an acquisition command to the industrial camera, and controls the industrial camera to acquire image data of a preset recognition area (i.e., a designated area on the conveyor belt carrying a suitcase).

[0043] Optionally, to ensure that the acquired images clearly present the key features of the suitcase, handle, and RFID tag, a linkage is established with the light source in the identification area. By adjusting the light intensity of the light source or adjusting the exposure parameters and focal length of the industrial camera, image blurring and feature loss due to excessively strong or weak light or improper exposure can be avoided, thereby obtaining initial image data that reflects the details of the suitcase and related targets.

[0044] After acquiring the initial image data, preprocessing is performed to remove interference information, standardize image specifications, and eliminate irrelevant factors for subsequent model recognition.

[0045] Optionally, the initial image is first filtered for noise to reduce image noise caused by changes in ambient light and camera sensor errors. Then, the image is cropped according to the actual scene requirements, retaining only the core area containing the suitcase and removing irrelevant backgrounds such as conveyor belt edges and equipment frames. Finally, the cropped image is scaled or filled according to the input specifications of the pre-trained suitcase recognition algorithm model to ensure that parameters such as image size and resolution meet the input requirements of the model, thus obtaining a target image that can be directly input into the model.

[0046] Step S102: Input the target image into the pre-trained suitcase recognition algorithm model to obtain prediction results including the state of the suitcase handle, the state of the RFID tag, and the relative position of the suitcase and the handle. The suitcase recognition algorithm model is a model trained based on deep learning for recognizing the suitcase handle and the RFID tag.

[0047] In an embodiment of the present invention, the preprocessed target image is input into a pre-trained suitcase recognition algorithm model. This model is built on deep learning technology and has been trained using a large amount of suitcase image data containing different styles, handle states, and RFID tag positions. It possesses the ability to recognize the handle state, RFID tag state, and the relative position of the suitcase and handle. By calling the model's computational interface, the model is driven to extract, analyze, and judge features from the target image. Finally, the model outputs a prediction result containing the aforementioned key information, completing the transformation from image data to specific recognition information.

[0048] Optionally, the overall structure of the luggage recognition algorithm model is an Encoder-Decoder format, including an encoder layer and a decoder layer. The encoder layer is used to encode the target image and extract its coarse and fine features. The decoder layer is used to fuse the coarse and fine features and predict the semantic segmentation mask to generate the prediction result. Specifically, by extracting coarse and fine features of the target image through the encoder layer, and then fusing multi-level features and predicting the semantic segmentation mask through the decoder layer, the algorithm can fully capture multi-dimensional image information of luggage, handles, and RFID tags. This effectively solves the problem in existing airport baggage handling automated pickup systems where traditional machine vision technology or single feature extraction methods are insufficient to comprehensively and accurately identify the key features of complex-shaped luggage.

[0049] Optionally, the encoder layer consists of three attention modules. These three attention modules process the input target image in a hierarchical manner, gradually reducing the resolution of the output feature maps at each layer to achieve multi-scale feature extraction. Each attention module includes a self-attention mechanism and a feedforward neural network. The self-attention mechanism uses localized window attention. By processing the target image through three hierarchically configured attention modules to gradually reduce the feature map resolution and achieve multi-scale feature extraction, and by combining a self-attention mechanism with a feedforward neural network in each module, this approach solves the problems of high computational cost and difficulty in balancing global information and local details in high-resolution feature processing in existing automated baggage handling systems at airports. It also accurately captures the features of suitcases, handles, and RFID tags of different sizes. Furthermore, by controlling computational complexity through localized window attention, it improves model efficiency while ensuring recognition accuracy.

[0050] Step S103: Based on the prediction result, control the robotic arm to perform a gripping operation or issue a warning signal.

[0051] In an embodiment of the present invention, after obtaining the prediction result output by the model, the prediction result is analyzed and judged, and corresponding control operations are executed according to the judgment conclusion. If the prediction result shows that the suitcase exists and the RFID tag is normal (i.e., present), the terminal device will synchronously send information including the relative position of the suitcase and the handle to the robotic arm controller, issue control instructions that meet the safety gripping requirements, and guide the robotic arm to avoid the handle and perform the gripping operation; if the prediction result shows that the suitcase does not have an RFID tag, the terminal device will trigger its own warning signal generation module or send a signal to a designated warning device to issue a warning of missing RFID tag, so as to prompt staff to handle it in time and realize dynamic control of the luggage handling process.

[0052] The beneficial effects of this invention compared to existing technologies are as follows: By using image acquisition, preprocessing, model recognition, and linkage control, it effectively solves the problems in traditional baggage handling where inaccurate identification of luggage handles and RFID tags leads to easy damage to luggage during robotic arm gripping and difficulty in detecting missing RFID tags. It achieves effective identification of the status of luggage handles, RFID tags, and their relative positions, thereby ensuring the safety of robotic arm gripping operations. At the same time, it promptly detects missing RFID tags, improving the reliability of airport baggage handling and reducing the risk of baggage damage and loss of tracking control.

[0053] In a complete example, data is collected using an industrial camera, requiring suitable external conditions such as lighting and exposure to ensure clear images of the suitcase, handle, and RFID features. After acquiring appropriate image data, it is cropped to retain only the image of the suitcase portion, removing irrelevant backgrounds to avoid adverse effects during training. The cropped image is then scaled and padded to 1024*2048 and input into the algorithm model for recognition.

[0054] The overall structure of the luggage recognition network follows an Encoder-Decoder format, comprising an encoder layer and a decoder layer. The encoder layer is a multi-layer encoder used to extract both coarse and fine image features, while the decoder layer is a lightweight All-MLP decoder used to directly fuse these multi-level features and predict the semantic segmentation mask. Images captured by an RGB camera first pass through the encoder layer for encoding, and the resulting feature maps are then fed to the decoder layer to complete decoding and output the corresponding prediction results.

[0055] The encoder layer consists of three attention modules that process the input image in a hierarchical manner to obtain multi-level features at different resolutions. In each layer of the encoder, the resolution of the feature maps gradually decreases. This multi-scale feature extraction strategy enables the model to extract image features at different scales, thus effectively recognizing objects of various sizes. Each attention module contains two main parts: a self-attention mechanism and a feed-forward network. The encoder combines local and global features through the stacked attention modules, providing rich image information.

[0056] Self-attention mechanism: The self-attention mechanism captures global dependencies by calculating a weighted sum of input features. For image data, this means the model can consider the information of the entire image when processing each pixel, thereby extracting the global contextual dependencies of the image. The network uses localized windowed attention to reduce computation on higher-resolution feature maps while maintaining the fluidity of global information. A major bottleneck of the above hierarchical feature representation is the complexity of secondary self-attention for long sequences of high-resolution features. In multi-head self-attention, each multi-head Q;K;V has the same dimension N×C, where N=H×W is the length of the sequence. The self-attention mechanism is as follows:

[0057]

[0058] Here, Q, K, and V represent the "Query," "Key," and "Value" matrices in the self-attention mechanism, respectively. All are obtained from the input features through linear transformations and are used to calculate the dependencies between features. All three have the same dimension N×C.

[0059] Specifically, N is the sequence length, equal to the product of the feature map's height H and width W (N = H × W), representing the number of tokens into which the input feature is segmented; C is the number of channels in the feature map, i.e., the feature dimension of each token. head The dimension C is the feature dimension of each attention head (i.e., the Q, K, and V feature dimensions in single-head attention). In multi-head self-attention, the total dimension C is evenly distributed across multiple attention heads, therefore... head =C / number of heads, used to scale attention weights and avoid gradient problems caused by excessive dimensionality.

[0060]

[0061] Where, x in The input features of the module represent the initial features that need to be processed.

[0062] MLP stands for Multi-Layer Perceptron, which consists of multiple linear transformation layers and non-linear activation functions. It is used to perform non-linear transformations and dimensionality adjustments on features, enhancing their expressive power. `mlp()` represents the processing function of a feedforward neural network (Multi-Layer Perceptron). The feedforward network consists of two linear layers and one non-linear activation function, using a 3×3 Conv to account for the impact of zero-padding on leaked location information. The feedforward neural network is responsible for further processing the features output by the self-attention mechanism. Feedforward networks can capture and express more complex feature information, thereby improving the model's expressive power.

[0063] Conv 3×3 This indicates that a 3×3 kernel is used for convolution operations to capture local dependencies in features (such as the association of adjacent features in space or channels) while reducing the loss of positional information caused by operations such as "zero padding".

[0064] GELU is an abbreviation for Gaussian Error Linear Unit, a non-linear activation function that can more smoothly introduce non-linearity and improve model performance.

[0065] For the output characteristics of the module (with x) in Dimensional matching is the final result after a series of transformations.

[0066] +x in This represents a residual connection, where the initial input x is... in Adding the features directly to the multi-layer transformed features can alleviate the gradient vanishing problem in deep networks and enhance information transmission and reuse.

[0067] The decoder layer is a lightweight MLP decoder, which allows it to accurately reconstruct image details while maintaining efficient computation. The implementation of the decoder can be divided into four main steps:

[0068] First, the multi-level feature maps (Fi) from the encoder are processed through a multilayer perceptron (MLP) layer to unify the channel dimension. The purpose of this step is to transform feature maps of different scales into feature representations with a consistent number of channels, which facilitates subsequent processing.

[0069] After channel unification, the feature maps are upsampled to 1 / 4 of the original image resolution (i.e., H / 4 × W / 4), and then these multi-level feature maps are stitched together. The stitching operation combines features at different scales, allowing the model to comprehensively utilize these features and improve recognition accuracy.

[0070] The concatenated feature maps are then input into another MLP layer for feature fusion. Through linear transformations and activation functions in the MLP layer, the model can extract more abstract and meaningful feature representations from the concatenated feature maps, further improving recognition performance.

[0071] Finally, the fused feature map is processed through another MLP layer to output the final segmentation mask (M). This recognition algorithm can efficiently generate classification results for each pixel while preserving details.

[0072] Based on the recognition results output by the algorithm model, the robotic arm controller will further analyze the information in the image to obtain the current status of the suitcase, handle, and RFID, as well as the relative position of the suitcase and handle. It will then determine whether the robotic arm needs to perform a gripping operation. If the RFID is not present, a warning will be issued.

[0073] While automated baggage handling systems at airports utilize machine vision technology, traditional image processing methods often fail to adequately preprocess the acquired images. Irrelevant backgrounds can interfere with feature extraction, and inconsistent image sizes across different acquisition scenarios can lead to misidentification or underidentification of features during subsequent algorithmic model recognition. Therefore, this invention proposes an optional embodiment. (Refer to...) Figure 2 , Figure 2 This is a schematic diagram of a specific embodiment of step S101 of the luggage handle and RFID tag identification method based on deep network vision in this invention. Step S101 also includes the following specific implementation methods.

[0074] Step S1011: Crop the image data, retain the suitcase portion of the image and remove the surrounding background, then scale and fill the cropped image to a preset size to complete the preprocessing of the image data and obtain the target image.

[0075] In an embodiment of the present invention, after acquiring the luggage image data captured by the industrial camera, the terminal device initiates an image cropping process. The purpose is to filter and retain only the image portion containing the luggage from the original image data, while removing the surrounding irrelevant background.

[0076] Optionally, a preset image region detection algorithm can be used to automatically locate the boundary range of the suitcase in the image, and then the precise cropping can be performed according to the boundary range to avoid errors caused by manually setting the cropping area and ensure that only valid target information is retained after cropping.

[0077] After image cropping is completed, scaling and filling operations are performed on the cropped suitcase image to adjust it to a preset size (this preset size matches the requirements of the subsequently input suitcase recognition algorithm model, such as 1024*2048 preferred in this embodiment).

[0078] During the scaling process, an interpolation algorithm may be used to ensure that the scaled image retains the key details of the suitcase, handle, and RFID tag when the resolution is adjusted, thus avoiding image distortion or feature blurring caused by scaling. If the cropped image size is smaller than the preset size, the image is supplemented to the preset size by filling preset pixel values ​​(such as black pixels, white pixels, or pixel values ​​consistent with the image background color) to ensure uniform image specifications and meet the input standards of the algorithm model. Finally, the preprocessing is completed and the target image is obtained.

[0079] In an optional embodiment of the present invention, in order to address the problem that irrelevant background interference and inconsistent size in image data may affect the recognition accuracy of subsequent algorithm models, the operation of first cropping and removing irrelevant background and then scaling and filling to a preset size effectively eliminates the adverse interference of non-target information on model training and recognition, while ensuring that the image specifications of the input model are consistent, avoiding deviations in feature extraction caused by differences in image size, and improving the reliability of model recognition results.

[0080] While automated baggage handling systems at airports utilize machine vision technology, traditional robotic arms often fail to accurately determine the position of the luggage handle, leading to accidental clamping of the handle and damage to the luggage's exterior. Therefore, this invention proposes an alternative embodiment. (Refer to...) Figure 3 , Figure 3 This is a schematic diagram of the first specific embodiment of step S103 of the luggage handle and RFID tag identification method based on deep network vision in this invention. Step S103 also includes the following specific implementation:

[0081] Step S1031: If, based on the luggage handle status, RFID tag status, and relative position of the luggage and handle in the prediction results, it is determined that the luggage handle status and RFID tag status are present, then, based on the relative position of the luggage and handle, the robotic arm is controlled to avoid the handle and perform the luggage clamping operation.

[0082] In the embodiments of the present invention, key state information in the prediction results is extracted and judged, focusing on confirming whether the luggage handle is present and whether the RFID tag is present. Only when both conditions are met simultaneously will the subsequent robotic arm gripping control process be entered; if either condition is not met, the operation logic corresponding to this claim is terminated and the process is switched to other adapted processing flow.

[0083] After confirming the presence of both the pull handle and the RFID tag, the relative position information of the suitcase and the pull handle is further extracted and analyzed from the prediction results. This information usually includes the specific orientation of the pull handle on the suitcase (such as extending from the side or top) and coordinate offset data.

[0084] Optionally, a preset coordinate transformation algorithm is used to convert the relative position data in the prediction results into coordinate parameters that match the motion coordinate system of the robotic arm, ensuring that the robotic arm can accurately identify the position boundary of the lever.

[0085] After completing the location information parsing, the terminal device sends control commands to the robotic arm controller. These commands include "motion path planning parameters to avoid the lever" and "suitcase gripping point parameters." Upon receiving the commands, the robotic arm controller drives the robotic arm to move along the planned path, precisely reaching the preset gripping position on the suitcase (such as a flat area on the side of the suitcase without a lever) while avoiding the lever area, and then performs the gripping action. After the gripping action is completed, the robotic arm controller sends a "gripping successful" status signal back to the terminal device, which records the operation result, completing the process.

[0086] In an optional embodiment of the present invention, by controlling the robotic arm to avoid gripping the lever based on relative position information in a scenario where both the lever and the RFID tag are present, the problem of damage to suitcases caused by the robotic arm's inability to identify the lever's position and the easy accidental gripping of the lever is accurately solved in existing airport automatic baggage pickup systems.

[0087] Optionally, refer to Figure 4 , Figure 4 This is a schematic diagram of a second specific embodiment of step S103 of the luggage handle and RFID tag identification method based on deep network vision in this invention. Step S103 further includes the following specific implementation:

[0088] Step S1032: Obtain the RFID tag status from the prediction results.

[0089] In step S1033, if the RFID tag status is "not present", a warning signal for missing RFID tag is issued, and the gripping operation of the robotic arm is not triggered.

[0090] In an embodiment of the present invention, if the determination result is that the RFID tag status is non-existent, a warning signal generation process is immediately triggered.

[0091] Optionally, a text alarm message containing the current identification area and identification time of the luggage is pushed to the central control terminal of the luggage handling system, so that staff can quickly locate and deal with the problem of missing RFID tags; at the same time, a control command to prohibit gripping is sent to the robotic arm controller.

[0092] In an optional embodiment of the present invention, by prioritizing the extraction of RFID tag status and triggering a warning and prohibiting the robotic arm from gripping the baggage in the scenario of "RFID tag not found", the problem of the lack of an effective detection mechanism for missing RFID tags in traditional baggage handling methods is accurately solved.

[0093] Optionally, step S103 further includes the following specific implementation:

[0094] Step S1034: If, based on the luggage handle status, RFID tag status, and relative position of the luggage and handle in the prediction results, it is determined that the luggage handle status is non-existent and the RFID tag status is present, then the robotic arm is controlled to directly perform the luggage clamping operation.

[0095] In the embodiments of the present invention, the RFID tag status is extracted and focused from the prediction results, and analyzed and judged separately. Other status information (such as lever status) is not relied upon for the time being, and the existence of the RFID tag is used as the core judgment basis for subsequent operations.

[0096] After extracting the RFID tag status, a logical judgment is made on the status. If the judgment result is that the RFID tag status does not exist, a warning signal generation mechanism is triggered.

[0097] Optionally, a control signal prohibiting gripping can be sent to the robotic arm controller to ensure that the robotic arm does not trigger any gripping operation, effectively preventing suitcases with missing RFID tags from entering the subsequent transfer process.

[0098] If the determination result indicates that the RFID tag status exists, the terminal device terminates the corresponding operation logic and instead enters the appropriate processing flow based on other status information (such as lever status).

[0099] In an optional embodiment of the present invention, by prioritizing the extraction of RFID tag status and triggering warnings and prohibiting clamping in scenarios where RFID is absent, the problem of the lack of an effective detection mechanism for missing RFID tags in traditional baggage handling methods is solved. Suitcases missing RFID tags will not be clamped and transported by robotic arms, avoiding the inability to effectively monitor and track suitcases during subsequent transportation and storage due to missing RFID tags, thereby improving the safety and reliability of baggage management and reducing the risk of baggage loss.

[0100] Optionally, step S103 further includes the following specific implementation:

[0101] Step S1035: If, based on the luggage handle status, RFID tag status, and relative position of the luggage and handle in the prediction results, it is determined that the luggage handle status and RFID tag status are present, then, based on the relative position of the luggage and handle, the robotic arm is controlled to avoid the handle and perform the luggage clamping operation.

[0102] In an embodiment of the present invention, after confirming that the pull rod is absent and the RFID tag is present, it is further determined whether special adjustments to the gripping position of the robotic arm are needed. Since the pull rod is absent, there is no need to avoid the pull rod area; therefore, a control command for direct gripping is generated based on the overall position information of the suitcase in the prediction result.

[0103] Optionally, by combining historical gripping data, a standard gripping point that is compatible with the current size and shape of the suitcase (such as a flat area without protrusions in the middle of the suitcase) is automatically matched to ensure gripping stability. Subsequently, the control command is sent to the robotic arm controller, which contains key information such as the gripping point coordinates and gripping force parameters.

[0104] After receiving the instruction, the robotic arm controller drives the robotic arm to move to the target gripping position according to the instruction parameters and execute the gripping action. After the gripping action is completed, the robotic arm controller sends a status signal of successful gripping to the terminal device and records the key data of this operation (such as operation time, suitcase recognition information, etc.) to complete the entire process.

[0105] In this embodiment of the invention, by controlling the robotic arm to directly perform the gripping operation in a scenario where the lever is absent but the RFID tag is present, the problem of low processing efficiency caused by over-reliance on fixed gripping processes in traditional baggage handling systems is solved.

[0106] Optionally, step S103 further includes the following specific implementation:

[0107] Step S1036: If, based on the luggage handle status, RFID tag status, and relative position of the luggage and handle in the prediction results, it is determined that the luggage handle status and RFID tag status are present, then, based on the relative position of the luggage and handle, the robotic arm is controlled to avoid the handle and perform the luggage clamping operation.

[0108] In an embodiment of the present invention, after confirming that the pull rod exists and the RFID tag does not exist, a warning signal generation mechanism is triggered.

[0109] Optionally, a continuous warning sound and flashing light can be emitted through a connected audible and visual alarm device, or a text alarm message containing the location of the luggage and the identification time can be pushed to the central control platform of the airport baggage handling system, so that staff can quickly locate and deal with the problem of missing RFID tags.

[0110] Optionally, a control command to prohibit gripping is sent to the robotic arm controller. This command contains explicit execution parameters to prevent gripping actions, ensuring that the robotic arm will not grip and transfer suitcases that are currently missing RFID tags, thus preventing them from entering subsequent processes and causing tracking to go out of control.

[0111] After receiving the command to prohibit gripping, the robotic arm controller sends a confirmation signal to the terminal device, indicating that the command has been received. The terminal device records key information of this alarm event (such as the image of the suitcase, the recognition result, and the alarm time) and stores it locally or uploads it to the system database, thus completing the entire process.

[0112] In this embodiment of the invention, by simultaneously issuing an RFID omission warning and prohibiting the robotic arm from gripping the luggage in scenarios where the handle is present but the RFID tag is absent, the dual problems of the current luggage handling method—that it cannot identify the risk of the handle and lacks detection of missing RFID tags—are precisely solved. This avoids damage to the suitcase caused by the robotic arm accidentally gripping the handle and prevents suitcases without RFID tags from entering the transfer process and becoming untraceable.

[0113] like Figure 5 The diagram illustrates a terminal device according to an embodiment of the present invention. The terminal device 500 may include a processor 501, a memory 502, and a computer program 503 stored in the memory 502 and executable on the processor 501, such as a deep network vision-based luggage handle and RFID tag identification program. When the processor 501 executes the computer program 503, it implements the steps described in the various deep network vision-based luggage handle and RFID tag identification embodiments.

[0114] A computer program can be divided into one or more modules / units. One or more modules / units are stored in memory 502 and executed by processor 501 to complete the present invention. One or more modules / units can be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in a terminal device.

[0115] The terminal device may include, but is not limited to, processor 501 and memory 502. Those skilled in the art will understand that... Figure 5 This is merely an example of a terminal device and does not constitute a limitation on the terminal device. It may include more or fewer components than shown in the figure, or combine certain components, or different components. For example, a terminal device may also include input / output devices, network access devices, buses, etc.

[0116] The processor 501 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0117] The memory 502 can be an internal storage unit of the terminal device, such as the hard drive or RAM of the terminal device. The memory 502 can also be an external storage device of the terminal device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory 502 can include both internal and external storage units of the terminal device. The memory 502 is used to store computer programs and other programs and data required by the terminal device. The memory 502 can also be used to temporarily store data that has been output or will be output.

[0118] It should be noted that, for the sake of convenience and brevity, the structure of the terminal device described above can also be referred to the specific description of the structure in the method embodiment, which will not be repeated here.

[0119] This invention also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it can implement the steps in the above-described method for identifying luggage handles and RFID tags based on deep network vision.

[0120] This invention provides a computer program product that, when run on a mobile terminal, enables the mobile terminal to implement the steps in the above-described method for identifying luggage handles and RFID tags based on deep network vision.

[0121] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0122] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for various specific applications, but such implementations should not be considered beyond the scope of this invention.

[0123] In the embodiments provided by this invention, it should be understood that the disclosed terminal devices and methods can be implemented in other ways. For example, the terminal device embodiments described above are merely illustrative. Furthermore, the couplings or direct couplings or communication connections shown or discussed may be indirect couplings or communication connections through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0124] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0125] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0126] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0127] The embodiments described above are merely illustrative of the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for identifying suitcase handles and RFID tags based on deep network vision, characterized in that, include: The industrial camera is controlled to acquire image data of suitcases within the recognition area, and the image data is preprocessed to obtain the target image; The target image is input into a pre-trained suitcase recognition algorithm model to obtain prediction results including the state of the suitcase handle, the state of the RFID tag, and the relative position of the suitcase and the handle. The suitcase recognition algorithm model is a model trained based on deep learning for recognizing the suitcase handle and RFID tag. Based on the prediction results, control the robotic arm to perform a gripping operation or issue a warning signal; The step of controlling the robotic arm to perform a gripping operation or issue a warning signal based on the prediction result includes: If, based on the luggage handle status, RFID tag status, and relative position of the luggage and handle in the prediction results, it is determined that the luggage handle status and RFID tag status are present, then based on the relative position of the luggage and handle, the robotic arm is controlled to avoid the handle and perform the luggage clamping operation. If the RFID tag is not found, a warning signal for missing RFID tag will be issued, and the robotic arm will not be triggered to grasp the tag. If, based on the predicted status of the suitcase handle, the RFID tag, and the relative position of the suitcase and the handle, it is determined that the suitcase handle is absent and the RFID tag is present, then the robotic arm is controlled to directly perform the suitcase clamping operation.

2. The method for identifying suitcase handles and RFID tags based on deep network vision as described in claim 1, characterized in that, The preprocessing of image data to obtain the target image includes: The image data is cropped, retaining the suitcase portion of the image and removing the surrounding background. The cropped image is then scaled and filled to a preset size to complete the preprocessing of the image data and obtain the target image.

3. The method for identifying suitcase handles and RFID tags based on deep network vision as described in claim 1, characterized in that, The overall structure of the luggage recognition algorithm model is in the Encoder-Decoder format, including an encoder layer and a decoder layer; the encoder layer is used to encode the target image and extract the coarse and fine features of the target image; The decoder layer is used to fuse coarse and fine features to predict semantic segmentation masks to generate prediction results.

4. The method for identifying suitcase handles and RFID tags based on deep network vision as described in claim 3, characterized in that, The encoder layer consists of three attention modules, which process the input target image in a hierarchical manner, so that the resolution of the output feature map of each layer is gradually reduced to achieve multi-scale feature extraction. Each attention module includes a self-attention mechanism and a feedforward neural network. The self-attention mechanism uses localized window attention.

5. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the luggage handle and RFID tag identification method based on deep network vision as described in any one of claims 1 to 4.

6. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the luggage handle and RFID tag identification method based on deep network vision as described in any one of claims 1 to 4.

Images