Luggage case pull rod and RFID tag identification method based on deep network vision, terminal equipment and storage medium

Abstract

The invention is suitable for the field of image identification, and discloses a luggage case pull rod and RFID tag identification method based on deep network vision, terminal equipment and a storage medium. The luggage case pull rod and RFID tag identification method based on deep network vision comprises the following steps: controlling an industrial camera to collect image data of a luggage case in an identification area, and preprocessing the image data to obtain a target image; inputting the target image into a pre-trained luggage case identification algorithm model to obtain a prediction result including a luggage case pull rod state, an RFID tag state and a relative position of the luggage case and the pull rod, the luggage case identification algorithm model being a model obtained based on deep learning training and used for identifying the luggage case pull rod and the RFID tag; and according to the prediction result, the mechanical arm is controlled to execute clamping operation or send out a warning signal. The reliability of the luggage processing system is remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the field of image recognition, and in particular relates to a luggage handle and RFID tag recognition method based on deep network vision, a terminal device, and a storage medium. Background Art

[0002] The efficiency of baggage handling systems has become a focus. Baggage needs to be handled reliably.

[0003] Because RFID tags and handles share certain visual similarities, traditional methods often misidentify RFID tags as handles, causing the robotic arm to miss a piece of luggage it should have grabbed. Furthermore, the point cloud generated by the RFID tag interferes with the accuracy of the luggage's position calculation, leading to misjudgment of the robotic arm's grasping position and severely impacting the reliability of luggage handling. A new technical approach is needed to address these issues. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a luggage handle and RFID tag recognition method, terminal device, and storage medium based on deep network vision, which can solve the problem of low reliability of luggage handling in luggage handling systems in related technologies.

[0005] The first aspect of the present invention provides a method for identifying a luggage handle and an RFID tag based on deep network vision, comprising: Control the industrial camera to collect image data of the suitcase in the recognition area, and pre-process the image data to obtain the target image; The target image is input into a pre-trained luggage recognition algorithm model to obtain a prediction result including the luggage handle status, RFID tag status, and the relative position of the luggage and the handle. The luggage recognition algorithm model is a model trained based on deep learning to identify luggage handles and RFID tags. Based on the prediction results, the robot arm is controlled to perform a gripping operation or issue a warning signal.

[0006] Optionally, in a first implementation of the first aspect of the present invention, preprocessing the image data to obtain the target image includes: The image data is cropped to retain the suitcase portion of the image and remove the surrounding background. The cropped image is then scaled and padded to a preset size to complete the preprocessing of the image data and obtain the target image.

[0007] Optionally, in a second implementation of the first aspect of the present invention, the overall structure of the suitcase recognition algorithm model is an Encoder-Decoder format, including an encoder layer and a decoder layer; the encoder layer is used to encode the target image and extract coarse features and fine features of the target image; the decoder layer is used to fuse the coarse features and fine features, predict the semantic segmentation mask to generate a prediction result.

[0008] Optionally, in a third implementation of the first aspect of the present invention, the encoder layer is composed of three attention modules, which process the input target image in a hierarchical manner, so that the resolution of the output feature map of each layer is gradually reduced to achieve multi-scale feature extraction; each attention module includes a self-attention mechanism and a feedforward neural network, and the self-attention mechanism uses localized window attention.

[0009] Optionally, in a fourth implementation of the first aspect of the present invention, the step of controlling the robotic arm to perform a gripping operation or issue a warning signal according to the prediction result includes: If the luggage handle status, RFID tag status, and the relative position of the luggage and the handle in the prediction results are judged to be present and the RFID tag status is present, then the robotic arm is controlled to avoid the handle and perform the luggage gripping operation based on the relative position of the luggage and the handle.

[0010] Optionally, in a fifth implementation of the first aspect of the present invention, the step of controlling the robotic arm to perform a gripping operation or issue a warning signal according to the prediction result includes: Get the RFID tag status in the prediction results; If the RFID tag status is not present, a warning signal of missing RFID tag is issued, and the gripping operation of the robot arm is not triggered.

[0011] Optionally, in a sixth implementation of the first aspect of the present invention, the step of controlling the robotic arm to perform a gripping operation or issue a warning signal according to the prediction result includes: If the luggage handle status, RFID tag status, and the relative position of the luggage and the handle in the prediction results indicate that the luggage handle status is present and the RFID tag status is absent, a warning signal indicating that the RFID tag is missing is issued, and the gripping operation of the robotic arm is not triggered.

[0012] Optionally, in a seventh implementation of the first aspect of the present invention, the step of controlling the robotic arm to perform a gripping operation or issue a warning signal according to the prediction result includes: If the luggage handle status, RFID tag status, and the relative position of the luggage and the handle in the prediction results indicate that the luggage handle status is present and the RFID tag status is absent, a warning signal indicating that the RFID tag is missing is issued, and the gripping operation of the robotic arm is not triggered.

[0013] In a second aspect, an embodiment of the present invention provides a terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the above-mentioned method for identifying suitcase handles and RFID tags based on deep network vision are implemented.

[0014] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps of the above-mentioned method for identifying suitcase handles and RFID tags based on deep network vision.

[0015] In a fourth aspect, an embodiment of the present invention provides a computer program product, which, when executed on a terminal device, enables the terminal device to execute the above-mentioned method for identifying suitcase handles and RFID tags based on deep network vision.

[0016] Compared with the existing technology, the embodiments of the present invention have the following advantages: by using a deep learning model to accurately identify the status of the luggage handle and its relative position to the luggage, damage to the luggage caused by the robot arm's misjudgment of the gripping position in the traditional method is avoided, operational errors, stalls, and delays in the luggage handling process are reduced, and luggage handling efficiency is improved. By using the model to identify the status of RFID tags, the traditional method overcomes the lack of an effective mechanism to detect missing RFID tags, preventing the problem of luggage being untrackable due to missing RFID tags. This significantly improves the reliability of the luggage handling system from the two aspects of reducing damage and avoiding errors, and detecting tags and ensuring tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 Schematic diagram of an embodiment of a method for identifying a luggage handle and an RFID tag based on deep network vision in an embodiment of the present invention; Figure 2This is a schematic diagram of a specific embodiment of step S101 of the method for identifying a luggage handle and an RFID tag based on deep network vision in an embodiment of the present invention; Figure 3 This is a schematic diagram of a first specific embodiment of step S103 of the method for identifying a luggage handle and an RFID tag based on deep network vision in an embodiment of the present invention; Figure 4 Schematic diagram of a second specific embodiment of step S103 of the method for identifying a luggage handle and an RFID tag based on deep network vision in an embodiment of the present invention; Figure 5 Schematic diagram of a terminal device in an embodiment of the present invention. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely used to explain the present invention and are not intended to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are protected by the present invention.

[0020] It should be noted that the terms "include", "comprising" and "having" and any variations thereof in the specification and claims of the present invention and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, terminal, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices. In the claims, specification and drawings of the present invention, relational terms such as "first" and "second" are merely used to distinguish one entity / operation / object from another entity / operation / object, and do not necessarily require or imply any such real-time relationship or order between these entities / operations / objects.

[0021] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present invention. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute a separate or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0022] The efficiency of baggage handling systems has become a focus. Baggage needs to be handled quickly and accurately.

[0023] Traditional baggage handling methods are inefficient and prone to errors. Suitcases can be damaged due to misjudgment of the robotic arm's gripping position. Even with the application of machine vision technology, inaccurate recognition and operational errors can occur when handling complex objects, such as extended luggage handles. Furthermore, there is a lack of effective detection mechanisms for missing RFID tags, leading to frequent stalls, delays, damaged luggage, and loss of tracking due to missing RFID tags. This demonstrates the low reliability of traditional baggage handling systems. A new technical solution is needed to address these issues.

[0024] In view of this, embodiments of the present invention provide a luggage handle and RFID tag identification method, terminal device, and storage medium based on deep network vision. By using a deep learning model to accurately identify the status of the luggage handle and its relative position to the luggage, this method avoids damage to the luggage caused by misjudgment of the gripping position by the robotic arm in traditional methods, reduces operational errors, stalls, and delays in the luggage handling process, and improves luggage handling efficiency. By using the model to identify the status of RFID tags, this method overcomes the traditional method's lack of an effective mechanism for detecting missing RFID tags, preventing the problem of luggage being untrackable due to missing RFID tags. This significantly improves the reliability of the luggage handling system from two aspects: reducing damage and avoiding errors, and detecting tags and ensuring tracking.

[0025] In order to illustrate the technical solution of the present invention, specific embodiments are provided below.

[0026] Figure 1 This section shows a flowchart of a method for identifying luggage handles and RFID tags based on deep network vision, as provided by an embodiment of the present invention. This method can be applied to a terminal device, such as a mobile phone, tablet computer, laptop computer, ultra-mobile personal computer (UMPC), or netbook.

[0027] Specifically, the above-mentioned luggage handle and RFID tag recognition method based on deep network vision may include the following steps S101 to S103.

[0028] Step S101: Control the industrial camera to collect image data of the suitcase within the recognition area, and pre-process the image data to obtain a target image.

[0029] In an embodiment of the present invention, the terminal device starts the image acquisition control process, sends an acquisition instruction to the industrial camera, and controls the industrial camera to acquire image data of a preset recognition area (i.e., the designated area where the conveyor belt carries the suitcase).

[0030] Optionally, to ensure that the collected image can clearly present the key features of the suitcase, handle and RFID tag, a linkage is established with the light source in the identification area. By adjusting the light intensity of the light source, or adjusting the exposure parameters and focal length of the industrial camera, image blur and feature loss caused by excessive or weak light or improper exposure can be avoided, thereby obtaining initial image data that can reflect the details of the suitcase and related targets.

[0031] After obtaining the initial image data, the image data is preprocessed to remove interference information, unify image specifications, and eliminate the influence of irrelevant factors for subsequent model recognition.

[0032] Optionally, the initial image is first filtered to reduce image noise caused by ambient light changes, camera sensor errors, etc., and then the image is cropped according to the actual scene requirements, retaining only the core area containing the suitcase, and removing irrelevant background such as the conveyor belt edge and the equipment frame. Finally, the cropped image is scaled or padded according to the input specifications of the pre-trained suitcase recognition algorithm model to ensure that parameters such as image size and resolution meet the input requirements of the model, and finally a target image that can be directly input into the model is obtained.

[0033] In step S102, the target image is input into a pre-trained luggage recognition algorithm model to obtain a prediction result including the luggage handle status, RFID tag status, and the relative position of the luggage and the handle. The luggage recognition algorithm model is a model trained based on deep learning and is used to identify luggage handles and RFID tags.

[0034] In this embodiment of the present invention, the preprocessed target image is input into a pre-trained luggage recognition algorithm model. This model, built using deep learning technology and trained on a large amount of luggage image data containing luggage of varying styles, handle states, and RFID tag positions, is capable of identifying the handle states, RFID tag status, and the relative position of the luggage and handle. By invoking the model's computational interface, the model is driven to extract, analyze, and identify features of the target image. Finally, the model outputs a prediction result containing the aforementioned key information, completing the conversion from image data to specific recognition information.

[0035] Optionally, the overall structure of the luggage recognition algorithm model adopts an encoder-decoder format, comprising an encoder layer and a decoder layer. The encoder layer encodes the target image and extracts both coarse and fine features. The decoder layer fuses these features and predicts a semantic segmentation mask to generate a prediction result. By extracting both coarse and fine features from the target image through the encoder layer and then fusing multi-level features and predicting a semantic segmentation mask through the decoder layer, the model can fully capture the multi-dimensional image information of the luggage, handle, and RFID tag. This effectively addresses the difficulty in fully and accurately identifying the key features of complex luggage in existing airport baggage handling and automatic picking systems, as traditional machine vision technology or single feature extraction methods struggle to do so.

[0036] Optionally, the encoder layer consists of three attention modules, which process the input target image in a hierarchical manner, gradually reducing the resolution of the feature maps output by each layer to achieve multi-scale feature extraction. Each attention module includes a self-attention mechanism and a feedforward neural network, and the self-attention mechanism uses localized window attention. The target image is processed by three hierarchical attention modules to gradually reduce the resolution of the feature maps to achieve multi-scale feature extraction. Each module combines the self-attention mechanism with a feedforward neural network. This not only solves the problems of high computational complexity and difficulty balancing global information and local details in existing machine vision technology for high-resolution feature processing in airport baggage handling automatic picking systems, but also accurately captures the features of suitcases, handles, and RFID tags of different sizes. Localized window attention is used to control computational complexity, improving model efficiency while ensuring recognition accuracy.

[0037] Step S103: Control the robotic arm to perform a gripping operation or issue a warning signal based on the prediction result.

[0038] In this embodiment of the present invention, after obtaining the prediction result output by the model, the prediction result is analyzed and judged, and corresponding control operations are executed based on the judgment conclusion. If the prediction result indicates that the suitcase is present and the RFID tag is normal (i.e., present), the terminal device will synchronously transmit information containing the relative position of the suitcase and the handle to the robotic arm controller, issuing control instructions that meet safe gripping requirements, instructing the robotic arm to avoid the handle and perform the gripping operation. If the prediction result indicates that the suitcase does not have an RFID tag, the terminal device will trigger its own warning signal generation module or send a signal to a designated early warning device to issue a warning that the RFID tag is missing, prompting staff to promptly address the problem, thereby achieving dynamic control of the baggage handling process.

[0039] Compared with the existing technology, the embodiments of the present invention have the following advantages: through image acquisition, preprocessing, model recognition, and linkage control, it effectively solves the problem in traditional baggage handling that inaccurate recognition of luggage handles and RFID tags leads to easy damage to luggage during robotic gripping and difficulty in detecting missing RFID tags. It achieves effective recognition of the luggage handle status, RFID tag status, and the relative position of the two, thereby ensuring the safety of the robotic gripping operation and timely detecting RFID missing situations, thereby improving the reliability of the airport baggage handling process and reducing the risk of luggage damage and tracking loss.

[0040] In a complete example, data is collected through an industrial camera, requiring appropriate external conditions such as lighting and exposure to ensure clear capture of the suitcase, handle, and RFID features. After collecting appropriate image data, it is cropped to retain only the image of the suitcase, removing irrelevant background to avoid adverse effects during training. The cropped image is then scaled and padded to 1024*2048 and input into the algorithm model for recognition.

[0041] The overall structure of the luggage recognition network is an encoder-decoder architecture, consisting of an encoder layer and a decoder layer. The encoder layer is a multi-layer encoder that extracts coarse and fine image features. The decoder layer is a lightweight All-MLP decoder that directly fuses these multi-level features and predicts semantic segmentation masks. Images captured by the RGB camera first pass through the encoder layer for encoding. The resulting feature map is then fed into the decoder layer for decoding and output of the corresponding prediction results.

[0042] The encoder layer consists of three attention modules, which process the input image in a hierarchical manner to obtain multi-level features at different resolutions. At each encoder layer, the resolution of the feature map decreases. This multi-scale feature extraction strategy enables the model to extract image features at different scales, effectively recognizing objects of various sizes. Each attention module consists of two main components: a self-attention mechanism and a feed-forward neural network. The encoder combines local and global features through stacked attention modules, providing rich image information.

[0043] Self-attention mechanism: The self-attention mechanism captures global dependencies by calculating the weighted sum of input features. For image data, this means that the model is able to consider the information of the entire image when processing each pixel, thereby extracting the global contextual dependencies of the image. The use of localized windowed attention (WindowedAttention) in the network reduces the amount of computation on higher-resolution feature maps while maintaining the fluidity of global information. A major bottleneck of the above-mentioned hierarchical feature representation is the complexity of quadratic self-attention for long sequence inputs of high-resolution features. In multi-head self-attention, each multi-head Q;K;V has the same dimension N×C, where N=H×W is the length of the sequence, then the self-attention mechanism is:

[0044] Among them, Q, K, and V represent the "Query", "Key", and "Value" matrices in the self-attention mechanism, respectively. They are all obtained by linear transformation of the input features and are used to calculate the dependencies between features. The three have the same dimension N×C. Specifically, N is the sequence length, which is equal to the product of the height H and width W of the feature map (N=H×W), representing the number of tokens into which the input feature is divided; C is the number of channels of the feature map, that is, the feature dimension of each token. head is the dimension of each attention head (i.e., the feature dimension of Q, K, V in single-head attention). In multi-head self-attention, the total dimension C will be evenly distributed to multiple attention heads, so d head = C / number of heads, used to scale attention weights to avoid gradient problems caused by high dimensions.

[0045]

[0046] Among them, x in It is the input feature of the module, representing the initial feature that needs to be processed.

[0047] MLP stands for Multi-Layer Perceptron. It consists of multiple linear transformation layers and nonlinear activation functions, performing nonlinear transformations and dimensionality adjustments on features to enhance their expressiveness. mlp() represents the processing function of a feedforward neural network (MLP). The feedforward network consists of two linear layers and a nonlinear activation function. A 3×3 ConvNet is used to account for the effects of zero-padding on leaked position information. The feedforward neural network further processes the features output by the self-attention mechanism. Feedforward networks can capture and express more complex feature information, thereby improving the model's expressiveness.

[0048] Conv3×3 Indicates that a convolution operation is performed with a convolution kernel of size 3×3, which is used to capture local dependencies on features (such as the association between adjacent features in space or channels) while reducing the loss of position information caused by operations such as "zero padding".

[0049] GELU is the abbreviation of Gaussian Error Linear Unit, which is a nonlinear activation function that can introduce nonlinearity more smoothly and improve model performance.

[0050] is the output feature of the module (with x in Dimension matching), is the final result after a series of transformations.

[0051] +x in Represents the residual connection (ResidualConnection), which takes the initial input x in Directly adding the features after multi-layer transformation can alleviate the gradient vanishing problem of deep networks and enhance the transmission and reuse of information.

[0052] The decoder layer is a lightweight MLP decoder, which enables it to accurately reconstruct image details while maintaining efficient computation. The implementation of the decoder can be divided into four main steps: First, the multi-level feature maps (Fi) from the encoder are unified in channel dimension through a multi-layer perceptron (MLP) layer. The purpose of this step is to convert feature maps of different scales into feature representations with a consistent number of channels for easy subsequent processing.

[0053] After channel unification, the feature maps are upsampled to 1 / 4 of the original image resolution (i.e., H / 4 × W / 4). These multi-level feature maps are then concatenated. This concatenation combines features at different scales, allowing the model to comprehensively utilize these features and improve recognition accuracy.

[0054] The concatenated feature maps are then fed into another MLP layer for feature fusion. Through the linear transformation and activation function of the MLP layer, the model can extract more abstract and meaningful feature representations from the concatenated feature maps, further improving recognition performance.

[0055] Finally, the fused feature map is processed through another MLP layer to output the final segmentation mask (M). This recognition algorithm can efficiently generate classification results for each pixel while preserving details.

[0056] Based on the recognition results output by the algorithm model, the robotic arm controller will further analyze the information in the image to obtain the current status of the suitcase, handle, RFID, and the relative position of the suitcase and handle, and determine whether the robotic arm needs to perform a gripping operation. If the RFID is not present, a warning will be issued.

[0057] Although the automatic baggage handling system at the airport uses machine vision technology, if the collected images are not effectively pre-processed in traditional image processing, irrelevant background in the image will interfere with feature extraction. At the same time, the image size is not uniform in different collection scenarios, which can easily lead to feature misjudgment and omission in subsequent algorithm model recognition. Based on this, the present invention proposes an optional embodiment. Figure 2 , Figure 2 Schematic diagram of a specific embodiment of step S101 of the method for identifying a suitcase handle and an RFID tag based on deep network vision in an embodiment of the present invention. Step S101 also includes the following specific implementation methods.

[0058] Step S1011 , cropping the image data, retaining the suitcase portion in the image and removing the surrounding background, and then scaling and padding the cropped image to a preset size to complete preprocessing of the image data and obtain a target image.

[0059] In this embodiment of the present invention, after acquiring the luggage image data captured by the industrial camera, the terminal device initiates the image cropping process. The purpose is to filter and retain only the image portion containing the luggage from the original image data, while removing the surrounding irrelevant background.

[0060] Optionally, a preset image area detection algorithm is used to automatically locate the boundary range of the suitcase in the image, and then precise cropping is performed according to the boundary range, avoiding errors caused by manually setting the cropping area and ensuring that only valid target information is retained after cropping.

[0061] After image cropping is completed, scaling and padding operations are further performed on the cropped suitcase image to adjust it to a preset size (the preset size matches the requirements of the luggage recognition algorithm model input subsequently, such as 1024*2048 in this embodiment).

[0062] During the scaling process, an interpolation algorithm may be optionally used to ensure that the scaled image retains key detail features of the suitcase, handle, and RFID tag when the resolution is adjusted, thereby avoiding image deformation or feature blurring caused by scaling. If the cropped image size is smaller than the preset size, the image is padded to the preset size by filling it with preset pixel values ​​(such as black pixels, white pixels, or pixel values ​​consistent with the image background color) to ensure uniform image specifications and meet the input standards of the algorithm model. Finally, preprocessing is completed and the target image is obtained.

[0063] In an optional embodiment of the present invention, in order to address the problem that irrelevant background interference and inconsistent sizes in image data may affect the accuracy of subsequent algorithm model recognition, the operation of first cropping to remove irrelevant background and then scaling to fill it to a preset size effectively eliminates the adverse interference of non-target information on model training and recognition. At the same time, the image specifications of the input model are consistent, avoiding deviations in feature extraction by the model due to differences in image size, and improving the reliability of the model recognition results.

[0064] Although the automatic baggage handling system at airports uses machine vision technology, traditional robotic arms often mis-clamp the handle due to the inability to determine the position of the handle, resulting in damage to the luggage case shell. Based on this, the present invention proposes an optional embodiment. Figure 3 , Figure 3 This is a schematic diagram of a first specific embodiment of step S103 of the method for identifying a luggage handle and an RFID tag based on deep network vision in an embodiment of the present invention. Step S103 also includes the following specific implementation methods: In step S1031, if the luggage handle state, the RFID tag state, and the relative position of the luggage and the handle in the prediction results determine that the luggage handle state is present and the RFID tag state is present, then according to the relative position of the luggage and the handle, the robotic arm is controlled to avoid the handle and perform the luggage gripping operation.

[0065] In an embodiment of the present invention, key status information in the prediction results is extracted and judged, focusing on confirming two core conditions: whether the luggage case pull rod status is present and whether the RFID tag status is present. Only when these two conditions are met at the same time, the subsequent robotic arm gripping control process is entered; if any condition is not met, the operation logic corresponding to this claim is terminated and turned to other adapted processing processes.

[0066] After confirming the presence of both the handle and the RFID tag, the relative position information of the suitcase and the handle is further extracted and parsed from the prediction results. This information usually includes data such as the specific position of the handle on the suitcase (such as the side or the top extension), coordinate offset, etc.

[0067] Optionally, the relative position data in the prediction result is converted into coordinate parameters that match the robot arm motion coordinate system through a preset coordinate conversion algorithm, ensuring that the robot arm can accurately identify the pull rod position boundary.

[0068] After parsing the position information, the terminal device sends a control command to the robotic arm controller. This command includes the "path planning parameters for avoiding the handle" and the "suitcase gripping point parameters." Upon receiving the command, the robotic arm controller drives the robotic arm along the planned path, avoiding the handle area and accurately reaching the preset gripping position (e.g., a flat area on the side of the suitcase without a handle) to perform the gripping action. Once the gripping action is complete, the robotic arm controller sends a "grip successful" status signal back to the terminal device, which records the result of the operation, completing the process.

[0069] In an optional embodiment of the present invention, by controlling the robotic arm to avoid clamping the pull rod based on relative position information when both the pull rod and the RFID tag are present, the problem of damage to the suitcase caused by the robotic arm's inability to identify the pull rod position and easily clamping the pull rod by mistake in the existing airport baggage automatic picking system is accurately solved.

[0070] Optionally, refer to Figure 4 , Figure 4 This is a schematic diagram of a second specific embodiment of step S103 of the method for identifying a luggage handle and an RFID tag based on deep network vision in an embodiment of the present invention. Step S103 also includes the following specific implementation methods: Step S1032: Obtain the RFID tag status in the prediction result.

[0071] Step S1033: If the RFID tag status is not present, a warning signal indicating that the RFID tag is missing is issued, and the gripping operation of the robotic arm is not triggered.

[0072] In an embodiment of the present invention, if the determination result is that the RFID tag status is not present, the warning signal generation process is immediately triggered.

[0073] Optionally, a text alert message containing the current luggage identification area and identification time is pushed to the central control terminal of the baggage handling system so that staff can quickly locate and handle the problem of missing RFID tags; at the same time, a control instruction prohibiting gripping is sent to the robotic arm controller.

[0074] In an optional embodiment of the present invention, by prioritizing the extraction of RFID tag status and triggering a warning for the "RFID tag not present" scenario and prohibiting the robotic arm from gripping, the problem of traditional baggage handling methods lacking an effective detection mechanism for RFID tag omissions is accurately solved.

[0075] Optionally, step S103 further includes the following specific implementations: In step S1034, if the luggage handle state, the RFID tag state, and the relative position of the luggage and the handle are determined to be absent and the RFID tag state is present based on the predicted results, the robotic arm is controlled to directly perform the luggage gripping operation.

[0076] In an embodiment of the present invention, the RFID tag status is extracted and focused from the prediction results, and is analyzed and judged separately without relying on other status information (such as the pull rod status). Only the existence of the RFID tag is used as the core judgment basis for subsequent operations.

[0077] After the extraction of the RFID tag status is completed, a logical judgment is performed on the status. If the judgment result is that the RFID tag status does not exist, the warning signal generation mechanism is triggered.

[0078] Optionally, a control signal prohibiting gripping is sent to the robotic arm controller to ensure that the robotic arm does not trigger any gripping operation, effectively preventing suitcases with missing RFID tags from entering the subsequent transfer process.

[0079] If the determination result is that the RFID tag status is present, the terminal device terminates the corresponding operation logic and enters the adapted processing flow according to other status information (such as the pull rod status).

[0080] In an optional embodiment of the present invention, by prioritizing the extraction of RFID tag status and triggering a warning and prohibiting the removal of luggage in scenarios where the RFID tag is not present, the problem of traditional baggage handling methods lacking an effective mechanism to detect missing RFID tags is resolved. Suitcases without RFID tags will not be picked up and transferred by the robotic arm, preventing the luggage from being effectively monitored and tracked during subsequent transportation and storage due to missing RFID tags. This improves the safety and reliability of baggage management and reduces the risk of luggage loss.

[0081] Optionally, step S103 further includes the following specific implementations: In step S1035, if the luggage handle state, the RFID tag state, and the relative position of the luggage and the handle in the prediction results determine that the luggage handle state is present and the RFID tag state is present, then according to the relative position of the luggage and the handle, the robotic arm is controlled to avoid the handle and perform the luggage gripping operation.

[0082] In this embodiment of the present invention, after confirming the absence of the handle and the presence of the RFID tag, a further determination is made as to whether the robotic arm's gripping position requires special adjustment. Since the handle is absent, there's no need to avoid the area around it. Therefore, a control instruction for direct gripping is generated based on the predicted overall position of the suitcase.

[0083] Optionally, based on historical gripping data, standard gripping points that are compatible with the current suitcase size and shape (such as a flat area without protrusions in the middle of the suitcase) are automatically matched to ensure gripping stability; then, the control instruction is sent to the robotic arm controller, which contains key information such as the gripping point coordinates and gripping force parameters.

[0084] After receiving the command, the robotic arm controller drives the robotic arm to move to the target gripping position according to the command parameters and executes the gripping action. After the gripping action is completed, the robotic arm controller feeds back a status signal of successful gripping to the terminal device and records key data of this operation (such as operation time, suitcase identification information, etc.), completing the entire process.

[0085] In an embodiment of the present invention, by controlling the robotic arm to directly perform a gripping operation when the pull rod is not present but the RFID tag is present, the problem of low processing efficiency in traditional baggage handling systems, which may be caused by over-reliance on fixed gripping processes, is solved.

[0086] Optionally, step S103 further includes the following specific implementations: In step S1036, if the luggage handle state, the RFID tag state, and the relative position of the luggage and the handle in the prediction results determine that the luggage handle state is present and the RFID tag state is present, then according to the relative position of the luggage and the handle, the robotic arm is controlled to avoid the handle and perform the gripping operation of the luggage.

[0087] In an embodiment of the present invention, after confirming that the pull rod exists and the RFID tag does not exist, a warning signal generation mechanism is triggered.

[0088] Optionally, a connected sound and light alarm device can emit a continuous warning sound and flashing light, or push a text alarm message containing the luggage box location and identification time to the central control platform of the airport baggage handling system, so that staff can quickly locate and handle the problem of missing RFID tags.

[0089] Optionally, a control instruction prohibiting gripping is sent to the robotic arm controller, which contains clear execution parameters for not triggering the gripping action to ensure that the robotic arm will not grip and transfer the suitcase that is currently missing an RFID tag, thereby preventing it from entering the subsequent process and causing tracking to be out of control.

[0090] After receiving the prohibition of gripping command, the robotic arm controller feeds back a confirmation signal that the command has been received to the terminal device. The terminal device records the key information of this alarm event (such as the suitcase image, recognition result, and alarm time) and stores it locally or uploads it to the system database, completing the entire process.

[0091] In this embodiment of the present invention, when a handle is present but an RFID tag is not, an RFID omission warning is issued simultaneously and the robotic arm is prohibited from grabbing the luggage. This precisely solves the dual problems of luggage handling methods that fail to identify the risk of handle presence and lack RFID tag omission detection. This not only avoids damage to the luggage due to the robotic arm accidentally grabbing the handle, but also prevents luggage without RFID tags from entering the transfer process and becoming untraceable.

[0092] like Figure 5 FIG2 is a schematic diagram of a terminal device according to an embodiment of the present invention. The terminal device 500 may include a processor 501, a memory 502, and a computer program 503 stored in the memory 502 and executable by the processor 501, such as a deep network vision-based luggage handle and RFID tag recognition program. When the processor 501 executes the computer program 503, the steps described in the aforementioned deep network vision-based luggage handle and RFID tag recognition embodiments are implemented.

[0093] The computer program can be divided into one or more modules / units, which are stored in the memory 502 and executed by the processor 501 to implement the present invention. One or more modules / units can be a series of computer program instruction segments that can perform specific functions. These instruction segments are used to describe the execution process of the computer program in the terminal device.

[0094] The terminal device may include, but is not limited to, a processor 501 and a memory 502. Those skilled in the art will appreciate that Figure 5 It is only an example of a terminal device and does not constitute a limitation of the terminal device. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal device may also include input and output devices, network access devices, buses, etc.

[0095] The processor 501 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0096] Memory 502 can be an internal storage unit of the terminal device, such as the terminal device's hard drive or memory. Memory 502 can also be an external storage device of the terminal device, such as a plug-in hard drive, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, etc. Furthermore, memory 502 can include both the terminal device's internal storage unit and an external storage device. Memory 502 is used to store computer programs and other programs and data required by the terminal device. Memory 502 can also be used to temporarily store data that has been output or is about to be output.

[0097] It should be noted that, for the convenience and brevity of description, the structure of the above-mentioned terminal device can also refer to the specific description of the structure in the method embodiment, which will not be repeated here.

[0098] An embodiment of the present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it can implement the steps in the above-mentioned method for identifying suitcase handles and RFID tags based on deep network vision.

[0099] An embodiment of the present invention provides a computer program product. When the computer program product is run on a mobile terminal, the mobile terminal can implement the steps in the above-mentioned method for identifying a suitcase handle and an RFID tag based on deep network vision.

[0100] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0101] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0102] In the embodiments provided herein, it should be understood that the disclosed terminal devices and methods can be implemented in other ways. For example, the terminal device embodiments described above are merely illustrative. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection via some interface, device, or unit, which may be electrical, mechanical, or other means.

[0103] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0104] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0105] If the integrated module / unit is implemented as a software functional unit and sold or used as a standalone product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention can implement all or part of the process steps in the above-mentioned method embodiments by using a computer program to instruct the relevant hardware. The computer program can be stored in a computer-readable storage medium. When executed by a processor, the computer program can implement the steps of each of the above-mentioned method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signals, telecommunication signals, and software distribution media. It should be noted that the content of the computer-readable medium can be appropriately increased or decreased based on the requirements of legislation and patent practice in a jurisdiction. For example, in some jurisdictions, based on legislation and patent practice, computer-readable media does not include electric carrier signals and telecommunication signals.

[0106] The above embodiments are intended only to illustrate the technical solutions of the present invention and are not intended to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may be modified or some of the technical features thereof may be replaced with equivalents. Such modifications or replacements do not deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention and are therefore intended to be included within the scope of protection of the present invention.

Claims

1. A luggage handle and RFID tag recognition method based on deep network vision, characterized in that: include: Control the industrial camera to collect image data of the suitcase in the recognition area, and pre-process the image data to obtain the target image; The target image is input into a pre-trained luggage recognition algorithm model to obtain a prediction result including the luggage handle status, RFID tag status, and the relative position of the luggage and the handle. The luggage recognition algorithm model is a model trained based on deep learning to identify luggage handles and RFID tags. Based on the prediction results, the robot arm is controlled to perform a gripping operation or issue a warning signal.

2. The luggage handle and RFID tag recognition method based on deep network vision according to claim 1, characterized in that: The preprocessing of the image data to obtain the target image includes: The image data is cropped to retain the suitcase portion of the image and remove the surrounding background. The cropped image is then scaled and padded to a preset size to complete the preprocessing of the image data and obtain the target image.

3. The luggage handle and RFID tag recognition method based on deep network vision according to claim 1, characterized in that: The overall structure of the luggage recognition algorithm model is an encoder-decoder format, including an encoder layer and a decoder layer. The encoder layer is used to encode the target image and extract the coarse and fine features of the target image. The decoder layer is used to fuse coarse features and fine features and predict semantic segmentation masks to generate prediction results.

4. The luggage handle and RFID tag recognition method based on deep network vision according to claim 3, characterized in that: The encoder layer consists of three attention modules, which process the input target image in a hierarchical manner, gradually reducing the resolution of the output feature map of each layer to achieve multi-scale feature extraction; each attention module includes a self-attention mechanism and a feedforward neural network, and the self-attention mechanism uses localized window attention.

5. The luggage handle and RFID tag recognition method based on deep network vision according to claim 1, characterized in that: The step of controlling the robotic arm to perform a gripping operation or issue a warning signal according to the prediction result includes: If the luggage handle status, RFID tag status, and the relative position of the luggage and the handle in the prediction results are judged to be present and the RFID tag status is present, then the robotic arm is controlled to avoid the handle and perform the luggage gripping operation based on the relative position of the luggage and the handle.

6. The luggage handle and RFID tag recognition method based on deep network vision according to claim 1, characterized in that: The step of controlling the robotic arm to perform a gripping operation or issue a warning signal according to the prediction result includes: Get the RFID tag status in the prediction results; If the RFID tag status is not present, a warning signal of missing RFID tag is issued, and the gripping operation of the robot arm is not triggered.

7. The luggage handle and RFID tag recognition method based on deep network vision according to claim 1, characterized in that: The step of controlling the robotic arm to perform a gripping operation or issue a warning signal according to the prediction result includes: If the luggage handle status, RFID tag status, and the relative position of the luggage and the handle are determined to be absent and the RFID tag status to be present based on the predicted results, the robotic arm is controlled to directly perform the luggage gripping operation.

8. The luggage handle and RFID tag recognition method based on deep network vision according to claim 1, characterized in that: The step of controlling the robotic arm to perform a gripping operation or issue a warning signal according to the prediction result includes: If the luggage handle status, RFID tag status, and the relative position of the luggage and the handle in the prediction results indicate that the luggage handle status is present and the RFID tag status is absent, a warning signal indicating that the RFID tag is missing is issued, and the gripping operation of the robotic arm is not triggered.

9. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method for identifying a suitcase handle and an RFID tag based on deep network vision are implemented as described in any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method for identifying a suitcase handle and an RFID tag based on deep network vision are implemented as described in any one of claims 1 to 8.

Images