Luggage towing vehicle with hand-eye integrated mechanical arm

Abstract

The application provides a luggage towing vehicle with a hand-eye integrated mechanical arm, which comprises a towing vehicle, a mechanical arm arranged on the towing vehicle and rotating on the towing vehicle, a monitoring module for monitoring a current front luggage position and establishing a three-dimensional space coordinate, and sending the front luggage position in the form of the coordinate to a control module, the control module for receiving the front luggage position coordinate, calculating the size of the current front luggage according to the position coordinate of the current front luggage, identifying a luggage type category through a convolutional neural network, planning and issuing a control instruction corresponding to a preset luggage type threshold according to a grabbing strategy, and an automatic operation module comprising a grabbing unit, the grabbing unit receiving the control instruction and grabbing and putting down the current luggage according to the control instruction, and a power module installed on the towing vehicle, the power module being wirelessly connected with a ground power supply device to improve the efficiency and flexibility of luggage loading.

Description

Technical Field

[0001] The present invention relates to the technical field of luggage tractors, and more particularly to a luggage tractor with a hand-eye integrated robotic arm. Background Art

[0002] The core technology of today's outbound baggage handling systems, particularly the loading process after baggage arrives at the sorting carousel, still relies heavily on traditional manual handling. Although some airports have introduced automated loading and unloading equipment, these systems generally suffer from inherent flaws such as complex structures, bulky size, and lack of flexibility, hindering their widespread adoption.

[0003] Manual handling forms the core of the process, resulting in long operation cycles and limited throughput. Furthermore, workers often bend over excessively, leading to frequent workplace accidents and significant labor and compensation costs. To alleviate this burden, assistive wearable devices such as exoskeletons have been introduced. However, these technologies are still essentially "human-assisted" models, requiring full manual intervention. Their labor-saving effects are limited by device performance, human compatibility, and ease of use, resulting in limited improvements in handling efficiency. Furthermore, they fail to address fundamental issues such as high labor costs, high error rates, and inconvenient charging, making them difficult to implement as a systemic solution.

[0004] Therefore, a reliable and convenient off-post luggage loading technology that does not require manpower is needed. Summary of the Invention

[0005] The present invention provides a luggage tractor with a hand-eye integrated robotic arm, aiming to solve the problems of the manpower burden required by the existing luggage handling mode and the large size of existing automated loading and unloading equipment, which leads to insufficient flexibility, and to improve the efficiency and flexibility of luggage loading.

[0006] To achieve the above objectives, the present invention provides a luggage tractor with a hand-eye integrated robotic arm, comprising:

[0007] tractor;

[0008] a mechanical arm, the mechanical arm being placed on the tractor and rotating on the tractor;

[0009] A monitoring module, which monitors the current position of the luggage ahead, establishes three-dimensional spatial coordinates, and sends the coordinates of the luggage ahead to the control module;

[0010] A control module receives the coordinates of the baggage ahead, calculates the size of the baggage ahead based on the coordinates, identifies the baggage type using a convolutional neural network, and issues control instructions corresponding to a preset baggage type threshold based on a grabbing strategy plan. The control instructions include grabbing instructions and unloading instructions.

[0011] An automatic operation module, the automatic operation module being placed on the robotic arm and comprising a grabbing unit, the grabbing unit receiving the grabbing instruction and the unloading instruction, and grabbing and placing down the current luggage according to the grabbing instruction and the unloading instruction;

[0012] A power supply module is installed on the tractor and is wirelessly connected to a ground power supply device.

[0013] In one embodiment, the monitoring module includes a three-dimensional camera, a laser radar, an inertial measurement unit, a control computing platform, and a calibration unit;

[0014] The three-dimensional camera acquires the depth information and surface features of the luggage by emitting and receiving infrared or laser signals, and generates a depth image and a color image of the luggage;

[0015] The laser radar generates three-dimensional point cloud data of the luggage and the surrounding environment by emitting a laser beam and measuring the return time of the reflected light;

[0016] The inertial measurement unit is used to measure the acceleration, angular velocity and posture information of the robotic arm and compensate for jitter and deviation during the movement of the robotic arm;

[0017] The control computing platform receives the depth image, color image, and three-dimensional point cloud data generated by the three-dimensional camera and the laser radar, and establishes three-dimensional space coordinates through a sensor fusion algorithm and a three-dimensional space coordinate calculation program;

[0018] The calibration unit is used to calibrate the three-dimensional camera and lidar to ensure that their data are accurately aligned in a unified coordinate system.

[0019] In one embodiment, the process of establishing three-dimensional space coordinates includes:

[0020] Data acquisition: A 3D camera scans the baggage storage area to obtain depth and color images of the baggage. A LiDAR scans the baggage and surrounding environment to obtain point cloud data. The collected depth, color, and point cloud data are pre-processed, and the data from the 3D camera, LiDAR, and inertial measurement unit are converted into a unified coordinate system, where the coordinate system is referenced by the baggage tractor's body coordinate system. The features in the depth image, color image, and point cloud data are fused to generate a comprehensive 3D feature description.

[0021] 3D Coordinate Calculation: This calculation is based on the depth image. For each pixel in the depth image, its coordinates in 3D space are calculated based on the camera's intrinsic parameters and depth value. This calculation is based on point cloud data. For each point in the point cloud data, the coordinates of the baggage in front of you in 3D space are calculated based on the LiDAR's coordinate system and scanning angle.

[0022] Coordinate fusion: The 3D coordinates calculated from the depth image and point cloud data are fused to generate more accurate 3D coordinates of the luggage.

[0023] In one embodiment, before establishing the three-dimensional space coordinates, the sensor is initialized and calibrated, including:

[0024] Turn on the 3D camera and run the self-test program to check whether the 3D camera is working properly. Turn on the LiDAR and run the self-test program to check whether the laser transmitter and receiver modules are working properly. Turn on the inertial measurement unit and run the self-test program to check whether the accelerometer and gyroscope modules are working properly.

[0025] The calibration plate is placed in the field of view of the 3D camera. The calibration algorithm is run by controlling the computing platform to calculate the rotation matrix and translation vector of the 3D camera. The zero bias and scale factor of the inertial measurement unit are calculated by collecting data in a static state.

[0026] In one embodiment, the control module uses a convolutional neural network to train depth patterns and color images of multiple luggage to automatically identify and classify different types of luggage;

[0027] The luggage types include suitcases, soft bags, and cartons.

[0028] In one embodiment, the crawling strategy planning specifically includes:

[0029] When the luggage type is a suitcase, the grabbing unit grabs it from both sides in parallel when approaching, keeps it horizontal after grabbing, and gently places it to the target location;

[0030] When the luggage type is a soft bag, the grabbing unit grabs it vertically from above, keeps it balanced after grabbing, and gently places it to the target location;

[0031] When the luggage type is a carton, the grabbing unit grabs it from the side, determines the corrugated direction of the carton before grabbing it, maintains balance after grabbing it, and gently places it to the target location.

[0032] In one embodiment, the grasping unit includes a metamorphic manipulator, and the morphology of the metamorphic manipulator includes a multi-finger metamorphic manipulator and a deformable manipulator;

[0033] The multi-finger metamorphic manipulator includes multiple fingers, the number of which is adjusted according to the type and size of the luggage to be grasped. When the type and size of the luggage exceeds the set threshold of the multi-finger metamorphic manipulator, the fingers are spread out to increase the stability of the grasping. When the type and size of the luggage are below the set threshold of the multi-finger metamorphic manipulator, the fingers are retracted.

[0034] The deformable manipulator adjusts the length, angle and spacing of the manipulator fingers. When the type and size of the luggage exceeds the set threshold of the deformable manipulator, the length of the fingers is extended to grab luggage of different types and sizes.

[0035] In one embodiment, the metamorphic manipulator is further provided with a sensor unit, which is provided with a gripping force threshold. The sensor unit monitors the gripping force in real time during the gripping process. When the gripping force threshold is exceeded, the PID control algorithm is dynamically adjusted to prevent damage to the luggage.

[0036] After the metamorphic manipulator grabs the bag in front of it, the monitoring module reconfirms the type and size of the bag it has grabbed, and determines whether it is the same as the originally determined baggage type and size. If not, the grabbing strategy is adjusted in real time.

[0037] LiDAR and depth cameras monitor the surrounding environment in real time and dynamically adjust the grasping path to avoid collisions.

[0038] In one embodiment, the grasping instruction includes: grasping direction, grasping form, number of manipulator fingers, finger length, finger spacing, and finger grasping angle;

[0039] The control module sets the multi-finger metamorphic manipulator threshold and the deformable manipulator threshold in various intervals according to the luggage type and size. The multi-finger metamorphic manipulator threshold in various intervals is set with the corresponding number of manipulator fingers, and the deformable manipulator threshold in various intervals is set with the corresponding grasping form, finger length and finger spacing.

[0040] In one embodiment, when the grabbing unit is grabbing, it first determines whether the baggage in front meets the conditions for automatic grabbing, including: checking whether there are obstacles in the surrounding environment. If there are obstacles, the obstacles need to be cleared first.

[0041] The present invention has the following beneficial effects:

[0042] High efficiency: The present invention can automatically determine whether the current luggage is within the grabbing range of the grabbing unit, thereby improving the efficiency and accuracy of luggage handling, reducing manual labor intensity and luggage damage, and reducing safety hazards.

[0043] High Flexibility: Compared to fixed robotic arm sorting devices, the robot arm, including multi-fingered metamorphic and transformable arms, can flexibly handle various types of baggage sorting and handling tasks. The monitoring module is integrated into the robotic arm, resulting in a highly integrated system and simplified maintenance. Furthermore, the same device can handle both outbound loading and inbound unloading, improving equipment utilization.

[0044] Convenient charging: The power module adopts wireless charging mode to charge, which means it can be charged while performing tasks, that is, "dynamic charging" mode, to achieve uninterrupted work. There is no need to make special trips to and from charging stations, which greatly improves work flexibility and efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is a schematic structural diagram of a luggage tractor with a hand-eye integrated robotic arm according to an embodiment of the present invention;

[0046] Figure 2 This is a schematic top view of the structure of a luggage tractor with a hand-eye integrated robotic arm according to one embodiment of the present invention;

[0047] Figure 3 This is a schematic diagram of the arriving baggage sorting structure of a baggage tractor with a hand-eye integrated robotic arm according to one embodiment of the present invention;

[0048] Figure 4 This is a schematic diagram of the outbound baggage sorting structure of a baggage tractor with a hand-eye integrated robotic arm according to one embodiment of the present invention;

[0049] Figure 5 This is a schematic structural diagram of the dynamic charging area of ​​a luggage tractor with a hand-eye integrated robotic arm according to an embodiment of the present invention.

[0050] Among them, 1 is the tractor; 2 is the robotic arm; 3 is the monitoring module; and 4 is the grasping unit. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solutions and advantages of the implementation of this application clearer, the technical solutions in the embodiments of this application will be described in more detail below in conjunction with the drawings in the embodiments of this application. In the drawings, the same or similar reference numerals represent the same or similar elements or elements with the same or similar functions. The described embodiments are part of the embodiments of this application, not all of the embodiments. The embodiments described below with reference to the drawings are exemplary and are intended to be used to explain this application, and should not be understood as limitations on this application. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0052] Figure 1This is a schematic structural diagram of a luggage tractor with a hand-eye integrated robotic arm according to an embodiment of the present invention. The luggage tractor with a hand-eye integrated robotic arm comprises:

[0053] Tractor 1;

[0054] A mechanical arm 2, the mechanical arm 2 is placed on the tractor 1, and the mechanical arm 2 rotates on the tractor 1;

[0055] Monitoring module 3, which monitors the current position of the luggage ahead, establishes three-dimensional spatial coordinates, and sends the coordinates of the luggage ahead to the control module;

[0056] A control module receives the coordinates of the baggage ahead, calculates the size of the baggage ahead based on the coordinates, identifies the baggage type using a convolutional neural network, and issues control instructions corresponding to a preset baggage type threshold based on a grabbing strategy plan. The control instructions include grabbing instructions and unloading instructions.

[0057] An automatic operation module, which is placed on the robotic arm 2 and includes a grabbing unit 4. The grabbing unit 4 receives the grabbing instruction and the unloading instruction, and grabs and puts down the current luggage according to the grabbing instruction and the unloading instruction;

[0058] A power supply module is installed on the tractor 1 and is wirelessly connected to a ground power supply unit.

[0059] Specifically, the power module is wirelessly connected to the ground power supply unit to form a dynamic wireless charging component, which mainly includes:

[0060] The ground terminal is set as Figure 5 The dynamic charging zone shown in the figure has a transmitting coil, inverter and control unit installed underneath. The transmitting coil is embedded in the copper coil array of the road to provide segmented power; the inverter can convert grid power into high-frequency alternating current; the control unit can detect the vehicle position in real time to activate the corresponding coil end.

[0061] The vehicle side is equipped with an on-board receiving coil, a rectifier and a battery management unit. The on-board receiving coil is installed under the chassis of the luggage tractor with a hand-eye integrated robotic arm and can resonantly couple with the transmitting coil on the ground side; the rectifier converts the received high-frequency alternating current into direct current, which is used directly for power supply or charging the battery; the battery management unit can adjust the charging power to avoid overcharging.

[0062] The charging process specifically includes:

[0063] When the battery level of the luggage tractor is less than 40%, the sensor built into the control unit detects the vehicle's position and activates the coil on the corresponding road section.

[0064] The transmitting coil and the on-board receiving coil transmit energy through the resonant magnetic field.

[0065] The electrical energy directly drives the motor or is stored in the battery, and the vehicle does not need to slow down.

[0066] In one embodiment, the monitoring module 3 includes a three-dimensional camera, a laser radar, an inertial measurement unit, a control computing platform, and a calibration unit;

[0067] The three-dimensional camera acquires the depth information and surface features of the luggage by emitting and receiving infrared or laser signals, and generates a depth image and a color image of the luggage;

[0068] The laser radar generates three-dimensional point cloud data of the luggage and the surrounding environment by emitting a laser beam and measuring the return time of the reflected light;

[0069] The inertial measurement unit is used to measure the acceleration, angular velocity and posture information of the robotic arm 2 and compensate for the jitter and deviation during the movement of the robotic arm 2;

[0070] The control computing platform receives the depth image, color image, and three-dimensional point cloud data generated by the three-dimensional camera and the laser radar, and establishes three-dimensional space coordinates through a sensor fusion algorithm and a three-dimensional space coordinate calculation program;

[0071] The calibration unit is used to calibrate the three-dimensional camera and lidar to ensure that their data are accurately aligned in a unified coordinate system.

[0072] In one embodiment, the process of establishing three-dimensional space coordinates includes:

[0073] Data acquisition: A 3D camera scans the baggage storage area to obtain depth and color images of the baggage. A LiDAR scans the baggage and surrounding environment to obtain point cloud data. The collected depth, color, and point cloud data are pre-processed, and the data from the 3D camera, LiDAR, and inertial measurement unit are converted into a unified coordinate system, where the coordinate system is referenced by the baggage tractor's body coordinate system. The features in the depth image, color image, and point cloud data are fused to generate a comprehensive 3D feature description.

[0074] 3D Coordinate Calculation: This calculation is based on the depth image. For each pixel in the depth image, its coordinates in 3D space are calculated based on the camera's intrinsic parameters and depth value. This calculation is based on point cloud data. For each point in the point cloud data, the coordinates of the baggage in front of you in 3D space are calculated based on the LiDAR's coordinate system and scanning angle.

[0075] Coordinate fusion: The 3D coordinates calculated from the depth image and point cloud data are fused to generate more accurate 3D coordinates of the luggage.

[0076] In one embodiment, before establishing the three-dimensional space coordinates, the sensor is initialized and calibrated, including:

[0077] Turn on the 3D camera and run the self-test program to check whether the 3D camera is working properly. Turn on the LiDAR and run the self-test program to check whether the laser transmitter and receiver modules are working properly. Turn on the inertial measurement unit and run the self-test program to check whether the accelerometer and gyroscope modules are working properly.

[0078] The calibration plate is placed in the field of view of the 3D camera. The calibration algorithm is run by controlling the computing platform to calculate the rotation matrix and translation vector of the 3D camera. The zero bias and scale factor of the inertial measurement unit are calculated by collecting data in a static state.

[0079] In one embodiment, the control module uses a convolutional neural network to train depth patterns and color images of multiple luggage to automatically identify and classify different types of luggage;

[0080] The luggage types include suitcases, soft bags, and cartons.

[0081] In one embodiment, the crawling strategy planning specifically includes:

[0082] When the luggage type is a suitcase, the grabbing unit 4 grabs it from both sides in parallel when approaching, keeps it horizontal after grabbing, and gently places it to the target location;

[0083] When the luggage is a soft bag, the grabbing unit 4 grabs it vertically from above, keeps it balanced, and gently places it to the target location.

[0084] When the luggage type is a carton, the grabbing unit 4 grabs it from the side, determines the corrugation direction of the carton and then grabs it, keeps the balance after grabbing it, and gently places it to the target location.

[0085] Furthermore, when the luggage type is a soft bag, the robotic arm 2 grabs it vertically from above, and a vacuum suction cup or soft gripper can be used to assist in grabbing.

[0086] In one embodiment, the gripping unit 4 includes a metamorphic manipulator, and the morphology of the metamorphic manipulator includes a multi-finger metamorphic manipulator and a deformable manipulator;

[0087] The multi-finger metamorphic manipulator includes multiple fingers, the number of which is adjusted according to the type and size of the luggage to be grasped. When the type and size of the luggage exceeds the set threshold of the multi-finger metamorphic manipulator, the fingers are spread out to increase the stability of the grasping. When the type and size of the luggage are below the set threshold of the multi-finger metamorphic manipulator, the fingers are retracted.

[0088] The deformable manipulator adjusts the length, angle and spacing of the manipulator fingers. When the type and size of the luggage exceeds the set threshold of the deformable manipulator, the length of the fingers is extended to grab luggage of different types and sizes.

[0089] Specifically, a certain number of fingers may be fixedly set within the threshold of the multi-finger metamorphic manipulator. When the threshold is exceeded, the number of fingers needs to be increased. When the threshold is lower than the threshold, the number of fingers needs to be reduced.

[0090] In one embodiment, the metamorphic manipulator is further provided with a sensor unit, which is provided with a gripping force threshold. The sensor unit monitors the gripping force in real time during the gripping process. When the gripping force threshold is exceeded, the PID control algorithm is dynamically adjusted to prevent damage to the luggage.

[0091] After the metamorphic manipulator grabs the bag in front of it, the monitoring module 3 reconfirms the type and size of the baggage it has grabbed, and determines whether it is the same as the originally determined type and size of the baggage. If not, the grabbing strategy is adjusted in real time.

[0092] LiDAR and depth cameras monitor the surrounding environment in real time and dynamically adjust the grasping path to avoid collisions.

[0093] Furthermore, when the robotic arm 2 performs an operation, it dynamically plans the optimal grasping path based on the luggage's position, posture, and surrounding environment, ensuring that the robotic arm 2 avoids obstacles and achieves efficient and stable grasping. The robotic arm 2 performs actions based on the planned path and grasping strategy, adjusting the joint angle and position in real time to ensure accurate grasping and placement. The robotic arm 2 is equipped with an adaptive control system, which can optimize the grasping posture and strength in real time for luggage of different weights and shapes, thereby improving the grasping success rate.

[0094] In one embodiment, the grasping instruction includes: grasping direction, grasping form, number of manipulator fingers, finger length, finger spacing, and finger grasping angle;

[0095] Unloading instructions include:

[0096] The control module sets the multi-finger metamorphic manipulator threshold and the deformable manipulator threshold in various intervals according to the luggage type and size. The multi-finger metamorphic manipulator threshold in various intervals is set with the corresponding number of manipulator fingers, and the deformable manipulator threshold in various intervals is set with the corresponding grasping form, finger length and finger spacing.

[0097] Specifically, the grasping methods are: Parallel grasping: For hard suitcases, the manipulator can grasp them in parallel from both sides to ensure the stability and reliability of the grasping. Vertical grasping: For soft bags or cartons, the manipulator can grasp them vertically from above, using vacuum suction cups or soft grippers to avoid squeezing or deformation of the luggage. Side grasping: For some luggage with special shapes, the manipulator can grasp them from the side to adapt to the shape and position of the luggage. Multi-point grasping: For larger luggage, the manipulator can use multiple grasping points at the same time to increase the stability and safety of the grasping. Adaptive grasping: The manipulator can automatically adjust the grasping force and grasping angle according to the shape, size and weight of the luggage to achieve the best grasping effect.

[0098] In one embodiment, when the grabbing unit 4 is grabbing, it first determines whether the luggage in front meets the conditions for automatic grabbing, including: checking whether there are obstacles in the surrounding environment. If there are obstacles, the obstacles need to be cleared first.

[0099] The grabbing capacity threshold of the grabbing unit 4 can be set. When the shape and size of the luggage detected by the monitoring module 3 exceeds the grabbing capacity threshold of the grabbing unit 4, manual grabbing is requested;

[0100] Check whether the surface characteristics of the luggage meet the requirements for grabbing. For example, liquid objects are not suitable for grabbing.

[0101] Specifically, shape recognition uses a 3D camera to capture a 3D model of the luggage and analyze its shape characteristics. For example, hard luggage typically has a regular rectangular shape, while soft luggage may have an irregular shape. Dimension measurement uses depth images and point cloud data to calculate the length, width, and height of the luggage. If the dimensions of the luggage exceed the gripping capacity threshold of the gripping unit 4 (for example, too large or too small), the luggage does not meet the gripping conditions.

[0102] Position detection: The 3D camera and LiDAR are used to determine the luggage's coordinate position in 3D space. If the luggage is beyond the reach of the robotic arm (for example, too far or too high), it does not meet the grasping conditions. Posture detection: The luggage's placement is analyzed, such as whether it is tilted, inverted, or partially obscured. If the luggage's posture is unfavorable for grasping by the robotic arm (for example, if it is inverted or partially obscured by other luggage), it does not meet the grasping conditions.

[0103] Surface Material: The vision system analyzes the surface material of luggage, such as hard plastic, soft fabric, or cardboard. Depending on the material, the appropriate gripping method must be selected (e.g., a gripper grips hard luggage, a vacuum cup grips soft luggage). Surface Wrinkles and Elasticity: Soft luggage may have wrinkles or elastic deformation. Excessive wrinkles or elasticity can lead to unstable gripping.

[0104] Obstacle Detection: LiDAR and depth cameras monitor the baggage's surroundings in real time to detect obstacles (such as other baggage, equipment, or people). Obstacles in the grasping path of Robot Arm 2 could create a collision risk. Space Constraints: Checks whether the available space around the baggage is sufficient for Robot Arm 2 to operate. If the space is too narrow, Robot Arm 2 may not be able to extend or rotate properly.

[0105] Pickup point selection: Choose the appropriate pickup point based on the shape, size and material of the luggage.

[0106] Pickup method selection: Select the appropriate pickup method based on the type of luggage. If the pickup method is not suitable for the type of luggage, it does not meet the pickup conditions.

[0107] The workflow of the luggage tractor with the hand-eye integrated robotic arm 2 of the present invention includes:

[0108] Power on and self-test the 3D camera, lidar, and inertial measurement unit. Run the calibration algorithm to ensure that the sensor data is accurately aligned in a unified coordinate system.

[0109] The 3D camera and LiDAR system acquires depth images, color images, and point cloud data of the luggage. Denoising, filtering, and feature extraction are performed on the data to generate a comprehensive 3D feature description.

[0110] Calculate the three-dimensional coordinates of the baggage in front, including position, posture, and size information.

[0111] Check the baggage's shape, size, position, posture, and surface features to see if they meet the requirements for handling. Check the surrounding environment for obstacles or space restrictions. Check for feasible grasping points and methods.

[0112] During the grasping process, the grasping force and position deviation are monitored in real time, and the grasping strategy is adjusted dynamically. If an abnormal situation occurs during the grasping process (such as the appearance of an obstacle or grasping failure), the grasping is paused and replanned.

[0113] The present invention has the following beneficial effects:

[0114] High efficiency: The present invention can automatically determine whether the current luggage is within the grabbing range of the grabbing unit, thereby improving the efficiency and accuracy of luggage handling, reducing manual labor intensity and luggage damage, and reducing safety hazards.

[0115] High Flexibility: Compared to fixed-arm sorting devices, the robotic arm, including multi-fingered metamorphic and deformable arms, can flexibly handle various types of baggage sorting and handling tasks. Furthermore, the monitoring module 3 is integrated into the robotic arm, resulting in a highly integrated system and simplified maintenance. Furthermore, the same device can handle both outbound loading and inbound unloading, improving equipment utilization.

[0116] Convenient Charging: The power module adopts wireless charging mode, which allows charging while performing tasks, that is, "dynamic charging" mode, to achieve uninterrupted work. There is no need to make special trips to and from charging stations, which greatly improves work flexibility and efficiency. It also avoids the risks of poor contact, short circuit and fire caused by traditional wired charging, and fundamentally enhances the safety of the working environment.

[0117] In the description of this application, it should be noted that the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the exemplary embodiments according to this application. For ease of description, the dimensions of the various parts shown in the drawings are not drawn according to the actual proportional relationship. Technologies, methods and equipment known to ordinary technicians in the relevant fields may not be discussed in detail, but where appropriate, the technologies, methods and equipment should be considered as part of the authorization specification. In all examples shown and discussed here, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of the exemplary embodiments may have different values. It should be noted that similar numbers and letters represent similar items in the following figures, so once an item is defined in one figure, it does not need to be further discussed in subsequent figures.

[0118] It should be noted that, in this application, the terms "comprises", "includes" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. It should also be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the opposite order according to the functions involved. For example, the described method may be performed in an order different from that described, and various steps may be added, omitted, or combined. In addition, features described with reference to certain examples may be combined in other examples.

[0119] In addition, it should be noted that, unless otherwise clearly stipulated and limited, words such as "connect" and "drive" used in the description of this application should be understood in a broad sense, which can be direct, through an intermediate medium, or the relationship between two elements. Technical personnel in the field can understand their specific meanings in this application based on specific circumstances.

[0120] The above embodiments are provided for persons familiar with the art to implement or use the present application. Personnel familiar with the art may make various modifications or changes to the above embodiments without departing from the application concept of the present application. Therefore, the scope of protection of the present application is not limited to the above embodiments, but should be the maximum scope of the innovative features mentioned in the claims.

Claims

1. The luggage tractor with hand-eye integrated robotic arm is characterized by: The luggage tractor with a hand-eye integrated robotic arm comprises: tractor; a mechanical arm, the mechanical arm being placed on the tractor and rotating on the tractor; A monitoring module, which monitors the current position of the luggage ahead, establishes three-dimensional spatial coordinates, and sends the coordinates of the luggage ahead to the control module; The control module receives the coordinates of the preceding baggage, calculates the size of the preceding baggage based on the coordinates, identifies the baggage type using a convolutional neural network, and issues control instructions corresponding to a preset baggage type threshold based on a grasping strategy. The control instructions include grasping instructions and unloading instructions. The control module uses a convolutional neural network to train depth graphs and color images of multiple bags to automatically identify and classify different types of baggage, including suitcases, soft bags, and cartons. The grasping strategy planning specifically includes: When the luggage type is a suitcase, the grabbing unit grabs it from both sides in parallel when approaching, keeps it horizontal after grabbing, and gently places it to the target location; When the luggage type is a soft bag, the grabbing unit grabs it vertically from above, keeps it balanced after grabbing, and gently places it to the target location; When the baggage is a carton, the gripping unit grabs it from the side, determines the direction of the corrugation of the carton, and then grabs it. After grabbing, it maintains balance and gently places it to the target location. An automatic operation module is placed on the robotic arm. The automatic operation module includes a grasping unit. The grasping unit receives the grasping instruction and the unloading instruction, and grasps and puts down the current luggage according to the grasping instruction and the unloading instruction. The grasping unit includes a metamorphic manipulator. The metamorphic manipulator includes a multi-finger metamorphic manipulator and a deformable manipulator. The multi-finger metamorphic manipulator includes multiple fingers, and the number of fingers is adjusted according to the type and size of the luggage to be grasped. When the type and size of the luggage exceeds the set threshold of the multi-finger metamorphic manipulator, the fingers are expanded to increase the stability of the grasping. When the type and size of the luggage is below the set threshold of the multi-finger metamorphic manipulator, the fingers are retracted. The deformable manipulator adjusts the length, angle and spacing of the manipulator fingers. When the type and size of the luggage exceeds the set threshold of the deformable manipulator, the finger length is extended to grasp luggage of different types and sizes. A power supply module is installed on the tractor and is wirelessly connected to a ground power supply device.

2. The luggage tractor with a hand-eye integrated robotic arm according to claim 1, characterized in that: The monitoring module includes a three-dimensional camera, a laser radar, an inertial measurement unit, a control computing platform and a calibration unit; The three-dimensional camera acquires the depth information and surface features of the luggage by emitting and receiving infrared or laser signals, and generates a depth image and a color image of the luggage; The laser radar generates three-dimensional point cloud data of the luggage and the surrounding environment by emitting a laser beam and measuring the return time of the reflected light; The inertial measurement unit is used to measure the velocity, angular velocity and posture information of the robotic arm and compensate for jitter and deviation during the movement of the robotic arm; The control computing platform receives the depth image, color image, and three-dimensional point cloud data generated by the three-dimensional camera and the laser radar, and establishes three-dimensional space coordinates through a sensor fusion algorithm and a three-dimensional space coordinate calculation program; The calibration unit is used to calibrate the three-dimensional camera and lidar to ensure that their data are accurately aligned in a unified coordinate system.

3. The luggage tractor with a hand-eye integrated robotic arm according to claim 2, characterized in that: The process of establishing three-dimensional space coordinates includes: Data acquisition: A 3D camera scans the baggage storage area to obtain depth and color images of the baggage. A LiDAR scans the baggage and surrounding environment to obtain point cloud data. The collected depth, color, and point cloud data are pre-processed, and the data from the 3D camera, LiDAR, and inertial measurement unit are converted into a unified coordinate system, where the coordinate system is referenced by the baggage tractor's body coordinate system. The features in the depth image, color image, and point cloud data are fused to generate a comprehensive 3D feature description. 3D space coordinate calculation: For each pixel in the depth image, the coordinates in 3D space are calculated based on the camera's intrinsic parameters and depth value. For each point in the point cloud data, the coordinates of the baggage in front are calculated based on the LiDAR's coordinate system and scanning angle. Coordinate fusion: The 3D coordinates calculated from the depth image and point cloud data are fused to generate more accurate 3D coordinates of the luggage.

4. The luggage tractor with a hand-eye integrated robotic arm according to claim 2, characterized in that: Before establishing the 3D space coordinates, the sensor is initialized and calibrated, including: Turn on the 3D camera and run the self-test program to check whether the 3D camera is working properly. Turn on the LiDAR and run the self-test program to check whether the laser transmitter and receiver modules are working properly. Turn on the inertial measurement unit and run the self-test program to check whether the accelerometer and gyroscope modules are working properly. The calibration plate is placed in the field of view of the 3D camera. The calibration algorithm is run by controlling the computing platform to calculate the rotation matrix and translation vector of the 3D camera. The zero bias and scale factor of the inertial measurement unit are calculated by collecting data in a static state.

5. The luggage tractor with a hand-eye integrated robotic arm according to claim 1, characterized in that: The metamorphic manipulator is also provided with a sensor unit, which is provided with a gripping force threshold. The sensor unit monitors the gripping force in real time during the gripping process. When the gripping force exceeds the set gripping force threshold, the PID control algorithm is dynamically adjusted to prevent damage to the luggage. After the metamorphic manipulator grabs the bag in front of it, the monitoring module reconfirms the type and size of the bag it has grabbed, and determines whether it is the same as the originally determined baggage type and size. If not, the grabbing strategy is adjusted in real time. LiDAR and depth cameras monitor the surrounding environment in real time and dynamically adjust the grasping path to avoid collisions.

6. The luggage tractor with a hand-eye integrated robotic arm according to claim 1, characterized in that: The grasping instructions include: grasping direction, grasping shape, number of manipulator fingers, finger length, finger spacing, and finger grasping angle; The control module sets the multi-finger metamorphic manipulator threshold and the deformable manipulator threshold in various intervals according to the luggage type and size. The multi-finger metamorphic manipulator threshold in various intervals is set with the corresponding number of manipulator fingers, and the deformable manipulator threshold in various intervals is set with the corresponding grasping form, finger length and finger spacing.

7. The luggage tractor with a hand-eye integrated robotic arm according to claim 1, characterized in that: When the grabbing unit grabs, it first determines whether the baggage in front meets the conditions for automatic grabbing, including: checking whether there are obstacles in the surrounding environment. If there are obstacles, they need to be cleared first.

Images