Robot tilt-push luggage gapless stacking control method, system and robot

Abstract

The application discloses a kind of robot tilt pushing type luggage gapless stacking control method and system based on 3D vision and force feedback, belong to robot intelligent stacking technical field.Method includes: through 3D vision module collection after falling position luggage image data and solution pose, calculate the minimum gap distance D of luggage and adjacent box and stacking trolley side wall;According to D and the distance from side wall adaptive calculation jaw tilt angle θ;Adjust robot end pose to make jaw form one side tilt pushing posture;Drive jaw pushing and real-time monitoring pushing displacement S and contact force F;When S reaches target gap value or F reaches preset maximum safety force, stop pushing;Finally, jaw resets and retreats back.The application realizes dense stacking without gap, collision and deformation by visual perception gap, intelligent decision tilt angle, one side tilt pushing and force-position double control, improves space utilization and transfer safety.

Description

Technical Field

[0001] This invention relates to the technical field of automated palletizing, and in particular to a method, system, and robot for seamless palletizing of tilting and pushing suitcases based on 3D vision and force feedback. Background Technology

[0002] Robotic palletizing is widely used in automated warehousing, airport baggage handling, and logistics sorting. Current technology typically involves a robot gripping a suitcase, placing it in the target location, and then releasing the gripper. However, this approach has several drawbacks: After placement, due to inertia, positioning errors, and suitcase movement, significant gaps can form between suitcases and between the suitcase and the cart's sidewalls. This results in loose stacks, low space utilization, and a high risk of tipping over during transport. When a suitcase approaches the cart's sidewall, the robot's gripper may collide with the wall, causing damage or shutdown. Furthermore, the lack of real-time 3D vision monitoring prevents adaptive adjustment of the pushing posture based on actual gaps and suitcase positions. The absence of feedback-based closed-loop control leads to uncontrollable pushing force, potentially damaging the suitcase's exterior. The lack of intelligent termination conditions for the pushing action also results in incomplete or excessive pushing.

[0003] In view of this, the present invention addresses the many shortcomings and inconveniences caused by the imperfections of existing robotic baggage palletizing methods, and has been developed through in-depth research, improvement and trial production. Summary of the Invention

[0004] The purpose of this invention is to provide a method, system, and robot for controlling the seamless stacking of tilting and pushing suitcases using a robot based on 3D vision and force feedback, in order to solve the problems in the prior art such as excessive gaps between suitcases and between suitcases and the trolley wall after the suitcases are placed, easy collision of the gripper with the trolley wall when the suitcases are close to the side wall of the trolley, uncontrollable pushing force, and easy damage to the suitcases.

[0005] To achieve the above objectives, the solution of the present invention is: A method for controlling the seamless stacking of tilting and pushing suitcases by a robot includes the following steps: S1. Collect image data of the suitcase after it is placed using the 3D vision module, calculate the actual pose of the suitcase, and calculate the minimum gap distance D between the suitcase and adjacent boxes and between the suitcase and the side wall of the stacking trolley. S2. Based on the minimum gap distance D and the distance between the suitcase and the side wall of the palletizing trolley, adaptively calculate the tilt angle θ of the gripper; S3. Adjust the robot end-effector pose according to the tilt angle θ so that the gripper forms a one-sided tilting and pushing posture. The one-sided tilting and pushing posture is that one side of the gripper is close to the side of the suitcase and the other side is facing the side wall of the palletizing trolley while maintaining a safe distance. S4. Control the robot to drive the gripper to push the side of the suitcase in a unilateral tilting and pushing posture, and obtain the pushing displacement S and contact force F in real time during the pushing process. S5. During the pushing process, it is determined in real time whether the pushing displacement S has reached the target gap value, and whether the contact force F has reached the preset maximum safety force Fmax. When the pushing displacement S reaches the target gap value, or the contact force F reaches the preset maximum safety force Fmax, the robot is controlled to stop pushing. S6. Control the gripper to return to its initial posture and drive the robot to retreat along a safe path to complete gapless palletizing.

[0006] Furthermore, in step S1, the 3D vision module simultaneously acquires RGB images and 3D point clouds, extracts the box outline through semantic segmentation, performs denoising, segmentation and clustering processing on the 3D point cloud, fits the box plane, and calculates the six-degree-of-freedom pose of the suitcase. Based on the deviation between the calculated actual pose and the target pose, positional deviation compensation is performed to correct the calculated value of the minimum gap distance D.

[0007] Furthermore, in step S2, the tilt angle θ is adaptively calculated according to the palletizing space partitioning strategy; the palletizing space partitioning strategy includes a central area and an edge high-risk area. When the suitcase is located in the edge high-risk area, the tilt angle θ is greater than the tilt angle when the suitcase is located in the central area; the tilt angle θ is negatively correlated with the distance between the suitcase and the side wall of the palletizing cart. The smaller the distance between the suitcase and the side wall of the palletizing cart, the larger the tilt angle θ.

[0008] Furthermore, in step S3, the gripper rotates around the X-axis of the tool coordinate system at the tilt angle θ, and the robot end posture is updated by the rotation matrix so that the pushing surface of the gripper fits against the side of the suitcase, and the side of the gripper facing the side wall of the palletizing trolley maintains a safety distance G greater than zero between it and the side wall.

[0009] Furthermore, in step S4, the robot drives the gripper to gently push the side of the suitcase at a preset low speed.

[0010] Furthermore, in step S5, during the pushing process, if the contact force F does not reach the preset maximum safety force Fmax, the pushing displacement S is controlled to not exceed the minimum gap distance D; if the pushing displacement S does not reach the target gap value, the contact force F is controlled to not exceed the preset maximum safety force Fmax; when the contact force F approaches the preset maximum safety force Fmax, the pushing speed is reduced; when the pushing displacement S approaches the target gap value, the pushing speed is reduced; the target gap value is zero or a preset dense gap value.

[0011] A robot tilting and pushing type gapless palletizing control system for suitcases, comprising: The 3D vision module is used to collect image data of the suitcase after it has been placed in its position. The pose calculation module is used to calculate the actual pose of the suitcase based on the image data, and to calculate the minimum gap distance D between the suitcase and adjacent suitcases and between the suitcase and the side wall of the stacking trolley. The tilt angle decision module is used to adaptively calculate the tilt angle θ of the gripper based on the minimum gap distance D and the distance between the luggage box and the side wall of the palletizing trolley. The pose transformation module is used to adjust the robot end pose according to the tilt angle θ, so that the gripper forms a unilateral tilting and pushing posture. The unilateral tilting and pushing posture is that one side of the gripper is close to the side of the suitcase and the other side is facing the side wall of the palletizing trolley while maintaining a safe distance. The pushing control module is used to control the robot to drive the gripper to push the side of the suitcase in the single-sided tilting pushing posture, and to acquire the pushing displacement S and contact force F in real time during the pushing process. The dual threshold protection module is used to determine in real time whether the pushing displacement S reaches the target gap value and whether the contact force F reaches the preset maximum safety force Fmax during the pushing process; when the pushing displacement S reaches the target gap value or the contact force F reaches the preset maximum safety force Fmax, the robot is controlled to stop pushing. The reset module is used to control the gripper to return to its initial posture and drive the robot to retreat along a safe path; The main controller is electrically connected to the 3D vision module, pose calculation module, tilt decision module, pose transformation module, push control module, dual threshold protection module and reset module.

[0012] Furthermore, the 3D vision module includes an RGB camera and a depth camera for simultaneously acquiring RGB images and 3D point clouds; the pose calculation module includes a semantic segmentation unit, a point cloud processing unit, and a plane fitting unit. The tilt angle decision module includes a spatial partitioning unit, which is used to divide the palletizing space into a central area and an edge high-risk area, and outputs a differentiated tilt angle θ for the central area and the edge high-risk area. The pose transformation module includes a rotation matrix calculation unit, which is used to generate a rotation matrix about the X-axis of the tool coordinate system based on the tilt angle θ, and update the robot end effector pose. The dual threshold protection module includes a coupling control unit, which is used to control the pushing displacement S to not exceed the minimum gap distance D when the contact force F does not reach the preset maximum safety force Fmax, and to control the contact force F to not exceed the preset maximum safety force Fmax when the pushing displacement S does not reach the target gap value. The pushing control module includes a force feedback sensor and a displacement encoder. The force feedback sensor is located between the gripper and the robot end effector to collect the contact force F in real time, and the displacement encoder is used to collect the pushing displacement S in real time.

[0013] The robot tilting and pushing luggage palletizing control system of the present invention further includes a positioning deviation compensation module, which is used to correct the minimum gap distance D according to the deviation between the actual position and the target position. The dual threshold protection module also includes a speed control unit, which is used to reduce the pushing speed when the contact force F approaches the preset maximum safety force Fmax or when the pushing displacement S approaches the target gap value.

[0014] A robot, characterized in that it comprises a six-degree-of-freedom robot body, a gripper with a force feedback sensor, a 3D vision module, and a robot tilting and pushing luggage seamless palletizing control method as described in any one of claims 1 to 6, or a robot tilting and pushing luggage seamless palletizing control system as described in any one of claims 7 to 9.

[0015] By adopting the above solution, the present invention has the following advantages over the prior art: This invention uses a 3D vision module in step S1 to collect real-time image data of the suitcase and calculate its pose, while simultaneously calculating the minimum gap distance D. This enables precise perception and compensation of the placement deviation, solving the problems of lack of real-time gap detection and blind pushing in existing technologies, making the pushing action more intelligent and precise.

[0016] This invention solves the technical problem of easy collision when the gripper pushes the suitcase flat near the side wall of the palletizing trolley by adaptively calculating the gripper tilt angle θ based on the minimum gap distance D and the distance between the suitcase and the side wall of the palletizing trolley in step S2, and forming a single-sided tilting and pushing posture in step S3, so that one side of the gripper is close to the side of the suitcase and the other side faces the side wall of the palletizing trolley while maintaining a safe distance. The tilt angle is automatically increased when the suitcase is close to the side wall, which effectively avoids collision, and the pushing is stable, without deviation or overturning of the suitcase.

[0017] This invention acquires the pushing displacement S and contact force F in real time during the pushing process through steps S4 and S5, and uses a dual-threshold joint triggering mechanism to control the termination of the pushing process. This solves the problems of uncontrollable pushing force, easy damage to the suitcase, and lack of intelligent termination conditions in the prior art. By using force feedback and displacement dual-threshold closed-loop control, it achieves flexible pushing, protects the suitcase from damage, and ensures that the gap is fully eliminated.

[0018] Through the coordinated operation of the above steps, this invention achieves dense stacking without gaps, collisions, or deformation, resulting in neat and stable stacking patterns. This significantly improves space utilization and transportation safety, and is highly automated, requiring no manual intervention. It is suitable for automated stacking of luggage in various scenarios such as airports, logistics, and warehousing. Attached Figure Description

[0019] Figure 1 This is a flowchart of the present invention.

[0020] Figure 2 This is a schematic diagram of the structure of the robot gripper of the present invention.

[0021] Figure 3 This is a schematic diagram of the inclined pushing action of the gripper in this invention. Detailed Implementation

[0022] To further explain the technical solution of the present invention, the present invention will be described in detail below through specific embodiments.

[0023] like Figures 1 to 3 As shown, this invention discloses a gapless palletizing control system for a robot tilting and pushing suitcase based on 3D vision and force feedback, including a gripper, a force feedback sensor, a six-degree-of-freedom robot, a 3D vision module, a main controller, and a palletizing cart. The 3D vision module includes an RGB camera and a depth camera. Figure 2 As shown, the gripper uses an integrated suction and clamping fixture, with a force feedback sensor mounted on the gripper to collect the contact force F between the gripper and the suitcase in real time. The main controller is electrically connected to the six-degree-of-freedom robot, the 3D vision module, and the force feedback sensor to coordinate the various modules to complete the palletizing control. The palletizing cart has side walls, and the suitcase rests on the carrying surface of the palletizing cart.

[0024] like Figure 1 The present invention also discloses a gapless palletizing control method for robot tilting and pushing suitcases based on 3D vision and force feedback, the specific workflow of which is as follows: S1. Multimodal data acquisition and box pose calculation.

[0025] After the suitcase is placed at the target position on the palletizing cart by the six-DOF robot, the 3D vision module simultaneously activates the RGB and depth cameras to acquire RGB images and 3D point clouds of the suitcase. The pose calculation module in the main controller performs semantic segmentation on the RGB images to extract the suitcase's outline; simultaneously, it performs denoising, segmentation, and clustering processing on the 3D point cloud, removing background point clouds and retaining the effective point cloud data of the suitcase surface. By fusing the outline information from the RGB images with the 3D point cloud, the suitcase's plane is fitted, and the six-DOF pose of the suitcase is accurately calculated, namely the spatial position coordinates x, y, z, and the rotation angles roll, pitch, and yaw around each axis.

[0026] Based on this, the pose calculation module calculates the minimum distance D1 between the suitcase and the adjacent stacked suitcases, and the minimum distance D2 between the side of the suitcase and the side wall of the stacking trolley, according to the fitted box plane. The smaller value between D1 and D2 is taken as the minimum gap distance D. At the same time, the pose calculation module compares the calculated actual pose with the target pose to obtain the placement deviation, and compensates and corrects the minimum gap distance D according to the placement deviation to obtain an accurate pushing target value.

[0027] S2, Adaptive calculation of gripper tilt angle.

[0028] The tilt angle decision module adaptively calculates the tilt angle θ of the gripper based on the minimum gap distance D obtained in step S1 and the distance between the luggage box and the side wall of the palletizing trolley.

[0029] In this embodiment, the main controller pre-stores a palletizing space partitioning strategy, dividing the palletizing trolley's bearing surface along its width into a central region and a high-risk edge region. The high-risk edge region is a preset width area close to the sidewall, for example, an area less than 150mm from the sidewall; the central region is the middle part of the bearing surface. When the suitcase is located in the high-risk edge region, the tilt angle decision module outputs a larger tilt angle θ, for example, 15° to 30°, giving the grippers greater clearance during pushing; when the suitcase is located in the central region, it outputs a smaller tilt angle θ, for example, 3° to 8°, to ensure smooth pushing.

[0030] The tilt angle θ is negatively correlated with the distance between the suitcase and the side wall; that is, the smaller the distance, the larger the tilt angle θ.

[0031] S3, robot end-effector pose transformation.

[0032] The pose transformation module adjusts the end effector pose of the six-DOF robot based on the tilt angle θ calculated in step S2. Specifically, the pose transformation module uses the X-axis of the tool coordinate system as the rotation axis and updates the robot's end effector pose through a rotation matrix, causing the gripper to assume a unilateral tilting and pushing posture. In this posture, the left gripping arm of the gripper rests against the side of the suitcase, while the right gripping arm faces the side wall of the palletizing cart, maintaining a safety clearance G greater than zero between the right gripping arm and the side wall. The safety clearance G is calculated in real time based on the tilt angle θ and the gripper's geometry to ensure that no collision occurs during the pushing process.

[0033] S4, force-displacement dual closed-loop push control.

[0034] The pushing control module controls the six-DOF robot to drive the gripper to gently push the side of the suitcase at a preset low speed. For example, the preset low speed is set to 5mm / s to 20mm / s to ensure a smooth pushing process and avoid impact. During the pushing process, the force feedback sensor collects the contact force F between the gripper and the suitcase in real time, and the joint encoder of the six-DOF robot or the external displacement sensor collects the pushing displacement S in real time, and feeds the contact force F and the pushing displacement S back to the main controller in real time.

[0035] S5, termination triggered by dual thresholds.

[0036] The dual-threshold protection module monitors the pushing displacement S and contact force F in real time during the pushing process and makes a joint judgment based on the dual thresholds.

[0037] In this embodiment, the dual-threshold protection module incorporates coupled control logic: during the pushing process, if the contact force F does not reach the preset maximum safety force Fmax, the pushing displacement S is controlled to not exceed the minimum gap distance D; if the pushing displacement S does not reach the target gap value, the contact force F is controlled to not exceed the preset maximum safety force Fmax. This coupled logic ensures that the pushing action is always performed under the dual constraints of the upper limit of displacement and the upper limit of force, preventing excessive pushing.

[0038] When the pushing displacement S reaches the target gap value, the dual threshold protection module sends a stop command to the six-degree-of-freedom robot, controlling the robot to stop pushing. The target gap value can be set to zero or a preset density gap value according to the palletizing compaction requirements.

[0039] When the contact force F reaches the preset maximum safety force Fmax, the dual threshold protection module also sends a stop command to prevent damage to the suitcase shell. The preset maximum safety force Fmax is pre-calibrated based on the suitcase's material and structural strength, for example, set to 50N.

[0040] To further improve the flexibility and safety of the pushing action, the dual threshold protection module also includes variable speed control logic: when the contact force F approaches the preset maximum safety force Fmax, or when the pushing displacement S approaches the target gap value, the pushing control module automatically reduces the pushing speed to 30% to 50% of the original speed to achieve flexible buffering and avoid sudden stop impact.

[0041] S6. Pushing completes and resets.

[0042] After the gap is eliminated, the reset module controls the gripper to return to its initial horizontal posture, and the six-degree-of-freedom robot returns to its initial position along the preset safety path, completing one gapless palletizing process and waiting for the next palletizing instruction.

[0043] This invention forms a complete closed-loop high-precision gap elimination mechanism through the coordinated cooperation of visually perceived gaps, intelligent decision-making tilt angles, unilateral tilting and pushing, and force-position dual control, achieving dense stacking of suitcases without gaps, collisions, or deformation.

[0044] The above embodiments and figures are not intended to limit the product form and style of the present invention. Any appropriate changes or modifications made by those skilled in the art should be considered as not departing from the patent scope of the present invention.

Claims

1. A method for controlling gapless stacking of robot-driven tilting and pushing suitcases, characterized in that, Includes the following steps: S1. Collect image data of the suitcase after it is placed using the 3D vision module, calculate the actual pose of the suitcase, and calculate the minimum gap distance D between the suitcase and adjacent boxes and between the suitcase and the side wall of the stacking trolley. S2. Based on the minimum gap distance D and the distance between the suitcase and the side wall of the palletizing trolley, adaptively calculate the tilt angle θ of the gripper; S3. Adjust the robot end-effector pose according to the tilt angle θ so that the gripper forms a one-sided tilting and pushing posture. The one-sided tilting and pushing posture is that one side of the gripper is close to the side of the suitcase and the other side is facing the side wall of the palletizing trolley while maintaining a safe distance. S4. Control the robot to drive the gripper to push the side of the suitcase in a unilateral tilting and pushing posture, and obtain the pushing displacement S and contact force F in real time during the pushing process. S5. During the pushing process, it is determined in real time whether the pushing displacement S has reached the target gap value, and whether the contact force F has reached the preset maximum safety force Fmax. When the pushing displacement S reaches the target gap value, or the contact force F reaches the preset maximum safety force Fmax, the robot is controlled to stop pushing. S6. Control the gripper to return to its initial posture and drive the robot to retreat along a safe path to complete gapless palletizing.

2. The robot tilting and pushing type gapless palletizing control method for suitcases as described in claim 1, characterized in that: In step S1, the 3D vision module simultaneously acquires RGB images and 3D point clouds, extracts the box outline through semantic segmentation, performs denoising, segmentation and clustering processing on the 3D point cloud, fits the box plane, and calculates the six-degree-of-freedom pose of the suitcase. Based on the deviation between the calculated actual pose and the target pose, positional deviation compensation is performed to correct the calculated value of the minimum gap distance D.

3. The robot tilting and pushing luggage palletizing control method as described in claim 1, characterized in that: In step S2, the tilt angle θ is adaptively calculated according to the palletizing space partitioning strategy. The palletizing space partitioning strategy includes a central area and an edge high-risk area. When the suitcase is located in the edge high-risk area, the tilt angle θ is greater than the tilt angle when the suitcase is located in the central area. The tilt angle θ is negatively correlated with the distance between the suitcase and the side wall of the palletizing cart. The smaller the distance between the suitcase and the side wall of the palletizing cart, the larger the tilt angle θ.

4. The robot tilting and pushing type gapless palletizing control method for suitcases as described in claim 1, characterized in that: In step S3, the gripper rotates around the X-axis of the tool coordinate system at the tilt angle θ, and the robot end posture is updated by the rotation matrix so that the pushing surface of the gripper is in contact with the side of the suitcase, and the side of the gripper facing the side wall of the palletizing trolley maintains a safety distance G greater than zero between it and the side wall.

5. The robot tilting and pushing type gapless palletizing control method for suitcases as described in claim 1, characterized in that: In step S4, the robot drives the gripper to gently push the side of the suitcase at a preset low speed.

6. The robot tilting and pushing type gapless palletizing control method for suitcases as described in claim 1, characterized in that: In step S5, if the contact force F does not reach the preset maximum safety force Fmax during the pushing process, the pushing displacement S is controlled to not exceed the minimum gap distance D. If the pushing displacement S does not reach the target gap value, the contact force F is controlled to not exceed the preset maximum safety force Fmax; When the contact force F approaches the preset maximum safety force Fmax, reduce the pushing speed; When the pushing displacement S approaches the target gap value, reduce the pushing speed; The target gap value is zero or a preset dense gap value.

7. A robot tilting and pushing type gapless palletizing control system for suitcases, characterized in that, include: The 3D vision module is used to collect image data of the suitcase after it has been placed in its position. The pose calculation module is used to calculate the actual pose of the suitcase based on the image data, and to calculate the minimum gap distance D between the suitcase and adjacent suitcases and between the suitcase and the side wall of the stacking trolley. The tilt angle decision module is used to adaptively calculate the tilt angle θ of the gripper based on the minimum gap distance D and the distance between the luggage box and the side wall of the palletizing trolley. The pose transformation module is used to adjust the robot end pose according to the tilt angle θ, so that the gripper forms a unilateral tilting and pushing posture. The unilateral tilting and pushing posture is that one side of the gripper is close to the side of the suitcase and the other side is facing the side wall of the palletizing trolley while maintaining a safe distance. The pushing control module is used to control the robot to drive the gripper to push the side of the suitcase in the single-sided tilting pushing posture, and to acquire the pushing displacement S and contact force F in real time during the pushing process. The dual threshold protection module is used to determine in real time whether the pushing displacement S reaches the target gap value and whether the contact force F reaches the preset maximum safety force Fmax during the pushing process; when the pushing displacement S reaches the target gap value or the contact force F reaches the preset maximum safety force Fmax, the robot is controlled to stop pushing. The reset module is used to control the gripper to return to its initial posture and drive the robot to retreat along a safe path; The main controller is electrically connected to the 3D vision module, pose calculation module, tilt decision module, pose transformation module, push control module, dual threshold protection module and reset module.

8. The robot tilting and pushing type gapless palletizing control system for suitcases as described in claim 7, characterized in that: The 3D vision module includes an RGB camera and a depth camera, used to simultaneously acquire RGB images and 3D point clouds; the pose calculation module includes a semantic segmentation unit, a point cloud processing unit, and a plane fitting unit. The tilt angle decision module includes a spatial partitioning unit, which is used to divide the palletizing space into a central area and an edge high-risk area, and outputs a differentiated tilt angle θ for the central area and the edge high-risk area. The pose transformation module includes a rotation matrix calculation unit, which is used to generate a rotation matrix about the X-axis of the tool coordinate system based on the tilt angle θ, and update the robot end effector pose. The dual threshold protection module includes a coupling control unit, which is used to control the pushing displacement S to not exceed the minimum gap distance D when the contact force F does not reach the preset maximum safety force Fmax, and to control the contact force F to not exceed the preset maximum safety force Fmax when the pushing displacement S does not reach the target gap value. The pushing control module includes a force feedback sensor and a displacement encoder. The force feedback sensor is located between the gripper and the robot end effector to collect the contact force F in real time, and the displacement encoder is used to collect the pushing displacement S in real time.

9. The robot tilting and pushing type gapless palletizing control system for suitcases as described in claim 7, characterized in that: It also includes a positioning deviation compensation module, used to correct the minimum gap distance D based on the deviation between the actual pose and the target pose; The dual threshold protection module also includes a speed control unit, which is used to reduce the pushing speed when the contact force F approaches the preset maximum safety force Fmax or when the pushing displacement S approaches the target gap value.

10. A robot, characterized in that, The system includes a six-degree-of-freedom robot body, a gripper with a force feedback sensor, a 3D vision module, and a robot tilting and pushing luggage seamless palletizing control method as described in any one of claims 1 to 6, or a robot tilting and pushing luggage seamless palletizing control system as described in any one of claims 7 to 9.

Images