Automatic stacking control system based on visual positioning
Through multimodal visual positioning and dynamic path planning, combined with three-axis cooperative robotic arms and adaptive fixtures, the positioning and path planning problems of the existing automatic palletizing system in complex environments are solved, and precise cargo grabbing and stacking are achieved, improving the adaptability and efficiency of the system.
Patent Information
- Application Number
- CN202510492684.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-18
- Publication Date
- 2025-07-04
- Estimated Expiration
- 2045-04-18
AI Technical Summary
When facing complex and diverse on-site working conditions, the existing automatic palletizing system lacks dynamic adjustment capabilities and relies on the positioning and fixed path planning of a single data dimension, resulting in inflexible movement path of the robotic arm and difficult to adapt to environmental changes.
The multi-modal visual positioning module is used to combine RGB cameras and lidar to obtain three-dimensional spatial information, the dynamic path planning module updates the path in real time, the three-axis cooperative robot arm cooperates with the adaptive fixture, and the error correction module compensates for errors in real time, achieving collision-free and optimal energy consumption grab and stacking operations.
It realizes accurate grabbing and placing of goods in complex environments, increases the adaptability and efficiency of the system, reduces the invalid movement of the robotic arm, and ensures the safety and accuracy of operation.
Smart Images

Figure CN120246695A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the technical field of palletizing control systems, and specifically to an automatic palletizing control system based on visual positioning. Background Art
[0002] Currently, most automatic palletizing systems use a preset program to control the operation of the robotic arm, rely on basic sensors (such as a single RGB camera or photoelectric switch) for rough positioning, and complete the stacking task through fixed path planning. Some solutions introduce simple visual recognition technology to achieve goods positioning through edge detection or template matching, but the data dimension is single (such as only relying on two-dimensional images), and the movement path of the robotic arm depends on manual teaching or offline programming, lacking the ability of dynamic adjustment.
[0003] For example, in the Chinese invention patent application with the application number CN202010407384.4, it discloses an automatic palletizing system, including: a robot suitable for grasping logistics packaging boxes; a packaging box conveyor suitable for conveying the logistics packaging boxes to a predetermined unloading position; a two-dimensional camera for identifying the logistics packaging boxes located at the predetermined unloading position to guide the robot to grasp the logistics packaging boxes; a pallet conveyor suitable for conveying the pallet for loading the logistics packaging boxes to a predetermined loading position; and a three-dimensional camera for identifying the pallet located at the predetermined loading position to guide the robot to place the grasped logistics packaging boxes on the pallet. In the present invention, the automatic palletizing system can automatically achieve the palletizing operation of logistics packaging boxes, saving labor costs and greatly improving the palletizing efficiency of logistics packaging boxes.
[0004] In the face of complex and diverse on-site working conditions, the palletizing systems in the prior art obviously cannot meet the user's usage requirements. Therefore, this application proposes an automatic palletizing control system based on visual positioning. Summary of the Invention
[0005] Therefore, this application provides an automatic palletizing control system based on visual positioning to solve the problems existing in the prior art.
[0006] To achieve the above object, this application provides the following technical solutions:
[0007] In a first aspect, an automatic palletizing control system based on visual positioning includes a multi-modal visual positioning module, a model establishment module, a dynamic path planning module, a robotic arm execution module, an error correction module, and a storage module;
[0008] The multi-modal visual positioning module includes an RGB camera, sensors, and a lidar, and is used to obtain the three-dimensional space information of the goods and the palletizing area in real time; the multi-modal visual positioning module is installed on one side of the palletizing area;
[0009] The model establishment module is used to establish the palletizing area, the position of the goods, and the position of the robotic arm in a three-dimensional space;
[0010] The dynamic path planning module is deployed in the path optimization engine of the industrial controller. According to the position of the goods, the target pallet position, and the position of the robotic arm, it generates a collision-free and energy-optimal grasping path and stacking path; the path planning supports online replanning. When environmental obstacles or pallet shape offsets are detected, the motion trajectory is updated in real time;
[0011] The robotic arm execution module includes a three-axis collaborative robotic arm, an adaptive fixture, and a force feedback unit; the three-axis collaborative robotic arm is used to move the robotic arm in the front-back, left-right, and up-down directions. The adaptive fixture supports the quick switching between a vacuum chuck and an electric gripper to adapt to the grasping of goods of different materials; the force feedback unit monitors the clamping force and contact pressure in real time to pick up the goods and avoid deformation of the goods;
[0012] The error correction module compares the real-time data of the multi-modal vision positioning module with the palletizing position on the palletizing area, and dynamically compensates for the robotic arm joint error, the deviation of the goods size, and the environmental vibration interference;
[0013] The storage module is used to store the grasping path and stacking path generated by the dynamic path planning module.
[0014] Preferably, the grasping path and stacking path in the storage module are saved in partitions, the effective grasping path and invalid grasping path in the grasping path are saved in partitions, the effective stacking path and invalid stacking path in the stacking path are saved in partitions, and at the same time, the goods position information, palletizing area position information, and robotic arm position information corresponding to the effective grasping path and effective stacking path are saved.
[0015] Preferably, when the dynamic path planning module plans the grasping path and stacking path of the goods, it compares the goods position information, palletizing area position information, and robotic arm position information collected by the multi-modal vision positioning module with the storage information in the storage module. And when the goods position information, palletizing area position information, and robotic arm position corresponding to the effective grasping path and effective stacking path stored in the storage module are the same as the collected goods position information, palletizing area position information, and robotic arm position, the effective grasping path and / or effective stacking path is directly retrieved.
[0016] Preferably, when storing the invalid grasping path, the invalid grasping path that can successfully grasp the goods after calibration is marked as the invalid grasping path one, and the others are the invalid grasping path two, and the calibration parameters, goods position, and robotic arm position information of the invalid grasping path one are saved;
[0017] When storing the invalid stacking path, the invalid stacking path that can successfully stack goods after calibration is marked as the invalid stacking path one, and the others are the invalid stacking path two. And the calibration parameters, the position of the robotic arm, and the information of the palletizing area position of the invalid stacking path one are saved.
[0018] Preferably, when the error correction module calibrates the robotic arm, it compares the robotic arm position, the goods position, and the palletizing area position information with the robotic arm position, the goods position, and the palletizing area position in the invalid grasping path one and the invalid stacking path one in the storage module. When the compared information is the same, the corresponding calibration parameters are directly retrieved.
[0019] Preferably, it further includes a human-machine interaction module, which can display the position of the palletizing area, the information on whether the palletizing area can receive goods, the position information of the robotic arm, the information on whether the robotic arm is in the working state, and the position of the goods.
[0020] Preferably, it further includes an emergency stop module, which is connected to the robotic arm execution module, and the emergency stop module can make the robotic arm execution module stop running.
[0021] Compared with the prior art, the present application has at least the following beneficial effects:
[0022] When the present system is used, it can obtain the position of the palletizing area and the information on whether the palletizing area can stack new goods, and then the three-axis collaborative robotic arm can move to any position in space, increasing the application range of the present system. Through the dynamic path planning module and the error correction module, the goods can be accurately transported to the position of the palletizing area. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] To more intuitively illustrate the prior art and the present application, the following exemplary drawings are given. It should be understood that the specific shapes and structures shown in the drawings generally should not be regarded as limiting conditions when implementing the present application; for example, those skilled in the art are capable of making routine adjustments or further optimizations to the addition / deletion / attribution division of certain units (components), specific shapes, positional relationships, connection methods, dimensional proportional relationships, etc. based on the technical concept disclosed in the present application and the exemplary drawings.
[0024] Figure 1 It is a module diagram of the automatic palletizing control system based on vision positioning provided by the present application;
[0025] Figure 2 It is a structural schematic diagram of the automatic palletizing control system based on vision positioning provided by the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0026] The following will further elaborate on the present application through specific embodiments in conjunction with the accompanying drawings.
[0027] As Figure 1-2 shown, an automatic palletizing control system based on visual positioning includes a multi-modal visual positioning module, a model establishment module, a dynamic path planning module, a robotic arm execution module, an error correction module, and a storage module;
[0028] The multi-modal visual positioning module includes an RGB camera, sensors, and a lidar, and is used to obtain the three-dimensional spatial information of the goods, the robotic arm, and the palletizing area in real time; the multi-modal visual positioning module is installed on one side of the palletizing area. Since the positions of the goods, the robotic arm, and the palletizing area are different each time, before palletizing the goods, it is necessary to obtain the three-dimensional spatial information of the goods, the robotic arm, and the palletizing area, so as to facilitate the subsequent planning of the grasping path and the stacking path. Moreover, through the combined use of the RGB camera, sensors, and lidar, the positions of the goods, the robotic arm, and the palletizing area can be accurately obtained;
[0029] The model establishment module is used to establish the palletizing area, the positions of the goods, and the position of the robotic arm in the same three-dimensional space, that is, to simulate the real spatial environment, and establish the relative positions of the palletizing area, the positions of the goods, and the robotic arm in the space. By establishing them in the three-dimensional space, when planning the grasping path and the stacking path, a three-dimensional coordinate system can be established based on the three-dimensional space, so that when planning the grasping path and the stacking path, it is more scalarized, ensuring that the grasping path and the stacking path are more accurate;
[0030] The dynamic path planning module is deployed in the path optimization engine of the industrial controller. According to the positions of the goods, the target pallet location, and the position of the robotic arm, it generates a collision-free and energy-optimal grasping path and stacking path. This can not only reduce the ineffective movement of the robotic arm during movement, but also quickly complete the palletizing operation; the path planning supports online replanning. When detecting environmental obstacles or pallet type offsets, in order to ensure the safety of the palletizing operation, the motion trajectory is updated in real time, so as to avoid damage to the robotic arm and the goods when other objects enter the working area during the implementation of this system;
[0031] The robotic arm execution module includes a three-axis collaborative robotic arm, an adaptive fixture, and a force feedback unit; the three-axis collaborative robotic arm is used to move the robotic arm in the front-back, left-right, and up-down directions, cooperate with the dynamic path planning module, and generate a collision-free and energy-optimal grasping path and stacking path, thereby ensuring that the robotic arm can move to any position. The adaptive fixture supports the quick switching between vacuum suction cups and electric grippers, and adapts to the grasping of goods of different materials. For example, when handling hard goods, electric grippers can be used, while for goods with a smooth surface and easy to break, vacuum suction cups are used. The use of vacuum suction cups and electric grippers enables the system to handle different types of goods, increasing the application range of the device. The force feedback unit monitors the clamping force and contact pressure in real time to clamp the goods and prevent the goods from deforming.
[0032] The error correction module compares the real-time data of the multi-modal vision positioning module with the palletizing positions on the palletizing area, and dynamically compensates for the robotic arm joint errors, goods size deviations, and environmental vibration interferences to ensure that the robotic arm can accurately grasp and stack the goods without error.
[0033] The storage module is used to store the grasping path and stacking path generated by the dynamic path planning module.
[0034] When the system is in use, it can obtain the position of the palletizing area and the information on whether new goods can be palletized in the palletizing area. Then, through the three-axis collaborative robotic arm, it can move to any position in space, increasing the application range of the system. Through the dynamic path planning module and the error correction module, the goods can be accurately transported to the position of the palletizing area.
[0035] The grasping path and stacking path in the storage module are saved in partitions. The valid grasping paths and invalid grasping paths in the grasping path are saved in partitions. The valid stacking paths and invalid stacking paths in the stacking path are saved in partitions. Saving the valid grasping paths and valid stacking paths allows the valid grasping paths and valid stacking paths to be directly retrieved and used when subsequent grasping operations and stacking operations occur, so as to speed up the stacking operation, and at the same time, the goods position information, palletizing area position information, and robotic arm position information corresponding to the valid grasping paths and valid stacking paths are saved.
[0036] In order to reduce the calculation of the dynamic path planning module and speed up the route planning speed, the following technical solutions are set: when the dynamic path planning module plans the grasping path and stacking path of the goods, according to the goods position information, palletizing area position information and robotic arm position information collected by the multi-modal vision positioning module, it compares them with the storage information in the storage module. And when the goods position information, palletizing area position information, and robotic arm position information corresponding to the effective grasping path and effective stacking path stored in the storage module are the same as the collected goods position information, palletizing area position information, and robotic arm position, the effective grasping path and / or effective stacking path is directly retrieved. This can effectively improve the efficiency of path generation and avoid the occurrence of invalid calculations.
[0037] When implementing the above solution, when the collected goods position information and the robotic arm position information are consistent with the information in the storage module, the corresponding effective grasping path is retrieved, and the robotic arm is made to grasp according to the effective grasping path; when the collected palletizing area position information and the robotic arm position information are consistent with the information in the storage module, the corresponding effective stacking path is retrieved, and the robotic arm is made to stack the goods according to the effective stacking path.
[0038] In order to reduce the calculation of the error correction module and speed up the reaction time of error calibration, the following technical solutions are set:
[0039] When storing the invalid grasping path, the invalid grasping path that can successfully grasp the goods after calibration is marked as invalid grasping path one, and the others are invalid grasping path two, and the calibration parameters, goods position, and robotic arm position information of the invalid grasping path one are saved;
[0040] When storing the invalid stacking path, the invalid stacking path that can successfully stack the goods after calibration is marked as invalid stacking path one, and the others are invalid stacking path two, and the calibration parameters, robotic arm position, and palletizing area position information of the invalid stacking path one are saved.
[0041] When the error correction module calibrates the robotic arm, it compares the robotic arm position, goods position, and palletizing area position information with the robotic arm position, goods position, and palletizing area position in the invalid grasping path one and invalid stacking path one in the storage module. When the compared information is the same, the corresponding calibration parameters are directly retrieved, and the robotic arm is calibrated according to the retrieved calibration parameters, thus ensuring the accuracy of the grasping and stacking routes.
[0042] It also includes a human-machine interaction module. The human-machine interaction module can display the position of the palletizing area, the information on whether the palletizing area can receive goods, the position information of the robotic arm, the information on whether the robotic arm is in a working state, and the position of the goods. Through the human-machine interaction module, users can conveniently view information such as the operating status of the system and master the situations of the palletizing area, the robotic arm, and the goods. During implementation, the staff can master the operating status (various data) of the system through the human-machine interaction module to master the operating conditions between various components.
[0043] It also includes an emergency stop module. The emergency stop module is connected to the robotic arm execution module. The emergency stop module can cause the robotic arm execution module to stop running. When using the system, in case of dangerous situations (such as when an obstacle appears in front of the robotic arm and the robotic arm is about to collide with the obstacle, or when the goods are damaged and palletizing operations cannot continue, or when the system fails, etc.), the emergency stop module can be used to stop the operation of this system to ensure the safety of this device.
[0044] The technical features of the above embodiments can be combined arbitrarily (as long as there is no contradiction in the combination of these technical features). For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described; these embodiments that are not explicitly written out should also be considered to be within the scope described in this specification.
Claims
1. An automatic palletizing control system based on visual positioning, characterized in that, It includes a multi-modal visual positioning module, a model establishment module, a dynamic path planning module, a robotic arm execution module, an error correction module, and a storage module; The multi-modal visual positioning module includes an RGB camera, sensors, and a lidar, and is used to obtain the three-dimensional space information of the goods and the palletizing area in real time; the multi-modal visual positioning module is installed on one side of the palletizing area; The model establishment module is used to establish the palletizing area, the position of the goods, and the position of the robotic arm in three-dimensional space; The dynamic path planning module is deployed in the path optimization engine of the industrial controller, and generates a collision-free and energy-optimal grasping path and stacking path according to the position of the goods, the target pallet position, and the position of the robotic arm; the path planning supports online replanning, and when environmental obstacles or pallet type offsets are detected, the motion trajectory is updated in real time; The robotic arm execution module includes a three-axis collaborative robotic arm, an adaptive fixture, and a force feedback unit; the three-axis collaborative robotic arm is used to move the robotic arm in the front-back, left-right, and up-down directions, and the adaptive fixture supports the quick switching between a vacuum suction cup and an electric gripper to adapt to the grasping of goods of different materials; the force feedback unit monitors the clamping force and contact pressure in real time to grasp the goods and prevent the goods from deforming; The error correction module compares the real-time data of the multi-modal visual positioning module with the palletizing position on the palletizing area, and dynamically compensates for the robotic arm joint error, the goods size deviation, and the environmental vibration interference; The storage module is used to store the grasping path and stacking path generated by the dynamic path planning module.
2. The automatic palletizing control system based on visual positioning according to claim 1, wherein The grasping path and stacking path in the storage module are stored in partitions, the valid grasping path and invalid grasping path in the grasping path are stored in partitions, the valid stacking path and invalid stacking path in the stacking path are stored in partitions, and at the same time, the goods position information, palletizing area position information, and robotic arm position information corresponding to the valid grasping path and valid stacking path are stored.
3. The automatic palletizing control system based on visual positioning according to claim 2, characterized in that, When the dynamic path planning module plans the grasping path and stacking path of the goods, it compares the goods position information, palletizing area position information, and robotic arm position information collected by the multi-modal visual positioning module with the storage information in the storage module. When the goods position information, palletizing area position information, and robotic arm position corresponding to the valid grasping path and valid stacking path stored in the storage module are the same as the collected goods position information, palletizing area position information, and robotic arm position, the valid grasping path and / or valid stacking path are directly retrieved.
4. The automatic palletizing control system based on visual positioning according to claim 2, wherein When storing the invalid grasping path, the invalid grasping path that can successfully grasp the goods after calibration is marked as the invalid grasping path one, and the others are the invalid grasping path two, and the calibration parameters, goods position, and robotic arm position information of the invalid grasping path one are stored; When storing the invalid stacking path, the invalid stacking path that can successfully stack the goods after calibration is marked as the invalid stacking path one, and the others are the invalid stacking path two, and the calibration parameters, robotic arm position, and palletizing area position information of the invalid stacking path one are stored.
5. The automatic palletizing control system based on visual positioning according to claim 4, characterized in that, When calibrating the robotic arm, the error correction module compares the robotic arm position, the cargo position, and the palletizing area position information with the robotic arm position, the cargo position, and the palletizing area position in the invalid grasping path 1 and the invalid stacking path 1 in the storage module. When the compared information is the same, the corresponding calibration parameters are directly retrieved.
6. The automatic palletizing control system based on visual positioning according to claim 1, characterized in that, It also includes a human-machine interaction module, which can display the position of the palletizing area, the information on whether the palletizing area can receive goods, the position information of the robotic arm, the information on whether the robotic arm is in a working state, and the position of the cargo.
7. The automatic palletizing control system based on visual positioning according to claim 1, wherein It also includes an emergency stop module, which is connected to the robotic arm execution module and can stop the operation of the robotic arm execution module.
Citation Information
Patent Citations
Mechanical arm path planning method and device based on space coordinate system
CN107671856A
Intelligent handling robotic arm system based on 3D vision and deep learning, and using method
CN111496770A
3D visual stacking system and method based on artificial intelligence technology
CN112047113A
Mechanical arm control method, device, electronic equipment and storage medium
CN113232019A
Redundant mechanical arm obstacle avoidance path planning method and device and storage device
CN116587272A
Cited By
MES cooperative scheduling method and system based on PLC and visual positioning
CN120742825A
Flexible stacking system for multi-variety bent finished products
CN120864254A
Multi-species bending finished product flexible stacking system
CN120864254B
Automatic building material stacking system based on industrial robot
CN121158528A
Stacking and discharging control method based on multiple sensors
CN121778431A
