Automatic palletizing control system based on visual positioning
The automated palletizing system, which utilizes multimodal visual positioning and dynamic path planning, solves the problem of inflexible movement paths of robotic arms in existing technologies. It enables precise palletizing and safe transportation in complex environments, improving the system's adaptability and efficiency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING XINGLU ECOLOGICAL FERTILIZER CO LTD
- Filing Date
- 2025-04-18
- Publication Date
- 2026-07-24
AI Technical Summary
Existing automated palletizing systems lack dynamic adjustment capabilities under complex and diverse on-site conditions, relying on visual recognition and manual teaching based on a single data dimension. This results in inflexible movement paths for robotic arms, making it difficult to meet user needs.
The system employs a multimodal vision positioning module combined with an RGB camera and LiDAR to acquire 3D spatial information. A dynamic path planning module generates collision-free and energy-optimized grasping and stacking paths. A three-axis collaborative robotic arm works in conjunction with an adaptive gripper. An error correction module compensates for robotic arm errors and environmental interference in real time. A storage module saves valid paths to support online replanning.
It enables accurate and error-free delivery and stacking of goods in complex environments, increasing the system's scope of use and flexibility, and improving palletizing efficiency and safety.
Smart Images
Figure CN120246695B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of palletizing control system technology, and specifically to an automatic palletizing control system based on vision positioning. Background Technology
[0002] Current automated palletizing systems mostly use preset programs to control robotic arm operations, relying on basic sensors (such as a single RGB camera or photoelectric switch) for coarse positioning and completing stacking tasks through fixed path planning. Some solutions introduce simple visual recognition technology to achieve cargo positioning through edge detection or template matching, but the data dimension is limited (e.g., relying only on two-dimensional images), and the robotic arm's movement path depends on manual teaching or offline programming, lacking dynamic adjustment capabilities.
[0003] For example, Chinese invention patent application CN202010407384.4 discloses an automatic palletizing system, comprising: a robot adapted to grasp logistics packaging boxes; a packaging box conveyor adapted to transport the logistics packaging boxes to a predetermined unloading position; a two-dimensional camera for identifying logistics packaging boxes located at the predetermined unloading position to guide the robot to grasp the logistics packaging boxes; a pallet conveyor adapted to transport a pallet for loading the logistics packaging boxes to a predetermined loading position; and a three-dimensional camera for identifying a pallet located at the predetermined loading position to guide the robot to place the grasped logistics packaging boxes onto the pallet. In this invention, the automatic palletizing system can automatically perform the palletizing operation of logistics packaging boxes, saving labor costs and greatly improving the palletizing efficiency of logistics packaging boxes.
[0004] However, given the complex and diverse on-site working conditions, existing palletizing systems clearly cannot meet the user's needs. Therefore, this application proposes an automatic palletizing control system based on vision positioning. Summary of the Invention
[0005] Therefore, this application provides an automated palletizing control system based on vision positioning to solve the problems existing in the prior art.
[0006] To achieve the above objectives, this application provides the following technical solution:
[0007] In the first aspect, an automatic palletizing control system based on vision positioning includes a multimodal vision positioning module, a model building module, a dynamic path planning module, a robotic arm execution module, an error correction module, and a storage module.
[0008] The multimodal visual positioning module includes an RGB camera, sensors, and a LiDAR, used to acquire real-time three-dimensional spatial information of the goods and the palletizing area; the multimodal visual positioning module is installed on one side of the palletizing area;
[0009] The model building module is used to establish the palletizing area, cargo location, and robotic arm location in three-dimensional space.
[0010] The dynamic path planning module is deployed within the path optimization engine of the industrial controller. Based on the location of the goods, the target stack position, and the position of the robotic arm, it generates collision-free and energy-efficient grasping and stacking paths. The path planning supports online replanning, updating the motion trajectory in real time when environmental obstacles or stack offsets are detected.
[0011] The robotic arm execution module includes a three-axis collaborative robotic arm, an adaptive gripper, and a force feedback unit. The three-axis collaborative robotic arm is used to enable the robotic arm to move in the forward, backward, left, right, and up and down directions. The adaptive gripper supports rapid switching between vacuum suction cups and electric grippers to adapt to the gripping of goods of different materials. The force feedback unit monitors the gripping force and contact pressure in real time to grip the goods and prevent them from deforming.
[0012] The error correction module compares the real-time data from the multimodal visual positioning module with the palletizing position in the palletizing area to dynamically compensate for robotic arm joint errors, cargo size deviations, and environmental vibration interference.
[0013] The storage module is used to store the crawling paths and stacking paths generated by the dynamic path planning module.
[0014] Preferably, the grab path and stacking path in the storage module are partitioned and saved separately, the valid grab path and invalid grab path in the grab path are partitioned and saved separately, the valid stacking path and invalid stacking path in the stacking path are partitioned and saved separately, and the cargo location information, palletizing area location information and robotic arm location information corresponding to the valid grab path and valid stacking path are saved simultaneously.
[0015] Preferably, when planning the grasping path and stacking path of goods, the dynamic path planning module compares the goods position information, palletizing area position information and robotic arm position information collected by the multimodal visual positioning module with the stored information in the storage module. When the goods position information, palletizing area position information and robotic arm position information corresponding to the valid grasping path and valid stacking path stored in the storage module are the same as the collected goods position information, palletizing area position information and robotic arm position, the valid grasping path and / or valid stacking path are directly retrieved.
[0016] Preferably, when storing invalid gripping paths, invalid gripping paths that can successfully grip the goods after calibration are marked as invalid gripping path one, and the others are invalid gripping paths two, and the calibration parameters, goods position and robotic arm position information of invalid gripping path one are saved.
[0017] When storing invalid stacking paths, invalid stacking paths that can be successfully stacked after calibration are marked as invalid stacking path one, and the others are marked as invalid stacking path two. The calibration parameters, robotic arm position, and palletizing area position information of invalid stacking path one are also saved.
[0018] Preferably, when calibrating the robotic arm, the error correction module compares the robotic arm position, cargo position, and palletizing area position information with the robotic arm position, cargo position, and palletizing area position in invalid grasping path one and invalid stacking path one in the storage module. When the compared information is the same, the corresponding calibration parameters are directly retrieved.
[0019] Preferably, the system further includes a human-machine interaction module, which can display the location of the palletizing area and whether the palletizing area can receive goods information, the location information of the robotic arm and whether the robotic arm is in working condition, and the location of the goods.
[0020] Preferably, the system further includes an emergency stop module, which is connected to the robotic arm execution module and enables the robotic arm execution module to stop operating.
[0021] Compared with the prior art, this application has at least the following beneficial effects:
[0022] When using this system, it can acquire information about the location of the palletizing area and whether the palletizing area can accommodate new goods. Then, through the three-axis collaborative robotic arm, it can move to any position in space, increasing the scope of application of this system. Through the dynamic path planning module and error correction module, it can accurately transport goods to the location of the palletizing area. Attached Figure Description
[0023] To more intuitively illustrate the prior art and this application, exemplary drawings are provided below. It should be understood that the specific shapes and structures shown in the drawings should not generally be regarded as limiting conditions for implementing this application; for example, based on the technical concept disclosed in this application and the exemplary drawings, those skilled in the art are able to easily make conventional adjustments or further optimizations to the addition / reduction / classification, specific shapes, positional relationships, connection methods, size ratios, etc. of certain units (components).
[0024] Figure 1 A block diagram of the vision-based automatic palletizing control system provided in this application;
[0025] Figure 2 A schematic diagram of the structure of the vision-based automatic palletizing control system provided in this application. Detailed Implementation
[0026] The present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0027] like Figure 1-2 As shown, an automatic palletizing control system based on vision positioning includes a multimodal vision positioning module, a model building module, a dynamic path planning module, a robotic arm execution module, an error correction module, and a storage module.
[0028] The multimodal visual positioning module includes an RGB camera, sensors, and a LiDAR, used to acquire real-time three-dimensional spatial information of the goods, robotic arm, and palletizing area. The multimodal visual positioning module is installed on one side of the palletizing area. Since the positions of the goods, robotic arm, and palletizing area are different each time, it is necessary to acquire the three-dimensional spatial information of the goods, robotic arm, and palletizing area before palletizing the goods. This facilitates the subsequent planning of the grasping path and stacking path. Furthermore, through the combined use of the RGB camera, sensors, and LiDAR, the positions of the goods, robotic arm, and palletizing area can be accurately acquired.
[0029] The model building module is used to establish the palletizing area, the cargo position, and the position of the robotic arm in the same three-dimensional space, which is equivalent to simulating the real spatial environment. It also establishes the relative positions of the palletizing area, the cargo position, and the robotic arm in the space. By establishing them in three-dimensional space, a three-dimensional coordinate system can be established based on the three-dimensional space when planning the grasping path and stacking path. This makes the planning of the grasping path and stacking path more standardized and ensures that the grasping path and stacking path are more accurate.
[0030] The dynamic path planning module is deployed within the path optimization engine of the industrial controller. Based on the location of the goods, the target stack position, and the position of the robotic arm, it generates collision-free and energy-optimized grasping and stacking paths. This not only reduces ineffective movements of the robotic arm but also enables rapid completion of the palletizing operation. The path planning supports online replanning. When environmental obstacles or stack offsets are detected, the movement trajectory is updated in real time to ensure the safety of the palletizing operation. This allows the robotic arm to make emergency avoidances when other objects accidentally enter the work area during the implementation of this system, thus preventing damage to the robotic arm and the goods.
[0031] The robotic arm execution module includes a three-axis collaborative robotic arm, an adaptive gripper, and a force feedback unit. The three-axis collaborative robotic arm enables the robotic arm to move in the forward, backward, left, right, and up / down directions. In conjunction with the dynamic path planning module, it generates collision-free, energy-optimized grasping and stacking paths, ensuring the robotic arm can move to any position. The adaptive gripper supports rapid switching between vacuum suction cups and electric grippers, adapting to the gripping of goods of different materials. For example, electric grippers can be used to handle hard goods, while vacuum suction cups are used for smooth and easily damaged goods. The use of both vacuum suction cups and electric grippers allows the system to handle different types of goods, increasing the device's applicability. The force feedback unit monitors the gripping force and contact pressure in real time to grip the goods and prevent deformation.
[0032] The error correction module compares the real-time data from the multimodal visual positioning module with the palletizing position on the palletizing area to dynamically compensate for joint errors of the robotic arm, deviations in cargo size, and environmental vibration interference, so as to ensure that the robotic arm can accurately grasp and stack cargo.
[0033] The storage module is used to store the crawling paths and stacking paths generated by the dynamic path planning module.
[0034] When using this system, it can acquire information about the location of the palletizing area and whether the palletizing area can accommodate new goods. Then, through the three-axis collaborative robotic arm, it can move to any position in space, increasing the scope of application of this system. Through the dynamic path planning module and error correction module, it can accurately transport goods to the location of the palletizing area.
[0035] The grab path and stacking path in the storage module are partitioned and saved separately. The valid grab path and invalid grab path in the grab path are partitioned and saved separately. The valid grab path and valid stacking path are saved separately. When grab and stacking operations occur, the valid grab path and valid stacking path can be directly retrieved and used to speed up the stacking operation. At the same time, the corresponding cargo position information, palletizing area position information and robotic arm position information of the valid grab path and valid stacking path are saved.
[0036] To reduce the computation of the dynamic path planning module and accelerate route planning, the following technical solution is provided: When planning the grasping and stacking paths for goods, the dynamic path planning module compares the goods position information, palletizing area position information, and robotic arm position information collected by the multimodal visual positioning module with the stored information in the storage module. If the goods position information, palletizing area position information, and robotic arm position information corresponding to the valid grasping path and valid stacking path stored in the storage module are the same as the collected goods position information, palletizing area position information, and robotic arm position information, then the valid grasping path and / or valid stacking path are directly retrieved. This effectively speeds up path generation and avoids invalid calculations.
[0037] When the above scheme is implemented, when the collected cargo location information, the robotic arm location information, and the information in the storage module are consistent, the corresponding valid grasping path is retrieved, and the robotic arm grasps the cargo according to the valid grasping path; when the collected palletizing area location information, the robotic arm location information, and the information in the storage module are consistent, the corresponding valid stacking path is retrieved, and the robotic arm stacks the cargo according to the valid stacking path.
[0038] To reduce the computation of the error correction module and speed up the response time of error calibration, the following technical solution is provided:
[0039] When storing invalid gripping paths, invalid gripping paths that can successfully grip the goods after calibration are marked as invalid gripping path one, and the others are marked as invalid gripping path two. The calibration parameters, goods position and robotic arm position information of invalid gripping path one are also saved.
[0040] When storing invalid stacking paths, invalid stacking paths that can be successfully stacked after calibration are marked as invalid stacking path one, and the others are marked as invalid stacking path two. The calibration parameters, robotic arm position, and palletizing area position information of invalid stacking path one are also saved.
[0041] When calibrating the robotic arm, the error correction module compares the robotic arm position, cargo position, and palletizing area position information with the positions of the robotic arm, cargo, and palletizing area in invalid grasping path one and invalid stacking path one stored in the storage module. When the compared information is the same, the corresponding calibration parameters are directly retrieved, and the robotic arm is calibrated according to the retrieved calibration parameters, thereby ensuring the accuracy of the grasping and stacking routes.
[0042] It also includes a human-machine interface (HMI) module. This HMI module displays information such as the location of the palletizing area, whether the palletizing area can receive goods, the location of the robotic arm, whether the robotic arm is in working order, and the location of the goods. Through the HMI module, users can easily view information such as the system's operating status and understand the status of the palletizing area, the robotic arm, and the goods. During implementation, staff can use the HMI module to understand the system's operating status (various data points) and monitor the operation of each component.
[0043] It also includes an emergency stop module, which is connected to the robotic arm execution module. The emergency stop module can stop the robotic arm execution module from running. When the system is in use, if a dangerous situation occurs (such as when there is an obstacle in front of the robotic arm and the robotic arm is about to collide with the obstacle, or when the goods are damaged and cannot continue the palletizing operation, or when the system malfunctions), the emergency stop module can be used to stop the system to ensure the safety of the device.
[0044] The technical features of the above embodiments can be combined in any way (as long as there is no contradiction in the combination of these technical features). For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described; these embodiments not explicitly written should also be considered to be within the scope of this specification.
Claims
1. An automatic palletizing control system based on vision positioning, characterized in that, It includes a multimodal vision positioning module, a model building module, a dynamic path planning module, a robotic arm execution module, an error correction module, and a storage module; The multimodal visual positioning module includes an RGB camera, sensors, and a LiDAR, used to acquire real-time three-dimensional spatial information of the goods and the palletizing area; the multimodal visual positioning module is installed on one side of the palletizing area; The model building module is used to establish the palletizing area, cargo location, and robotic arm location in three-dimensional space. The dynamic path planning module is deployed within the path optimization engine of the industrial controller. Based on the location of the goods, the target stack position, and the position of the robotic arm, it generates collision-free and energy-efficient grasping and stacking paths. The path planning supports online replanning, updating the motion trajectory in real time when environmental obstacles or stack offsets are detected. The robotic arm execution module includes a three-axis collaborative robotic arm, an adaptive gripper, and a force feedback unit. The three-axis collaborative robotic arm is used to enable the robotic arm to move in the forward, backward, left, right, and up and down directions. The adaptive gripper supports rapid switching between vacuum suction cups and electric grippers to adapt to the gripping of goods of different materials. The force feedback unit monitors the gripping force and contact pressure in real time to grip the goods and prevent them from deforming. The error correction module compares the real-time data from the multimodal visual positioning module with the palletizing position in the palletizing area to dynamically compensate for robotic arm joint errors, cargo size deviations, and environmental vibration interference. The storage module is used to store the crawling paths and stacking paths generated by the dynamic path planning module; The grab path and stacking path in the storage module are partitioned and saved separately. The valid grab path and invalid grab path in the grab path are partitioned and saved separately. The valid stacking path and invalid stacking path in the stacking path are partitioned and saved separately. At the same time, the cargo location information, palletizing area location information and robotic arm location information corresponding to the valid grab path and valid stacking path are saved. When storing the invalid gripping paths, invalid gripping paths that can successfully grip the goods after calibration are marked as invalid gripping path one, and the others are invalid gripping paths two. The calibration parameters, goods position and robotic arm position information of invalid gripping path one are also saved. When storing invalid stacking paths, invalid stacking paths that can be successfully stacked after calibration are marked as invalid stacking path one, and the others are marked as invalid stacking path two. The calibration parameters, robotic arm position, and palletizing area position information of invalid stacking path one are also saved. When calibrating the robotic arm, the error correction module compares the robotic arm position, cargo position, and palletizing area position information with the robotic arm position, cargo position, and palletizing area position in invalid grasping path one and invalid stacking path one stored in the storage module. When the compared information is the same, the corresponding calibration parameters are directly retrieved.
2. The automatic palletizing control system based on vision positioning according to claim 1, characterized in that, When planning the grasping and stacking paths for goods, the dynamic path planning module compares the goods position information, palletizing area position information, and robotic arm position information collected by the multimodal visual positioning module with the information stored in the storage module. If the goods position information, palletizing area position information, and robotic arm position information corresponding to the valid grasping path and valid stacking path stored in the storage module are the same as the collected goods position information, palletizing area position information, and robotic arm position information, then the valid grasping path and / or valid stacking path are directly retrieved.
3. The automatic palletizing control system based on vision positioning according to claim 1, characterized in that, It also includes a human-machine interaction module, which can display the location of the palletizing area and whether the palletizing area can receive goods information, the location information of the robotic arm and whether the robotic arm is in working status, and the location of the goods.
4. The automatic palletizing control system based on vision positioning according to claim 1, characterized in that, It also includes an emergency stop module, which is connected to the robotic arm execution module and can stop the robotic arm execution module from running.