A multi-penetration box automatic deviation correction method and system based on fusion of vision and laser pulse
By integrating multi-source data from pulse encoders, laser sensors, and vision sensors, and combining it with a graded correction method, the problem of bin misalignment caused by vibration in smart warehousing was solved, achieving efficient and reliable picking operations.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- 湖南德荣医链数智科技有限公司
- Filing Date
- 2026-02-09
- Publication Date
- 2026-05-29
AI Technical Summary
In the dense storage racks of smart warehouses, the bins may shift due to equipment vibration and inertia. Existing single sensors are unable to simultaneously meet the positioning efficiency during high-speed operation and the high-precision attitude compensation requirements during end-of-line retrieval, leading to retrieval failures or safety hazards.
By fusing pulse encoders, laser sensors, and vision sensors, and through a graded correction method of coarse and fine adjustment, the comprehensive offset of the bin is obtained, and the actuator is driven to perform graded correction actions, including a coarse adjustment stage and a fine adjustment compensation stage, to ensure the accurate positioning of the picking mechanism.
It achieves accurate detection and automatic correction of the bin's posture, improving the operating efficiency and picking reliability of multi-pass vehicles in complex vibration environments, and avoiding safety hazards such as mechanical collisions and bin falling.
Smart Images
Figure CN122111008A_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of intelligent warehousing and industrial robot technology, specifically involving an automatic correction method and system for multi-car material boxes based on vision and laser pulse fusion. Background Technology
[0002] In the dense storage racking of smart warehouses, multi-pass carts, as the core equipment of automated storage / retrieval systems, are widely used to handle loose boxes and increase storage capacity. However, due to vibrations generated during long-term high-speed operation, minor deformations of the racks, and the inertial effects of acceleration and deceleration, the boxes stored in the storage locations may experience cumulative offset.
[0003] This displacement typically includes lateral offset of the bin in the horizontal plane, forward and backward offset in the depth direction, and tilting and rotation relative to the horizontal reference. If the offset exceeds the tolerance range of the multi-cart picking mechanism, it will cause the multi-cart to have difficulty aligning with the bin during picking, resulting in picking failure, and may even cause safety hazards such as mechanical collisions or bin falling. Existing single-sensor positioning methods often cannot simultaneously meet the positioning efficiency requirements of high-speed operation and the high-precision attitude compensation requirements of end-point picking. Summary of the Invention
[0004] To address the aforementioned technical issues, this application provides an automatic deviation correction method and system for multi-car material boxes based on the fusion of vision and laser pulses, which can achieve accurate detection and automatic deviation correction of the material box posture.
[0005] According to a first aspect of this application, an automatic correction method for a multi-cart hopper is provided, comprising: acquiring real-time coarse positioning pulse data of the multi-cart via a pulse encoder configured on the multi-cart, and driving the multi-cart to move towards a target storage location based on the real-time coarse positioning pulse data; after determining that the multi-cart has reached a preset detection range of the target storage location based on the real-time coarse positioning pulse data, scanning the hopper at the target storage location using a laser sensor and a vision sensor to acquire the attitude data of the hopper, wherein the attitude data includes the planar offset, depth distance, and tilt angle of the hopper. The system integrates the real-time coarse positioning pulse data, the depth distance obtained by the laser sensor, and the planar offset and tilt angle obtained by the vision sensor. The overall offset of the material box relative to the picking mechanism of the multi-cart is calculated; based on the overall offset, the actuator of the multi-cart is driven to perform a graded correction action, which includes a coarse adjustment stage based on the real-time coarse positioning pulse data and a fine adjustment compensation stage based on the attitude data. After the correction is completed, the picking operation is performed.
[0006] Preferably, the fusion of the real-time coarse positioning pulse data, the depth distance, the planar offset, and the tilt angle... The steps for calculating the overall offset specifically include: converting the real-time coarse positioning pulse data into a first displacement coordinate as a basic reference for the position of the material box; using the depth distance measured by the laser sensor as the component of the overall offset on the depth direction Z-axis; using the offset of the material box contour center relative to the center of the picking mechanism in the horizontal plane as identified by the vision sensor as the component of the overall offset on the plane XY-axis; and using the tilt angle measured by the vision sensor. The rotation component of the overall offset.
[0007] Preferably, the visual sensor acquires the tilt angle. The specific steps include: acquiring a front image of the material bin using the vision sensor; identifying the upper edge contour line of the material bin in the front image using an edge detection algorithm; obtaining the projection of the horizontal baseline corresponding to the coordinate system fixed by the multi-carriage in the front image; calculating the angle between the upper edge contour line and the projection of the horizontal baseline, and determining the angle as the tilt angle. The fine-tuning compensation stage is based on the tilt angle. The tilt of the hopper is compensated by rotating the picking mechanism.
[0008] Preferably, in the graded correction action, the trigger condition for switching from the coarse adjustment stage to the fine adjustment compensation stage is as follows: In the coarse adjustment stage, the multi-cart continues to move at high speed, while the laser sensor monitors the distance to the material box in real time; when the distance value monitored by the laser sensor is less than a preset first distance threshold for the first time... At that time, the multi-vehicle decelerates and stops moving at high speed, and then switches to the fine-tuning compensation stage.
[0009] Preferably, the internal execution logic of the fine-tuning compensation stage includes the following sub-steps executed sequentially: a first compensation sub-step, in which the actuator initially responds only to the planar offset and the tilt angle. The action drives the picking mechanism to translate horizontally and rotate around an axis until the planar offset and tilt angle fed back by the vision sensor are equal. The residuals are all less than their respective preset accuracy thresholds; in the second compensation sub-step, after the first compensation sub-step is completed, the plane and rotational attitude of the picking mechanism are locked, and then, in response to the depth distance, the picking mechanism is driven to move forward along the depth direction until the distance value fed back by the laser sensor reaches the preset picking engagement distance. .
[0010] According to a second aspect of this application, an automatic alignment system for a multi-car hopper is provided, comprising: a multi-car body; a pulse encoder disposed on the transmission mechanism of the multi-car body for acquiring real-time coarse positioning pulse data of the multi-car body; a laser sensor disposed on the multi-car body for acquiring the depth distance of the hopper at the target storage location; and a vision sensor disposed on the multi-car body for acquiring the planar offset and tilt angle of the hopper. The document includes: a picking mechanism and an actuator for driving the picking mechanism; and a processor electrically connected to the pulse encoder, the laser sensor, the vision sensor and the actuator, the processor being configured to perform the above-described automatic deviation correction method for multi-car material boxes.
[0011] Preferably, the processor is further configured to: convert the real-time coarse positioning pulse data into first displacement coordinates; and convert the depth distance, the planar offset, and the tilt angle into... These are combined into a multi-dimensional state vector, and then combined with the first displacement coordinate to generate the final comprehensive offset control command.
[0012] Preferably, the processor is further configured to: maintain a control state machine, the control state machine including a coarse-tuning state and a fine-tuning state; when the system is in the coarse-tuning state, control the actuator based solely on data from the pulse encoder; when the distance detected by the laser sensor is less than a first distance threshold... When the time is right, the control state machine is driven to switch from the coarse adjustment state to the fine adjustment state.
[0013] Preferably, the laser sensor and the vision sensor are integrated together at the front end of the picking mechanism to ensure that the measurement coordinate system of the laser sensor and the vision sensor maintains a fixed relative relationship with the action coordinate system of the picking mechanism.
[0014] Preferably, the actuator includes: a first drive unit configured to drive the picking mechanism to perform planar XY-axis translation; and a unit configured to drive the picking mechanism to rotate about a vertical axis to compensate for the tilt angle. The processor sends independent control signals to the first drive unit, the second drive unit, and the third drive unit, respectively.
[0015] The technical solution provided in this application achieves the perception of the left and right, front and back, and tilt posture of the material box by integrating three heterogeneous data: pulse, laser, and vision. Combined with the graded correction method of "coarse adjustment + fine adjustment", it overcomes the shortcomings of a single sensor that cannot take into account both high-speed real-time performance and end-point accuracy, and improves the operating efficiency and picking reliability of multi-pass vehicles in complex vibration environments. Attached Figure Description
[0016] Figure 1 This is a structural block diagram of the automatic correction system for multi-car material boxes provided in the embodiments of this application.
[0017] Figure 2 This is a flowchart illustrating the automatic correction method for multi-car material boxes provided in the embodiments of this application.
[0018] Figure 3 This is a schematic diagram of the layout of the sensor and the picking mechanism provided in the embodiments of this application.
[0019] Figure 4 This is a schematic diagram of the application scenario and equipment configuration of the multi-penetration vertical warehouse provided in the embodiments of this application. Detailed Implementation
[0020] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.
[0021] See Figure 1 This embodiment provides an automatic deviation correction system for a multi-car material box. The system mainly includes a multi-car body (not shown in the figure), a processor (101), a pulse encoder (102), a laser sensor (103), a vision sensor (104), and an actuator (105).
[0022] The processor (101), serving as the control core, is electrically connected to the pulse encoder (102), laser sensor (103), and vision sensor (104) to receive and fuse multi-source sensor data. The pulse encoder (102) is mounted on the walking transmission mechanism of the multi-vehicle body and obtains real-time coarse positioning pulse data of the multi-vehicle body by monitoring the rotation steps of the drive motor. The laser sensor (103) preferably adopts a high-precision TOF ranging sensor to obtain the front-to-back distance of the material box in the depth direction at the target storage location. The vision sensor (104) is configured as an industrial camera and identifies the outline of the material box through image processing algorithms, thereby obtaining the planar offset of the material box in the horizontal plane and the tilt angle of rotation around the axis. .
[0023] The actuator (105) is electrically connected to the picking mechanism. In this embodiment, the actuator (105) includes a first drive unit for XY axis translation and a unit for compensating for rotation angle. The system includes a second drive unit and a third drive unit for Z-axis extension. The processor (101) drives the picking mechanism to perform fine pose adjustments by sending independent control signals (such as PWM pulse signals) to each drive unit. Through this structure, the system can establish a deep mapping from the perception layer to the execution layer, providing a hardware foundation for subsequent automatic correction.
[0024] See Figure 2 This embodiment also provides an automatic correction method for multi-car material boxes, which is implemented through a spatiotemporal hierarchical collaborative mechanism of "perception-action". The specific steps are as follows:
[0025] Step S201: Acquire real-time coarse positioning pulse data and perform coarse adjustment. After receiving the picking instruction, the processor (101) of the multi-cart obtains displacement feedback provided by the pulse encoder (102). At this stage, the system is in coarse adjustment state. Due to the extremely low response delay of the pulse data, the processor (101) controls the drive motor to rotate at high speed, so that the multi-cart quickly approaches the target storage location.
[0026] Step S202: Multi-source data scanning and attitude perception. When the processor (101) determines the preset detection range of the multi-cart to the target storage location based on real-time coarse positioning pulse data, the laser sensor (103) and the vision sensor (104) are activated. In this embodiment, the preset detection range is set to a distance of 500mm to 1000mm from the target storage location. This range should ensure that the field of view (FOV) of the vision sensor (104) can completely cover the outline of the hopper, and the multi-cart is in the deceleration transition phase from high-speed operation to low-speed crawling. Subsequently, the vision sensor (104) extracts the feature outline of the hopper through an edge detection algorithm. In a specific embodiment, the vision sensor (104) first acquires a front image of the hopper. The processor (101) identifies the upper edge outline line of the hopper through an edge detection operator (such as the Canny operator) and compares it with the horizontal baseline.
[0027] Specifically, the processor (101) executes the tilt angle. The calculation logic involves determining the tilt angle based on the mapping relationship between the slope of the upper edge contour line in the image coordinate system and the horizontal baseline. In a preferred embodiment, the above calculation logic is implemented using the following formula:
[0028] = arctan( ) -
[0029] in, Indicates the tilt angle of the hopper relative to the multi-pass vehicle; This represents the slope of the straight line representing the upper edge contour of the bin as identified by the vision sensor. The projection angle of the horizontal reference line corresponding to the coordinate system fixed to the multi-carriage in the frontal image is expressed as follows: This horizontal reference line is pre-calibrated during system installation. Using this formula, the system can quantify the degree of hopper deflection caused by vibration, providing a basis for rotation compensation.
[0030] Step S203: Calculate the overall offset. The processor (101) converts the pulse data into basic displacement coordinates and combines them with the depth distance measured by the laser sensor. Planar offset measured by a vision sensor and tilt angle These are combined into a multidimensional state vector. The combined offset can be represented as a matrix containing translation and rotation components.
[0031] Step S204: Graded correction and fine-tuning compensation. The system determines whether to switch states based on the distance value monitored in real time by the laser sensor (103). When the distance value fed back by the laser sensor is less than the preset first distance threshold for the first time... When the speed reaches 500mm, the vehicle stops traveling at high speed and switches to the fine-tuning compensation stage.
[0032] During the fine-tuning and compensation phase, the system executes a second approximation logic, including:
[0033] In the first compensation sub-step, the processor (101) drives the first and second drive units in the actuator to make the picking mechanism translate in the horizontal plane and rotate about the vertical axis to compensate for the planar offset and tilt angle. .
[0034] The second compensation sub-step, after attitude alignment, locks the rotation and horizontal coordinates in response to depth distance. The third drive unit controls the picking mechanism to move forward along the depth direction until the picking engagement distance is reached. .
[0035] This timing control method of "aligning the posture first, then extending and retracting to pick up the goods" can effectively avoid the phenomenon of the robot arm getting stuck or colliding due to the tilt of the material box.
[0036] See Figure 3 This embodiment further defines the hardware layout scheme. The laser sensor (103) and the vision sensor (104) are integrated together at the front end of the picking mechanism. This integrated design ensures that the measurement coordinate system of the laser and vision sensors maintains a fixed relative transformation relationship with the motion coordinate system of the picking mechanism, eliminating the dynamic interference of vehicle body vibration on the measurement values. It should be noted that the above integration method is only a preferred example. In other embodiments, the sensors can also be arranged separately, as long as data unification can be achieved through coordinate system transformation matrix.
[0037] See Figure 4 The diagram illustrates an application scenario for multi-pass automated storage and retrieval systems (AS / RS). AS / RS operate in dense storage environments, including cold storage and cool storage areas, where the shelving layers are numerous and extremely dense. In this scenario, the vibrations generated by the equipment operation can have a significant cumulative impact on the position of the storage bins. Figure 4 The configuration table shows the configuration of multiple goods-to-person stations, demonstrating the importance of efficient goods retrieval in dense storage and distribution scenarios. The method in this embodiment can achieve intelligent self-correction using existing hardware without increasing manual intervention, improving the system's robustness in multi-temperature environments.
[0038] Furthermore, this embodiment also includes a redundant calibration logic: when the processor (101) detects a logical contradiction between the displacement coordinates converted by the pulse encoder (102) and the distance value of the laser sensor (103) (for example, the absolute value of the difference is greater than a preset deviation threshold value, usually caused by drive wheel slippage), the system automatically switches to visual absolute positioning mode. That is, it uses the visual sensor to identify the static barcode or QR code on the storage location as an absolute coordinate reference to clear the pulse accumulation error online. Through this calibration mechanism, positioning failure caused by single-point sensor failure can be effectively prevented.
[0039] In summary, this application utilizes deep mapping through multi-source data fusion and hierarchical execution methods, leveraging low-latency pulse data to drive high-speed vehicle approach and high-precision visual / laser data to drive pose compensation of the end effector. This hierarchical collaborative mechanism of "perception-action" not only solves the problem of hopper displacement caused by vibration but also achieves a balance between operational efficiency and grasping accuracy.
[0040] It should be noted that, in this document, the terms "comprising," "including," and any other variations are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Specific examples have been used in this document to illustrate the principles and implementation methods of the present invention. These examples are merely for the purpose of helping to understand the method and core ideas of the present invention. The above descriptions are only preferred embodiments of the present invention. It should be pointed out that, due to the limitations of written expression and the objective existence of infinite specific structures, those skilled in the art can make several improvements, modifications, or variations without departing from the principles of the present invention, and can also combine the above technical features in an appropriate manner. These improvements, modifications, variations, or combinations, or the direct application of the concept and technical solution of the present invention to other situations without modification, should all be considered within the scope of protection of the present invention.
Claims
1. An automatic deviation correction method for multi-car material boxes based on vision and laser pulse fusion, characterized in that, include: The pulse encoder configured on the multi-vehicle is used to obtain the real-time coarse positioning pulse data of the multi-vehicle, and the multi-vehicle is driven to move towards the target storage location based on the real-time coarse positioning pulse data. Once the multi-vehicle has reached the preset detection range of the target storage location based on the real-time coarse positioning pulse data, the material bins at the target storage location are scanned using laser sensors and vision sensors to obtain the attitude data of the material bins. The attitude data includes the planar offset, depth distance, and tilt angle of the material bins. ; The data is fused together with the real-time coarse positioning pulse data, the depth distance obtained by the laser sensor, and the planar offset and tilt angle obtained by the vision sensor. Calculate the overall offset of the material bin relative to the picking mechanism of the multi-cart. Based on the comprehensive offset, the actuator of the multi-vehicle is driven to perform a graded correction action. The graded correction action includes a coarse adjustment stage based on the real-time coarse positioning pulse data and a fine adjustment compensation stage based on the attitude data. After the correction is completed, the cargo retrieval operation is performed.
2. The method according to claim 1, characterized in that, The data is obtained by fusing the real-time coarse positioning pulse data, the depth distance, the planar offset, and the tilt angle. The steps for calculating the overall offset specifically include: The real-time coarse positioning pulse data is converted into a first displacement coordinate, which serves as the basic reference for the position of the material box. The depth distance measured by the laser sensor is taken as the component of the overall offset on the depth direction Z-axis; The offset of the bin outline center relative to the center of the picking mechanism in the horizontal plane, as identified by the vision sensor, is taken as the component of the comprehensive offset on the XY axis of the plane. The tilt angle measured by the vision sensor The rotation component of the overall offset.
3. The method according to claim 2, characterized in that, The visual sensor acquires the tilt angle. The specific steps include: The visual sensor captures a frontal image of the material bin; An edge detection algorithm is used to identify the straight line of the upper edge contour of the hopper in the frontal image; Obtain the projection of the horizontal baseline corresponding to the coordinate system fixed to the multi-vehicle into the frontal image; Calculate the angle between the upper edge contour line and the projection of the horizontal baseline, and determine the angle as the tilt angle. ; The fine-tuning compensation stage is based on the tilt angle. The tilt of the hopper is compensated by rotating the picking mechanism.
4. The method according to claim 1, characterized in that, In the graded correction action, the trigger condition for switching from the coarse adjustment stage to the fine adjustment compensation stage is: During the coarse adjustment phase, the multi-cart continues to move at high speed, while the laser sensor monitors the distance to the material bin in real time; When the distance value detected by the laser sensor is less than a preset first distance threshold for the first time At that time, the multi-vehicle decelerates and stops moving at high speed, and then switches to the fine-tuning compensation stage.
5. The method according to claim 4, characterized in that, The internal execution logic of the fine-tuning and compensation phase includes the following sub-steps executed sequentially: In the first compensation sub-step, the actuator initially responds only to the planar offset and the tilt angle. The action drives the picking mechanism to translate horizontally and rotate around an axis until the planar offset and tilt angle fed back by the vision sensor are equal. The residuals are all less than their respective preset accuracy thresholds; The second compensation sub-step involves locking the plane and rotational attitude of the picking mechanism after the first compensation sub-step is completed. Then, in response to the depth distance, the picking mechanism is driven to move forward along the depth direction until the distance value fed back by the laser sensor reaches the preset picking engagement distance. .
6. An automatic deviation correction system for multi-car material boxes based on vision and laser pulse fusion, characterized in that, include: Multiple vehicle body; A pulse encoder is installed on the transmission mechanism of the multi-vehicle body to acquire real-time coarse positioning pulse data of the multi-vehicle body. A laser sensor, mounted on the multi-vehicle body, is used to obtain the depth distance of the loading bin at the target storage location; A vision sensor, mounted on the multi-car body, is used to acquire the planar offset and tilt angle of the material bin. ; A pickup mechanism and an actuator for driving the pickup mechanism; A processor, electrically connected to the pulse encoder, the laser sensor, the vision sensor, and the actuator, is configured to perform the method as described in claim 1.
7. The system according to claim 6, characterized in that, The processor is further configured to: The real-time coarse positioning pulse data is converted into first displacement coordinates; The depth distance, the plane offset, and the tilt angle are used. These are combined into a multi-dimensional state vector, and then combined with the first displacement coordinate to generate the final comprehensive offset control command.
8. The system according to claim 6, characterized in that, The processor is further configured to: Maintain a control state machine, which includes a coarse-tuning state and a fine-tuning state; When the system is in the coarse adjustment state, the actuator is controlled solely based on the data from the pulse encoder; When the distance detected by the laser sensor is less than the first distance threshold When the time is right, the control state machine is driven to switch from the coarse adjustment state to the fine adjustment state.
9. The system according to claim 6, characterized in that, The laser sensor and the vision sensor are integrated together at the front end of the picking mechanism to ensure that the measurement coordinate system of the laser sensor and the vision sensor maintains a fixed relative relationship with the action coordinate system of the picking mechanism.
10. The system according to claim 6, characterized in that, The implementing mechanism includes: A first drive unit configured to drive the picking mechanism to perform planar XY-axis translation; Configured to drive the picking mechanism to rotate about a vertical axis to compensate for the tilt angle. The second drive unit; A third drive unit configured to drive the picking mechanism to extend and retract along the depth Z-axis direction; The processor sends independent control signals to the first driving unit, the second driving unit, and the third driving unit, respectively.