Control system and control method for self-adaptive pallet recognition and fork entering of forklift

Abstract

The invention discloses a control system and control method for forklift self-adaptive pallet recognition and fork entering, and the control method comprises the steps: controlling a forklift portal frame to be lifted to a target height, and collecting point cloud data of a front environment; processing the point cloud data, calculating the nearest distance between the forklift and the front target pallet in real time, and performing segmented control on the advancing speed of the forklift based on the nearest distance; when the nearest distance is not larger than a preset triggering threshold value, pallet recognition is started, the processed point cloud data is matched with a pre-stored pallet point cloud template, and the pose of the pallet relative to the forklift is calculated according to the matching result; and according to the calculated forward distance and the calculated transverse deviation, a forklift portal frame and / or a forklift body are / is controlled to move, and fork entering operation is completed. The invention further comprises a forklift self-adaptive pallet recognition and fork entering control system. The problems that in a traditional scheme, the visual field is blocked, the sensor is prone to being damaged, and the positioning precision is insufficient are effectively solved, and the reliability and efficiency of forklift operation are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic control of unmanned forklifts, and in particular to a forklift adaptive pallet recognition and forking control system and method. BACKGROUND

[0002] The rapid evolution of modern logistics systems is pushing the material handling mode from traditional manual operation to intelligent and automated direction. The original forklift operation mode relying on manual driving is increasingly showing limitations in efficiency, cost and adaptability. On the one hand, high-intensity and repetitive handling operations are prone to cause manual fatigue, affecting operation accuracy and response speed, and it is difficult to ensure the stability and efficiency of the logistics process. On the other hand, the continuous rise of labor costs and the shortage of professional drivers also prompt enterprises to seek more economically viable and sustainable solutions. Against this background, intelligent unmanned forklifts and other automated devices that integrate advanced navigation, perception and scheduling technologies have gradually become a key path to improving the overall efficiency of logistics systems, helping enterprises to achieve cost reduction and efficiency improvement and digitalization upgrade in the warehouse and production links.

[0003] However, in actual logistics loading scenarios, the placement of pallets often has many uncertainties. The randomness of the operator, the non-standardization of the carriage layout, and human factors in cargo loading all lead to the fact that the front-to-back and left-to-right intervals between pallets are not fixed, and there are often cases of uneven spacing and skewed rows. SUMMARY

[0004] The purpose of the present application is to overcome the above-mentioned deficiencies of the prior art and provide a forklift adaptive pallet recognition and forking control system and method with high positioning accuracy, wide applicability, and sensors that are not easily damaged.

[0005] The technical solution of the present application is: The forklift adaptive pallet recognition and forking control method of the present application comprises the following steps: S1: control the forklift mast to lift to a target height, and collect point cloud data of the front environment; S2: process the point cloud data, calculate the closest distance between the forklift and the front target pallet in real time, and control the forklift travel speed in sections based on the closest distance; S3: when the closest distance is not greater than a preset trigger threshold, start the pallet recognition unit, match the processed point cloud data with a pre-stored pallet point cloud template, and according to the matching result, calculate the pose of the pallet relative to the forklift, the pose at least including a forward distance and a lateral deviation; S4: Based on the calculated positive distance and lateral deviation, control the movement of the forklift mast and / or vehicle body to align the forks with the pallet insertion holes and complete the fork entry operation.

[0006] Furthermore, in S1, when the forklift mast is in a low position, the 3D vision sensing module installed below the forklift mast is blocked; the forklift mast is controlled to rise, so that the 3D vision sensing module is exposed and collects point cloud data of the environment in front.

[0007] Furthermore, S2 specifically includes: Preprocess the collected raw point cloud; Based on the geometric features of the stack, the corresponding point cloud clusters are segmented from the processed point cloud. Real-time calculation of the nearest distance between the point cloud cluster and the current position of the forklift. d 1. and based on the nearest distance d 1. Implement segmented speed control: when d 1> When the first distance value is reached, control the forklift to travel at the first preset speed; When the second distance value < d When 1 ≤ the first distance value, control the forklift to travel at the second preset speed; when d When 1 ≤ the second distance value, control the forklift to move slowly at the third preset speed; Among them, the first preset speed > the second preset speed > the third preset speed.

[0008] Furthermore, in S3, when the stack identification unit is triggered, the unit executes the following steps: S31: Point cloud template matching: Perform a registration operation between the real-time acquired point cloud and the pre-stored stack point cloud template, and obtain the spatial transformation matrix of the stack relative to the 3D vision sensing module based on the optimal matching result. S32: Pose calculation: Based on the spatial transformation matrix obtained from the matching, calculate the key pose parameters used to control the forklift to perform fork entry operation, including the forward distance and lateral deviation.

[0009] Furthermore, in S3, when the forklift is unable to move laterally, it is determined whether the lateral deviation is greater than the maximum lateral travel of the forklift mast. If it is greater, the fork tip width and the horizontal position relationship of the pallet baffle area between the two insertion holes of the pallet are calculated to obtain the minimum lateral deviation that satisfies the fork entry condition. If the minimum lateral deviation is not greater than the maximum lateral travel of the forklift mast, it is determined that the fork can enter; otherwise, the fork cannot enter.

[0010] Furthermore, in S4, based on the lateral deviation, the forklift mast and / or vehicle body are controlled to move laterally so that the center line of the forks is aligned with the center line of the insertion hole on the pallet; based on the forward distance and the preset safe stopping distance, the forklift is controlled to move and stop at the target position, and the forks are controlled to be horizontally inserted into the insertion hole on the pallet; wherein, during the insertion process, the relative position of the forks and the insertion hole is continuously monitored based on point cloud data, and the fork entry action is adjusted in real time based on the monitoring results to ensure smooth insertion.

[0011] One aspect of the present invention is a control system for adaptive pallet recognition and fork entry of a forklift, comprising: The 3D vision sensing module is used to collect point cloud data of the environment in front. The data processing module is used to preprocess the point cloud data, and also to detect and calculate the closest distance between the forklift and the target pallet in front in real time; and when the closest distance is less than or equal to a preset trigger threshold, the pallet recognition process is started, the preprocessed point cloud data is matched with the pre-stored pallet point cloud template, and the pose of the pallet relative to the forklift is calculated based on the matching result, wherein the pose includes at least the positive distance and the lateral deviation. The motion control module is used to control the forklift's travel speed in segments based on the distance information output by the data processing module; and to control the forklift to stop when a preset safe stopping distance is reached. The mast adjustment module is used to generate adjustment commands based on the pose calculated by the data processing module, and control the forklift mast to tilt forward and backward and / or move laterally, so that the forks are accurately aligned with the slots on the pallet.

[0012] Furthermore, the 3D vision sensing module is installed at the lower front of the forklift mast via a height adjustment mechanism. The forklift mast is equipped with a protective plate at the position corresponding to the 3D vision sensing module. When the forklift mast is not raised, the 3D vision sensing module can be protected by the protective plate. When the forklift mast is raised to a certain height, the 3D vision sensing module is exposed from behind the protective plate, obtaining an unobstructed frontal view to collect point cloud data of the pallet.

[0013] Furthermore, the data processing module includes: The point cloud preprocessing unit is used to preprocess the point cloud data acquired by the 3D vision sensing module. The distance detection unit is used to detect and calculate the closest distance between the forklift and the target pallet in front in real time, and send the distance information to the motion control module; it is also used to send a trigger command to the pallet recognition unit when the closest distance is less than or equal to a preset trigger threshold. The pallet recognition unit is activated upon receiving a trigger command from the distance detection unit. It matches the pre-processed point cloud data with a pre-stored pallet point cloud template, calculates the pallet's pose relative to the forklift based on the matching result, and the pose includes at least the positive distance and lateral deviation. The pose information is then sent to the gantry adjustment module.

[0014] Furthermore, the segmented control strategy of the motion control module is as follows: When the nearest distance d 1> When the first distance value is reached, control the forklift to travel at the first preset speed; When the second distance value < d When 1 ≤ the first distance value, control the forklift to travel at the second preset speed; when d When 1 ≤ the second distance value, control the forklift to move slowly at the third preset speed; Among them, the first preset speed > the second preset speed > the third preset speed.

[0015] The beneficial effects of this invention are: (1) By installing a 3D vision sensing module under the forklift mast and using the forklift mast lifting mechanism, the 3D vision sensing module is protected by the protective plate when the mast is low, and obtains unobstructed forward point cloud data at a specific height. This realizes the automatic and reliable switching between the sensor being mechanically protected in the non-working state and obtaining an unobstructed panoramic view in the working state, which fundamentally avoids the sensor being damaged by collision when moving or not working, and ensures the best observation field of view during operation. (2) The segmented speed control strategy is adopted so that the system only performs coarse distance detection and speed control at long distances, avoiding the continuous operation of high-load recognition algorithms, thereby significantly reducing the overall computing power consumption of the system; the high-precision stack recognition unit is triggered only when it approaches the preset trigger threshold, ensuring that the complex recognition algorithm runs in a stable state when the vehicle is stationary or at low speed, eliminating motion blur and vibration interference. (3) The pallet recognition unit adopts point cloud template matching technology. By registering the real-time point cloud with the template, the transformation matrix is ​​directly calculated to obtain the complete pose of the pallet, instead of relying on preset size for screening. This greatly improves the robustness and accuracy of positioning and has stronger adaptability to scenarios where the pallet placement interval is not fixed and the position is random, effectively ensuring the accurate alignment of the forks and the pallet sockets. (4) In scenarios where the forklift cannot move laterally, the introduction of a secondary intelligent decision-making mechanism for lateral over-limit situations significantly improves the success rate of operations and the level of system intelligence in scenarios with poor pallet placement and non-ideal conditions. Attached Figure Description

[0016] Figure 1This is a flowchart illustrating the control method of an embodiment of the present invention. Detailed Implementation

[0017] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0018] A control system for adaptive pallet recognition and forklift entry in a forklift includes a 3D vision sensing module, a data processing module, a motion control module, and a mast adjustment module. In this embodiment, the 3D vision sensing module includes a 3D camera (e.g., a binocular stereo vision camera or a multi-line LiDAR). The camera is mounted on the lower front of the forklift mast via a height adjustment mechanism, for example, the forklift mast is movably connected to the forklift body, and an angle control mechanism for controlling the tilt angle of the forklift mast is provided between the two. The forklift mast includes an outer mast and an inner mast. The inner mast can move up and down along the outer mast. The 3D camera is mounted on the bottom of the lower crossbeam of the inner mast. A fork carriage is mounted on the inner mast, and the fork carriage connects two forks, which can drive the forks to move up and down through the inner mast. The fork carriage is connected to the inner mast via a connecting plate, and the fork carriage is slidably mounted on the connecting plate laterally, allowing the fork carriage to move left and right.

[0019] The forks are positioned above the 3D camera to avoid obstructing its view. The outer mast has a protective plate at the location corresponding to the 3D camera. When the inner mast is not raised, the 3D camera is shielded and protected by the outer mast's protective plate, preventing collisions with goods. When the inner mast is raised to a certain height, the 3D camera emerges from behind the protective plate, obtaining an unobstructed forward view to collect point cloud data of the pallet. Understandably, since the height adjustment mechanism and angle control mechanism of the forklift mast are already mature technologies, they will not be described in detail here.

[0020] In this embodiment, the data processing module is responsible for processing the point cloud data acquired by the 3D camera, and mainly includes a point cloud preprocessing unit, a distance detection unit, and a pallet recognition unit. The point cloud preprocessing unit filters, denoises, and segments the raw point cloud; the distance detection unit detects and calculates the distance between the forklift and the pallet in front in real time, and triggers the pallet recognition unit when this distance reaches a preset trigger threshold (e.g., 1 meter); the pallet recognition unit, after being triggered, identifies the pallet using point cloud template matching technology and calculates the forward distance and lateral deviation of the pallet relative to the distance detection unit.

[0021] In this embodiment, the motion control module dynamically adjusts the forklift's travel speed using a segmented control strategy based on the distance information output by the data processing module. This segmented control strategy involves traveling at medium to high speeds when the distance is relatively long to improve efficiency; switching to low speeds when the distance approaches a threshold to improve positioning accuracy; and stopping when a preset safe stopping distance is reached.

[0022] In this embodiment, the mast adjustment module is used to generate adjustment commands based on the position deviation calculated by the pallet recognition unit, control the forklift mast to tilt forward and backward, and control the fork carriage to move left and right, so that the forks are accurately aligned with the insertion holes of the pallet, and finally complete the fork entry operation.

[0023] like Figure 1 As shown, the control method in this embodiment specifically includes the following steps: S101: Initial positioning and environmental perception: When the forklift mast is in a low position, the 3D camera is blocked; control the forklift mast to raise, so that the 3D camera is exposed and collects point cloud data of the environment in front.

[0024] Specifically, the forklift, following instructions from the upper-level scheduling system, travels to the preset working area of ​​the target pallet via its navigation system (such as a 3D SLAM-based navigation system); upon reaching the approximate location, the forklift pauses. At this point, the inner mast of the forklift mast is raised. During the raising process, the inner mast drives the camera and fork carriage upwards. When the inner mast is raised to a predetermined height (e.g., 1.5 meters), a proximity switch is triggered, and the 3D camera collects 3D point cloud data of the surrounding environment at a certain frequency (e.g., 10Hz).

[0025] S102: Distance detection and speed control: Process the point cloud data, calculate the shortest distance between the forklift and the target pallet in front in real time, and control the forklift's travel speed in segments based on the shortest distance.

[0026] Specifically, the distance detection unit of the data processing module preprocesses the acquired raw point cloud, including downsampling using voxel grid filtering and statistical filtering to remove outlier noise points. Then, using plane fitting or obstacle clustering algorithms, it segments point cloud clusters that may be pallets from the point cloud and calculates the nearest distance between the point cloud cluster and the current position of the forklift in real time. d 1. It is understandable that the aforementioned point cloud filtering and segmentation are existing technologies, and will not be elaborated on here.

[0027] The motion control module is based on the nearest distance d 1. Implement segmented speed control: when d When the distance is greater than 3m, control the forklift to travel at a higher speed (e.g., 0.8m / s); When 1m < d When 1 ≤ 3m, control the forklift to travel at a medium speed (e.g., 0.4m / s); when d When 1 ≤ 1m (threshold is configurable), control the forklift to move slowly at a low speed (e.g., 0.1m / s) and prepare to trigger pallet recognition.

[0028] This segmented control strategy, implemented by a PID controller, effectively ensures that the forklift approaches the pallet smoothly and accurately.

[0029] S103: Pallet recognition and pose calculation: When the nearest distance is not greater than the preset trigger threshold, the pallet recognition unit is activated; the processed point cloud data is matched with the pre-stored pallet point cloud template, and the pose of the pallet relative to the forklift is calculated based on the matching result, where the pose includes the forward distance and the lateral deviation.

[0030] Specifically, when the closest distance d 1. When the pallet recognition unit enters a preset trigger threshold range (e.g., 1 meter), it is activated. This unit performs the following steps: (1) Point cloud template matching: The real-time acquired point cloud is matched with the stack point cloud template obtained through offline learning. The template matching algorithm can use the Iterative Closest Point (ICP) algorithm or its variants to find the optimal rigid body transformation (translation and rotation) so that the real-time point cloud is optimally aligned with the template. Since the template matching algorithm is an existing technology, it will not be described in detail here.

[0031] (2) Pose calculation: Based on the transformation matrix obtained from the matching, the pose of the pallet relative to the 3D camera is calculated. This includes the three-dimensional coordinates of the pallet center in the camera coordinate system, which are then converted into key parameters required by the forklift: forward distance ( d (Vertical distance from the front of the pallet to the fork plane) and lateral deviation (△) x (The horizontal offset between the pallet centerline and the fork centerline). There is also a method for handling extreme cases, namely, when the forklift cannot move laterally, and the lateral deviation Δ... x Greater than the maximum stroke of the fork carriage lateral movement ( S max When calculating the fork tip width and the horizontal position relationship between the fork tip width and the pallet baffle area based on the width of the baffle area between the two insertion holes on the pallet, the minimum lateral deviation (Δ) that satisfies the fork entry condition is obtained. x min If the minimum lateral deviation is less than the maximum stroke of the fork carriage lateral movement. S max When the lateral deviation Δx is equal to the maximum stroke of the gantry lateral displacement, then... S max Otherwise, the forks cannot be inserted. That is, when the forks cannot be perfectly aligned with the two holes on the pallet, the system calculates how much translation the fork carriage needs to make so that the two forks can barely be inserted, thereby greatly improving the success rate and robustness of the system in non-ideal lateral movement scenarios.

[0032] S104: Based on the calculated positive distance and lateral deviation, control the movement of the forklift mast and / or vehicle body to align the forks with the pallet insertion holes and complete the fork entry operation.

[0033] Specifically, when the positive distance is obtained d and lateral deviation △ x Then, the gantry adjustment module generates control commands: for lateral deviation △ x By controlling the lateral movement of the forklift mast and / or body, the center line of the forks is aligned with the center line of the insertion hole on the pallet; based on the forward distance and the preset safe stopping distance (e.g., 0.15 meters), the forklift is controlled to move and stop at the target position, and the forks are controlled to be horizontally inserted into the insertion hole on the pallet; during the insertion process, the relative position of the forks and the insertion hole can be continuously monitored based on point cloud data, and the fork entry action can be adjusted in real time according to the monitoring results to ensure smooth insertion.

[0034] In summary, this embodiment, by installing a 3D vision sensing module under the forklift mast and processing point cloud data in real time, enables the system to sequentially perform distance detection, pallet recognition, and pose positioning, ultimately controlling the forklift to complete precise fork entry. This effectively solves the problems of field of view obstruction, sensor damage, and insufficient positioning accuracy in traditional solutions, significantly improving the reliability and efficiency of forklift operations.

Claims

1. A control method for adaptive pallet recognition and fork entry of a forklift, characterized in that, Includes the following steps: S1: Control the forklift mast to rise to the target height and collect point cloud data of the environment in front; S2: Process the point cloud data, calculate the shortest distance between the forklift and the target pallet in front in real time, and control the forklift's travel speed in segments based on the shortest distance; S3: When the nearest distance is not greater than the preset trigger threshold, the pallet recognition unit is activated to match the processed point cloud data with the pre-stored pallet point cloud template. Based on the matching result, the pose of the pallet relative to the forklift is calculated. The pose includes at least the positive distance and the lateral deviation. S4: Based on the calculated positive distance and lateral deviation, control the movement of the forklift mast and / or vehicle body to align the forks with the pallet insertion holes and complete the fork entry operation.

2. The forklift adaptive pallet recognition and fork entry control method according to claim 1, characterized in that, In S1, when the forklift mast is in a low position, the 3D vision sensing module installed below the forklift mast is blocked; control the forklift mast to raise, so that the 3D vision sensing module is exposed and can collect point cloud data of the environment in front.

3. The forklift adaptive pallet recognition and fork entry control method according to claim 1, characterized in that, S2 specifically includes: Preprocess the collected raw point cloud; Based on the geometric features of the stack, the corresponding point cloud clusters are segmented from the processed point cloud. Real-time calculation of the nearest distance between the point cloud cluster and the current position of the forklift. d 1. and based on the nearest distance d 1. Implement segmented speed control: when d 1> When the first distance value is reached, control the forklift to travel at the first preset speed; When the second distance value < d When 1 ≤ the first distance value, control the forklift to travel at the second preset speed; when d When 1 ≤ the second distance value, control the forklift to move slowly at the third preset speed; Among them, the first preset speed > the second preset speed > the third preset speed.

4. The forklift adaptive pallet recognition and fork entry control method according to claim 2, characterized in that, In S3, when the pallet identification unit is triggered, the unit performs the following steps: S31: Point cloud template matching: Perform a registration operation between the real-time acquired point cloud and the pre-stored stack point cloud template, and obtain the spatial transformation matrix of the stack relative to the 3D vision sensing module based on the optimal matching result. S32: Pose calculation: Based on the spatial transformation matrix obtained from the matching, calculate the key pose parameters used to control the forklift to perform fork entry operation, including the forward distance and lateral deviation.

5. The forklift adaptive pallet recognition and fork entry control method according to claim 1 or 4, characterized in that, In S3, when the forklift is unable to move laterally, it is determined whether the lateral deviation is greater than the maximum lateral travel of the forklift mast. If it is greater, the fork tip width and the horizontal position relationship of the pallet baffle area between the two insertion holes of the pallet are calculated to obtain the minimum lateral deviation that satisfies the fork entry condition. If the minimum lateral deviation is not greater than the maximum lateral travel of the forklift mast, it is determined that the fork can enter; otherwise, the fork cannot enter.

6. The forklift adaptive pallet recognition and fork entry control method according to claim 1, characterized in that, In S4, based on the lateral deviation, the forklift mast and / or vehicle body are controlled to move laterally so that the center line of the forks is aligned with the center line of the insertion hole on the pallet; based on the forward distance and the preset safe stopping distance, the forklift is controlled to move and stop at the target position, and the forks are controlled to be horizontally inserted into the insertion hole on the pallet; wherein, during the insertion process, the relative position of the forks and the insertion hole is continuously monitored based on point cloud data, and the fork entry action is adjusted in real time according to the monitoring results to ensure smooth insertion.

7. A control system for adaptive pallet recognition and forklift entry of a forklift, characterized in that, include: The 3D vision sensing module is used to collect point cloud data of the environment in front. The data processing module is used to preprocess the point cloud data, and also to detect and calculate the closest distance between the forklift and the target pallet in front in real time; and when the closest distance is less than or equal to a preset trigger threshold, the pallet recognition process is started, the preprocessed point cloud data is matched with the pre-stored pallet point cloud template, and the pose of the pallet relative to the forklift is calculated based on the matching result, wherein the pose includes at least the positive distance and the lateral deviation. The motion control module is used to control the forklift's travel speed in segments based on the distance information output by the data processing module; and to control the forklift to stop when a preset safe stopping distance is reached. The mast adjustment module is used to generate adjustment commands based on the pose calculated by the data processing module, and control the forklift mast to tilt forward and backward and / or move laterally, so that the forks are accurately aligned with the slots on the pallet.

8. The control system for adaptive pallet recognition and fork entry of a forklift according to claim 1, characterized in that, The 3D vision sensing module is installed on the lower front of the forklift mast via a height adjustment mechanism. The forklift mast is equipped with a protective plate at the position corresponding to the 3D vision sensing module. When the forklift mast is not raised, the 3D vision sensing module can be protected by the protective plate. When the forklift mast is raised to a certain height, the 3D vision sensing module is exposed from behind the protective plate, obtaining an unobstructed frontal view to collect point cloud data of the pallet.

9. The control system for forklift adaptive pallet recognition and fork entry according to claim 7 or 8, characterized in that, The data processing module includes: The point cloud preprocessing unit is used to preprocess the point cloud data acquired by the 3D vision sensing module. The distance detection unit is used to detect and calculate the closest distance between the forklift and the target pallet in front in real time, and send the distance information to the motion control module; it is also used to send a trigger command to the pallet recognition unit when the closest distance is less than or equal to a preset trigger threshold. The pallet recognition unit is activated upon receiving a trigger command from the distance detection unit. It matches the pre-processed point cloud data with a pre-stored pallet point cloud template, calculates the pallet's pose relative to the forklift based on the matching result, and the pose includes at least the positive distance and lateral deviation. The pose information is then sent to the gantry adjustment module.

10. The control system for forklift adaptive pallet recognition and fork entry according to claim 7 or 8, characterized in that, The segmented control strategy of the motion control module is as follows: When the nearest distance d 1> When the first distance value is reached, control the forklift to travel at the first preset speed; When the second distance value < d When 1 ≤ the first distance value, control the forklift to travel at the second preset speed; when d When 1 ≤ the second distance value, control the forklift to move slowly at the third preset speed; Among them, the first preset speed > the second preset speed > the third preset speed.

Images