Forking operation track generation method and device for forklift

Abstract

The invention discloses a forking operation track generation method and device for a forklift, and the method comprises the steps: determining the pose of a target tray when the distance between the forklift and the target tray reaches a first distance; according to the target tray pose, the current fork tooth pose and the forking environment illumination intensity, the first pose deviation between the target tray and fork teeth is determined, and a first forklift linear speed and angular speed instruction is obtained through calculation; in the process that the forklift travels to the second distance, the ground friction coefficient and the vibration compensation amount are determined according to the friction-vibration joint estimation model and the first forklift linear speed and the first forklift angular speed; when the second distance is reached, determining a second pose deviation between the target tray and the prongs; and simulating the obtained parameters to obtain a second linear velocity and angular velocity instruction so as to complete the forking operation. Through rough adjustment and fine adjustment of the two distances, high-precision track planning of the forklift is achieved, and through calculation of the ground friction coefficient and the vibration compensation amount, the curve insertion precision of the tray is further improved.

Description

Technical Field

[0001] The present application relates to the technical field of unmanned forklift control, and in particular to a method and device for generating a forklift operation trajectory. Background Art

[0002] In the fields of warehousing and logistics, intelligent manufacturing, etc., the pallet picking operations of unmanned forklifts require extremely high efficiency and precision.

[0003] However, for existing unmanned forklift pallet picking methods, pallet picking operations in complex dynamic environments are easily affected by sensor errors, mechanical vibrations, and ground disturbances, resulting in insufficient pallet picking accuracy (picking accuracy is only ±10mm), which cannot meet the needs of high-value scenarios such as cold chain and semiconductors with strict precision requirements. Summary of the Invention

[0004] The embodiments of the present application provide a method and device for generating a forklift operation trajectory for a forklift, which are used to solve the problem of insufficient pallet forklift accuracy in the prior art.

[0005] The embodiments of this application adopt the following technical solutions:

[0006] On the one hand, an embodiment of the present application provides a method for generating a forklift operation trajectory for a forklift, the method comprising: when the distance between the forklift and the target pallet reaches a preset first distance, determining the target pallet posture according to the target pallet main direction and the target pallet three-dimensional coordinates; determining a first posture deviation between the target pallet and the fork tine according to the target pallet posture, the first current fork tine posture, and the first light intensity of the forklift environment, inputting the first posture deviation into a PID controller to obtain a first forklift linear speed and angular speed instruction; and driving the forklift to travel to the distance between the forklift and the target pallet according to the first forklift linear speed and angular speed instruction. the distance reaches a preset second distance; in the process of traveling to the preset second distance, the ground friction coefficient and the IMU vibration compensation amount are determined according to the preset friction-vibration joint estimation model and the first forklift linear speed and angular speed; when the preset second distance is reached, the second posture deviation between the target pallet and the fork tine is determined according to the target pallet posture, the second current fork tine posture, and the second light intensity of the forking environment; the ground friction coefficient, the IMU vibration compensation amount, and the second posture deviation are simulated to obtain the second linear speed and angular speed instructions to drive the forklift to perform the forking operation based on the second linear speed and angular speed instructions.

[0007] In one example, before the distance between the forklift and the target pallet reaches a preset first distance, the method also includes: when a fork-picking task is received, performing spatiotemporal calibration on the binocular camera, the line laser radar and the single-line laser radar; generating a global navigation path from the current position of the forklift to the preset first distance by performing global positioning on the warehouse electronic map; when the forklift moves according to the global navigation path, the line laser radar point cloud is fused with the binocular camera image data, and static obstacles and dynamic obstacles are identified through the semantic segmentation network and the fused data; the trajectory of the dynamic obstacle is predicted by the Kalman algorithm to update the global navigation path; when a sudden obstacle is identified based on the single-line laser radar point cloud, if the distance between the forklift and the sudden obstacle is less than a preset distance threshold and the time to reach the distance is less than a preset time threshold, a preset emergency avoidance instruction is triggered to complete the avoidance of the sudden obstacle.

[0008] In one example, the first pose deviation between the target pallet and the fork tine is determined based on the target pallet posture, the first current fork tine posture, and the first light intensity of the forking environment, specifically including: when the first light intensity is greater than the preset light intensity, according to the preset first weight distribution, the binocular camera and the line laser radar point cloud weights are allocated to determine the first pose deviation between the target pallet and the fork tine; when the first light intensity is less than the preset light intensity, according to the preset second weight distribution, the binocular camera and the line laser radar point cloud weights are allocated to determine the first pose deviation between the target pallet and the fork tine.

[0009] In one example, the ground friction coefficient and the IMU vibration compensation amount are determined based on a preset friction-vibration joint estimation model and the linear velocity and angular velocity of the first forklift. Specifically, the method includes: inputting the motor current and linear velocity of the forklift during driving, as well as the preset torque constant and forklift wheel diameter of the forklift, into the preset friction-vibration joint estimation model at the same time to determine the ground friction coefficient; and inputting the amplitude and frequency of the IMU vibration during driving of the forklift into the friction-vibration joint estimation model to generate the IMU vibration compensation amount.

[0010] In one example, the ground friction coefficient, IMU vibration compensation amount, and second posture deviation are simulated to obtain the second linear velocity and angular velocity instructions, specifically including: inputting the ground friction coefficient, IMU vibration compensation amount, and second posture deviation into the Actor network to obtain continuous fine-tuning instructions for linear velocity and angular velocity; inputting the ground friction coefficient, IMU vibration compensation amount, second posture deviation, and continuous fine-tuning instructions for linear velocity and angular velocity into the Critic network for simulation, and obtaining the second linear velocity and angular velocity instructions through the preset reward function scoring mechanism during the simulation process.

[0011] In one example, a second linear velocity and angular velocity instruction is obtained through a preset reward function scoring mechanism in the simulation process, specifically including: controlling a forklift to perform a simulated forking operation according to the ground friction coefficient, the IMU vibration compensation amount, the second posture deviation, and the linear velocity and angular velocity continuous fine-tuning instruction; judging whether the forklift successfully forks according to a preset contact force threshold; determining the forklift score corresponding to the forklift when the forklift is successfully forked; judging whether the forklift collides during the forklift process according to a preset impact acceleration threshold; determining the collision deduction score corresponding to the forklift when a collision occurs; deducting points from the forklift's existing forklift score according to a preset algorithm for how much time the forklift takes from the start of forking to successful forking; adding points to the forklift's existing forklift score according to a preset algorithm for the distance between the forklift and the target pallet; when the forklift's existing forklift score reaches a preset total score threshold, stopping the simulation and obtaining the second linear velocity and angular velocity instruction.

[0012] In one example, the first posture deviation is input into the PID controller to obtain the first forklift linear speed and angular speed instructions, specifically including: decomposing the first posture deviation into a straight-line distance deviation and an angle deviation; inputting the straight-line distance deviation into the linear speed PID controller for weighted summation to obtain the first forklift linear speed instruction; inputting the angle deviation into the angular speed PID controller for weighted summation to obtain the first forklift angular speed instruction.

[0013] In one example, when the distance between the forklift and the target pallet reaches a preset first distance, the target pallet posture is determined based on the main direction of the target pallet and the three-dimensional coordinates of the target pallet, specifically including: when the forklift enters the preset first distance, identifying the binocular camera image through the target detection algorithm, determining the pixel coordinates of the target pallet, and converting the target pallet pixel coordinates into the three-dimensional coordinates of the target pallet through the binocular camera intrinsic parameter matrix; extracting the radar point cloud of the target pallet area within the preset first distance, and performing covariance matrix calculation on the radar point cloud to determine the main direction of the target pallet; and aligning the radar point cloud with the preset standard pallet point cloud through the ICP algorithm based on the main direction of the target pallet and the three-dimensional coordinates of the target pallet to obtain the target pallet posture.

[0014] In one example, after simulating the ground friction coefficient, IMU vibration compensation amount, and second posture deviation to obtain second linear velocity and angular velocity instructions, the method also includes: driving the forklift to perform a forking operation according to the second linear velocity and angular velocity; when the fork tines are inserted into the pallet, the contact area image of the fork tines and the pallet is captured by a binocular camera, and the contact area image is subjected to feature extraction by a pre-trained convolutional neural network to verify the forking state; if the verification fails, the ground friction coefficient, IMU vibration compensation amount, and second posture deviation are re-input into the Actor-Critic architecture, and a new forklift second linear velocity and angular velocity instruction is obtained through a preset reward function scoring mechanism.

[0015] On the other hand, an embodiment of the present application provides a forklift operation trajectory generation device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any one of the above-mentioned methods for generating a forklift operation trajectory.

[0016] On the other hand, an embodiment of the present application further provides a non-volatile computer storage medium for generating a forklift's forking operation trajectory, which stores computer-executable instructions. The computer-executable instructions can execute any of the above-mentioned methods for generating a forklift's forking operation trajectory.

[0017] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects:

[0018] This application constructs a coarse and fine double-layer adjustment. In the coarse adjustment stage, vision and laser fusion positioning and PID control are used to quickly correct the posture deviation, so as to achieve efficient operation results by compressing the path planning time from >3 seconds to less than 0.5 seconds and supporting the coordinated scheduling of 100 forklift clusters without lag; in the fine adjustment stage, the DDPG reinforcement learning algorithm is introduced to output continuous fine-tuning actions based on the friction coefficient and vibration amplitude, so as to achieve high-precision operation results with the fork picking accuracy increased from ±10mm to ±1mm and the pallet damage rate reduced from 1.2% to 0.05%; through the fusion of multimodal perception system and spatiotemporal calibration technology, vision is dominant in strong light and laser is dominant in dim light, so as to achieve full-scene reliable perception effect with pallet positioning error <3mm in dynamic environment and near-field obstacle detection blind spot reduced to 0.3 meters. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solution of the present application, some embodiments of the present application will be described in detail below with reference to the accompanying drawings, in which:

[0020] Figure 1 A flowchart of a method for generating a forklift operation trajectory for a forklift provided in an embodiment of the present application;

[0021] Figure 2 A schematic structural diagram of a forklift operation trajectory generation device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] To make the objectives, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0023] Some embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0024] Figure 1 This is a flow chart of a method for generating a forklift pick-up operation trajectory for an embodiment of the present application. This method can be applied to various business areas. Certain input parameters or intermediate results in this process can be manually adjusted to help improve accuracy.

[0025] The analysis method involved in the embodiments of the present application can be implemented by a terminal device or a server, and the present application does not impose any special restrictions on this. For ease of understanding and description, the following embodiments are described in detail using a controller as an example.

[0026] It is necessary to explain in advance that:

[0027] A Kalman filter is an algorithm that uses linear system state equations and observation data from the system's input and output to optimally estimate the system's state. In obstacle prediction scenarios, this application uses information such as the position and speed of dynamic targets (people and vehicles) as the system state, continuously updating the state estimate using sensor measurements to predict their future trajectory.

[0028] The PID (Proportional-Integral-Derivative) controller is a common feedback control algorithm that reduces the error by taking a weighted sum of the proportional, integral, and differential components of the error and outputting a control signal. In this scenario, the PID controller's input is the deviation (Δx, Δy, Δθ) between the pallet's target position and the fork's current position. The output is the forklift's linear velocity v and angular velocity ω commands.

[0029] Based on this, Figure 1 The process in may include the following steps:

[0030] S101: When the distance between the forklift and the target pallet reaches a preset first distance, the position and posture of the target pallet is determined according to the main direction of the target pallet and the three-dimensional coordinates of the target pallet.

[0031] It should be noted that, in some embodiments of the present application, when a forklift receives a forking task, the installed binocular camera, line laser radar and single-line laser radar are activated.

[0032] The binocular camera uses RGB-D data acquisition, outputting 1280×720 resolution color images and depth maps at 60 FPS. The line-shaped lidar performs 360° horizontal scanning at a rate of 300,000 points per second, completing point cloud collection within a 50-meter radius within 5 seconds. For example, at the warehouse entrance, the radar quickly constructs a global framework to identify aisle widths and shelf arrangement orientations. The single-line lidar performs high-frequency scanning (100Hz) in the near-field (0.3-5 meters) to detect obstacles around the equipment in real time. For example, when the equipment approaches the warehouse door, the single-line radar detects the threshold at 0.5 meters, triggering an obstacle avoidance strategy and avoiding the collision risk caused by the near-field blind spot of traditional 360° radars.

[0033] After the installed equipment is powered on, the ROS tf2 toolchain is used to achieve spatiotemporal synchronization of the binocular camera, line-aligned LiDAR, and single-line LiDAR, with a calibration error of <0.5mm. Global positioning is then performed on the warehouse electronic map uploaded by the user, generating a global navigation path from the forklift's current position (e.g., the warehouse entrance) to a preset first distance.

[0034] Furthermore, when the forklift moves according to the global navigation path, the line lidar point cloud is fused with the binocular camera image data, and static obstacles and dynamic obstacles are identified through the semantic segmentation network and the fused data; then the trajectory of the dynamic obstacle is predicted by the Kalman algorithm to update the global navigation path; and when the single-line lidar point cloud identifies a sudden obstacle, if the distance between the forklift and the sudden obstacle is less than a preset distance threshold and the time to reach the distance is less than a preset time threshold, the preset emergency avoidance instruction is triggered to complete the avoidance of the sudden obstacle (such as turning or other operations).

[0035] When the forklift enters the preset first distance according to the global navigation path (such as ≤5 meters from the target pallet), the binocular camera image is recognized by the YOLOv11 target detection algorithm. When the recognition confidence is greater than the preset confidence (detection confidence ≥0.95), it is determined to be the target pallet and the pixel coordinates (u, v) of the target pallet are output.

[0036] Furthermore, the target pallet pixel coordinates (u, v) are converted to the target pallet three-dimensional coordinates (Xc, Yc, Zc) through the binocular camera intrinsic parameter matrix (pre-calibrated by the checkerboard calibration method, with a reprojection error of <1mm). The formula is:

[0037] Where K is the camera intrinsic parameter matrix, (Zc) is the depth value of the corresponding pixel in the depth map (directly output by the binocular camera).

[0038] Furthermore, the radar point cloud of the target pallet area within a preset first distance is extracted, and the covariance matrix calculation (PCA principal component analysis) is performed on the radar point cloud to obtain the long axis direction of the pallet (angle error <0.5°) to determine the main direction of the target pallet; then, the three-dimensional coordinates (Xc, Yc, Zc) of the target pallet are used as the initial transformation matrix, and the radar point cloud of the pallet is aligned with the pre-stored standard pallet point cloud model (which has been calibrated with high precision in a laboratory environment) through the iterative closest point (ICP) algorithm, and the optimized target pallet pose (position error <2mm, angle error <0.3°) is output. The optimized target pallet pose is then converted into the precise pose (x, y, θ) in the world coordinate system through the extrinsic parameter matrix (the relative pose of the camera and the forklift body, which has been pre-calibrated). This is also the target pallet pose in the world coordinate system.

[0039] S102: Determine a first posture deviation between the target pallet and the fork tine based on the target pallet posture, the first current fork tine posture, and the first light intensity of the forking environment, input the first posture deviation into a PID controller, and obtain first forklift linear speed and angular speed instructions.

[0040] It should be noted that in some embodiments of the present application, the control weights of vision and laser are adjusted according to the ambient light intensity (calculated in real time through the brightness value of the camera image). When the first light intensity is greater than the preset light intensity, the binocular camera and the line laser radar point cloud weights are allocated according to the preset first weight distribution to determine the first pose deviation (Δx, Δy, Δθ) between the target pallet and the fork tines; when the first light intensity is less than the preset light intensity, the binocular camera and the line laser radar point cloud weights are allocated according to the preset second weight distribution to determine the first pose deviation (Δx, Δy, Δθ) between the target pallet and the fork tines. For example, when the light intensity is >500 lux, the vision weight is 0.7 and the laser radar weight is 0.3; when the light intensity is ≤500 lux, the laser radar weight is increased to 0.8, and vision is only used for posture verification.

[0041] Furthermore, the first posture deviation (Δx, Δy, Δθ) is decomposed into a linear distance deviation and an angular deviation. The linear distance deviation is then input into the linear velocity PID controller for weighted summation to obtain the first forklift linear velocity instruction. The angular deviation is input into the angular velocity PID controller for weighted summation to obtain the first forklift angular velocity instruction. The formula is:

[0042]

[0043] ω=Kp·Δθ+Ki·∫Δθdt

[0044] Where K is the gain coefficient of the PID controller, which is the key parameter used to adjust the control effect.

[0045] S103: driving the forklift according to the first forklift linear velocity and angular velocity instructions until the distance between the forklift and the target pallet reaches a preset second distance; the first distance is greater than the second distance.

[0046] It should be noted that in some embodiments of the present application, the first distance and the second distance are both user-defined distances. The first distance is usually taken as the distance at which the binocular camera can clearly scan the target pallet, and the second distance is the distance between the forklift forks and the target pallet that is less than or equal to 0.5m.

[0047] S104: During the process of traveling to the preset second distance, determining a ground friction coefficient and an IMU vibration compensation amount according to a preset friction-vibration joint estimation model and the linear velocity and angular velocity of the first forklift.

[0048] It should be noted that in some embodiments of the present application, the motor current and linear speed of the forklift during driving, as well as the preset torque constant and wheel diameter of the forklift, are simultaneously input into a preset friction-vibration joint estimation model to reversely infer the ground friction coefficient; the formula is:

[0049]

[0050] where K t is the torque constant, and r is the wheel diameter.

[0051] At the same time, the amplitude and frequency of the IMU vibration during the forklift's driving process are input into the friction-vibration joint estimation model to generate the IMU vibration compensation amount. The formula is:

[0052] δ' x =δ x +A·sin(2πft)

[0053] Where A is the amplitude and f is the frequency.

[0054] S105: When the preset second distance is reached, a second posture deviation between the target pallet and the fork tine is determined according to the posture of the target pallet, the second current posture of the fork tine, and the second illumination intensity of the forking environment.

[0055] It should be noted that, in some embodiments of the present application, when the forklift travels to a preset second distance at a first linear velocity and angular velocity, the second posture deviation s = [Δx1, Δy1, Δθ1] between the target pallet and the fork teeth is obtained according to the same steps as S102.

[0056] S106: Simulating the ground friction coefficient, the IMU vibration compensation amount, and the second posture deviation to obtain second linear velocity and angular velocity instructions to drive the forklift to perform a forking operation based on the second linear velocity and angular velocity instructions.

[0057] It should be noted that in some embodiments of the present application, the calculated ground friction coefficient, IMU vibration compensation amount, and second posture deviation are input into the Actor network (structure is a fully connected layer) for offline pre-training; for example, 10 typical disturbance scenarios (such as ground tilt 3°, vibration noise) are loaded in the Gazebo simulation environment, and 5000 rounds of training are performed to obtain the continuous fine-tuning instructions a = [δx, δy, δθ] of the linear velocity and angular velocity.

[0058] Among them, δx, δy∈[-1,1]mm / step, δθ∈[-0.5°,0.5°] / step.

[0059] Furthermore, the ground friction coefficient, IMU vibration compensation, second posture deviation, linear velocity and angular velocity continuous fine-tuning instructions are input into the Critic network for simulation learning, and the mechanism is obtained by the preset reward function:

[0060]

[0061] Based on the preset contact force threshold, determine whether the forklift has successfully picked up the fork;

[0062] When the forklift is successfully forked, a forklift score corresponding to the forklift is determined;

[0063] Based on the preset impact acceleration threshold, determine whether the forklift has collided during the forking process;

[0064] When a collision occurs, determining the collision deduction points corresponding to the forklift;

[0065] According to the preset algorithm, the time it takes for the forklift to successfully fork is used to deduct the forklift score.

[0066] Based on the preset algorithm of the distance between the forklift and the target pallet, the forklift's points are added to the forklift's points.

[0067] When the forklift's fork-picking score reaches the preset total score threshold, the simulation is stopped and the second linear velocity and angular velocity instructions are obtained.

[0068] Furthermore, after obtaining the second linear velocity and angular velocity instructions, the forklift is driven to perform the forking operation according to the second linear velocity and angular velocity. After the fork tines are inserted into the pallet, the contact area image of the fork tines and the pallet is captured by the binocular camera, and the features of the contact area image are extracted by the pre-trained convolutional neural network to verify the forking state. Specifically, the convolution layer of ResNet-18 (including 7×7 convolution, maximum pooling, and 4 residual blocks) is used to extract key features in the image, including the edge contour of the fork tines and the pallet, the contact gap, and the insertion depth of the fork hole. The model then outputs three-dimensional probability values ​​((P_{full}, P_{partial}, P_{none})), corresponding to the three states of "full insertion", "partial insertion", and "not inserted" respectively.

[0069] Then, the average probability of the inference results of the three consecutive frames is taken. If the highest probability category (such as (P_{full})) is ≥ 0.9, the category is output; if the highest probability is < 0.9, it is marked as an "uncertain" state.

[0070] When the verification result is full insertion (P_{full}≥0.9), the verification is successful. Other states are verification failures. If the verification fails, an alarm is triggered (buzzer + LED flashing), and the process returns to S106 for fine-tuning. A maximum of 3 retries are performed. If it still fails, it is marked as an abnormal task.

[0071] It should be noted that although the embodiments of this application are based on Figure 1 Steps S101 to S106 are described in sequence, but this does not mean that steps S101 to S106 must be performed in a strict order. Figure 1 The order shown in FIG1 is to introduce and explain steps S101 to S106 in order to facilitate those skilled in the art to understand the technical solutions of the embodiments of the present application. In other words, in the embodiments of the present application, the order of steps S101 to S106 can be adjusted appropriately according to actual needs.

[0072] pass Figure 1In the coarse adjustment stage, vision and laser fusion positioning and PID control are used to quickly correct the posture deviation, so as to achieve efficient operation effect by compressing the path planning time from >3 seconds to less than 0.5 seconds and supporting the coordinated scheduling of 100 forklifts in a cluster without lag; in the fine adjustment stage, the DDPG reinforcement learning algorithm is introduced to output continuous fine-tuning actions based on the friction coefficient and vibration amplitude, so as to achieve high-precision operation effect with the fork picking accuracy increased from ±10mm to ±1mm and the pallet damage rate reduced from 1.2% to 0.05%; through the fusion of multimodal perception system and spatiotemporal calibration technology, vision is used as the main method under strong light and laser is used as the main method under dark light, so as to achieve full-scene reliable perception effect with pallet positioning error <3mm in dynamic environment and near-field obstacle detection blind spot reduced to 0.3 meters.

[0073] Figure 2 A schematic structural diagram of a forklift operation trajectory generation device provided in an embodiment of the present application includes:

[0074] at least one processor; and,

[0075] a memory communicatively connected to at least one processor; wherein,

[0076] The memory stores instructions that can be executed by at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any one of the above-mentioned methods for generating a forklift operation trajectory.

[0077] Some embodiments of the present application provide a non-volatile computer storage medium for generating a forklift operation trajectory, which stores computer-executable instructions. The computer-executable instructions can execute any of the above-mentioned methods for generating a forklift operation trajectory.

[0078] The various embodiments in this application are described in a progressive manner. Similar portions between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the device and medium embodiments are generally similar to the method embodiments, so their descriptions are relatively simple. For relevant portions, refer to the descriptions of the method embodiments.

[0079] The devices and media provided in the embodiments of the present application correspond one-to-one to the methods. Therefore, the devices and media also have similar beneficial technical effects to their corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be repeated here.

[0080] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0081] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0082] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0083] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0084] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0085] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM), and non-volatile memory such as read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0086] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0087] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0088] The above are merely embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modifications, equivalent replacements, improvements, etc. made within the technical principles of the present application should fall within the scope of protection of the present application.

Claims

1. A method for generating a forklift operation trajectory, characterized in that: The method comprises: When the distance between the forklift and the target pallet reaches a preset first distance, the position of the target pallet is determined according to the main direction of the target pallet and the three-dimensional coordinates of the target pallet; Determining a first posture deviation between the target pallet and the fork tines according to the target pallet posture, the first current fork tine posture, and a first light intensity of the forking environment, inputting the first posture deviation into a PID controller to obtain first forklift linear speed and angular speed instructions; According to the first forklift linear velocity and angular velocity instructions, the forklift is driven to travel until the distance from the target pallet reaches a preset second distance; the first distance is greater than the second distance; During the process of traveling to the preset second distance, determining the ground friction coefficient and the IMU vibration compensation amount according to a preset friction-vibration joint estimation model and the linear velocity and angular velocity of the first forklift; When the preset second distance is reached, determining a second posture deviation between the target pallet and the fork tine according to the target pallet posture, the second current fork tine posture, and a second light intensity of the forking environment; The ground friction coefficient, the IMU vibration compensation amount, and the second posture deviation are simulated to obtain second linear velocity and angular velocity instructions to drive the forklift to perform a forking operation based on the second linear velocity and angular velocity instructions.

2. The method according to claim 1, characterized in that Before the distance between the forklift and the target pallet reaches a preset first distance, the method further includes: When receiving a fork-picking task, the binocular camera, line laser radar and single-line laser radar are calibrated in time and space; By performing global positioning on the warehouse electronic map, a global navigation path is generated from the current position of the forklift to the preset first distance; When the forklift moves according to the global navigation path, the line laser radar point cloud is fused with the binocular camera image data, and static obstacles and dynamic obstacles are identified through the semantic segmentation network and the fused data; Predicting the trajectory of dynamic obstacles using the Kalman algorithm to update the global navigation path; When a sudden obstacle is identified based on the single-line lidar point cloud, if the distance between the forklift and the sudden obstacle is less than the preset distance threshold and the time to reach the distance is less than the preset time threshold, the preset emergency avoidance instruction is triggered to complete the avoidance of the sudden obstacle.

3. The method according to claim 1, characterized in that The determining, based on the target pallet posture, the first current fork tine posture, and the first illumination intensity of the forking environment, a first posture deviation between the target pallet and the fork tine specifically includes: When the first light intensity is greater than the preset light intensity, the binocular camera and the alignment laser radar point cloud weights are allocated according to a preset first weight allocation to determine the first pose deviation between the target pallet and the fork tines; When the first light intensity is less than the preset light intensity, the binocular camera and the line laser radar point cloud weights are allocated according to the preset second weight distribution to determine the first pose deviation between the target pallet and the fork tines.

4. The method according to claim 1, wherein Determining the ground friction coefficient and the IMU vibration compensation amount based on the preset friction-vibration joint estimation model and the linear velocity and angular velocity of the first forklift specifically includes: The motor current and linear speed of the forklift during travel, as well as the preset torque constant and wheel diameter of the forklift, are input into a preset friction-vibration joint estimation model to determine the ground friction coefficient; The amplitude and frequency of the IMU vibration during the forklift's driving process are input into the friction-vibration joint estimation model to generate the IMU vibration compensation amount.

5. The method according to claim 1, characterized in that The simulating of the ground friction coefficient, the IMU vibration compensation amount, and the second posture deviation to obtain the second linear velocity and angular velocity instructions specifically includes: The ground friction coefficient, IMU vibration compensation, and second pose deviation are input into the Actor network to obtain continuous fine-tuning instructions for linear velocity and angular velocity. The ground friction coefficient, IMU vibration compensation, second posture deviation, linear velocity and angular velocity continuous fine-tuning instructions are input into the Critic network for simulation, and the second linear velocity and angular velocity instructions are obtained through the preset reward function scoring mechanism during the simulation process.

6. The method according to claim 5, characterized in that The second linear velocity and angular velocity instructions are obtained through the preset reward function scoring mechanism during the simulation process, specifically including: The forklift is controlled to perform simulated forklift operations based on the ground friction coefficient, IMU vibration compensation, second posture deviation, linear speed and angular speed continuous fine-tuning instructions; Based on the preset contact force threshold, determine whether the forklift has successfully picked up the fork; When the forklift is successfully forked, a forklift score corresponding to the forklift is determined; Based on the preset impact acceleration threshold, determine whether the forklift has collided during the forking process; When a collision occurs, determining the collision deduction points corresponding to the forklift; According to the preset algorithm, the time it takes for the forklift to successfully fork is used to deduct the forklift score. Based on the preset algorithm of the distance between the forklift and the target pallet, the forklift's points are added to the forklift's points. When the forklift's fork-picking score reaches the preset total score threshold, the simulation is stopped and the second linear velocity and angular velocity instructions are obtained.

7. The method according to claim 1, characterized in that Inputting the first posture deviation into a PID controller to obtain first forklift linear speed and angular speed instructions specifically includes: Decomposing the first posture deviation into a straight-line distance deviation and an angle deviation; Inputting the straight-line distance deviation into the linear speed PID controller for weighted summation to obtain a first forklift linear speed instruction; The angle deviation is input into the angular velocity PID controller for weighted summation to obtain a first forklift angular velocity instruction.

8. The method according to claim 1, characterized in that When the distance between the forklift and the target pallet reaches a preset first distance, determining the position of the target pallet according to the main direction of the target pallet and the three-dimensional coordinates of the target pallet specifically includes: When the forklift enters the preset first distance, the binocular camera image is recognized by the target detection algorithm to determine the pixel coordinates of the target pallet, and the pixel coordinates of the target pallet are converted into the three-dimensional coordinates of the target pallet through the binocular camera intrinsic parameter matrix; Extracting a radar point cloud of a target pallet region within a preset first distance, and performing covariance matrix calculation on the radar point cloud to determine a main direction of the target pallet; According to the main direction of the target pallet and the three-dimensional coordinates of the target pallet, the radar point cloud is aligned with the preset standard pallet point cloud through the ICP algorithm to obtain the target pallet position and posture.

9. The method according to claim 1, characterized in that After simulating the ground friction coefficient, the IMU vibration compensation amount, and the second posture deviation to obtain second linear velocity and angular velocity instructions, the method further includes: driving the forklift to perform a forking operation according to the second linear velocity and angular velocity; When the fork tines are inserted into the pallet, the contact area between the fork tines and the pallet is captured by a binocular camera, and features are extracted from the contact area image using a pre-trained convolutional neural network to verify the forking status. If the verification fails, the ground friction coefficient, IMU vibration compensation, and second posture deviation are re-input into the Actor-Critic architecture, and the new second linear velocity and angular velocity instructions of the forklift are obtained through the preset reward function scoring mechanism.

10. A device for generating a forklift operation trajectory, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method for generating a forklift operation trajectory according to any one of claims 1 to 8.

Images