A method, system, storage medium and program product for pallet picking

Abstract

A pallet picking method, system, storage medium and program product, which relate to the field of general control or regulation systems. The method includes: collecting an image of the pallet area to obtain the pallet position coordinates and the cargo distribution data; performing a centroid calculation to obtain a predicted value of the cargo plane centroid, calculating the force compensation coefficient of the scissor lift module, and determining the initial force value; determining the motion trajectory parameters of the robot and generating a motion control instruction to control the fork of the robot to move to a preset position; generating an initial lifting instruction to control the scissor lift module of the robot to perform a lifting motion; obtaining the real-time force data of the scissor lift module and determining the horizontal attitude deviation value of the pallet, and generating a lifting compensation instruction to adjust the real-time force data so that the target pallet is adjusted to a horizontal state; generating a vehicle body forward movement instruction and a module return instruction to control the target pallet to be placed on the cargo vehicle body. Implementing this application can improve the stability of the robot in picking and moving pallets.

Description

Technical Field

[0001] The present application relates to the field of general control or regulation systems, and particularly to a method, system, storage medium and program product for pallet picking and placing. Background Art

[0002] With the rapid development of intelligent manufacturing and intelligent warehousing, the material handling demand in automated stereoscopic warehouses is increasing day by day. AMR (Autonomous Mobile Robot) has been widely used in the logistics field due to its flexibility and intelligence, especially playing an important role in material handling scenarios in narrow aisles and complex environments.

[0003] In the related art, the AMR device uses a front - rear moving fork and a scissor - lift module for pallet handling operations. Specifically, the fork extends into the bottom of the pallet, and the scissor - lift module is used to lift the pallet to a predetermined height, and then the vehicle body moves forward to place the pallet. During the lifting of the fork, the scissor - lift module is lifted synchronously according to preset parameters, and the force supported by the pallet is controlled by a preset fixed value.

[0004] However, in practical applications, due to differences in factors such as the placement position and weight distribution of the goods on the pallet, relying solely on preset parameters for lifting control often leads to unstable pallet loading. Summary of the Invention

[0005] The present application provides a method, system, storage medium and program product for pallet picking and placing, which are used to improve the stability of the robot in picking and moving the pallet.

[0006] In a first aspect, the present application provides a method for pallet picking, which is applied to a picking control system. The method includes: collecting an image of a pallet area containing a target pallet within a collection area, and performing edge extraction on the pallet area image to obtain pallet position coordinates and cargo distribution data; calculating the center of gravity based on the cargo distribution data to obtain a predicted value of the center of gravity of the cargo plane in the horizontal direction; calculating the force compensation coefficients of each scissor fork module of the robot according to the predicted value of the center of gravity of the cargo plane, and determining the initial force values of each scissor fork module; determining the motion trajectory parameters of the robot according to the pallet position coordinates, and generating a motion control instruction to control the forklift of the robot to move to a preset position of the target pallet; generating an initial lifting instruction according to the initial force value to control the scissor fork module of the robot to perform a lifting motion; obtaining the real-time force data of the scissor fork module, and determining the horizontal attitude deviation value of the pallet according to the pallet area image and the inclination data collected by the attitude sensor; generating a lifting compensation instruction based on the horizontal attitude deviation value of the pallet to adjust the real-time force data so that the target pallet is adjusted to a horizontal state; after the target pallet is lifted to a preset height, generating a vehicle body forward movement instruction and a module return instruction to control the cargo-carrying vehicle body of the robot to move below the target pallet, and controlling the scissor fork module to return to place the target pallet on the cargo-carrying vehicle body.

[0007] In the above embodiment, the picking control system performs edge extraction based on the pallet area image to obtain the pallet position and cargo distribution information, combines the center of gravity calculation to obtain the predicted value of the center of gravity of the plane, and calculates the force compensation coefficient of the scissor fork module accordingly; during the lifting process, the picking control system realizes the dynamic adjustment of the horizontal attitude of the pallet through the real-time force data and the attitude sensor data; this closed-loop control method based on visual information and force feedback can effectively solve the tilt problem caused by uneven cargo distribution during the pallet handling process, and improve the stability and safety of the entire picking process.

[0008] Combined with some embodiments of the first aspect, in some embodiments, the step of determining the motion trajectory parameters of the robot according to the pallet position coordinates and generating a motion control instruction to control the forklift of the robot to move to a preset position of the target pallet specifically includes: converting the pallet position coordinates into target position data in the local coordinate system of the robot; obtaining the contour features of the target pallet, and identifying the pallet type and corresponding standard size parameters of the target pallet; calculating the relative position relationship between the scissor fork module of the robot and the pallet support point according to the target position data, contour features and standard size parameters; determining the motion trajectory parameters of the forklift based on the relative position relationship, and generating a corresponding motion control instruction to control the forklift of the robot to move to a preset position of the target pallet.

[0009] In the above embodiments, the fork-taking control system converts the tray position coordinates into the local coordinate system of the robot, and combines the tray type and standard size parameters to accurately calculate the relative position relationship between the fork and the tray support point, which can ensure that the fork accurately inserts into the empty space at the bottom of the tray, avoiding collisions or position deviations. Through contour feature recognition and standard parameter matching, adaptive fork-taking control for different types of trays is achieved.

[0010] Combined with some embodiments of the first aspect, in some embodiments, the target tray is a cross-shaped tray; the step of calculating the relative position relationship between the scissor fork module of the robot and the tray support point according to the target position data, contour features and standard size parameters specifically includes: determining the empty space area of the cross-shaped tray according to the contour features and standard size parameters; based on the target position data and the empty space area, calculating the optimal support positions corresponding to each scissor fork module of the robot; determining the relative position relationship according to the optimal support positions to avoid the scissor fork module from touching the bottom plane of the cross-shaped tray.

[0011] In the above embodiments, for the special structure of the cross-shaped tray, the fork-taking control system will determine the empty space area according to the contour features and standard size, and calculate the optimal support position for each scissor fork module, avoiding interference between the scissor fork module and the bottom structure of the tray, while ensuring uniform distribution of the bearing capacity and improving the reliability of fork-taking the cross-shaped tray.

[0012] Combined with some embodiments of the first aspect, in some embodiments, the step of calculating the predicted value of the center of gravity of the cargo plane in the horizontal direction based on the cargo distribution data specifically includes: obtaining the type information and mass attribute data of the cargo on the target tray stored in the cargo area management system; determining the placement position of the cargo according to the type information and the tray area image; calculating the predicted value of the center of gravity of the cargo plane according to the mass attribute data, the placement position and the centroid calculation formula.

[0013] In the above embodiments, by combining the cargo information in the cargo area management system and the visual data collected in real time, the fork-taking control system can accurately obtain the type, mass and spatial distribution information of the cargo, and through the centroid calculation formula, obtain a more accurate predicted value of the plane center of gravity, improving the accuracy of the entire fork-taking process.

[0014] Combined with some embodiments of the first aspect, in some embodiments, there are multiple pieces of cargo; the step of determining the placement position of the cargo according to the type information and the tray area image specifically includes: extracting the cargo contour data in the tray area image, performing target detection and image segmentation operations to generate the spatial coordinate data and boundary data of each piece of cargo; calculating the relative distance and overlap degree between the cargoes according to the spatial coordinate data and the boundary data; substituting the relative distance and overlap degree into a preset spatial relationship model to generate a three-dimensional coordinate set representing the placement position of the cargoes.

[0015] In the above embodiments, for the case of multiple goods, the fork-taking control system analyzes the spatial position relationship of each good through target detection and image segmentation technologies, calculates the relative distance and overlap degree, and generates a three-dimensional coordinate set using the spatial relationship model, which can more accurately predict the center-of-gravity position and improve the balance during the fork-taking process.

[0016] In combination with some embodiments of the first aspect, in some embodiments, before the step of determining the motion trajectory parameters of the robot according to the pallet position coordinates and generating a motion control instruction to control the fork of the robot to move to a preset position of the target pallet, the method further includes: obtaining the pallet weight data of the target pallet; reading the fork specification data of the robot to determine the rated load parameter and the supporting force data of the scissor fork module; when the pallet weight data is simultaneously less than the rated load parameter and the supporting force data, generating a fork-taking permission signal to enable the robot to perform the fork-taking action.

[0017] In the above embodiments, before the fork-taking control system performs the fork-taking action, the system performs a safety check on the pallet weight, the rated load of the robot, and the supporting force, which can effectively avoid overloading operations, ensure the safety of the equipment and goods, and improve the fork-taking reliability.

[0018] In combination with some embodiments of the first aspect, in some embodiments, after the step of generating a vehicle body forward movement instruction and a module return instruction to control the load-carrying vehicle body of the robot to move below the target pallet and controlling the scissor fork module to return so that the target pallet is placed on the load-carrying vehicle body after the target pallet is lifted to a preset height, the method further includes: determining the planar center-of-gravity coordinates of the target pallet according to the predicted value of the planar center of gravity of the goods and the real-time force data; determining the spatial center-of-gravity coordinates of the target pallet according to the planar center-of-gravity coordinates and the goods distribution data; calculating the steering compensation coefficients of the robot in each direction based on the spatial center-of-gravity coordinates; and determining the steering speed value of the robot based on the steering compensation coefficients.

[0019] In the above embodiments, the fork-taking control system can correct the predicted value of the center of gravity according to the actual force data, calculate more accurate spatial center-of-gravity coordinates, and dynamically adjust the steering compensation coefficients of the robot accordingly, which can optimize the steering performance during transportation and improve the motion stability in the loaded state.

[0020] In a second aspect, an embodiment of the present application provides a fork-taking control system, which includes: one or more processors and a memory; the memory is coupled to the one or more processors, and the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the fork-taking control system to execute the method described in the first aspect and any possible implementation manner in the first aspect.

[0021] In a third aspect, an embodiment of the present application provides a computer program product including instructions. When the computer program product runs on a fork-taking control system, the fork-taking control system is caused to execute the method described in the first aspect and any possible implementation manner in the first aspect.

[0022] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium including instructions. When the instructions run on a fork-taking control system, the fork-taking control system is caused to execute the method described in the first aspect and any possible implementation manner in the first aspect.

[0023] It can be understood that the fork-taking control system provided in the second aspect, the computer program product provided in the third aspect, and the computer storage medium provided in the fourth aspect are all used to execute the method provided in the embodiment of the present application. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method, and will not be elaborated here.

[0024] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0025] 1. Due to the adoption of a closed-loop control scheme based on visual information and force feedback, the system acquires position coordinates and cargo distribution data by collecting images of the tray area, and realizes precise control by combining center-of-gravity calculation and force compensation techniques; during the execution process, by real-time monitoring the force data and tray attitude of the scissor lift module, compensation instructions are dynamically generated for adjustment; therefore, it can accurately perceive the spatial state of the tray and the cargo, adjust the lifting parameters in real time, dynamically compensate for attitude deviations, effectively solve the problem in the related art that only relying on preset parameters cannot cope with uneven loads, and thus realize a stable and reliable tray fork-taking and handling process, significantly improving the operation efficiency and safety of automated material handling.

[0026] 2. Due to the adoption of a precise positioning scheme for multi-dimensional information fusion, the system maps the tray position to the local coordinate system of the robot through coordinate system conversion, and combines tray type recognition and standard size parameters for contour feature analysis to accurately calculate the relative position relationship between the fork and the tray support point; therefore, it can automatically adjust the fork-taking strategy according to the structural characteristics of different types of trays, precisely control the movement trajectory of the fork, effectively solve the problems in the related art of poor adaptability to different tray types and low positioning accuracy, and thus realize precise fork-taking of various standard trays, greatly reducing the collision risk during the fork-taking process and improving the operation reliability.

[0027] 3. Since the system adopts a collaborative analysis solution for the cargo area management system and real-time visual data, it can obtain complete cargo information, including type, quality attributes, and actual placement location, and obtain an accurate predicted value of the planar center of gravity through the centroid calculation formula. Therefore, it can comprehensively grasp the spatial distribution state of the cargo on the pallet, provide accurate reference data for force compensation and attitude adjustment, effectively solve the problem of inaccurate center of gravity prediction due to lack of cargo information in the related technology, and further achieve more accurate fork-taking control, significantly improving the balance and stability in the loaded state. Description of the Drawings

[0028] Figure 1 is a schematic diagram of an application scenario of the pallet fork-taking method in an embodiment of the present application;

[0029] Figure 2 is a schematic flowchart of the pallet fork-taking method in an embodiment of the present application;

[0030] Figure 3 is another schematic flowchart of the pallet fork-taking method in an embodiment of the present application;

[0031] Figure 4 is a schematic structural diagram of an entity device of the fork-taking control system in an embodiment of the present application.

[0032] Description of the Reference Numerals in the Drawings:

[0033] 101, AMR device; 102, picking fork; 103, scissor fork module; 104, target pallet; 105, pallet cargo. Detailed Embodiments

[0034] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. As used in the specification of the present application, the singular forms "a", "an", "above-mentioned", "the", and "this" are also intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in the present application refers to any or all possible combinations including one or more of the listed items.

[0035] Hereinafter, the terms "first" and "second" are only used for descriptive purposes, and cannot be construed as implying or suggesting relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of the present application, unless otherwise specified, the meaning of "a plurality" is two or more.

[0036] For ease of understanding, the application scenarios of the embodiments of the present application are introduced below.

[0037] Please refer to Figure 1 , Figure 1 which is a schematic diagram of an application scenario of the pallet picking method in the embodiment of the present application. In Figure 1 , a static action diagram of the AMR device 101 for pallet picking in the present application is shown. First, the composition of the AMR device 101 is introduced. The AMR device 101 includes a picking fork 102 that can move back and forth. The picking fork 102 is divided into two fork arms on the left and right, which can be used to pick up the target pallet 104 and the pallet goods 105. Two scissor lift modules 103 are loaded on the fork arm of a single picking fork 102, and the principle is similar to that of a jack, which can be used to lift the picking fork 102. When the pallet picks up the pallet goods 105, it is divided into three steps, as shown in Figure 1 . That is: Action ①, the picking fork 102 moves forward and forks into the bottom of the target pallet 104 in the shape of a three-way or four-way; the target pallet 104 is placed with irregularly shaped pallet goods 105; Action ②, control the scissor lift module 103 to lift the target pallet 104 upward to a certain height; Action ③, the vehicle body of the AMR device 101 moves forward to directly below the target pallet 104; at this time, the scissor lift module 103 is retracted, and the target pallet 104 moves downward, and then it can be completely placed on the vehicle body of the AMR device 101. Since the target pallet 104 is placed above the vehicle body of the AMR device 101, the overall turning radius is small, which can meet the flat-bottom handling of AMR in narrow aisles, and the application scenarios are relatively rich.

[0038] However, in daily operations, different-sized box pallet goods 105 may be placed on a target pallet 104 at the same time, and the weight distribution is uneven. When the AMR device 101 picks up according to the preset program, since the center of gravity position of the pallet goods 105 cannot be accurately judged, the target pallet 104 often tilts. Especially during the rapid lifting process, the center of gravity offset will cause the target pallet 104 to shake, and in severe cases, the pallet goods 105 will even fall.

[0039] In the related art, the basic pallet picking function can be achieved by using fixed parameter control and simple visual detection methods. This method mainly relies on preset standard parameters and basic image recognition algorithms, and cannot be dynamically adjusted according to different load conditions, nor does it have the ability to monitor and compensate the pallet posture in real time. The following introduces the scenario of using the pallet picking method in the related art.

[0040] In the related art, preset fixed parameters and simple visual detection methods are usually adopted to solve the problem of pallet picking. For example, the AMR device 101 used in a certain logistics center is equipped with a basic visual system that can identify the position of the pallet and perform simple contour detection. The system has preset the center of gravity position and lifting parameters of the standard target pallet 104, and all pallet picking operations are performed according to these fixed parameters. However, when encountering unevenly distributed or heavy goods, the preset parameters often cannot meet the actual needs. In a picking task, the load on one side of the target pallet 104 is significantly greater than that on the other side, resulting in obvious tilting during the lifting process and requiring manual intervention for adjustment. This method cannot perceive and respond to load changes in real time, nor does it have the ability to dynamically adjust the posture of the pallet, reducing the reliability of automated operations.

[0041] By adopting the pallet picking method in the embodiment of the present application, through real-time image acquisition and edge detection technology, the position of the target pallet 104 and the distribution state of the pallet goods 105 are accurately identified. Combining dynamic center of gravity calculation and intelligent force compensation mechanism, intelligent control of the pallet picking process is realized, which can not only ensure the stability of the picking process, but also adapt to various complex goods distribution situations. The following introduces the scenario where the pallet picking method in the present application is used.

[0042] After adopting the pallet picking method of the present application, the picking operation efficiency of a certain intelligent warehouse has been significantly improved. The system accurately identifies the pallet position and the goods distribution situation through real-time image acquisition and edge detection. For example, for a target pallet 104 loaded with multiple boxes of different weights, the system can calculate the accurate planar center of gravity position through the goods distribution data and dynamically adjust the force compensation coefficients of each scissor lift module 103 accordingly. During the picking process, the system continuously monitors the posture of the target pallet 104. When a slight tilt is detected, a compensation instruction is immediately generated to adjust the lifting force to ensure that the target pallet 104 always remains horizontal. This intelligent control method enables stable and reliable picking operations even when the weight distribution of the pallet goods 105 is uneven, greatly reducing the need for manual intervention.

[0043] It can be seen that by adopting the pallet picking method in the embodiment of the present application, while realizing the basic pallet picking function, it can effectively solve the problems of unstable lifting and easy tilting of the target pallet 104 under the traditional fixed parameter control method, and thus achieve a more efficient and reliable automated picking operation.

[0044] For easy understanding, the method provided in this embodiment will be described in terms of its process in combination with the above scenario. Please refer to Figure 2 , which is a schematic flow diagram of the pallet picking method in the embodiment of the present application.

[0045] S201. Collect the tray area image containing the target tray 104 within the collection area, and perform edge extraction on the tray area image to obtain the tray position coordinates and the cargo distribution data.

[0046] Among them, the tray area refers to a specific spatial range that includes the target tray 104 and its surrounding environment; the tray area image refers to a two-dimensional digital image collected by the vision sensor on the robot (i.e., the AMR device 101, which will be referred to as the robot for convenience of description hereinafter) or captured by the monitoring camera in this area that is interconnected with the robot; edge extraction refers to the process of extracting the object contour from the image using image processing algorithms; the tray position coordinates are used to represent the spatial position of the target tray 104 in the robot coordinate system; the cargo distribution data represents the spatial arrangement and density information of the tray cargo 105 on the target tray 104.

[0047] Before starting to execute the tray picking task, the picking control system needs to obtain the accurate position and cargo distribution of the target tray 104. Specifically, the picking control system first activates the vision sensor on the robot to collect high-resolution images within a certain range. Subsequently, noise and light interference are eliminated through image preprocessing, and edge detection algorithms (such as Canny or Sobel operators) are applied to extract the object contours in the image. The target tray 104 is identified through contour feature matching, and its three-dimensional spatial coordinates are calculated in combination with depth information. At the same time, image segmentation and object detection technologies are used to analyze the placement state of the tray cargo 105 on the target tray 104, and a data matrix representing the spatial distribution of the tray cargo 105 is generated.

[0048] In some embodiments, the positioning of the target tray 104 and the analysis of the cargo distribution can be achieved in multiple ways: Optionally, a binocular stereo vision system is used to calculate the three-dimensional position of the target, the depth information is calculated through the disparity map, and coordinate transformation is performed in combination with the image coordinates and camera calibration parameters to finally obtain the accurate position of the target tray 104 in the robot coordinate system; Optionally, structured light or time-of-flight (TOF) cameras are used to directly obtain depth images, point cloud registration and ICP algorithms are used for pose estimation, and at the same time, the image is semantically segmented through a deep learning model to identify the tray cargo 105 area and analyze its distribution characteristics. It can be understood that other visual sensing methods and image processing algorithms can also be used to achieve object detection and position estimation, which are not limited here.

[0049] In practical applications, there are situations of occlusion or uneven illumination in the pallet area, which affect the image quality and the accuracy of feature extraction. To address this, the picking control system adopts a multi-angle image acquisition and fusion solution: First, multiple frames of images are acquired from different perspectives, and the corresponding relationships between the images are established through feature point matching; then, the RANSAC algorithm is used to eliminate abnormal matching points to achieve robust pose estimation; finally, the multi-view geometric constraints are used to optimize the reconstruction result to improve the positioning accuracy. At the same time, the system can also adjust the camera parameters in real time in combination with the ambient light sensor to ensure the image quality.

[0050] S202. Calculate the center of gravity based on the cargo distribution data to obtain the predicted value of the center of gravity of the cargo plane in the horizontal direction.

[0051] Among them, the center of gravity of the pallet cargo plane refers to the position of the mass center of the pallet cargo 105 in the horizontal plane; the predicted value of the center of gravity represents the estimated center of gravity coordinates calculated based on the current data; the horizontal direction refers to the XY plane parallel to the ground.

[0052] After obtaining the cargo distribution data, the picking control system needs to calculate the center of gravity position of the pallet cargo 105 to guide the subsequent picking actions. Specifically, the picking control system first converts the cargo distribution data into a discrete mass distribution matrix, and each grid cell corresponds to a local mass value. By dividing the entire pallet area into a grid, a density distribution model is established. Considering the stacking situation of the pallet cargo 105, the system also needs to calculate the three-dimensional mass distribution in combination with the height information. Finally, the center of gravity coordinates in the horizontal plane are calculated by the method of weighted average.

[0053] In some embodiments, the calculation of the center of gravity of the cargo can be achieved in various ways: Optionally, a three-dimensional voxel grid is constructed based on the image segmentation result, a corresponding density value is assigned to each voxel, and the total mass and the center of gravity position are calculated through integration; Optionally, using a pre-calibrated mass feature library of the pallet cargo 105, the identified cargo type is associated with the standard mass data, and the center of gravity is calculated in combination with the actual placement position. It can be understood that other mathematical models and calculation methods can also be used to achieve the center of gravity prediction, which is not limited here.

[0054] During the implementation process, when the stacking structure of the cargo is complex or there are irregularly shaped objects, the accuracy of the center of gravity calculation will be affected. To address this, the picking control system adopts a hierarchical analysis solution: First, the pallet cargo 105 is scanned layer by layer to obtain the planar projection contour of each layer; then, the layer mass is calculated according to the contour area and the preset density; then, the spatial relationship between the layers is restored through a three-dimensional reconstruction algorithm; finally, considering the mass distribution of each layer comprehensively, a more accurate predicted value of the center of gravity is obtained through iterative optimization.

[0055] S203. Calculate the force compensation coefficients of each scissor lift module 103 of the robot based on the predicted value of the center of gravity of the goods plane, and determine the initial force values of each scissor lift module 103.

[0056] Among them, the scissor lift module 103 represents the mechanical execution unit of the robot for lifting the target pallet 104; the force compensation coefficient refers to the torque adjustment parameter calculated according to the center of gravity position; the initial force value represents the reference force value that each scissor lift module 103 needs to apply; the torque balance point represents the ideal force application point required to keep the target pallet 104 horizontal.

[0057] After obtaining the predicted value of the center of gravity, the fork picking control system needs to adjust the force distribution of each scissor lift module 103 accordingly. Specifically, the fork picking control system first establishes a mechanical model of the target pallet 104 - pallet goods 105 system, and projects the center of gravity position onto the force application plane of the scissor lift module 103. Through static analysis, calculate the balanced forces required for each support point. Considering the uneven force caused by the offset of the center of gravity of the goods, the system assigns corresponding compensation coefficients to each scissor lift module 103 to ensure that the target pallet 104 remains horizontal during the lifting process. The calculation of the compensation coefficient needs to consider the distance from the center of gravity to each support point, the spatial distribution of the support points, and the structural characteristics of the system.

[0058] In some embodiments, the force compensation calculation can be implemented in various ways: Optionally, establish a torque balance equation of the pallet system based on the principle of virtual work, solve the optimal force distribution of each support point by the least squares method, and determine the compensation parameters in combination with the safety factor; Optionally, adopt a dynamic programming algorithm, predict the ideal force curve of each module based on historical lifting data and the current center of gravity position, and adjust the compensation coefficient in real time. It can be understood that other mechanical modeling and optimization methods can also be used to achieve force compensation, which is not limited here.

[0059] In practical applications, the center of gravity of the goods may change dynamically, affecting the accuracy of force compensation. To this end, the fork picking control system adopts an adaptive compensation strategy: First, establish a force state evaluation model to monitor the force changes of each module in real time; then predict the center of gravity drift trend through the Kalman filter algorithm; finally, dynamically adjust the compensation coefficient to ensure that the system is always in the optimal force state. At the same time, set a torque threshold, and start the protection mechanism in time when abnormal fluctuations are detected.

[0060] S204. Determine the motion trajectory parameters of the robot according to the pallet position coordinates, and generate a motion control instruction to control the forklift of the robot to move to the preset position of the target pallet 104.

[0061] Among them, the motion trajectory parameters include motion control quantities such as speed, acceleration, and path points; the motion control instruction refers to the command sequence for controlling the picking fork 102 to perform specific actions; the preset position represents the target position suitable for the picking operation.

[0062] The picking control system needs to plan a safe and efficient motion path according to the pallet position. Specifically, the picking control system first performs a spatial analysis, considers the distribution of obstacles in the working environment, and plans a collision-free motion path. Then, the path is decomposed into a sequence of key path points, and corresponding motion parameters are configured for each path point. The system needs to consider the kinematic constraints, dynamic characteristics, and safety limitations of the robot to generate a smooth and continuous speed curve. Finally, the motion parameters are converted into low-level control instructions to ensure that the picking fork 102 can accurately reach the preset position.

[0063] In some embodiments, trajectory planning can be achieved in various ways: Optionally, an improved RRT algorithm is used for path search, a feasible path is generated considering kinematic constraints, and the trajectory is smoothed through Bezier curves; Optionally, an environmental model is constructed based on the artificial potential field method, the optimal path is found by combining the gradient descent algorithm, and the motion parameters are adjusted in real time. It can be understood that other path planning and trajectory optimization methods can also be adopted, which are not limited here.

[0064] During the implementation process, there are dynamic obstacles around the target pallet 104, which affect the execution of the preset trajectory. In this regard, the picking control system adopts a dynamic trajectory adjustment scheme: The environment is scanned in real time by a lidar to construct a local occupancy grid map; the local path planning is carried out by using the dynamic window method; according to the prediction of the motion trend of the obstacles, the motion trajectory is adjusted in time to ensure the safety of the motion process.

[0065] S205. Generate an initial lifting instruction according to the initial force value to control the scissor lift module 103 of the robot to perform a lifting motion.

[0066] Among them, the initial lifting instruction refers to the control command for controlling the scissor lift module 103 to start the lifting action; the lifting motion represents the process of vertically lifting the target pallet 104 upward; the lifting height represents the vertical distance of the target pallet 104 relative to the ground.

[0067] After the picking control system determines the initial force values of each module, it needs to perform a synchronous lifting action. Specifically, the picking control system first converts the initial force value into the drive parameters of each actuator, including control quantities such as motor speed and oil pressure. Then, a segmented lifting control sequence is generated, and the entire lifting process is divided into four stages: start, acceleration, constant speed, and deceleration. The system needs to ensure the synchronization of each module, control the continuity of the lifting speed curve, and monitor the execution status of each stage in real time. During the lifting process, it is also necessary to perform status checks according to the preset safety threshold.

[0068] In some embodiments, the lifting control can be achieved in various ways: Optionally, a fuzzy PID control algorithm is adopted to dynamically adjust the control parameters according to the current height and speed states to achieve smooth lifting; Optionally, based on neural network predictive control, a dynamic model of the lifting process is established to optimize the control instruction sequence and improve the lifting accuracy. It can be understood that other control algorithms can also be used to achieve precise control of the lifting action, which is not limited here.

[0069] In practical applications, there are mechanical clearances or synchronization errors in the scissor lift module 103, which affect the stability of lifting. To this end, the fork-taking control system adopts a multi-feedback control strategy: A high-precision feedback network is constructed through displacement sensors and angle encoders; The Kalman filter algorithm is used to fuse multi-source sensing data; The relative position deviation of each module is calculated in real time; The drive parameters are dynamically adjusted according to the deviation value to ensure the synchronization and stability of the lifting process.

[0070] S206. Obtain the real-time force data of the scissor lift module 103, and determine the horizontal attitude deviation value of the tray according to the inclination angle data collected by the tray area image and the attitude sensor.

[0071] Among them, the real-time force data represents the actual force value currently borne by the scissor lift module 103; The attitude sensor refers to a measuring device for measuring the inclination angle of the target tray 104; The inclination angle data represents the deflection angle of the target tray 104 relative to the horizontal plane; The tray horizontal attitude deviation value indicates the degree to which the target tray 104 deviates from the horizontal state.

[0072] The fork-taking control system needs to continuously monitor the force state and attitude changes of the target tray 104. Specifically, the fork-taking control system collects the force data of each support point in real time through force sensors, and at the same time obtains the three-axis inclination angle information of the target tray 104 from the attitude sensor. The system registers these data with the tray area image in space-time to establish a dynamic evaluation model of the tray attitude. Through multi-dimensional data fusion, the deviation values of the target tray 104 in the pitch and roll directions are calculated, providing a basis for subsequent attitude adjustment.

[0073] In some embodiments, the attitude monitoring can be achieved in various ways: Optionally, a multi-sensor data fusion algorithm is adopted to construct a KF attitude estimator by combining IMU and visual data to achieve high-precision attitude measurement; Optionally, based on deep learning methods, attitude features are extracted from the image sequence, and a prediction model is established in combination with physical constraints to improve the robustness of attitude estimation. It can be understood that other sensing methods and estimation algorithms can also be used to achieve attitude monitoring, which is not limited here.

[0074] During the implementation process, there is noise or delay in the sensor data, which affects the accuracy of attitude estimation. To address this, the fork-taking control system adopts an adaptive filtering scheme: First, preprocess various types of sensing data to remove outliers and high-frequency noise; then use an extended Kalman filter to fuse multi-source data and dynamically adjust the measurement weights; finally, perform data smoothing through a sliding time window to improve the timeliness and accuracy of attitude estimation.

[0075] S207. Generate a lifting compensation instruction based on the horizontal attitude deviation value of the tray to adjust the real-time force data, so that the target tray 104 is adjusted to a horizontal state.

[0076] Among them, the lifting compensation instruction represents a correction control command for adjusting the tray attitude; the real-time force data adjustment indicates the dynamic adjustment process of the forces on each support point; the horizontal state refers to the state where the target tray 104 is in an ideal balanced position.

[0077] The fork-taking control system performs dynamic compensation according to the detected attitude deviation. Specifically, the fork-taking control system first establishes a mapping relationship between the tray attitude and the supporting force, and analyzes the deviation direction and degree. Then, based on the attitude deviation value, calculate the torque compensation amount required for each scissor fork module 103, and generate a corresponding adjustment instruction sequence. The system adopts a segmented compensation strategy, first performing a rough adjustment in the main deviation direction, and then achieving fine-tuning of the attitude through fine adjustment. The entire process needs to maintain the stability of the target tray 104 to avoid violent vibration or secondary inclination.

[0078] In some embodiments, attitude compensation can be achieved in multiple ways: Optionally, adopt a model predictive control algorithm, establish a tray attitude dynamics model, predict the effect of the compensation instruction, and optimize the adjustment strategy; Optionally, based on adaptive fuzzy control, dynamically adjust the control rules according to the change trend of the attitude deviation to improve the accuracy of compensation. It can be understood that other control methods can also be used to achieve attitude compensation, which is not limited here.

[0079] In practical applications, the tray system has non-linear characteristics or time-varying parameters, which affect the compensation effect. To address this, the fork-taking control system adopts a hybrid compensation strategy: Combine feedback control and feed-forward compensation to establish a non-linear dynamic model of the tray system; use the recursive least squares method to identify the system parameters in real time; design a robust controller based on the Lyapunov stability theory to ensure the convergence and stability of the compensation process.

[0080] S208. After the target tray 104 is lifted to a preset height, generate a vehicle body forward movement instruction and a module return instruction, control the cargo-carrying vehicle body of the robot to move below the target tray 104, and control the scissor fork module 103 to return to place the target tray 104 on the cargo-carrying vehicle body.

[0081] Among them, the preset height represents the target lifting height that the target tray 104 needs to reach; the vehicle body forward movement instruction is used to control the horizontal movement of the load-carrying vehicle body; the module homing instruction controls the scissor lift module 103 to return to the initial position; the load-carrying vehicle body refers to the platform structure of the robot for carrying the target tray 104.

[0082] After the pallet lifting is completed, the picking and placing control system needs to perform the final placing action. Specifically, the picking and placing control system first confirms that the target tray 104 has reached the preset height and is in a stable horizontal state. Then it calculates the motion parameters of the load-carrying vehicle body, including the moving distance, speed curve, etc., and generates a smooth motion trajectory. The system coordinates the vehicle body forward movement and the module lowering actions to ensure that the target tray 104 can be smoothly transferred to the load-carrying platform. The entire process needs to monitor the position and attitude of the target tray 104 in real time to ensure the safety and accuracy of the placing process.

[0083] In some embodiments, the pallet placement can be achieved in various ways: Optionally, a phased control strategy is adopted, and the placement process is divided into four stages: alignment, movement, lowering, and detachment, and dedicated control parameters are configured for each stage; Optionally, based on visual servo control, the vehicle body position is adjusted using real-time image feedback to achieve precise alignment and placement. It can be understood that other control schemes can also be adopted to achieve pallet placement, which is not limited herein.

[0084] During the implementation process, the target tray 104 may shake or shift when being transferred from the scissor lift module 103 to the load-carrying vehicle body. In response to this, the picking and placing control system adopts a synchronous coordination control scheme: it monitors the tray state in real time through multi-sensor fusion; establishes a speed synchronization model for the vehicle body movement and the module lowering; dynamically adjusts the motion parameters of the two mechanisms using an adaptive control algorithm; sets a buffer transition interval to ensure the smooth transition of the target tray 104. At the same time, the system also establishes a complete exception handling mechanism, which can interrupt the operation in time and perform protection when an abnormal condition is detected.

[0085] In the above embodiments, by calculating the predicted value of the center of gravity of the cargo plane in real time and dynamically adjusting the force compensation coefficient of the scissor lift module 103, the stability of the pallet picking process is ensured. In practical applications, this method can automatically adjust the picking parameters according to the distribution characteristics of different pallet goods 105, significantly improving the adaptability and reliability of the picking operation. The following supplements the scenario of this embodiment.

[0086] After this technology was introduced into a cross-border e-commerce warehouse, not only was the basic stable fork-lifting achieved, but also optimization was carried out according to actual needs. The system added an automatic identification function for pallet types and could adapt to target pallets 104 of different specifications. When encountering palletized goods 105 with special shapes, through spatial center-of-gravity calculation and steering compensation mechanisms, the stability during transportation was ensured. For example, when dealing with a batch of palletized goods 105 with special-shaped packaging, the system could calculate the spatial center-of-gravity coordinates in real time and optimize the steering speed of the robot accordingly, maintaining stability even during high-speed transportation. This intelligent control method greatly improved the level of warehousing automation, reduced operation risks, and enhanced the overall efficiency.

[0087] After combining the above scenarios, the following is a further and more specific process description of the method provided in this embodiment. Please refer to Figure 3 , which is another process schematic diagram of the pallet fork-lifting method in the embodiments of the present application.

[0088] S301. Collect the pallet area image containing the target pallet 104 in the collection area, and perform edge extraction on the pallet area image to obtain the pallet position coordinates and the cargo distribution data.

[0089] Referring to step S201, the fork-lifting control system will determine the pallet position coordinates and the cargo distribution data.

[0090] S302. Obtain the type information and quality attribute data of the goods on the target pallet 104 stored in the cargo area management system.

[0091] Among them, the cargo area management system refers to a database system used to manage the cargo information in the warehousing area; the goods type information represents the basic attributes such as the specification model, shape, and packaging of the palletized goods 105; the quality attribute data is used to represent the physical characteristics such as the weight, density, and mass distribution of the palletized goods 105; storage refers to the process of storing data in a structured form in the database.

[0092] After the fork-lifting control system obtains the visual data, it needs to supplement information by combining the historical data of the cargo area management system. Specifically, the fork-lifting control system first queries the cargo area management system database through the pallet ID or location information to obtain the detailed information of all palletized goods 105 on the target pallet 104. The system needs to parse the SKU code of the palletized goods 105 and extract the specification parameters of the palletized goods 105, including dimensions, weight, packaging type, etc. For stacked palletized goods 105, the hierarchical relationship and stacking rules also need to be obtained. At the same time, the system will verify the timeliness of the data to ensure that the obtained information matches the current cargo status.

[0093] In some embodiments, the acquisition of goods information can be achieved in various ways: Optionally, first establish an association data model between the target pallet 104 and the pallet goods 105, obtain the basic information through SQL query, then call the material master data interface to supplement the detailed parameters, and finally perform data format conversion and verification; Optionally, adopt a distributed cache architecture, pre-load the frequently accessed goods information into the in-memory database, and maintain data synchronization through an asynchronous update mechanism to achieve fast retrieval and real-time update. It can be understood that other data management and query methods can also be used to achieve the acquisition of goods information, which is not limited here.

[0094] In practical applications, the data of the cargo area management system may be lagged or inaccurate. In this regard, the fork-taking control system adopts a multi-source data verification scheme: First, obtain the real-time identification of the pallet goods 105 through barcode scanning or RFID reading; then compare the management system data with the visual recognition results; use a confidence evaluation model to screen reliable data; when data inconsistency is found, trigger the manual confirmation process and update the database. At the same time, establish a data quality evaluation mechanism to regularly clean and update abnormal data.

[0095] S303. Determine the placement position of the goods according to the type information and the pallet area image.

[0096] Among them, the placement position refers to the spatial positioning information of the pallet goods 105 on the plane of the target pallet 104; the type information represents the goods feature data used for auxiliary positioning; and the image matching refers to the process of comparing the known goods features with the image content.

[0097] The fork-taking control system needs to accurately locate the spatial position of each pallet goods 105 on the target pallet 104. Specifically, the fork-taking control system first constructs a feature template library according to the goods type information, which includes the appearance features, size ratios, etc. of the pallet goods 105 from different perspectives. The system divides the pallet area image into multiple regions of interest and applies the target detection algorithm to each region. Determine the goods type through feature matching, and reconstruct the three-dimensional position in combination with the depth information. For stacked pallet goods 105, the system also needs to analyze the inter-layer relationship and establish a complete spatial layout model.

[0098] In some embodiments, the goods positioning can be achieved in various ways: Optionally, use a deep learning model to perform semantic segmentation on the image, extract the contour and position features of each pallet goods 105, restore the spatial coordinates through a three-dimensional reconstruction algorithm, and finally verify the rationality of the reconstruction result; Optionally, establish a reference library based on the template matching method, perform pose estimation in combination with geometric constraints, and improve the positioning accuracy through iterative optimization. It can be understood that other computer vision methods can also be used to achieve goods positioning, which is not limited here.

[0099] During the implementation process, there is occlusion or overlap between the pallet goods 105, which affects the accuracy of position recognition. In response to this, the fork-taking control system adopts a multi-view fusion solution: collect images from different angles through multiple cameras; establish the correspondence between images using feature point matching; apply stereo vision algorithms to reconstruct the occluded parts; and combine geometric constraints and physical rules to verify the effectiveness of the reconstruction results. The system also establishes an occlusion detection mechanism to make reasonable inferences about areas that cannot be directly observed.

[0100] In some embodiments, the fork-taking control system accurately locates the specific placement position of each pallet good 105 on the target pallet 104. That is, when there are multiple goods, the fork-taking control system extracts the contour data of the goods in the pallet area image, performs object detection and image segmentation operations, and generates the spatial coordinate data and boundary data of each good; according to the spatial coordinate data and boundary data, calculates the relative distance and overlap degree between the goods; and substitutes the relative distance and overlap degree into a preset spatial relationship model to generate a three-dimensional coordinate set representing the placement position of the goods.

[0101] Among them, the goods contour data represents the external boundary information of the pallet good 105; object detection refers to the process of identifying and locating the pallet good 105 in the image; image segmentation is used to represent the process of dividing the image into different semantic regions; the spatial coordinate data represents the position information of the pallet good 105 in the three-dimensional space; the boundary data refers to the geometric features describing the contour of the pallet good 105; the relative distance represents the spatial interval between the pallet goods 105; the overlap degree is used to represent the overlapping degree between the pallet goods 105; and the spatial relationship model is a mathematical model describing the position relationship between the pallet goods 105.

[0102] After the fork-taking control system obtains the pallet area image, it needs to perform an accurate analysis of the spatial relationship of the pallet goods 105. Specifically, the fork-taking control system first preprocesses and enhances the image, and uses a deep learning object detection network to identify each pallet good 105 entity. Then, through the instance segmentation algorithm, it generates an accurate mask for each pallet good 105 and extracts the contour feature point set. The system fuses the two-dimensional image information with the depth data to reconstruct the three-dimensional spatial position of the pallet goods 105. It obtains the relative position relationship by calculating the Euclidean distance between the bounding boxes of the pallet goods 105, and at the same time calculates the overlap degree using the projection overlap area ratio. Finally, the system inputs these spatial relationship parameters into a pre-trained spatial relationship inference model to generate accurate three-dimensional coordinate data describing the distribution of the goods.

[0103] In some embodiments, the spatial relationship analysis of the pallet goods 105 can be achieved in various ways: Optionally, first, an improved Mask R-CNN network is used for detecting and segmenting the pallet goods 105, then a point cloud registration algorithm is adopted to fuse depth information, outliers are removed through the RANSAC algorithm, and finally, a graph optimization method is used to refine the 3D reconstruction result; Optionally, based on the multi-view geometry method, image sequences are collected by multiple cameras, and the 3D shape of the pallet goods 105 is reconstructed using structured light, and the spatial position estimation is optimized by combining physical constraints. It can be understood that other computer vision methods can also be used to achieve the spatial relationship analysis, which is not limited herein.

[0104] During the implementation process, there are complex occlusion relationships between the pallet goods 105, which affect the accuracy of the spatial position estimation. In this regard, the picking control system adopts a multi-modal fusion strategy: combining RGB-D camera and lidar data to construct a complete scene point cloud; using a voxelization method to represent the spatial occupancy; inferring the geometry of the occluded part through a probabilistic graph model; establishing a dynamic update mechanism to adapt to the position changes of the pallet goods 105. The system also establishes a spatial relationship verification mechanism to ensure that the generated 3D coordinates meet the physical feasibility constraints.

[0105] S304. According to the quality attribute data, placement position, and centroid calculation formula, the predicted value of the plane centroid of the goods is calculated.

[0106] Among them, the centroid calculation formula represents a mathematical equation for calculating the centroid position based on physical principles; the quality attribute data refers to the weight and density distribution characteristics of the pallet goods 105; the predicted value of the plane centroid is used to represent the mass center coordinates of the pallet goods 105 on the horizontal plane projection.

[0107] The picking control system needs to calculate the overall centroid position according to the actual distribution of the pallet goods 105. Specifically, the picking control system first divides the space of the target pallet 104 into grid cells and establishes a 3D coordinate system. For each pallet good 105, the system calculates the local centroid according to its quality attributes and spatial position. Considering the mass distribution characteristics of different pallet goods 105, the system adopts a hierarchical integration method to calculate the composite centroid. For irregularly shaped pallet goods 105, it needs to be decomposed into basic geometric bodies for segmented calculation. Finally, the 3D centroid is projected onto the horizontal plane to obtain the predicted value of the plane centroid.

[0108] In some embodiments, the center-of-gravity calculation can be achieved in various ways: Optionally, first establish a mass distribution model of the pallet goods 105, use the Gaussian integration method to calculate the center-of-gravity position of each geometric body, solve the total center of gravity of the system through the principle of moment superposition, and finally perform coordinate transformation to obtain the plane projection; Optionally, discretize the pallet goods 105 based on the finite element method, assign mass attributes to each element, calculate the center-of-gravity coordinates through numerical integration, and perform grid convergence analysis. It can be understood that other numerical calculation methods can also be used to achieve center-of-gravity prediction, which is not limited here.

[0109] In practical applications, the mass distribution of the pallet goods 105 may be uneven or there may be voids, affecting the accuracy of the center-of-gravity calculation. To this end, the fork-taking control system adopts an adaptive calculation strategy: First, construct a hierarchical density model, considering material characteristics and packaging forms; use the Monte Carlo method to evaluate the influence of parameter uncertainties; update the density distribution through Bayesian estimation; and finally calculate the confidence interval to ensure the reliability of the center-of-gravity prediction.

[0110] S305. Calculate the force compensation coefficients of each scissor fork module 103 of the robot according to the predicted value of the plane center of gravity of the goods, and determine the initial force values of each scissor fork module 103.

[0111] Referring to step S203, the fork-taking control system will determine the initial force values of each scissor fork module 103.

[0112] S306. Convert the pallet position coordinates into target position data in the local coordinate system of the robot.

[0113] Among them, the local coordinate system refers to a relative coordinate system established with the robot as a reference; the target position data represents the position and attitude information of the target pallet 104 in the robot coordinate system; the coordinate transformation refers to the process of mapping position data in different reference systems.

[0114] The fork-taking control system needs to convert the pallet position in the global coordinate system to the local coordinate system of the robot. Specifically, the fork-taking control system first determines the current pose of the robot and establishes a transformation matrix from the world coordinate system to the local coordinate system. Then, obtain the position data of the target pallet 104 in the world coordinate system, including spatial coordinates and direction angles. The system needs to consider the scale relationship and rotation deviation between coordinate systems and calculate the relative position of the target pallet 104 in the robot coordinate system through rigid body transformation. At the same time, it is also necessary to verify the accuracy of the conversion result to ensure the reliability of subsequent motion planning.

[0115] In some embodiments, coordinate transformation can be achieved in various ways: Optionally, first establish a complete coordinate transformation chain, calculate the rotation matrix and translation vector, perform coordinate mapping through homogeneous transformation, and finally perform error analysis and compensation; Optionally, use the dual quaternion method to represent spatial transformation, combine Kalman filtering to optimize pose estimation, and achieve real-time coordinate update. It can be understood that other coordinate transformation algorithms can also be used to achieve position transformation, which is not limited here.

[0116] During the implementation process, the positioning error of the robot may cause inaccurate coordinate transformation. In this regard, the picking control system adopts a multi-sensor fusion scheme: construct an environmental map through lidar and vision systems; use feature matching for position correction; establish an error compensation model; and update coordinate transformation parameters in real time. At the same time, the system will also set a position tolerance, and trigger the repositioning process when the threshold is exceeded.

[0117] S307. Obtain the contour features of the target tray 104, and identify the tray type of the target tray 104 and the corresponding standard size parameters.

[0118] Among them, the contour features represent the key geometric features of the shape of the target tray 104, including the outer boundary, internal slots, and structural feature points; the tray type refers to the types of trays that meet different standard specifications, such as European standard target trays, American standard target trays, etc.; the standard size parameters are used to represent the standardized size data of the target tray 104, including length, width, height, and load-bearing grade; feature recognition refers to the process of extracting and analyzing tray features from images.

[0119] After the picking control system determines the tray position, it needs to perform tray type recognition and parameter acquisition. Specifically, the picking control system first performs image enhancement and noise suppression on the tray area image to improve the image quality. Then, it extracts the contour line of the target tray 104 through an edge detection algorithm and obtains a set of feature points using corner detection. The system analyzes the geometric features of the contour, including aspect ratio, symmetry, and internal structure features. Through feature vector matching, the extracted features are compared with the pre-established tray template library to determine the tray type. According to the recognition result, the corresponding specification parameters are retrieved from the standard parameter database. At the same time, the system also needs to verify the consistency between the actual size and the standard parameters to ensure the accuracy of subsequent operations.

[0120] In some embodiments, tray recognition and parameter acquisition can be achieved in various ways: Optionally, first use a deep learning model to perform semantic segmentation on the image, extract the region mask of the target tray 104, then apply a shape description algorithm to extract geometric features, establish a feature vector, perform classification and recognition through a support vector machine, and finally query the parameter database according to the recognition result to obtain standard parameters; Optionally, based on traditional image processing methods, use the Hough transform to detect line features, combine distance transformation to extract internal structures, perform type recognition through a template matching algorithm, and at the same time use a depth camera to obtain the actual size for parameter verification. It can be understood that other computer vision methods can also be used to achieve the recognition and parameter acquisition of the target tray 104, which is not limited here.

[0121] During the implementation process, there are stains or partial occlusions on the surface of the target tray 104, which affect the accuracy of feature extraction. In this regard, the fork-taking control system adopts a multi-feature fusion recognition strategy: First, construct a multi-scale feature extraction network to process local and global features simultaneously; use the attention mechanism to highlight the key region features; establish a feature completion model to infer and reconstruct the missing features; screen reliable features through a confidence evaluation mechanism. The system also establishes a recognition result verification mechanism, which triggers a multi-angle re-recognition or manual confirmation process when the confidence level is insufficient. For non-standard trays, the system will record their feature parameters and establish a temporary template to ensure the accuracy of subsequent operations. At the same time, the system continuously updates and expands the tray feature library through online learning to improve the adaptability and robustness of the recognition system.

[0122] S308. According to the target position data, contour features, and standard size parameters, calculate the relative position relationship between the scissor fork module 103 of the robot and the tray support points.

[0123] Among them, the relative position relationship represents the spatial position correspondence between the scissor fork module 103 and the tray support points; the tray support points refer to the key contact positions where the target tray 104 is carried by the scissor fork module 103; the contour features are used to locate the geometric boundaries and structural features of the target tray 104.

[0124] The fork-taking control system needs to accurately calculate the spatial matching relationship between the grasping mechanism and the target tray 104. Specifically, the fork-taking control system first constructs a three-dimensional geometric model of the target tray 104 according to the standard size parameters and actual contour features of the target tray 104. Then determine the positions of the support grooves at the bottom of the tray, which are the target points for the scissor fork module 103 to insert. The system needs to consider the orientation and tilt angle of the target tray 104, calculate the precise coordinates of each support point in the robot coordinate system. At the same time, it is also necessary to analyze the force characteristics of the support points to ensure the stability of the fork-taking process.

[0125] In some embodiments, the calculation of the positional relationship can be achieved in various ways: Optionally, first establish a geometric constraint model of the tray-robot, determine the correspondence of key feature points, optimize the relative position through the least squares registration algorithm, and finally verify the accuracy and reliability of the registration result; Optionally, adopt an image-based visual servo method, adjust the position deviation through real-time image feedback, and combine depth information to achieve precise alignment. It can be understood that other positioning algorithms can also be used to calculate the positional relationship, which is not limited here.

[0126] In practical applications, the target tray 104 is deformed or damaged, affecting the accuracy of the support point position. In this regard, the fork-taking control system adopts an adaptive positioning strategy: Obtain the actual shape of the target tray 104 through multi-sensor scanning; establish a deformation detection model; dynamically adjust the support point position; set a safety margin to ensure the reliability of the fork insertion. At the same time, the system also establishes an anomaly detection mechanism to give an alarm in time when an abnormal tray state is found.

[0127] In some embodiments, the fork-taking control system will ensure that the fork arm can safely and reliably support the target tray 104. That is, when the target tray 104 is a cross-shaped tray, the fork-taking control system will determine the empty area of the cross-shaped tray according to the contour features and standard size parameters; Based on the target position data and the empty area, calculate the optimal support positions corresponding to each scissor lift module 103 of the robot; Determine the relative position relationship according to the optimal support positions to avoid the scissor lift module 103 from touching the bottom plane of the cross-shaped tray.

[0128] Among them, the contour feature represents the geometric shape feature of the outer shape of the target tray 104; the standard size parameter refers to the normalized size data of the target tray 104; the cross-shaped tray is used to represent a standard tray type with a cross-shaped bottom structure; the empty area represents the area suitable for placing the scissor lift module 103; the optimal support position refers to the ideal position point for the scissor lift module 103 to support the target tray 104; the relative position relationship is used to represent the spatial correspondence between the scissor lift module 103 and the target tray 104.

[0129] After the fork-taking control system determines the tray type, it needs to precisely plan the support position. Specifically, the fork-taking control system first analyzes the structural characteristics of the cross-shaped tray and identifies the distribution of the bottom cross beams and longitudinal beams. Extract the empty positions at the bottom of the tray through an image processing algorithm, and establish a detailed empty area model in combination with the standard size parameters. Then, the fork-taking control system maps the target position data to the tray coordinate system, considers the motion constraints of the robot and the load-bearing requirements of the target tray 104, and calculates the best insertion position and attitude angle for each scissor lift module 103. The system also needs to perform collision detection analysis to ensure a safe distance between the scissor lift module 103 and the structure of the target tray 104 during the support process.

[0130] In some embodiments, the support position optimization can be achieved in various ways: Optionally, first establish a three-dimensional grid model of the bottom of the tray, calculate the feasibility score of each grid point, use the genetic algorithm to search for the optimal combination of support points, and verify the support stability through dynamic simulation; Optionally, adopt a graph-based planning method, construct the empty space of the target tray 104 into a constraint graph network, use a heuristic search algorithm to find a support position solution that meets multi-objective optimization, and perform real-time path planning. It can be understood that other optimization algorithms can also be used to implement the support position planning, which is not limited here.

[0131] During the implementation process, the bottom structure of the tray is deformed or irregularly changed, affecting the accuracy of the support position. In response, the fork-taking control system adopts an adaptive planning strategy: scan the actual shape of the bottom of the tray through a laser profiler; establish a deformation evaluation model; dynamically adjust the support position; set the position compensation amount. The system also establishes a support reliability evaluation mechanism to adjust the planning scheme in a timely manner when abnormal conditions are detected.

[0132] S309. Determine the motion trajectory parameters of the fork based on the relative position relationship, generate corresponding motion control instructions, and use them to control the fork of the robot to move to the preset position of the target tray 104.

[0133] Among them, the motion trajectory parameters include motion control quantities such as speed, acceleration, and path points; the motion control instructions refer to the command sequence for controlling the picking fork 102 to perform specific actions; the preset position represents the target position suitable for the fork-taking operation.

[0134] The fork-taking control system needs to plan the optimal motion trajectory based on the calculated position relationship. Specifically, the fork-taking control system first analyzes the distribution of obstacles in the working space to determine the safe motion area. Then, according to the starting position and the target position, a path planning algorithm is used to generate a collision-free trajectory. The system needs to consider the kinematic constraints and dynamic characteristics of the robot to optimize the trajectory and generate a smooth speed curve. Finally, the trajectory parameters are converted into low-level control instructions to ensure that the picking fork 102 can accurately reach the preset position.

[0135] In some embodiments, the trajectory planning can be achieved in various ways: Optionally, use the probabilistic roadmap method to construct a motion space model, search for the optimal path through the A* algorithm, smooth the path with B-spline, and finally generate a time-optimal speed plan; Optionally, perform real-time trajectory planning based on the dynamic window method, consider the dynamic constraints of the robot, and optimize the execution effect through model predictive control. It can be understood that other planning algorithms can also be used to implement the trajectory generation, which is not limited here.

[0136] During the implementation process, dynamic obstacles may appear in the working environment, affecting the execution of the preset trajectory. To address this, the picking control system adopts a real-time planning strategy: continuously monitors environmental changes through lidar; constructs a local cost map; dynamically adjusts the trajectory using the elastic band algorithm; and sets a safety buffer area to avoid collisions. The system also establishes an emergency obstacle avoidance mechanism that can promptly adjust the motion trajectory when detecting sudden obstacles.

[0137] In some embodiments, the picking control system ensures the safety of the picking process, that is, the picking control system first obtains the tray weight data of the target tray 104; then reads the fork specifications data of the robot to determine the rated load parameter and the supporting force data of the scissor fork module 103; when the tray weight data is simultaneously less than the rated load parameter and the supporting force data, a picking permission signal is generated, enabling the robot to perform the picking action.

[0138] Among them, the tray weight data represents the total mass of the target tray 104 and the tray goods 105; the fork specifications data refers to the performance parameters of the robot's picking fork 102; the rated load parameter represents the maximum weight that the robot is allowed to carry; the supporting force data is used to represent the maximum load-bearing capacity of the scissor fork module 103; and the picking permission signal is a control instruction that allows the execution of the picking action.

[0139] Before the picking control system executes the picking action, safety verification is required. Specifically, the picking control system first obtains the actual weight value of the target tray 104 through a weight sensor and simultaneously reads the relevant specification parameters from the robot configuration database. The system makes multiple comparisons between the tray weight and the robot's rated load and the upper limit of the supporting force of each module to ensure the safety margin of the picking operation. When all safety conditions are met, the system generates a picking permission signal to start the subsequent execution process.

[0140] In some embodiments, safety verification can be achieved in various ways: Optionally, first establish a multi-layer safety inspection model, collect weight data in real time through a pressure sensor, calculate the force on each support point in combination with the load distribution, establish a dynamic safety factor evaluation module, and finally generate a picking decision through a fuzzy logic controller; Optionally, adopt a verification method based on risk assessment, construct a state space including multi-dimensional parameters such as weight, speed, and attitude, use a predictive control algorithm to evaluate the operation risk, and dynamically adjust the safety threshold. It can be understood that other safety verification methods can also be used to achieve picking permission control, which is not limited here.

[0141] During the implementation process, the weight of the pallet has dynamic changes or measurement errors, which affects the reliability of safety verification. In response, the fork-taking control system adopts an adaptive safety control strategy: constructing a distributed measurement system through a multi-point weight sensing network; fusing multi-source data using the Kalman filtering algorithm; establishing a weight prediction model to evaluate the change trend; setting up a hierarchical early warning mechanism. The system also establishes an emergency braking mechanism that can interrupt the operation in a timely manner when an overload risk is detected.

[0142] S310. Generate an initial lifting command according to the initial force value to control the scissor lift module 103 of the robot to perform a lifting movement.

[0143] Referring to step S205, the fork-taking control system will control the scissor lift module 103 of the robot to lift.

[0144] S311. Obtain the real-time force data of the scissor lift module 103, and determine the horizontal attitude deviation value of the pallet based on the inclination data collected by the pallet area image and the attitude sensor.

[0145] Referring to step S206, the fork-taking control system will determine the horizontal attitude deviation value of the pallet.

[0146] S312. Generate a lifting compensation command based on the horizontal attitude deviation value of the pallet to adjust the real-time force data, so that the target pallet 104 is adjusted to a horizontal state.

[0147] Referring to step S207, the fork-taking control system will adjust the target pallet 104 to a horizontal state.

[0148] S313. After the target pallet 104 is lifted to a preset height, generate a vehicle body forward movement command and a module return command, control the cargo-carrying vehicle body of the robot to move under the target pallet 104, and control the scissor lift module 103 to return to its original position, so that the target pallet 104 is placed on the cargo-carrying vehicle body.

[0149] Referring to step S208, the fork-taking control system will place the target pallet 104 on the cargo-carrying vehicle body.

[0150] In some embodiments, the fork-taking control system will optimize and adjust the steering speed of the robot to ensure stability during transportation. That is, the fork-taking control system will determine the plane center of gravity coordinates of the target pallet 104 according to the predicted value of the plane center of gravity of the cargo and the real-time force data; determine the spatial center of gravity coordinates of the target pallet 104 according to the plane center of gravity coordinates and the cargo distribution data; calculate the steering compensation coefficient of the robot in each direction based on the spatial center of gravity coordinates; determine the steering speed value of the robot based on the steering compensation coefficient.

[0151] Among them, the planar centroid coordinates represent the position of the mass center of the pallet goods 105 in the horizontal plane; the spatial centroid coordinates refer to the three-dimensional mass center considering height information; the steering compensation coefficient is used to represent the adjustment parameter for correcting the steering action; the steering speed value refers to the angular velocity command of the robot during the steering process.

[0152] After determining the goods distribution, the fork-taking control system needs to optimize the steering control strategy. Specifically, the fork-taking control system first fuses the centroid prediction value and the force sensor data, and calculates the current planar centroid position through weighted average. Then, combined with the spatial distribution information of the pallet goods 105, a three-dimensional centroid dynamic model is established, considering the influence of goods stacking. Based on the spatial centroid position, the system calculates the torque distribution at different steering angles, generates the compensation coefficients in each direction. Finally, the steering speed curve is dynamically adjusted according to the compensation coefficients to ensure stability during the steering process.

[0153] In some embodiments, the steering control optimization can be achieved in various ways: Optionally, first establish the dynamic model of the robot-goods system, design the speed planner through the optimal control theory, and use model predictive control to optimize the steering parameters in real time to achieve smooth steering control; Optionally, based on the reinforcement learning method, construct a reward function including states such as centroid position, steering angle, and speed, and learn the optimal steering strategy through the policy gradient algorithm. It can be understood that other control algorithms can also be used to achieve steering optimization, which is not limited here.

[0154] During the implementation process, the centroid of the goods may shift during the steering process, affecting the steering stability. To this end, the fork-taking control system adopts a real-time compensation strategy: monitor the attitude change through the inertial measurement unit; establish a centroid drift prediction model; dynamically adjust the compensation parameters using the adaptive control algorithm; set the stability evaluation index. The system also establishes an anti-tip protection mechanism, which automatically reduces the steering speed or stops steering when an unstable trend is detected. At the same time, the system continuously optimizes the steering control model through online learning to improve the adaptability to different load conditions.

[0155] In the embodiments of the present application, due to the adoption of the tray position recognition technology based on real-time image acquisition and edge detection, combined with dynamic center-of-gravity calculation and intelligent force compensation mechanism, it is possible to accurately sense the tray position and the state of cargo distribution, and adjust the force parameters of the scissor lift module 103 in real time, effectively solving the problems of unstable lifting and easy tilting of the target tray 104 under the traditional fixed-parameter control method, and thus realizing the intelligent control and stable operation of the tray picking process. The picking control system ensures that the target tray 104 always maintains a horizontal state through real-time attitude monitoring and compensation control, greatly improving the picking reliability; optimizes the stability during transportation through spatial center-of-gravity calculation and steering compensation mechanism; at the same time, the adaptive characteristics of this method enable it to handle different types of trays and irregular cargo distribution situations, significantly improving the level of warehouse automation and operation efficiency.

[0156] The picking control system in the embodiments of the present invention application will be described from the perspective of hardware processing. Please refer to Figure 4 , which is a schematic structural diagram of an entity device of the picking control system in the embodiments of the present application.

[0157] It should be noted that Figure 4 The structure of the picking control system shown is only an example and should not impose any limitations on the functions and usage scope of the embodiments of the present invention.

[0158] As Figure 4 shown, the picking control system includes a CPU 401, which can perform various appropriate actions and processes according to the program stored in the ROM 402 or the program loaded into the RAM 403 from the storage section 408, such as executing the method described in the above embodiments. In the RAM 403, various programs and data required for system operation are also stored. The CPU 401, ROM 402, and RAM 403 are connected to each other through a bus 404. The I / O interface 405 is also connected to the bus 404.

[0159] The following components are connected to the I / O interface 405: an input section 406 including an audio input device, a button switch, etc.; an output section 407 including a liquid crystal display (Liquid Crystal Display, LCD) and an audio output device, an indicator light, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A driver 410 is also connected to the I / O interface 405 as needed. A removable medium 411, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the driver 410 as needed so that the computer program read from it can be installed into the storage section 408 as needed.

[0160] Specifically, according to an embodiment of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, an embodiment of the present invention includes a computer program product that includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication section 409, and / or installed from the removable medium 411. When the computer program is executed by the CPU 401, various functions defined in the present invention are executed.

[0161] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present invention. Among them, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the above-mentioned module, program segment, or part of code includes one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order from that marked in the accompanying drawings.

[0162] Specifically, the fork picking control system of this embodiment includes a processor and a memory. A computer program is stored on the memory. When the computer program is executed by the processor, the pallet fork picking method provided in the above embodiment is implemented.

[0163] As another aspect, the present invention also provides a computer-readable storage medium, which may be included in the fork picking control system described in the above embodiment; or it may exist separately without being assembled into the fork picking control system. The above storage medium carries one or more computer programs. When the one or more computer programs are executed by a processor of the fork picking control system, the fork picking control system is enabled to implement the pallet fork picking method provided in the above embodiment.

[0164] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the various embodiments of the present application.

[0165] As used in the foregoing embodiments, depending on the context, the term "when" can be interpreted to mean "if" or "after" or "in response to determining" or "in response to detecting". Similarly, depending on the context, the phrase "upon determining" or "if (the stated condition or event) is detected" can be interpreted to mean "if determined" or "in response to determining" or "when (the stated condition or event) is detected" or "in response to detecting (the stated condition or event)".

Claims

1. A pallet picking method, characterized in that, Applied to a fork-taking control system, the method includes: Collect an image of a tray area containing a target tray in the collection area, and perform edge extraction on the tray area image to obtain tray position coordinates and cargo distribution data; Based on the cargo distribution data, calculate the center of gravity to obtain a predicted value of the center of gravity of the cargo plane in the horizontal direction; According to the predicted value of the center of gravity of the cargo plane, calculate the force compensation coefficients of each scissor fork module of the robot, and determine the initial force values of each scissor fork module; According to the tray position coordinates, determine the motion trajectory parameters of the robot, and generate a motion control instruction to control the forklift of the robot to move to a preset position of the target tray; Generate an initial lifting instruction according to the initial force value to control the scissor fork module of the robot to perform a lifting motion; Obtain the real-time force data of the scissor fork module, and determine the horizontal attitude deviation value of the tray according to the tray area image and the inclination angle data collected by the attitude sensor; Generate a lifting compensation instruction based on the horizontal attitude deviation value of the tray to adjust the real-time force data so that the target tray is adjusted to a horizontal state; After the target tray is lifted to a preset height, generate a body forward movement instruction and a module return instruction to control the cargo-carrying body of the robot to move below the target tray, and control the scissor fork module to return so that the target tray is placed on the cargo-carrying body.

2. The method according to claim 1, characterized in that The step of determining the motion trajectory parameters of the robot according to the tray position coordinates and generating a motion control instruction to control the forklift of the robot to move to a preset position of the target tray specifically includes: Convert the tray position coordinates into target position data in the local coordinate system of the robot; Obtain the contour features of the target tray, and identify the tray type and corresponding standard size parameters of the target tray; According to the target position data, the contour features, and the standard size parameters, calculate the relative position relationship between the scissor fork module of the robot and the tray support point; Based on the relative position relationship, determine the motion trajectory parameters of the forklift, and generate a corresponding motion control instruction to control the forklift of the robot to move to a preset position of the target tray.

3. The method according to claim 2, characterized in that, The target tray is a cross-shaped tray; The step of calculating the relative position relationship between the scissor fork module of the robot and the tray support point according to the target position data, the contour features, and the standard size parameters specifically includes: According to the contour features and the standard size parameters, determine the empty space area of the cross-shaped tray; Based on the target position data and the empty space area, calculate the optimal support positions corresponding to each scissor fork module of the robot; Determine the relative position relationship according to the optimal support positions to avoid the scissor fork module from touching the bottom plane of the cross-shaped tray.

4. The method according to claim 1, wherein The step of calculating the center of gravity based on the cargo distribution data to obtain a predicted value of the center of gravity of the cargo plane in the horizontal direction specifically includes: Obtain the type information and quality attribute data of the goods on the target tray stored in the cargo area management system; Determine the placement position of the goods according to the type information and the image of the tray area; Calculate the predicted value of the planar center of gravity of the goods based on the quality attribute data, the placement position, and the center of gravity calculation formula.

5. The method according to claim 4, wherein There are multiple goods; The step of determining the placement position of the goods according to the type information and the image of the tray area specifically includes: Extract the contour data of the goods in the image of the tray area, perform object detection and image segmentation operations, and generate the spatial coordinate data and boundary data of each good. Calculate the relative distance and overlap degree between the goods according to the spatial coordinate data and the boundary data. Substitute the relative distance and overlap degree into a preset spatial relationship model to generate a three-dimensional coordinate set representing the placement position of the goods.

6. The method according to claim 1, wherein Before the step of determining the motion trajectory parameters of the robot according to the tray position coordinates and generating a motion control instruction to control the forklift of the robot to move to a preset position of the target tray, the method further includes: Obtain the tray weight data of the target tray; Read the forklift specification data of the robot to determine the rated load parameter and the supporting force data of the scissor fork module; When the tray weight data is simultaneously less than the rated load parameter and the supporting force data, generate a fork-taking permission signal to enable the robot to perform a fork-taking action.

7. The method according to claim 1, wherein After the step of generating a vehicle body forward movement instruction and a module reset instruction after the target tray is lifted to a preset height, controlling the cargo-carrying vehicle body of the robot to move below the target tray, and controlling the scissor fork module to reset so that the target tray is placed on the cargo-carrying vehicle body, the method further includes: Determine the planar center of gravity coordinates of the target tray according to the predicted value of the planar center of gravity of the goods and the real-time force data; Determine the spatial center of gravity coordinates of the target tray according to the planar center of gravity coordinates and the goods distribution data; Calculate the steering compensation coefficient of the robot in each direction based on the spatial center of gravity coordinates; Determine the steering speed value of the robot based on the steering compensation coefficient.

8. A fork-taking control system, characterized in that, The fork-taking control system includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the fork-taking control system to execute the method according to any one of claims 1-7.

9. A computer-readable storage medium, comprising instructions, characterized in that, When the instruction runs on the fork-taking control system, enable the fork-taking control system to execute the method according to any one of claims 1-7.

10. A computer program product, characterized in that, When the computer program product runs on the fork-taking control system, enable the fork-taking control system to execute the method according to any one of claims 1-7.

Images