Intelligent logistics forklift pallet pose estimation method and system for optimizing 3D coordinates

Abstract

The invention belongs to the technical field of visual detection. According to the intelligent logistics forklift tray pose estimation method and system for optimizing the 3D coordinates, the SolvePnP algorithm and an error calculation and backstepping mechanism are combined, and the pose of a tray is calculated through the known 3D coordinates and initial 2D pixel coordinates; by comparing the measured rotation matrix and the error between the translation vector and the actual value, a cost function is defined to quantify the error; a gradient descent method is adopted to carry out iterative optimization on 3D coordinates, the gradient of a cost function is calculated, the coordinates are updated, and iterative updating is carried out to minimize errors, so that more accurate pose estimation is realized; pixel-level errors in deep learning key point detection can be effectively overcome, and the accuracy and reliability of tray pose estimation are improved; the method has wide application prospects in the field of intelligent logistics, the operation efficiency of the forklift can be remarkably improved, and powerful technical support is provided for development of automatic logistics.

Description

Technical Field

[0001] The present invention relates to the field of visual detection technology, and in particular to a method for estimating the posture of a pallet of an intelligent logistics forklift by optimizing 3D coordinates, a system for estimating the posture of a pallet of an intelligent logistics forklift by optimizing 3D coordinates, a computer device, a computer-readable storage medium and a computer program product. Background Art

[0002] The statements in this section merely provide background art related to the present invention and do not necessarily constitute prior art.

[0003] With the development of intelligent logistics, forklifts are increasingly used in warehousing and transportation. Accurate pallet pose estimation is crucial for forklift navigation and object recognition. In the prior art, the SolvePnP (Perspective-n-Point) algorithm is widely used in camera positioning and object tracking. It can calculate the external parameters of the camera through the known 3D coordinates and the corresponding 2D pixel coordinates, thereby inferring the pose of the object in the camera coordinate system.

[0004] In the scenario of intelligent logistics forklifts, the 3D coordinates and corresponding 2D pixel coordinates of each corner point of the pallet are known, and the SolvePnP algorithm can be used to effectively estimate the pose of the pallet. However, existing methods rely on deep learning key point detection to obtain the 2D pixel coordinates of the pallet, but due to the existence of pixel-level errors, the 2D pixel coordinates cannot be perfectly matched with the corresponding 3D coordinates, affecting the accurate estimation of the pallet pose. Summary of the invention

[0005] In order to address the shortcomings of the prior art, the present invention provides a method and system for estimating the pallet pose of an intelligent logistics forklift that optimizes 3D coordinates. By calculating the errors of rotation and translation and iteratively adjusting the 3D coordinates through the gradient descent method, the accuracy of pallet pose estimation is improved, providing effective support for the development of automated logistics.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] In a first aspect, the present invention provides a method for estimating the position and posture of a pallet of an intelligent logistics forklift by optimizing 3D coordinates.

[0008] A method for estimating the position and posture of a pallet of an intelligent logistics forklift by optimizing 3D coordinates includes the following steps:

[0009] Obtain the 3D coordinates of each corner point of the pallet and the 2D pixel coordinates of the pallet, and establish a one-to-one correspondence between the 3D coordinates and the 2D pixel coordinates so that each 3D point has a corresponding 2D pixel point in the image;

[0010] Generate an N×3 matrix according to the 3D coordinates of each corner point of the pallet, where N is the number of points, generate an N×2 matrix according to the 2D pixel coordinates of the pallet, and obtain the rotation matrix solution result and the translation vector solution result according to the N×3 matrix, the N×2 matrix and the SolvePnP algorithm;

[0011] Obtaining a rotation error according to a rotation matrix solution result and a true rotation matrix, obtaining a translation error according to a translation vector solution result and a true translation vector, and calculating a cost function according to the rotation error and the translation error;

[0012] The gradient of the cost function relative to the 3D coordinates is calculated. According to the obtained gradient, the 3D coordinates are updated using the gradient descent method. The update iterative process continues until the change in the cost function is less than the set threshold or the maximum number of iterations is reached. The pose estimation result is obtained based on the final optimized 3D coordinates.

[0013] As a further limitation of the first aspect of the present invention, the rotation error R is obtained according to the rotation matrix solution result and the true rotation matrix. error ,include:

[0014]

[0015] Among them, R measured is the result of rotation matrix solution, R actual is the true rotation matrix.

[0016] As a further limitation of the first aspect of the present invention, the translation error T is obtained according to the translation vector solution result and the real translation vector error ,include:

[0017] T error =T actual -T measured ;

[0018] Among them, T actual is the real translation vector, T measured is the result of translation vector solution.

[0019] As a further limitation of the first aspect of the present invention, calculating the cost function J according to the rotation error and the translation error includes:

[0020] J=α·||R error || 2 +β·||T error || 2 ;

[0021] Among them, α and β are weight coefficients, R error is the rotation error, T error is the translation error.

[0022] As a further limitation of the first aspect of the present invention, the gradient of the cost function relative to the 3D coordinates is calculated According to the obtained gradient include:

[0023]

[0024] Among them, for each 3D coordinate (u i , v i ,0),u i represents the horizontal coordinate of the i-th point, v i represents the ordinate of the i-th point, is the cost function J with respect to the horizontal coordinate u of the i-th point i The partial derivative of is the ordinate v of the cost function J with respect to the i-th point i The partial derivative of , where n is the number of points on the tray.

[0025] As a further limitation of the first aspect of the present invention, updating the 3D coordinates using a gradient descent method comprises:

[0026]

[0027] Among them, 3D corrdinate is the updated 3D coordinate, η is the learning rate, 3D Corrdinate is the 3D coordinate before updating.

[0028] In a second aspect, the present invention provides an intelligent logistics forklift pallet posture estimation system that optimizes 3D coordinates.

[0029] An intelligent logistics forklift pallet posture estimation system for optimizing 3D coordinates, comprising:

[0030] A data acquisition unit is configured to: acquire 3D coordinates of each corner point of the tray and 2D pixel coordinates of the tray, and establish a one-to-one correspondence between the 3D coordinates and the 2D pixel coordinates so that each 3D point has a corresponding 2D pixel point in the image;

[0031] The data solving unit is configured to: generate an N×3 matrix according to the 3D coordinates of each corner point of the pallet, wherein N is the number of points, generate an N×2 matrix according to the 2D pixel coordinates of the pallet, and obtain a rotation matrix solving result and a translation vector solving result according to the N×3 matrix, the N×2 matrix and the SolvePnP algorithm;

[0032] The cost function calculation unit is configured to: obtain a rotation error according to a rotation matrix solution result and a true rotation matrix, obtain a translation error according to a translation vector solution result and a true translation vector, and calculate a cost function according to the rotation error and the translation error;

[0033] The pose estimation unit is configured to: calculate the gradient of the cost function relative to the 3D coordinates, and update the 3D coordinates using the gradient descent method based on the obtained gradient. The updating iterative process continues until the change in the cost function is less than the set threshold or the maximum number of iterations is reached, and the pose estimation result is obtained based on the final optimized 3D coordinates.

[0034] In a third aspect, the present invention provides a computer device, comprising: a processor and a computer-readable storage medium;

[0035] a processor adapted to execute a computer program;

[0036] A computer-readable storage medium having a computer program stored therein, wherein the computer program, when executed by the processor, implements the method for estimating the pallet posture of an intelligent logistics forklift by optimizing 3D coordinates as described in the first aspect of the present invention.

[0037] In a fourth aspect, the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program is suitable for being loaded by a processor and executing the intelligent logistics forklift pallet pose estimation method for optimizing 3D coordinates as described in the first aspect of the present invention.

[0038] In a fifth aspect, the present invention provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the intelligent logistics forklift pallet pose estimation method for optimizing 3D coordinates as described in the first aspect of the present invention.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] The present invention innovatively designs a pallet pose estimation method based on an optimization algorithm, aiming to improve the positioning accuracy of intelligent logistics forklifts in pallet operations. By combining the SolvePnP algorithm with the error calculation and back-deduction mechanism, the pallet pose is calculated using known 3D coordinates and initial 2D pixel coordinates; by comparing the errors between the measured rotation matrix and translation vector and the actual values, a cost function is defined to quantify these errors; the gradient descent method is used to iteratively optimize the 3D coordinates, calculate the gradient of the cost function and update the coordinates, and iteratively update to minimize the error, thereby achieving more accurate pose estimation, which can effectively overcome the pixel-level error in deep learning key point detection, improve the accuracy and reliability of pallet pose estimation, and has broad application prospects in the field of intelligent logistics, can significantly improve the operating efficiency of forklifts, and provide strong technical support for the development of automated logistics.

[0041] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0043] Figure 1 A schematic diagram of the overall process of the method for estimating the position and posture of a pallet of an intelligent logistics forklift with optimized 3D coordinates provided in Example 1 of the present invention;

[0044] Figure 2 A structural diagram of the initial posture setting provided in Example 1 of the present invention;

[0045] Figure 3 A flowchart of iterative optimization provided in Example 1 of the present invention;

[0046] Figure 4 A schematic diagram of an intelligent logistics forklift pallet posture estimation system for optimizing 3D coordinates provided in Example 2 of the present invention;

[0047] Figure 5 A schematic diagram of a computer device provided in Example 3 of the present invention. DETAILED DESCRIPTION

[0048] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0049] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0050] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0051] Embodiment 1:

[0052] This implementation proposes a method for estimating the position of a pallet for an intelligent logistics forklift by optimizing 3D coordinates, aiming to improve the positioning accuracy of the intelligent logistics forklift in pallet operation. Figure 1 As shown in the figure, the method includes multiple steps: first, by setting the initial pose and performing camera calibration, ensure that the captured image is consistent with the actual scene. Then, prepare the 3D coordinates of the pallet and the corresponding 2D pixel coordinates to prepare for the subsequent SolvePnP algorithm input. After solving the initial pose, by calculating the errors of rotation and translation, the 3D coordinates are iteratively adjusted using an optimization algorithm to minimize the deviation between the measurement and the actual; this process is achieved through the gradient descent method, which ultimately improves the accuracy of the pallet pose estimation and provides effective support for the development of automated logistics.

[0053] More specifically, the process includes the following:

[0054] Step 1: Initial pose setting, specifically including:

[0055] like Figure 2 As shown in the figure, an initial pose is set when the tray is facing the camera. The translation parameters of this pose are (x, y, z) = (0, 0, z), where x, y, z represent the translation of the tray in the camera coordinate system, the X, Y, and Z axes respectively, z is a fixed value, and the rotation parameter offset is 0, indicating that the tray remains horizontal in the camera coordinate system. This setting can ensure the stability of each corner point of the tray in the camera field of view during the initial calculation. To ensure the accuracy of the initial pose, the camera must be calibrated before the tray pose estimation to obtain its internal parameters and distortion parameters. Through camera calibration, the impact of lens distortion can be reduced, ensuring the high consistency of the captured image with the actual scene, and then adjusting the position of the tray and the camera of the camera by detecting the pose of the chessboard in the center of the tray to ensure the accuracy of the pose setting point.

[0056] Step 2: Data preparation, specifically, including:

[0057] In the process of pallet pose estimation, data preparation is a key step. First, the position of the pallet in the world coordinate system is clarified. With the center of the pallet or a specific reference point as the origin, the 3D coordinates of each corner point of the pallet are obtained using high-precision measurement tools and recorded in a three-dimensional vector format (X, Y, Z). Among them, X: represents the lateral position of the pallet on the horizontal plane; Y: represents the longitudinal position of the pallet on the horizontal plane; Z: represents the horizontal distance between the center of the pallet and the camera; secondly, the pallet image is captured by a camera to ensure that the angle between the camera and the pallet is appropriate, and the deep learning key point detection algorithm is applied to identify the corner points of the pallet, and the corresponding 2D pixel coordinates are extracted from them; then, a one-to-one correspondence between 3D coordinates and 2D pixel coordinates is established to ensure that each 3D point has a corresponding 2D pixel point in the image, and its accuracy is verified by visual inspection; finally, the collected 3D and 2D pixel coordinates are organized into a structured data format to facilitate subsequent input into the SolvePnP algorithm for calculation, laying a solid foundation for pallet pose estimation.

[0058] Step 3: Solve the initial pose, specifically including:

[0059] Input data preparation and calling SolvePnP algorithm. First, collect the 3D coordinates of the corner points of the pallet in the world coordinate system to form an N×3 matrix, where N is the number of points, containing the (X, Y, Z) coordinates of each point; at the same time, according to the initial key point detection results, collect the corresponding 2D pixel coordinates to form an N×2 matrix, containing the (u, v) coordinates of each point in the image, where u represents the horizontal coordinate in the image and v represents the vertical coordinate in the image; these 3D and 2D pixel coordinate sets are passed into the SolvePnP algorithm, which outputs the rotation matrix (i.e., the rotation matrix solution result) and the translation vector (i.e., the translation vector solution result). The rotation matrix describes the rotation of the pallet relative to the camera coordinate system, and the translation vector represents the position of the pallet in the camera coordinate system; this process lays the foundation for subsequent error optimization and precise positioning.

[0060] Step 4: Iterative optimization, such as Figure 3 As shown, specifically, including:

[0061] Iterative optimization first calculates the error, then estimates the error, and finally adjusts the 3D coordinates. Calculate the error: Calculate the rotation matrix solution R measured The translation vector solution result T measured With the true rotation matrix R actual and the true translation vector T actual The error between the rotation matrices can be obtained by calculating their relative rotation. The rotation error R can usually be calculated using the following method error :

[0062]

[0063] By adding R actual The transpose of R measured The relative rotation matrix can be obtained.

[0064] The translation error T between the translation vectors error It can be calculated directly by the following formula:

[0065] T error =T actual -T measured (2).

[0066] Error estimation: For rotation error, a common measurement method is to calculate the angle of the rotation error. The magnitude of the rotation error can be obtained by converting the error matrix into an axis-angle representation using the following formula:

[0067]

[0068] Where θ is the angle of rotation error.

[0069] For the translation error, its norm can be directly calculated:

[0070] d translation =‖T error ||(4);

[0071] Adjusting 3D coordinates: After obtaining the rotation error and translation error, the 3D coordinates need to be adjusted through an optimization algorithm to minimize these errors. The following strategy is adopted: Set a cost function J, which is in the form of:

[0072] J=α·||R error || 2 +β·||T error || 2 (5);

[0073] Among them, α and β are weight coefficients. Adjusting them can balance the impact of rotation and translation errors on the overall cost. Calculate the gradient of the cost function J with respect to the 3D coordinates The gradient reflects the direction and rate of change of the cost function at the current point. The steps to calculate the gradient include: for each 3D coordinate (u i , v i ,0),u i represents the horizontal coordinate of the i-th point, v i Represents the ordinate of the i-th point. Calculate the partial derivative of the cost function with respect to this coordinate: and The overall gradient is calculated using the chain rule by combining the partial derivatives of the rotation and translation errors:

[0074]

[0075] in, is the cost function J with respect to the horizontal coordinate u of the i-th point i The partial derivative of is the ordinate v of the cost function J with respect to the i-th point i The partial derivative of n is the number of points on the tray. Use the gradient descent method to update the 3D coordinates. The update formula is:

[0076]

[0077] Among them, 3D corrdinate is the updated 3D coordinate, 3D Corrdinate is the 3D coordinate before updating, η is the learning rate, which determines the step size of each iterative update, is the gradient of the current cost function, indicating the direction of the steepest descent. Before starting the iteration, initialize the 3D coordinates and set the learning rate η. In each iteration, calculate the value J and gradient of the current cost function Then the 3D coordinates are adjusted according to the update formula. The iteration process continues until the value of the cost function no longer decreases significantly (that is, the change is less than the set threshold) or the maximum number of iterations is reached; after several iterations, the optimized 3D coordinates are obtained, and the matching degree between these coordinates and the 2D pixel coordinates is improved, thereby improving the accuracy of the pallet pose estimation.

[0078] Embodiment 2:

[0079] like Figure 4 As shown, this implementation method provides an intelligent logistics forklift pallet posture estimation system for optimizing 3D coordinates, including:

[0080] A data acquisition unit is configured to: acquire 3D coordinates of each corner point of the tray and 2D pixel coordinates of the tray, and establish a one-to-one correspondence between the 3D coordinates and the 2D pixel coordinates so that each 3D point has a corresponding 2D pixel point in the image;

[0081] The data solving unit is configured to: generate an N×3 matrix according to the 3D coordinates of each corner point of the pallet, wherein N is the number of points, generate an N×2 matrix according to the 2D pixel coordinates of the pallet, and obtain a rotation matrix solving result and a translation vector solving result according to the N×3 matrix, the N×2 matrix and the SolvePnP algorithm;

[0082] The cost function calculation unit is configured to: obtain a rotation error according to a rotation matrix solution result and a true rotation matrix, obtain a translation error according to a translation vector solution result and a true translation vector, and calculate a cost function according to the rotation error and the translation error;

[0083] The pose estimation unit is configured to: calculate the gradient of the cost function relative to the 3D coordinates, and update the 3D coordinates using the gradient descent method based on the obtained gradient. The updating iterative process continues until the change in the cost function is less than the set threshold or the maximum number of iterations is reached, and the pose estimation result is obtained based on the final optimized 3D coordinates.

[0084] The specific working methods of each unit are described in Example 1 and will not be repeated here.

[0085] It is understandable that the above-mentioned various units can be separately or all combined into one or several other units to constitute, or some (some) units therein can also be split into multiple smaller units in function to constitute, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above-mentioned units are divided based on logical functions. In practical applications, the function of a unit can also be realized by multiple units, or the function of multiple units can be realized by one unit. In other embodiments of the present application, the system can also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented by the collaboration of multiple units.

[0086] According to another embodiment of the present application, the system described in this embodiment can be constructed, and the method of Example 1 of the present application can be implemented by running a computer program (including program code) capable of executing the steps involved in the corresponding method described in Example 1 on a general computing device such as a computer including processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM). The computer program can be recorded on, for example, a computer-readable recording medium, and loaded into the above-mentioned computing device through the computer-readable recording medium and run therein.

[0087] Embodiment 3:

[0088] like Figure 5 As shown, this implementation provides an electronic device, which includes a processor 1001, a communication interface 1002, and a computer-readable storage medium 1003. The processor 1001, the communication interface 1002, and the computer-readable storage medium 1003 may be connected via a bus or other means.

[0089] Among them, the communication interface 1002 is used to receive and send data, the computer-readable storage medium 1003 can be stored in the memory of the electronic device, the computer-readable storage medium 1003 is used to store a computer program, the computer program includes program instructions, and the processor 1001 is used to execute the program instructions stored in the computer-readable storage medium 1003.

[0090] Processor 1001 (or CPU (Central Processing Unit)) is the computing core and control core of the electronic device, which is suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions to implement corresponding method processes or corresponding functions.

[0091] The processor 1001 is configured to execute the following process:

[0092] Obtain the 3D coordinates of each corner point of the pallet and the 2D pixel coordinates of the pallet, and establish a one-to-one correspondence between the 3D coordinates and the 2D pixel coordinates so that each 3D point has a corresponding 2D pixel point in the image;

[0093] Generate an N×3 matrix according to the 3D coordinates of each corner point of the pallet, where N is the number of points, generate an N×2 matrix according to the 2D pixel coordinates of the pallet, and obtain the rotation matrix solution result and the translation vector solution result according to the N×3 matrix, the N×2 matrix and the SolvePnP algorithm;

[0094] Obtaining a rotation error according to a rotation matrix solution result and a true rotation matrix, obtaining a translation error according to a translation vector solution result and a true translation vector, and calculating a cost function according to the rotation error and the translation error;

[0095] The gradient of the cost function relative to the 3D coordinates is calculated. According to the obtained gradient, the 3D coordinates are updated using the gradient descent method. The update iterative process continues until the change in the cost function is less than the set threshold or the maximum number of iterations is reached. The pose estimation result is obtained based on the final optimized 3D coordinates.

[0096] The specific working method is described in Example 1 and will not be repeated here.

[0097] Embodiment 4:

[0098] This implementation provides a computer-readable storage medium (Memory), which is a memory device in an electronic device for storing programs and data. It is understandable that the computer-readable storage medium here can include both built-in storage media in the electronic device and, of course, extended storage media supported by the electronic device. The computer-readable storage medium provides a storage space that stores the processing system of the electronic device.

[0099] In addition, the storage space also stores one or more instructions suitable for being loaded and executed by the processor, and these instructions may be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here may be a high-speed RAM memory, or a non-volatile memory, such as at least one disk memory; optionally, it may also be at least one computer-readable storage medium located away from the aforementioned processor.

[0100] In one embodiment, the computer-readable storage medium stores one or more instructions; the processor loads and executes the one or more instructions stored in the computer-readable storage medium to implement the following process:

[0101] Obtain the 3D coordinates of each corner point of the pallet and the 2D pixel coordinates of the pallet, and establish a one-to-one correspondence between the 3D coordinates and the 2D pixel coordinates so that each 3D point has a corresponding 2D pixel point in the image;

[0102] Generate an N×3 matrix according to the 3D coordinates of each corner point of the pallet, where N is the number of points, generate an N×2 matrix according to the 2D pixel coordinates of the pallet, and obtain the rotation matrix solution result and the translation vector solution result according to the N×3 matrix, the N×2 matrix and the SolvePnP algorithm;

[0103] Obtaining a rotation error according to a rotation matrix solution result and a true rotation matrix, obtaining a translation error according to a translation vector solution result and a true translation vector, and calculating a cost function according to the rotation error and the translation error;

[0104] The gradient of the cost function relative to the 3D coordinates is calculated. According to the obtained gradient, the 3D coordinates are updated using the gradient descent method. The update iterative process continues until the change in the cost function is less than the set threshold or the maximum number of iterations is reached. The pose estimation result is obtained based on the final optimized 3D coordinates.

[0105] The specific working method is described in Example 1 and will not be repeated here.

[0106] Embodiment 5:

[0107] The present implementation provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device performs the following process:

[0108] Obtain the 3D coordinates of each corner point of the pallet and the 2D pixel coordinates of the pallet, and establish a one-to-one correspondence between the 3D coordinates and the 2D pixel coordinates so that each 3D point has a corresponding 2D pixel point in the image;

[0109] Generate an N×3 matrix according to the 3D coordinates of each corner point of the pallet, where N is the number of points, generate an N×2 matrix according to the 2D pixel coordinates of the pallet, and obtain the rotation matrix solution result and the translation vector solution result according to the N×3 matrix, the N×2 matrix and the SolvePnP algorithm;

[0110] Obtaining a rotation error according to a rotation matrix solution result and a true rotation matrix, obtaining a translation error according to a translation vector solution result and a true translation vector, and calculating a cost function according to the rotation error and the translation error;

[0111] The gradient of the cost function relative to the 3D coordinates is calculated. According to the obtained gradient, the 3D coordinates are updated using the gradient descent method. The update iterative process continues until the change in the cost function is less than the set threshold or the maximum number of iterations is reached. The pose estimation result is obtained based on the final optimized 3D coordinates.

[0112] The specific working method is described in Example 1 and will not be repeated here.

[0113] A person skilled in the art can appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0114] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loading and executing computer program instructions on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instruction can be stored in a computer-readable storage medium or transmitted by a computer-readable storage medium. The computer instruction can be transmitted from a website site, a computer, a server or a data center to another website site, a computer, a server or a data center by wired (for example, coaxial cable, optical fiber, digital line (DSL)) or wireless (for example, infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that a computer can access or a data processing device such as a server, a data center, etc. that contains one or more available media integration. Available media can be magnetic media (for example, floppy disk, hard disk, tape), optical media (for example, DVD), or semiconductor media (for example, solid-state drive (Solid State Disk, SSD)) and the like.

[0115] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for estimating the position and posture of a pallet of an intelligent logistics forklift by optimizing 3D coordinates, characterized in that: The process includes: Obtain the 3D coordinates of each corner point of the pallet and the 2D pixel coordinates of the pallet, and establish a one-to-one correspondence between the 3D coordinates and the 2D pixel coordinates so that each 3D point has a corresponding 2D pixel point in the image; Generate an N×3 matrix according to the 3D coordinates of each corner point of the pallet, where N is the number of points, generate an N×2 matrix according to the 2D pixel coordinates of the pallet, and obtain the rotation matrix solution result and the translation vector solution result according to the N×3 matrix, the N×2 matrix and the SolvePnP algorithm; Obtaining a rotation error according to a rotation matrix solution result and a true rotation matrix, obtaining a translation error according to a translation vector solution result and a true translation vector, and calculating a cost function according to the rotation error and the translation error; The gradient of the cost function relative to the 3D coordinates is calculated. According to the obtained gradient, the 3D coordinates are updated using the gradient descent method. The update iterative process continues until the change in the cost function is less than the set threshold or the maximum number of iterations is reached. The pose estimation result is obtained based on the final optimized 3D coordinates.

2. The method for estimating the position and posture of a pallet of an intelligent logistics forklift by optimizing 3D coordinates according to claim 1, characterized in that: The rotation error R is obtained based on the rotation matrix solution result and the true rotation matrix. error ,include: Among them, R measured is the result of rotation matrix solution, R actual is the true rotation matrix.

3. The method for estimating the position and posture of a pallet of an intelligent logistics forklift by optimizing 3D coordinates according to claim 1 or 2, characterized in that: The translation error T is obtained based on the translation vector solution result and the real translation vector. error ,include: T error =T actual -T measured ; Among them, T actual is the real translation vector, T measured is the result of translation vector solution.

4. The method for estimating the position and posture of a pallet of an intelligent logistics forklift by optimizing 3D coordinates according to claim 1, characterized in that: Calculating a cost function J according to the rotation error and the translation error includes: J=α·||R error || 2 +β·||T error || 2 ; Among them, α and β are weight coefficients, R error is the rotation error, T error is the translation error.

5. The method for estimating the position and posture of a pallet of an intelligent logistics forklift by optimizing 3D coordinates according to claim 4, characterized in that: Compute the gradient of the cost function with respect to the 3D coordinates According to the obtained gradient include: Among them, for each 3D coordinate (u i , v i ,0),u i represents the horizontal coordinate of the i-th point, v i represents the ordinate of the i-th point, is the cost function J with respect to the horizontal coordinate u of the i-th point i The partial derivative of is the ordinate v of the cost function J with respect to the i-th point i The partial derivative of , where n is the number of points on the tray.

6. The method for estimating the position and posture of a pallet of an intelligent logistics forklift by optimizing 3D coordinates as claimed in claim 5, characterized in that: Update the 3D coordinates using gradient descent, including: Among them, 3D corrdinate is the updated 3D coordinate, η is the learning rate, 3D Corrdinate is the 3D coordinate before updating.

7. An intelligent logistics forklift pallet posture estimation system with optimized 3D coordinates, characterized in that: include: A data acquisition unit is configured to: acquire 3D coordinates of each corner point of the tray and 2D pixel coordinates of the tray, and establish a one-to-one correspondence between the 3D coordinates and the 2D pixel coordinates so that each 3D point has a corresponding 2D pixel point in the image; The data solving unit is configured to: generate an N×3 matrix according to the 3D coordinates of each corner point of the pallet, wherein N is the number of points, generate an N×2 matrix according to the 2D pixel coordinates of the pallet, and obtain a rotation matrix solving result and a translation vector solving result according to the N×3 matrix, the N×2 matrix and the SolvePnP algorithm; The cost function calculation unit is configured to: obtain a rotation error according to a rotation matrix solution result and a true rotation matrix, obtain a translation error according to a translation vector solution result and a true translation vector, and calculate a cost function according to the rotation error and the translation error; The pose estimation unit is configured to: calculate the gradient of the cost function relative to the 3D coordinates, and update the 3D coordinates using the gradient descent method based on the obtained gradient. The updating iterative process continues until the change in the cost function is less than the set threshold or the maximum number of iterations is reached, and the pose estimation result is obtained based on the final optimized 3D coordinates.

8. A computer device, characterized in that: include: a processor and a computer readable storage medium; a processor adapted to execute a computer program; A computer-readable storage medium having a computer program stored therein, wherein when the computer program is executed by the processor, the method for estimating the pallet posture of an intelligent logistics forklift with optimized 3D coordinates as described in any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which is suitable for being loaded by a processor and executing the intelligent logistics forklift pallet pose estimation method for optimizing 3D coordinates as described in any one of claims 1 to 6.

10. A computer program product, characterized in that The computer program product includes a computer program, and when the computer program is executed by a processor, it implements the intelligent logistics forklift pallet pose estimation method for optimizing 3D coordinates as described in any one of claims 1 to 6.

Images