Forklift posture adjustment method and device

Abstract

This invention provides a forklift posture adjustment method and apparatus, relating to the field of forklift intelligent control technology. The method includes: determining the tilt angle and position of a target pallet; determining the central axis of the target pallet based on its tilt angle and position; determining a position to be adjusted for the forklift on the central axis, wherein the central axis passes through the pickup side of the target pallet; controlling the forklift to move to the position to be adjusted; and controlling the forklift to rotate according to the tilt angle. This invention adjusts the forklift posture by determining the position of the target pallet, thereby improving the accuracy of target pallet pickup.

Description

Technical Field

[0001] This invention relates to the field of intelligent control technology for forklifts, and in particular to a method and device for adjusting the position and posture of a forklift. Background Technology

[0002] With the rapid development of computer and robotics technologies, intelligent forklifts have been widely used in the logistics industry, marking the development of warehousing and logistics technologies toward automation and intelligence.

[0003] In the complex environment of unstructured warehouses, the position and posture of pallets are highly uncertain due to factors such as work processes, equipment precision, and manual operations, affecting the flexible pallet picking by intelligent forklifts. Especially when forklift navigation and control precision is limited, accurate pallet picking becomes extremely difficult. Summary of the Invention

[0004] To address the problems existing in the prior art, the present invention provides a forklift position adjustment method and device.

[0005] This invention provides a forklift posture adjustment method, including: determining the tilt angle and position of the target pallet;

[0006] Based on the tilt angle and position of the target pallet, the centerline of the target pallet is determined, and the position to be adjusted for the forklift is determined on the centerline, wherein the centerline passes through the pickup side of the target pallet;

[0007] Control the forklift to move to the position to be adjusted, and control the rotation of the forklift according to the tilt angle.

[0008] According to a forklift posture adjustment method provided by the present invention, the step of determining the tilt angle and position of the target pallet includes:

[0009] The color and depth images of the target tray are acquired from the pickup side or the side opposite to the pickup side using an image sensor;

[0010] The color image is segmented to obtain the tray region in the color image;

[0011] Convert the depth image into a point cloud image;

[0012] Based on the point cloud map and the pallet area, determine the tilt angle of the target pallet;

[0013] The location of the target pallet is determined based on the point cloud map and the pallet area.

[0014] According to a forklift posture adjustment method provided by the present invention, the step of determining the tilt angle of the target pallet based on the point cloud map and the pallet area includes:

[0015] Determine the left midpoint and right midpoint of the tray area;

[0016] The point cloud coordinates corresponding to the left midpoint and the right midpoint are obtained from the point cloud map, and the coordinate system of the point cloud map is the coordinate system of the image sensor.

[0017] Based on the point cloud coordinates corresponding to the left midpoint and the right midpoint, the tilt angle of the target tray rotating about the Y-axis of the coordinate system is obtained.

[0018] According to a forklift positioning method provided by the present invention, the step of determining the position of the target pallet based on the point cloud map and the pallet area includes:

[0019] Determine the center point of the tray area;

[0020] Obtain the point cloud coordinates corresponding to the center point from the point cloud map;

[0021] The point cloud coordinates corresponding to the center point are used as the position of the target tray.

[0022] According to a forklift posture adjustment method provided by the present invention, the step of determining the position to be adjusted of the forklift on the central axis includes:

[0023] The point where the central axis intersects the front of the forklift is taken as the position to be adjusted for the forklift.

[0024] According to a forklift posture adjustment method provided by the present invention, the step of determining the position to be adjusted of the forklift on the central axis includes:

[0025] The position to be adjusted of the forklift is determined on the central axis, such that after the forklift is rotated by the tilt angle at the position to be adjusted, the front left or front right of the forklift is located in front of the forklift at its original position.

[0026] According to a forklift pose adjustment method provided by the present invention, the image sensor is installed at the midpoint of the front of the forklift;

[0027] The step of determining the position to be adjusted of the forklift on the central axis includes:

[0028] Based on the position of the target tray, obtain the absolute value of the X coordinate and the Z coordinate of the target tray in the coordinate system of the image sensor, and use the absolute value of the X coordinate as the first distance;

[0029] Determine the intersection point between the central axis and the X-axis of the coordinate system, and the projection point of the target tray's position on the X-axis;

[0030] Based on the Z-coordinate and the tilt angle, obtain the second distance between the intersection point and the projection point;

[0031] Based on the width of the forklift and the tilt angle, obtain the third distance between the intersection point and the front left or front right side;

[0032] Based on the width of the forklift and the tilt angle, obtain the fourth distance between the projection point of the position to be adjusted on the X-axis and the front left or front right.

[0033] The travel distance of the forklift in the X-axis direction is determined based on the first distance, the second distance, the third distance, and the fourth distance.

[0034] The travel distance of the forklift in the Z-axis direction is determined based on the width and tilt angle of the forklift.

[0035] The direction of travel of the forklift in the X-axis direction is determined based on the relative position between the intersection point and the image sensor.

[0036] The direction of travel of the forklift in the Z-axis direction is determined to be the negative direction of the Z-axis;

[0037] The position to be adjusted for the forklift is determined based on the forklift's travel distance and direction in the X-axis direction, and the travel distance and direction in the Z-axis direction.

[0038] According to a forklift pose adjustment method provided by the present invention, the step of segmenting the color image and obtaining the pallet region in the color image includes:

[0039] The color image is input into a semantic segmentation network, and the tray region in the color image is output based on the semantic segmentation network.

[0040] The semantic segmentation network is trained using the color image of the sample tray as a sample and the tray area marked in the color image of the sample tray as a label.

[0041] The present invention also provides a forklift position adjustment device, comprising:

[0042] The determination module is used to determine the tilt angle and position of the target pallet;

[0043] The calculation module is used to determine the centerline of the target pallet based on the tilt angle and position of the target pallet, and to determine the position to be adjusted of the forklift on the centerline, wherein the centerline passes through the pickup side of the target pallet;

[0044] The adjustment module is used to control the forklift to move to the position to be adjusted, and to control the rotation of the forklift according to the tilt angle.

[0045] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the forklift posture adjustment method as described above.

[0046] The forklift position adjustment method and device provided by the present invention adjusts the position and posture of the forklift according to the tilt angle and position of the target pallet, so that the central axis of the forklift is consistent with the central axis of the target pallet, thereby improving the accuracy of picking up the target pallet. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0048] Figure 1 This is a flowchart illustrating the forklift position adjustment method provided by the present invention;

[0049] Figure 2 This is a schematic diagram of the camera coordinate system in the forklift pose adjustment method provided by the present invention;

[0050] Figure 3 This is a schematic diagram of the tilt angle of the target pallet in the forklift posture adjustment method provided by the present invention;

[0051] Figure 4 This is one of the schematic diagrams illustrating the relative position of the target pallet and the forklift in the forklift posture adjustment method provided by this invention;

[0052] Figure 5 This is the second schematic diagram of the relative position of the target pallet and the forklift in the forklift posture adjustment method provided by the present invention;

[0053] Figure 6 This is the third schematic diagram of the relative position of the target pallet and the forklift in the forklift posture adjustment method provided by the present invention;

[0054] Figure 7This is the fourth schematic diagram of the relative position of the target pallet and the forklift in the forklift posture adjustment method provided by the present invention;

[0055] Figure 8 This is the fifth schematic diagram showing the relative position of the target pallet and the forklift in the forklift posture adjustment method provided by the present invention;

[0056] Figure 9 This is the sixth schematic diagram showing the relative position of the target pallet and the forklift in the forklift posture adjustment method provided by the present invention;

[0057] Figure 10 This is a schematic diagram of the forklift posture adjustment device provided by the present invention;

[0058] Figure 11 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0060] The following is combined Figures 1 to 9 The forklift pose adjustment method of the present invention includes:

[0061] Step 101: Determine the tilt angle and position of the target pallet;

[0062] In this embodiment, the forklift is mounted on an AGV (Automated Guided Vehicle).

[0063] The tilt angle of the target pallet is the rotation angle of the target pallet relative to the specified placement direction, and the position of the target pallet is the spatial location where the target pallet is placed. This embodiment does not limit the method for determining the tilt angle and position of the target pallet.

[0064] Because the tilt angle and position of the target pallet may change, it is difficult to pick up the pallet accurately if the forklift is set up as before.

[0065] In this embodiment, after the forklift travels to the front of the target pallet according to its own navigation, the forklift's posture is adjusted according to the tilt angle and position of the target pallet.

[0066] Optionally, the tilt angle and position of the target pallet can be determined by taking an image of the target pallet and then analyzing the image.

[0067] Step 102: Determine the central axis of the target pallet based on its tilt angle and position, and determine the position to be adjusted for the forklift on the central axis, wherein the central axis passes through the pickup side of the target pallet;

[0068] The tilt angle of the target pallet is the tilt angle of the target pallet's central axis. The straight line passing through the center of the target pallet and tilted at the same angle as the target pallet's tilt angle is the central axis of the target pallet.

[0069] Determine the position to be adjusted for the forklift on the centerline so that the forklift is directly in front of the target pallet.

[0070] To avoid collisions between the forklift forks and the target pallet, the distance between the position to be adjusted and the target pallet should be greater than the length of the forklift forks.

[0071] Step 103: Control the forklift to move to the position to be adjusted, and control the rotation of the forklift according to the tilt angle.

[0072] The forklift's current position, the position to be adjusted, and the tilt angle of the target pallet are input into the forklift's AGV system. The AGV system controls the forklift to move to the position to be adjusted and controls the forklift to rotate, with the rotation angle equal to the tilt angle of the target pallet.

[0073] Specifically, the AGV system plans a path based on the forklift's current position and the position to be adjusted. It controls the forklift to move backward and then forward to the position to be adjusted according to the planned path. It also controls the forklift to rotate during the forward movement so that the forklift's centerline at the position to be adjusted is aligned with the centerline of the target pallet, thereby accurately picking up the target pallet.

[0074] This invention improves the accuracy of picking up the target pallet by adjusting the position and posture of the forklift according to the tilt angle and position of the target pallet, so that the central axis of the forklift is aligned with the central axis of the target pallet.

[0075] In the forklift posture adjustment method of the present invention, the step of determining the tilt angle and position of the target pallet includes:

[0076] The color and depth images of the target tray are acquired from the pickup side or the side opposite to the pickup side using an image sensor;

[0077] Optionally, the image sensor is a TOF (Time of Flight, Depth Camera) capable of acquiring depth information of the image, i.e., the distance between each pixel in the image and the shooting plane.

[0078] When a TOF camera is mounted on a forklift, it acquires an image of the target pallet from the pickup side of the target pallet.

[0079] The forklift travels to a position about 2 meters in front of the target pallet, activates the TOF camera, and sets the trigger mode of the TOF camera to continuous trigger to capture the RGB data stream, depth data stream, and camera intrinsic parameters of the target pallet.

[0080] Optionally, distortion correction is performed on the RGB data, and the depth image is aligned with the distortion-corrected color image.

[0081] The color image is segmented to obtain the tray region in the color image;

[0082] The color image is segmented using an image segmentation algorithm to separate the target tray from the background.

[0083] Optionally, the segmentation results are subjected to morphological dilation to merge the tray regions into one, and the boundaries of the tray regions are optimized by performing quadrilateral fitting of the boundary points.

[0084] Convert the depth image into a point cloud image;

[0085] Based on the camera parameters, the coordinates of each pixel in the depth image are mapped from the pixel coordinate system to the point cloud coordinate system through indexing, that is, the depth map is converted into a point cloud map.

[0086] Optionally, before converting the depth image, the depth image can be filtered to remove noise.

[0087] Based on the point cloud map and the pallet area, determine the tilt angle of the target pallet;

[0088] The location of the target pallet is determined based on the point cloud map and the pallet area.

[0089] The pixels in the color image of the tray area correspond one-to-one with the pixels in the depth image. By indexing the coordinate points in the point cloud map through the pixels in the depth image corresponding to the tray area, the three-dimensional coordinates of each pixel in the tray area can be determined.

[0090] The tilt angle of the target pallet can be calculated using the point cloud coordinates of the pallet area. This embodiment does not limit the method for calculating the tilt angle.

[0091] This embodiment processes the color and depth images of the target pallet to accurately determine its three-dimensional coordinates, thereby improving the accuracy of the calculation of the target pallet's tilt angle and position.

[0092] In the forklift posture adjustment method of the present invention, the step of determining the tilt angle of the target pallet based on the point cloud map and the pallet area includes:

[0093] Determine the left midpoint and right midpoint of the tray area;

[0094] Since the image sensor acquires the color image of the target tray from the pickup side or the opposite side of the pickup side, the tray area obtained by segmenting the color image is the front view of the target tray.

[0095] The midpoint of the left side of the pallet area is the midpoint of the left side of the pallet area, and the midpoint of the right side of the pallet area is the midpoint of the right side of the pallet area.

[0096] The point cloud coordinates corresponding to the left midpoint and the right midpoint are obtained from the point cloud map, and the coordinate system of the point cloud map is the coordinate system of the image sensor.

[0097] With the center of the TOF camera lens as the origin, the horizontal axis of its shooting interface as the X-axis, the vertical axis as the Y-axis, and the axis perpendicular to the shooting interface as the Z-axis, the camera coordinate system is established as follows: Figure 2 As shown.

[0098] Obtain the point cloud coordinates of the left and right midpoints of the tray area from the point cloud map.

[0099] Based on the point cloud coordinates corresponding to the left midpoint and the right midpoint, the tilt angle of the target tray rotating about the Y-axis of the coordinate system is obtained.

[0100] The tilt angle of the target pallet is calculated using the coordinates of the left and right midpoints of the pallet area in the XZ plane of the point cloud coordinate system.

[0101] like Figure 3 As shown, the coordinates of the left midpoint A on the XZ plane are (X... a Z a The point cloud coordinates of the right midpoint B on the XZ plane are (X... b Z b The distance between the left midpoint A and the right midpoint B along the Z-axis is |Z b -Z a The distance along the X-axis is |X. b -X a |

[0102] The tangent of the tilt angle of the target pallet is |Z b -Z a | / |X b -X a The tilt angle of the target tray can be calculated using inverse trigonometric functions.

[0103] In the forklift pose adjustment method of the present invention, the step of determining the position of the target pallet based on the point cloud map and the pallet area includes:

[0104] Determine the center point of the tray area;

[0105] Optionally, the intersection of the lines connecting the diagonals of the pallet area can be used as the center point of the pallet area.

[0106] Obtain the point cloud coordinates corresponding to the center point from the point cloud map;

[0107] The point cloud coordinates corresponding to the center point are used as the position of the target tray.

[0108] Based on the point cloud coordinate system, the point cloud coordinates corresponding to the center point C are determined to be (x...). c y C , z c This is used as the target pallet position to calculate the forklift's adjustment position.

[0109] In the forklift posture adjustment method of the present invention, the step of determining the position to be adjusted of the forklift on the central axis includes:

[0110] The point where the central axis intersects the front of the forklift is taken as the position to be adjusted for the forklift.

[0111] The forklift front refers to the side of the forklift body facing the target pallet. The point D where the centerline of the target pallet intersects the forklift front is the position to be adjusted for the forklift.

[0112] Take the midpoint E of the forklift's front as the forklift position, control the forklift to move from point E to the position to be adjusted, point D, and control the forklift to rotate at an angle equal to the tilt angle of the target pallet.

[0113] Reference Figure 4 The distance point E moves to point D Therefore, based on the coordinates and distance of point E Determine the position to be adjusted.

[0114] In the forklift posture adjustment method of the present invention, the step of determining the position to be adjusted of the forklift on the central axis includes:

[0115] The position to be adjusted of the forklift is determined on the central axis, such that after the forklift is rotated by the tilt angle at the position to be adjusted, the front left or front right of the forklift is located in front of the forklift at its original position.

[0116] like Figure 4 As shown, the front left or front right of the forklift is the left or right side of the forklift closest to the target pallet.

[0117] When the target pallet rotates clockwise along the Y-axis, after controlling the forklift's rotation tilt angle, the front left side of the forklift is located in front of the forklift's original position, i.e., on the extension line of DE.

[0118] When the target pallet rotates counterclockwise along the Y-axis, after controlling the rotation tilt angle of the forklift, the front right side of the forklift is located in front of the original position of the forklift, that is, on the extension line of DE.

[0119] This embodiment determines the position of the forklift to be adjusted so that the left or right front of the forklift is in front of the forklift in its original position, thereby controlling the position of the forklift within a suitable range of the target pallet and reserving a certain space for the operation of the forklift's forks.

[0120] In the forklift posture adjustment method of the present invention, the image sensor is installed in front of the forklift and located at the midpoint of the width of the forklift;

[0121] The image sensor is positioned in front of the forklift at point E, which is the midpoint of the forklift's width. When the forklift is in its original position, point E is the origin of the point cloud coordinate system and also the location of the image sensor, facilitating the calculation of the forklift's desired adjustment position.

[0122] The step of determining the position to be adjusted of the forklift on the central axis includes:

[0123] Based on the position of the target tray, obtain the absolute value of the X coordinate and the Z coordinate of the target tray in the coordinate system of the image sensor, and use the absolute value of the X coordinate as the first distance;

[0124] like Figure 4 As shown, the coordinates of the target pallet's position C are (x... c y C , z c ), first distance

[0125] Determine the intersection point between the centerline of the target tray and the X-axis, and the projection point of the target tray's position on the X-axis;

[0126] The intersection of the central axis of the target pallet and the X-axis is point D, and the projection of point C onto the X-axis is point F.

[0127] Based on the Z-coordinate and the tilt angle, obtain the second distance between the intersection point and the projection point;

[0128] The angle between the central axis of the target pallet and the Z-axis of the point cloud coordinate system is the tilt angle α of the target pallet. The second distance between intersection point D and point F.

[0129] Based on the width of the forklift and the tilt angle, obtain the third distance between the intersection point and the front left or front right side;

[0130] Let point G be the position to be adjusted for the forklift. Let d represent the width of the forklift. Then, the third distance can be calculated using trigonometric formulas.

[0131] Based on the width of the forklift and the tilt angle, obtain the fourth distance between the projection point of the position to be adjusted on the X-axis and the front left or front right.

[0132] The projection point of the position to be adjusted on the X-axis is point I. The fourth distance is calculated using trigonometric functions.

[0133] The travel distance of the forklift in the X-axis direction is determined based on the first distance, the second distance, the third distance, and the fourth distance.

[0134] The distance the forklift needs to travel along the X-axis to move to the position to be adjusted is ΔX, which is calculated as follows.

[0135] The calculation method for the forklift's travel distance in the X-axis direction differs depending on the relative position of the forklift and the target pallet, as detailed below:

[0136] When the target pallet rotates clockwise along the Y-axis, there are three possibilities:

[0137] like Figure 4 As shown, when point E is to the left of point D,

[0138] like Figure 5 As shown, when point E is located between points D and F,

[0139] like Figure 6 As shown, when point E is to the right of point F,

[0140] When the target pallet rotates counterclockwise along the Y-axis, there are three possibilities:

[0141] like Figure 7 As shown, when point E is to the right of point D,

[0142] like Figure 8 As shown, when point E is located between points D and F,

[0143] like Figure 9 As shown, when point E is to the left of point F,

[0144] The travel distance of the forklift in the Z-axis direction is determined based on the width and tilt angle of the forklift.

[0145] The distance the forklift travels along the Z-axis to move to the desired adjustment position is ΔZ. This distance is calculated using trigonometric functions based on the forklift's width d and tilt angle α.

[0146] The direction of travel of the forklift in the X-axis direction is determined based on the relative position between the intersection point and the image sensor.

[0147] If the intersection point D is to the left of point E, then the forklift will travel from point E to point G along the negative X-axis.

[0148] If the intersection point D is to the right of point E, then the forklift will travel from point E to point G along the positive X-axis.

[0149] The direction of travel of the forklift in the Z-axis direction is determined to be the negative direction of the Z-axis.

[0150] Regardless of whether the forklift travels in the positive or negative direction on the X-axis, it always travels in the negative direction on the Z-axis, thus maintaining a certain distance between the forklift's forks and the target pallet, reserving space for fork operation.

[0151] The position to be adjusted for the forklift is determined based on the forklift's travel distance and direction in the X-axis direction, and the travel distance and direction in the Z-axis direction.

[0152] Based on the forklift's travel distance and direction along the X-axis, as well as its travel distance and direction along the Z-axis, the position to be adjusted can be determined. Then, the forklift's AGV system can be used to control the forklift to move to the position to be adjusted.

[0153] This embodiment determines the position of the forklift to be adjusted so that the left or right front of the forklift is in front of the forklift in its original position, thereby controlling the position of the forklift within a suitable range of the target pallet and reserving a certain space for the operation of the forklift's forks.

[0154] In the forklift pose adjustment method of the present invention, the step of segmenting the color image to obtain the pallet region in the color image includes:

[0155] The color image is input into a semantic segmentation network, and the tray region in the color image is output based on the semantic segmentation network.

[0156] Optionally, the semantic segmentation network is the DDRNet network.

[0157] Before using a semantic segmentation network to segment color images, the collected color image dataset is subjected to enhancement processing such as rotation, translation, and scaling to expand the number of samples and improve sample diversity.

[0158] The augmented dataset is then input into the semantic segmentation network for training. There is no need to resample the image size, but it is necessary to ensure that the input image size is consistent.

[0159] Optionally, the number of samples is more than 400, the initial learning rate is 0.0001, the batch size is set to 2, and the training is completed after a total of 200 rounds.

[0160] After the semantic segmentation network is trained, the distortion-corrected color image is input into the trained semantic segmentation network. Without resampling the image, the segmentation result detected by the semantic segmentation network is obtained, and the tray region in the color image is separated.

[0161] The semantic segmentation network is trained using the color image of the sample tray as a sample and the tray area marked in the color image of the sample tray as a label.

[0162] Images of trays under different postures, lighting conditions, and scenes were collected. The trays were manually labeled and then created using Labelme software. The labels were converted to VOC format. The labels for the tray images were raster data, with the tray area attribute value being 1 and the background area attribute value being 0.

[0163] This embodiment achieves accurate segmentation of color images by using a semantic segmentation network.

[0164] The forklift posture adjustment device provided by the present invention is described below. The forklift posture adjustment device described below and the forklift posture adjustment method described above can be referred to in correspondence.

[0165] like Figure 10 As shown, the device includes a determining module 1001, a calculating module 1002, and an adjusting module 1003, wherein:

[0166] The determination module 1001 is used to determine the tilt angle and position of the target pallet;

[0167] In this embodiment, the forklift is mounted on an AGV (Automated Guided Vehicle).

[0168] The tilt angle of the target pallet is the rotation angle of the target pallet relative to the specified placement direction, and the position of the target pallet is the spatial location where the target pallet is placed. This embodiment does not limit the method for determining the tilt angle and position of the target pallet.

[0169] Because the tilt angle and position of the target pallet may change, it is difficult to pick up the pallet accurately if the forklift is set up as before.

[0170] In this embodiment, after the forklift travels to the front of the target pallet according to its own navigation, the forklift's posture is adjusted according to the tilt angle and position of the target pallet.

[0171] Optionally, the tilt angle and position of the target pallet can be determined by taking an image of the target pallet and then analyzing the image.

[0172] The calculation module 1002 is used to determine the centerline of the target pallet based on the tilt angle and position of the target pallet, and to determine the position to be adjusted of the forklift on the centerline, wherein the centerline passes through the pickup side of the target pallet;

[0173] The tilt angle of the target pallet is the tilt angle of the target pallet's central axis. The straight line passing through the center of the target pallet and tilted at the same angle as the target pallet's tilt angle is the central axis of the target pallet.

[0174] Determine the position to be adjusted for the forklift on the centerline so that the forklift is directly in front of the target pallet.

[0175] To avoid collisions between the forklift forks and the target pallet, the distance between the position to be adjusted and the target pallet should be greater than the length of the forklift forks.

[0176] The adjustment module 1003 is used to control the forklift to run to the position to be adjusted, and to control the rotation of the forklift according to the tilt angle.

[0177] The forklift's current position, the position to be adjusted, and the tilt angle of the target pallet are input into the forklift's AGV system. The AGV system controls the forklift to move to the position to be adjusted and controls the forklift to rotate, with the rotation angle equal to the tilt angle of the target pallet.

[0178] Specifically, the AGV system plans a path based on the forklift's current position and the position to be adjusted. It controls the forklift to move backward and then forward to the position to be adjusted according to the planned path. It also controls the forklift to rotate during the forward movement so that the forklift's centerline at the position to be adjusted is aligned with the centerline of the target pallet, thereby accurately picking up the target pallet.

[0179] This invention improves the accuracy of picking up the target pallet by adjusting the position and posture of the forklift according to the tilt angle and position of the target pallet, so that the central axis of the forklift is aligned with the central axis of the target pallet.

[0180] Figure 11 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 11 As shown, the electronic device may include a processor 1110, a communications interface 1120, a memory 1130, and a communication bus 1140. The processor 1110, communications interface 1120, and memory 1130 communicate with each other via the communication bus 1140. The processor 1110 can call logical instructions in the memory 1130 to execute a forklift posture adjustment method. This method includes: determining the tilt angle and position of the target pallet; and determining the travel distance of the forklift based on the width of the forklift and the tilt angle and position of the target pallet.

[0181] Furthermore, the logical instructions in the aforementioned memory 1130 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0182] On the other hand, the present invention also provides a computer program product, the computer program product including a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the forklift posture adjustment method provided by the above methods. The method includes: determining the tilt angle and position of the target pallet; and determining the travel distance of the forklift based on the width of the forklift, the tilt angle and position of the target pallet.

[0183] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a forklift pose adjustment method provided by the methods described above, the method comprising: determining the tilt angle and position of a target pallet; and determining the travel distance of the forklift based on the width of the forklift, the tilt angle and position of the target pallet.

[0184] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0185] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0186] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A forklift position adjustment method, characterized in that, include: Determine the tilt angle and position of the target pallet; Based on the tilt angle and position of the target pallet, the centerline of the target pallet is determined, and the position to be adjusted for the forklift is determined on the centerline, wherein the centerline passes through the pickup side of the target pallet; Control the forklift to move to the position to be adjusted, and control the rotation of the forklift according to the tilt angle; The steps for determining the tilt angle and position of the target pallet include: The color and depth images of the target tray are acquired from the pickup side or the side opposite to the pickup side using an image sensor; The color image is segmented to obtain the tray region in the color image; Convert the depth image into a point cloud image; Based on the point cloud map and the pallet area, determine the tilt angle of the target pallet; The location of the target pallet is determined based on the point cloud map and the pallet area; The image sensor is mounted at the midpoint of the front of the forklift. The step of determining the position to be adjusted of the forklift on the central axis includes: Based on the position of the target tray, obtain the absolute value of the X coordinate and the Z coordinate of the target tray in the coordinate system of the image sensor, and use the absolute value of the X coordinate as the first distance; Determine the intersection point between the central axis and the X-axis of the coordinate system, and the projection point of the target tray's position on the X-axis; Based on the Z-coordinate and the tilt angle, obtain the second distance between the intersection point and the projection point; Based on the width of the forklift and the tilt angle, obtain the third distance between the intersection point and the left or right side in front; Based on the width of the forklift and the tilt angle, obtain the fourth distance between the projection point of the position to be adjusted on the X-axis and the front left or front right. The travel distance of the forklift in the X-axis direction is determined based on the first distance, the second distance, the third distance, and the fourth distance. Based on the width and tilt angle of the forklift, determine the travel distance of the forklift in the Z-axis direction of the coordinate system; The direction of travel of the forklift in the X-axis direction is determined based on the relative position between the intersection point and the image sensor. The direction of travel of the forklift in the Z-axis direction is determined to be the negative direction of the Z-axis; The position to be adjusted for the forklift is determined based on the forklift's travel distance and direction in the X-axis direction, and the travel distance and direction in the Z-axis direction.

2. The forklift position adjustment method according to claim 1, characterized in that, The step of determining the tilt angle of the target pallet based on the point cloud map and the pallet area includes: Determine the left midpoint and right midpoint of the tray area; The point cloud coordinates corresponding to the left midpoint and the right midpoint are obtained from the point cloud map, and the coordinate system of the point cloud map is the coordinate system of the image sensor. Based on the point cloud coordinates corresponding to the left midpoint and the right midpoint, the tilt angle of the target tray rotating about the Y-axis of the coordinate system is obtained.

3. The forklift position adjustment method according to claim 1, characterized in that, The step of determining the location of the target tray based on the point cloud map and the tray area includes: Determine the center point of the tray area; Obtain the point cloud coordinates corresponding to the center point from the point cloud map; The point cloud coordinates corresponding to the center point are used as the position of the target tray.

4. The forklift position adjustment method according to any one of claims 1-3, characterized in that, The step of determining the position to be adjusted of the forklift on the central axis includes: The point where the central axis intersects the front of the forklift is taken as the position to be adjusted for the forklift.

5. The forklift position adjustment method according to any one of claims 1-3, characterized in that, The step of determining the position to be adjusted of the forklift on the central axis includes: The position to be adjusted of the forklift is determined on the central axis, such that after the forklift is rotated by the tilt angle at the position to be adjusted, the front left or front right of the forklift is located in front of the forklift at its original position.

6. The forklift position adjustment method according to any one of claims 1-3, characterized in that, The step of segmenting the color image to obtain the tray region in the color image includes: The color image is input into a semantic segmentation network, and the tray region in the color image is output based on the semantic segmentation network. The semantic segmentation network is trained using the color image of the sample tray as a sample and the tray area marked in the color image of the sample tray as a label.

7. A forklift position adjustment device, characterized in that, include: The determination module is used to determine the tilt angle and position of the target pallet; The calculation module is used to determine the centerline of the target pallet based on the tilt angle and position of the target pallet, and to determine the position to be adjusted of the forklift on the centerline, wherein the centerline passes through the pickup side of the target pallet; An adjustment module is used to control the forklift to move to the position to be adjusted, and to control the rotation of the forklift according to the tilt angle; The steps for determining the tilt angle and position of the target pallet include: The color and depth images of the target tray are acquired from the pickup side or the side opposite to the pickup side using an image sensor; The color image is segmented to obtain the tray region in the color image; Convert the depth image into a point cloud image; Based on the point cloud map and the pallet area, determine the tilt angle of the target pallet; The location of the target pallet is determined based on the point cloud map and the pallet area; The image sensor is mounted at the midpoint of the front of the forklift. The step of determining the position to be adjusted of the forklift on the central axis includes: Based on the position of the target tray, obtain the absolute value of the X coordinate and the Z coordinate of the target tray in the coordinate system of the image sensor, and use the absolute value of the X coordinate as the first distance; Determine the intersection point between the central axis and the X-axis of the coordinate system, and the projection point of the target tray's position on the X-axis; Based on the Z-coordinate and the tilt angle, obtain the second distance between the intersection point and the projection point; Based on the width of the forklift and the tilt angle, obtain the third distance between the intersection point and the left or right side in front; Based on the width of the forklift and the tilt angle, obtain the fourth distance between the projection point of the position to be adjusted on the X-axis and the front left or front right. The travel distance of the forklift in the X-axis direction is determined based on the first distance, the second distance, the third distance, and the fourth distance. Based on the width and tilt angle of the forklift, determine the travel distance of the forklift in the Z-axis direction of the coordinate system; The direction of travel of the forklift in the X-axis direction is determined based on the relative position between the intersection point and the image sensor. The direction of travel of the forklift in the Z-axis direction is determined to be the negative direction of the Z-axis; The position to be adjusted for the forklift is determined based on the forklift's travel distance and direction in the X-axis direction, and the travel distance and direction in the Z-axis direction.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the forklift position adjustment method as described in any one of claims 1 to 6.

Images