Stacking detection method, forking mobile robot, device, electronic equipment and medium

Abstract

The embodiment of the invention provides a stack detection method, a forking mobile robot, a device, electronic equipment and a medium, and relates to the technical field of robots, the method is applied to the forking mobile robot, and a fork tooth of the robot is provided with a laser radar. Controlling the prongs to lift upwards from the first height; when the prongs are located at the first height, fork holes corresponding to goods at the bottommost layer are located in the detection range of the laser radar; detecting the direction of the to-be-detected stack through a laser radar in the lifting process of the prongs until the lifting stopping condition of the prongs is met, and obtaining to-be-utilized point cloud data; when the lifting stopping condition is met, the detected range of the laser radar comprises the fork hole corresponding to the cargo at the highest layer; based on the to-be-utilized point cloud data, the poses of fork holes corresponding to all the layers of goods are determined, and to-be-utilized information serving as a basis for disassembling all the layers of goods in the to-be-detected stack is obtained. And the pose information of the stack can be obtained.

Description

Technical Field

[0001] The present application relates to the field of robot technology, and in particular to a stacking detection method, a fork-picking mobile robot, a device, an electronic device and a medium. Background Art

[0002] In warehousing and logistics scenarios, goods are often placed in stacks. In order to improve the efficiency of sorting goods and save labor costs, the Fork Mobile Robot (FMR) can be used to remove the goods that need to be transported from the stack and transport the removed goods to the target location. In order to ensure that the Fork Mobile Robot can accurately remove goods from the stack, it is often necessary to obtain the position information of the stack. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide a stack detection method, a fork-picking mobile robot, a device, an electronic device and a medium, so as to obtain the posture information of the stack and ensure that the fork-picking mobile robot can accurately remove the goods from the stack.

[0004] The specific technical solutions are as follows:

[0005] In a first aspect of an embodiment of the present application, a stacking detection method is provided, which is applied to a fork picking mobile robot, wherein a laser radar is provided on the fork tines of the fork picking mobile robot, and the method comprises:

[0006] When the fork picking mobile robot reaches a designated detection position near the stack to be detected, the fork tine is controlled to be lifted upward from a first height; wherein, when the fork tine is at the first height, the fork hole corresponding to the bottom layer of the goods in the stack to be detected is within the detection range of the laser radar;

[0007] During the lifting process of the fork teeth, the laser radar is used to detect the direction of the stack to be detected until the lifting condition of the fork teeth is reached, and the point cloud data to be used of the stack to be detected is obtained; wherein, when the lifting condition is reached, the range detected by the laser radar includes the fork hole corresponding to the highest layer of goods in the stack to be detected;

[0008] Based on the point cloud data to be utilized, the positions and postures of the fork holes corresponding to each layer of goods in the stack to be detected are determined, and the information to be utilized as a basis for removing each layer of goods in the stack to be detected is obtained.

[0009] Optionally, the position and posture of the fork hole corresponding to any layer of goods include: the height of the fork hole corresponding to the layer of goods, the position of the projection of the fork hole corresponding to the layer of goods on the horizontal plane, and the deflection angle of the fork hole corresponding to the layer of goods; wherein the deflection angle of the fork hole corresponding to the layer of goods represents: the angle between the projection of the surface of the stack to be detected to which the fork hole corresponding to the layer of goods belongs on the horizontal plane and the projection of the designated plane on the horizontal plane; the designated plane is perpendicular to the direction in which the fork tine points;

[0010] After determining the positions of the fork holes corresponding to each layer of goods in the to-be-detected stack based on the to-be-utilized point cloud data and obtaining the to-be-utilized information as a basis for removing each layer of goods in the to-be-detected stack, the method further includes:

[0011] When receiving a removal instruction, determining the position of the fork hole corresponding to the layer where the goods to be removed indicated by the removal instruction are located from the information to be used;

[0012] Based on the position of the fork hole corresponding to the layer where the goods to be removed are located and the position of the fork-picking mobile robot, path planning is performed, and the fork teeth are controlled to move to the height of the fork hole corresponding to the layer where the goods to be removed are located;

[0013] The steering wheel of the fork-picking mobile robot is controlled to move according to the planned path so that the fork teeth are inserted into the fork holes corresponding to the layer where the goods to be removed are located, and the fork teeth are controlled to be lifted to carry the goods to be removed.

[0014] Optionally, before performing path planning based on the posture of the fork hole corresponding to the layer where the goods to be removed are located and the posture of the fork-picking mobile robot, and controlling the fork teeth to move to the height of the fork hole corresponding to the layer where the goods to be removed are located, the method further includes:

[0015] Based on the position of the fork hole corresponding to the layer where the goods to be removed are projected on the horizontal plane and the current position of the fork tine projected on the horizontal plane, determine the current distance between the fork tine and the fork hole corresponding to the layer where the goods to be removed are located in a specified horizontal direction to obtain the position deviation; wherein the specified horizontal direction is perpendicular to the direction of the fork tine;

[0016] When the deflection angle of the fork hole corresponding to the layer where the goods to be removed are located is greater than a preset angle, and / or the position deviation is greater than a preset distance, an alarm is issued;

[0017] Based on the posture of the fork hole corresponding to the layer where the goods to be removed are located and the posture of the fork-picking mobile robot, path planning is performed, and the fork teeth are controlled to move to the height of the fork hole corresponding to the layer where the goods to be removed are located, including:

[0018] When the deflection angle of the fork hole corresponding to the layer where the goods to be picked are located is not greater than the preset angle, and the position deviation is not greater than the preset distance, path planning is performed based on the posture of the fork hole corresponding to the layer where the goods to be picked are located and the posture of the fork mobile robot, and the fork teeth are controlled to move to the height of the fork hole corresponding to the layer where the goods to be picked are located.

[0019] Optionally, the lifting stop condition includes:

[0020] The fork tines are lifted to a maximum liftable height;

[0021] or,

[0022] The projection of the highest three-dimensional point currently detected by the laser radar on the horizontal plane is outside a specified range; wherein the specified range represents: the portion of the projection of the stack to be detected on the horizontal plane, close to one side of the fork mobile robot.

[0023] Optionally, during the process of the fork tine lifting, the laser radar is used to detect the direction of the stack to be detected until the fork tine stops lifting, and the point cloud data to be used of the stack to be detected is obtained, including:

[0024] During the lifting process of the fork tine, the laser radar detects the direction of the stack to be detected according to a preset detection cycle until the lifting condition of the fork tine is reached, and the point cloud data to be processed detected by the laser radar each time the preset detection cycle is reached;

[0025] The obtained multiple point cloud data to be processed are fused to obtain the point cloud data to be used of the stack to be detected.

[0026] Optionally, during the process of the fork tine lifting, the laser radar detects the direction of the stack to be detected according to a preset detection cycle until the fork tine stops lifting, and obtains the point cloud data to be processed detected by the laser radar each time the preset detection cycle is reached, including:

[0027] During the lifting process of the fork tine, the laser radar detects the direction of the stack to be detected according to a preset detection cycle until the lifting condition of the fork tine is reached, and the initial point cloud data detected by the laser radar each time the preset detection cycle is reached is obtained; wherein the initial point cloud data obtained by each detection includes: the position of each detected three-dimensional point projected on the horizontal plane;

[0028] For the initial point cloud data obtained in each detection, combined with the height of the fork tine during the detection, the height of each three-dimensional point detected in the detection is calculated;

[0029] The point cloud data to be processed of this detection is obtained by combining the position and height of the projection of each three-dimensional point detected this time on the horizontal plane.

[0030] Optionally, the laser radar is a three-dimensional laser radar; the height to which the fork tine is lifted within a preset detection cycle is not greater than the detection height of the laser radar during one detection.

[0031] A second aspect of the embodiments of the present application further provides a fork picking mobile robot, comprising: a laser radar and a processor arranged on the fork tine;

[0032] The laser radar is used to detect the front of the fork mobile robot during the lifting process of the fork tine;

[0033] The processor is used to execute any of the above-mentioned stacking detection methods.

[0034] According to a third aspect of the embodiments of the present application, a stacking detection device is provided, which is applied to a fork picking mobile robot, wherein a laser radar is provided on the fork tines of the fork picking mobile robot, and the device comprises:

[0035] a fork tine control module, configured to control the fork tine to be lifted upward from a first height when the fork picking mobile robot reaches a designated detection position near the stack to be detected; wherein, when the fork tine is at the first height, the fork hole corresponding to the bottom layer of goods in the stack to be detected is within the detection range of the laser radar;

[0036] A radar detection module, used to detect the direction of the stack to be detected by the laser radar during the lifting process of the fork teeth, until the lifting stop condition of the fork teeth is reached, and obtain the point cloud data to be used of the stack to be detected; wherein, when the lifting stop condition is reached, the range detected by the laser radar includes the fork hole corresponding to the highest layer of goods in the stack to be detected;

[0037] The posture determination module is used to determine the posture of the fork holes corresponding to each layer of goods in the stack to be detected based on the point cloud data to be used, and obtain the information to be used as a basis for removing each layer of goods in the stack to be detected.

[0038] Optionally, the position of the fork hole corresponding to any layer of goods includes: the height of the fork hole corresponding to the layer of goods, the position of the projection of the fork hole corresponding to the layer of goods on the horizontal plane, and the deflection angle of the fork hole corresponding to the layer of goods; wherein the deflection angle of the fork hole corresponding to the layer of goods represents: the angle between the projection of the surface of the stack to be detected to which the fork hole corresponding to the layer of goods belongs on the horizontal plane and the projection of the designated plane on the horizontal plane; the designated plane is perpendicular to the direction in which the fork teeth point; the device also includes:

[0039] a posture acquisition module, for determining the posture of the fork hole corresponding to each layer of the goods in the to-be-detected stack based on the to-be-utilized point cloud data, and obtaining the to-be-utilized information as a basis for removing the goods in each layer of the to-be-detected stack, and when receiving a removal instruction, determining the posture of the fork hole corresponding to the layer where the goods to be removed are located indicated by the removal instruction from the to-be-utilized information;

[0040] A path planning module, used for performing path planning based on the posture of the fork hole corresponding to the layer where the goods to be removed are located and the posture of the fork-picking mobile robot, and controlling the fork teeth to move to the height of the fork hole corresponding to the layer where the goods to be removed are located;

[0041] The mobile control module is used to control the steering wheel of the fork-picking mobile robot to move according to the planned path, so that the fork teeth are inserted into the fork holes corresponding to the layer where the goods to be removed are located, and control the lifting of the fork teeth to carry the goods to be removed.

[0042] Optionally, the device further comprises:

[0043] A position deviation determination module is used to perform path planning based on the position of the fork hole corresponding to the layer where the goods to be removed are located and the position of the fork-picking mobile robot, and before controlling the fork tine to move to the height of the fork hole corresponding to the layer where the goods to be removed are located, determine the distance between the fork tine and the fork hole corresponding to the layer where the goods to be removed are located in a specified horizontal direction based on the position of the fork hole corresponding to the layer where the goods to be removed are located on the horizontal plane and the current position of the fork tine projected on the horizontal plane, so as to obtain the position deviation; wherein the specified horizontal direction is perpendicular to the direction of the fork tine;

[0044] An alarm module, used for giving an alarm when the deflection angle of the fork hole corresponding to the layer where the goods to be removed are located is greater than a preset angle, and / or the position deviation is greater than a preset distance;

[0045] The path planning module is specifically used to perform path planning based on the posture of the fork hole corresponding to the layer where the goods to be picked up are located and the posture of the fork-picking mobile robot, when the deflection angle of the fork hole corresponding to the layer where the goods to be picked up are located is not greater than a preset angle, and the position deviation is not greater than the preset distance, and control the fork teeth to move to the height of the fork hole corresponding to the layer where the goods to be picked up are located.

[0046] Optionally, the lifting stop condition includes:

[0047] The fork tines are lifted to a maximum liftable height;

[0048] or,

[0049] The projection of the highest three-dimensional point currently detected by the laser radar on the horizontal plane is outside a specified range; wherein the specified range represents: the portion of the projection of the stack to be detected on the horizontal plane, close to one side of the fork mobile robot.

[0050] Optional, radar detection module, including:

[0051] A periodic detection submodule is used to detect the direction of the stack to be detected by the laser radar according to a preset detection cycle during the lifting process of the fork tine until the lifting stop condition of the fork tine is reached, and obtain the point cloud data to be processed detected by the laser radar each time the preset detection cycle is reached;

[0052] The point cloud data fusion submodule is used to fuse the multiple point cloud data to be processed to obtain the point cloud data to be used of the stack to be detected.

[0053] Optionally, a periodic detection submodule is specifically used to detect the direction of the stack to be detected by the laser radar according to a preset detection cycle during the process of the fork lifting, until the condition for stopping the lifting of the fork is reached, and obtain the initial point cloud data detected by the laser radar each time the preset detection cycle is reached; wherein the initial point cloud data obtained by each detection includes: the position of each detected three-dimensional point projected on the horizontal plane;

[0054] For the initial point cloud data obtained in each detection, combined with the height of the fork tine during the detection, the height of each three-dimensional point detected in the detection is calculated;

[0055] The point cloud data to be processed of this detection is obtained by combining the position and height of the projection of each three-dimensional point detected this time on the horizontal plane.

[0056] Optionally, the laser radar is a three-dimensional laser radar; the height to which the fork tine is lifted within a preset detection cycle is not greater than the detection height of the laser radar during one detection.

[0057] According to a fourth aspect of the embodiments of the present application, an electronic device is provided, including:

[0058] Memory, used to store computer programs;

[0059] The processor is used to implement any of the above-mentioned stacking detection methods when executing the program stored in the memory.

[0060] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any of the above-mentioned stacking detection methods is implemented.

[0061] An embodiment of the present application also provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute any of the above-mentioned stacking detection methods.

[0062] Beneficial effects of the embodiments of the present application:

[0063] A stack detection method provided in an embodiment of the present application is applied to a fork-picking mobile robot, wherein a laser radar is arranged on the fork tine of the fork-picking mobile robot, and when the fork-picking mobile robot reaches a designated detection position near the stack to be detected, the fork tine is controlled to lift upward from a first height; wherein, when the fork tine is at the first height, the fork hole corresponding to the bottom layer of goods in the stack to be detected is within the detection range of the laser radar; during the lifting process of the fork tine, the direction of the stack to be detected is detected by the laser radar until the lifting condition of the fork tine is reached, and the point cloud data to be used of the stack to be detected is obtained; wherein, when the lifting condition is reached, the range detected by the laser radar includes the fork hole corresponding to the top layer of goods in the stack to be detected; based on the point cloud data to be used, the posture of the fork hole corresponding to each layer of goods in the stack to be detected is determined, and the information to be used as the basis for removing each layer of goods in the stack to be detected is obtained.

[0064] Based on the above processing, the laser radar deployed on the fork tine of the fork picking mobile robot can detect the stack to be detected during the process of the fork tine being lifted upward from the first height. During the process of the fork tine being lifted, the detection range of the laser radar can include the fork holes corresponding to the highest layer of goods and the lowest layer of goods in the stack to be detected, so that the detected point cloud data to be used can include the posture information of all fork holes in the stack to be detected. And because the fork holes in the stack to be detected will not reflect the detection signal emitted by the laser radar, the laser radar will not receive the signal reflected by the fork holes. Accordingly, it can be determined that the part of the point cloud data to be used of the stack to be detected that does not contain three-dimensional points is the fork hole. Accordingly, according to the point cloud data to be used, the posture of the fork holes corresponding to each layer of goods in the stack to be detected can be determined to obtain the information to be used. That is, the laser radar set on the fork tine of the fork picking mobile robot can be used to obtain the posture information of the stack to be detected. Subsequently, the fork picking mobile robot can accurately remove the goods from the stack to be detected according to the information to be used.

[0065] Of course, implementing any product or method of the present application does not necessarily require achieving all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0067] Figure 1 A schematic diagram of a first process flow of a stacking detection method provided in an embodiment of the present application;

[0068] Figure 2 A top view of the positional relationship between a stack to be detected and a fork-picking mobile robot in the stack detection method provided in an embodiment of the present application;

[0069] Figure 3 A schematic diagram of point cloud data to be used in the stacking detection method provided in an embodiment of the present application;

[0070] Figure 4 In the stacking detection method provided in the embodiment of the present application Figure 3 A schematic diagram of a physical object corresponding to the point cloud data to be utilized is shown;

[0071] Figure 5 A schematic diagram of state transition of a stacking detection state machine in a stacking detection method provided in an embodiment of the present application;

[0072] Figure 6 A second schematic flow chart of the stacking detection method provided in an embodiment of the present application;

[0073] Figure 7 A third flow chart of the stacking detection method provided in the embodiment of the present application;

[0074] Figure 8 A schematic diagram of the structure of a stacking detection device provided in an embodiment of the present application;

[0075] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0076] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field based on the present application belong to the scope of protection of the present application.

[0077] In warehousing and logistics scenarios, goods are often placed in stacks. In order to improve the efficiency of sorting goods and save labor costs, FMR can be used to remove the goods that need to be transported from the stack and transport the removed goods to the target location. In order to ensure that the fork-picking mobile robot can accurately remove goods from the stack, it is often necessary to obtain the position information of the stack.

[0078] In order to obtain the position information of the stack to ensure that the fork-picking mobile robot can accurately remove the goods from the stack, the embodiment of the present application provides a stack detection method, see Figure 1 , Figure 1 A first flow chart of a stacking detection method provided in an embodiment of the present application is provided. The method is applied to a fork picking mobile robot. For example, the method can be applied to a processor of the fork picking mobile robot. A laser radar is provided on the fork tines of the fork picking mobile robot. The method includes:

[0079] Step S101: When the fork-picking mobile robot reaches a designated detection position near a stack to be detected, the fork tines are controlled to lift upward from a first height.

[0080] When the fork tine is at the first height, the fork hole corresponding to the bottom layer of the cargo in the stack to be detected is within the detection range of the laser radar.

[0081] Step S102: During the process of fork tine lifting, the laser radar is used to detect the direction of the stack to be detected until the fork tine stops lifting, thereby obtaining the point cloud data to be used of the stack to be detected.

[0082] Among them, when the lifting stop condition is reached, the range detected by the laser radar includes the fork hole corresponding to the highest layer of cargo in the stack to be detected.

[0083] Step S103: Based on the point cloud data to be utilized, the positions and postures of the fork holes corresponding to the layers of goods in the stack to be inspected are determined, and the information to be utilized as a basis for removing the layers of goods in the stack to be inspected is obtained.

[0084] Based on the above processing, the laser radar deployed on the fork tine of the fork picking mobile robot can detect the stack to be detected during the process of the fork tine being lifted upward from the first height. During the process of the fork tine being lifted, the detection range of the laser radar can include the fork holes corresponding to the highest layer of goods and the lowest layer of goods in the stack to be detected, so that the detected point cloud data to be used can include the posture information of all fork holes in the stack to be detected. And because the fork holes in the stack to be detected will not reflect the detection signal emitted by the laser radar, the laser radar will not receive the signal reflected by the fork holes. Accordingly, it can be determined that the part of the point cloud data to be used of the stack to be detected that does not contain three-dimensional points is the fork hole. Accordingly, according to the point cloud data to be used, the posture of the fork holes corresponding to each layer of goods in the stack to be detected can be determined to obtain the information to be used. That is, the laser radar set on the fork tine of the fork picking mobile robot can be used to obtain the posture information of the stack to be detected. Subsequently, the fork picking mobile robot can accurately remove the goods from the stack to be detected according to the information to be used.

[0085] For step S101, the stack for which the posture information is currently required can be referred to as the stack to be detected. The stack to be detected can include one or more layers of goods, and a loading platform (such as a pallet or a pallet) for carrying each layer of goods. The loading platform corresponding to each layer of goods has a fork hole, which can be referred to as the fork hole corresponding to the layer of goods. RCS (Robots Control Service) can control the fork-picking mobile robot to move to a specified detection position (also referred to as an identification point) near the stack to be detected.

[0086] For example, RCS can determine the stack to which the goods indicated by the currently received handling task belong (i.e., the stack to be detected), and can determine the front facade of the stack to be detected, and the front facade can be any vertical face of the stack to be detected. For example, if each layer of goods in the stack to be detected is a cube, then the stack to be detected can also be approximately a cube, and the front facade can be any one of the four vertical faces of the stack to be detected. And it can determine the position with a specified distance from the projection of the front facade of the stack to be detected on the horizontal plane as the designated detection position, such as the designated distance can be 0.5 meters or 1 meter. Correspondingly, when the fork-picking mobile robot reaches the designated detection position, the fork teeth can point to the front facade of the stack to be detected, so that the laser radar can detect the front facade of the stack to be detected. And RCS can determine the front facade of the stack to be detected in combination with the environment near the stack to be detected. For example, it can be determined whether there are obstacles near each vertical face of the stack to be detected, causing the fork-picking mobile robot to be unable to move to the vicinity of the vertical face. Accordingly, any vertical plane can be selected from the vertical planes where there are no obstacles that affect the movement of the fork-picking mobile robot as the front facade of the stack to be detected, so as to ensure that the fork-picking mobile robot can safely reach the designated detection position near the front facade of the stack to be detected.

[0087] The laser radar is arranged on the fork tine, and accordingly, the relative position of the laser radar and the fork tine remains unchanged. For example, the laser radar may be a 2D (two-dimensional) laser radar or a 3D (three-dimensional) laser radar. The first height may be set in advance according to the detection range of the laser radar and the height of the fork hole corresponding to the bottom layer of the goods in the stack to be detected (which may be referred to as the second height), so that when the fork tine is at the first height, the fork hole corresponding to the bottom layer of the goods in the stack to be detected is located within the detection range of the laser radar. For example, since the two-dimensional laser radar can only detect in the horizontal dimension, when the laser radar is a two-dimensional laser radar, the first height is not greater than the second height. Since the three-dimensional laser radar can detect in both horizontal and vertical dimensions, that is, the three-dimensional laser radar has a detection height of a preset size in the vertical direction each time it detects, therefore, when the laser radar is a three-dimensional laser radar, the first height may not be greater than the second height, or the first height may also be greater than the second height, as long as it is ensured that when the fork tine is at the first height, the second height is within the detection range of the laser radar.

[0088] And because the fork hole in the stack to be detected does not reflect the detection signal emitted by the laser radar, the laser radar will not receive the signal reflected by the fork hole. Accordingly, it can be determined that the part of the point cloud data to be used of the stack to be detected detected by the laser radar that does not contain three-dimensional points is the fork hole. Accordingly, based on the point cloud data to be used detected by the laser radar during the process of the fork tine being lifted upward from the first height, the position of the fork hole corresponding to the bottom layer of the goods can be determined, which can ensure that the point cloud data to be used has a high integrity.

[0089] The first height may also be referred to as a preset low position. For example, the first height may be the lowest height to which the tines can be lowered. If the tines are not at the first height when the designated detection position is reached, the fork picking mobile robot may control the tines to move to the first height, and then control the tines to lift upward from the first height. If the tines are at the first height when the designated detection position is reached, the fork picking mobile robot may directly control the tines to lift upward from the first height.

[0090] With respect to step S102, during the process of the fork tine being lifted upward from the first height, the laser radar can move along with the fork tine and can detect the direction of the stack to be detected. The detection direction of the laser radar is the direction in which the fork tine is directed, for example, the direction in which the fork tine is directed toward the front elevation of the stack to be detected. For example, the direction in which the fork tine is directed may be perpendicular to the projection of the front elevation of the stack to be detected on the horizontal plane. It is understandable that, since there may be deviations in the movement of the fork picking mobile robot, there may also be certain deviations between the direction in which the fork tine is directed and the projection of the front elevation of the stack to be detected on the horizontal plane, and they are not completely perpendicular.

[0091] When the condition for stopping the lifting of the fork tine is reached, the fork-picking mobile robot can stop the laser radar detection and stop the lifting of the fork tine to obtain the point cloud data to be used for the stack to be detected. The specific method of obtaining the point cloud data to be used through laser radar detection can refer to the relevant description of steps 1 and 2 in the subsequent embodiments. When the condition for stopping the lifting is reached, the range detected by the laser radar includes the fork holes corresponding to the highest layer of goods in the stack to be detected. This means that in this case, the position and posture of the fork holes corresponding to the goods in the bottom layer, the fork holes corresponding to the goods in the highest layer, and the fork holes corresponding to the goods between the bottom layer and the highest layer can be determined based on the point cloud data to be used, that is, the position and posture of the fork holes corresponding to all layers of goods in the stack to be detected can be determined based on the point cloud data to be used.

[0092] For example, the conditions for stopping lifting may include: the fork tines are lifted to the maximum lifting height, in which case the forks cannot be lifted any further. And in order to ensure that the fork-picking mobile robot can fork the goods in the stacks in the warehouse, when each stack in the warehouse is placed, the height of the fork hole of the highest layer of goods is not greater than the maximum lifting height of the fork tines. For example, each stack in the warehouse can be placed by the fork-picking mobile robot, so the height of the fork hole of the highest layer of goods in each stack is not greater than the maximum lifting height of the fork tines. Accordingly, when the fork tines are lifted to the maximum lifting height, the range detected by the laser radar can include the fork hole corresponding to the highest layer of goods in the stack to be detected.

[0093] Alternatively, the condition for stopping lifting may also include: the projection of the highest three-dimensional point currently detected by the laser radar on the horizontal plane is outside the specified range. The specified range represents: the part of the projection of the stack to be detected on the horizontal plane, close to the side of the fork-picking mobile robot. In this case, the highest three-dimensional point currently detected by the laser radar no longer belongs to the stack to be detected, which means that the range detected by the laser radar includes the highest point of the stack to be detected, that is, in order to complete the detection of the stack to be detected, the detected range may also include the fork hole corresponding to the highest layer of cargo in the stack to be detected, and there is no need to continue to lift the fork teeth upward so that the laser radar can continue to detect higher positions. For example, the specified range may represent: the projection of the stack to be detected on the horizontal plane, close to the half of the fork-picking mobile robot, or the specified range may also represent: the projection of the stack to be detected on the horizontal plane. For example Figure 2 As shown, Figure 2 A top view of the positional relationship between the stack to be detected and the fork-picking mobile robot in the stack detection method provided in the embodiment of the present application. The projection of the part filled with oblique lines in the stack to be detected 21 on the horizontal plane represents the specified range. The fork-picking mobile robot 22 includes: fork teeth 221 and a main body 222.

[0094] If the height of the stack to be detected is low, when the fork tines are not lifted to the maximum liftable height, the range detected by the laser radar includes the highest point of the stack to be detected, that is, the projection of the highest three-dimensional point currently detected by the laser radar on the horizontal plane is already outside the specified range, and the condition for stopping the lifting of the fork tines is met. If the height of the stack to be detected is high, when the projection of the highest three-dimensional point currently detected by the laser radar on the horizontal plane is still within the specified range, the fork tines have been lifted to the maximum liftable height, and the condition for stopping the lifting of the fork tines is met.

[0095] With respect to step S103, the detected range of the laser radar may also include the environment near the stack to be detected, that is, the detection result of the laser radar includes point cloud data representing the environment near the stack to be detected, and the fork-picking mobile robot may also determine the point cloud data to be used of the stack to be detected from the detection result of the laser radar. For example, the alternative range of the stack to be detected may be set in advance according to the position of the stack to be detected. For example, the neighborhood range of the projection of the stack to be detected on the horizontal plane may be set as the alternative range. Accordingly, the three-dimensional points whose projection on the horizontal plane belongs to the alternative range may be determined from the detection result of the laser radar to obtain the point cloud data to be used of the stack to be detected.

[0096] Since the fork hole in the stack to be detected does not reflect the detection signal emitted by the laser radar, the laser radar will not receive the signal reflected by the fork hole. Accordingly, it can be determined that the part of the point cloud data to be used of the stack to be detected obtained by the laser radar that does not contain three-dimensional points is the fork hole. And in the process of fork lifting, the detection range of the laser radar may include the fork holes corresponding to the highest layer of goods and the lowest layer of goods in the stack to be detected. Accordingly, according to the point cloud data to be used, the position and posture of the fork holes corresponding to each layer of goods in the stack to be detected can be determined to obtain the information to be used. When the range detected by the laser radar includes the highest point of the stack to be detected, the point cloud data to be used can include the point cloud data of all goods in the stack to be detected. Accordingly, according to the point cloud data to be used, the total height of each layer of goods in the stack to be detected and the number of goods in each layer of the stack to be detected can also be determined. That is, the information to be used can also include the total height of the goods in that layer and the number of goods in that layer.

[0097] The information to be used can be used as a basis for removing the goods from each layer of the stack to be inspected. Later, when it is necessary to remove the goods in the stack to be inspected (which can be called the goods to be removed), the position of the fork hole corresponding to the layer where the goods to be removed are located can be determined according to the information to be used, and the path planning can be performed in combination with the position of the fork picking mobile robot to control the fork picking mobile robot to move and carry the goods to be removed. In this way, it can be ensured that the fork picking mobile robot can accurately remove the goods from the stack.

[0098] In one embodiment, the laser radar can detect the direction of the stack to be detected multiple times according to a preset detection cycle, and the fork-picking mobile robot can obtain the point cloud data to be used based on the point cloud data (i.e., the point cloud data to be processed) detected each time the preset detection cycle is reached. Step S102 includes:

[0099] Step 1: During the lifting process of the fork tine, the laser radar detects the direction of the stack to be detected according to a preset detection cycle until the lifting condition of the fork tine is reached, and the point cloud data to be processed detected by the laser radar each time the preset detection cycle is reached;

[0100] Step 2: Fuse the multiple point cloud data to be processed to obtain the point cloud data to be used for the stack to be detected.

[0101] In an embodiment of the present application, during the process of fork tine lifting, the laser radar can perform multiple detections on the direction of the stack to be detected according to a preset detection cycle. The cycle duration of the preset detection cycle can be set as needed and is not specifically limited. For example, the cycle duration of the preset detection cycle can be 0.1 seconds or 0.2 seconds. Taking the cycle duration of the preset detection cycle as 0.1 seconds as an example, that is, the scanning frequency of the laser radar is 10 Hz (Hertz), which means that the laser radar performs 10 detections per second. The point cloud data to be processed detected by the laser radar each time the preset detection cycle is reached only includes the local three-dimensional points of the stack to be detected. Therefore, it is necessary to fuse the multiple point cloud data to be processed obtained by multiple detections to obtain the point cloud data to be used of the stack to be detected.

[0102] For example, multiple point cloud data to be processed obtained by multiple detections can be spliced ​​in the order of detection time, and the three-dimensional points representing the same position in each two adjacent detection times of the point cloud data to be processed can be fused to obtain the three-dimensional points representing the position in the point cloud data to be used. For example, the preset DLR (Deep Learning Recognition, deep learning algorithm) library can be used to fuse the point cloud data to be processed detected multiple times to obtain the point cloud data to be used. Before controlling the fork teeth (also known as the actuator) to lift upward from the first height, the fork-picking mobile robot can set the recognition parameters of the DLR library. For example, the recognition parameters can include the specified range in the above embodiment, and accordingly, the DLR library can identify whether the projection of the highest three-dimensional point currently detected by the laser radar on the horizontal plane is outside the specified range; the recognition parameters can also include the alternative range in the above embodiment, and accordingly, the DLR library can determine the three-dimensional points belonging to the stack to be detected from the detection results of the laser radar in combination with the alternative range to obtain the point cloud data to be used. The fork-picking mobile robot may be provided with a data acquisition thread (also referred to as a detection result acquisition thread), and the data acquisition thread may poll the DLR library to obtain the point cloud data to be used. When the point cloud data to be used is obtained, the data acquisition thread may call the control mechanism inside the fork-picking mobile robot to stop the detection of the laser radar and exit the thread.

[0103] Based on the above processing, during the process of fork lifting, the laser radar can detect the direction of the stack to be detected multiple times according to the preset detection cycle, and can obtain the point cloud data to be used based on the point cloud data to be processed obtained from the multiple detections. In this way, the integrity of the point cloud data to be used can be further guaranteed, and accordingly, the integrity of the information to be used obtained later based on the point cloud data to be used can be guaranteed, so as to ensure that the position information of the stack to be detected can be obtained. Furthermore, it is guaranteed that the fork-picking mobile robot can be controlled to accurately remove the goods from the stack based on the information to be used.

[0104] In one embodiment, the laser radar is a three-dimensional laser radar; the height to which the fork tines are lifted within a preset detection cycle is no greater than the detection height of the laser radar during one detection.

[0105] In the embodiment of the present application, the laser radar may be a three-dimensional laser radar, and the laser radar may have a detection height of a preset size each time it detects, that is, the laser radar has a detection range of a preset size in the vertical direction each time it detects. When the height of the fork tine lifted in a preset detection cycle is greater than the detection height of the laser radar during one detection, the lifting speed of the fork tine is too fast, and there is a gap between the detection ranges of two adjacent detections of the laser radar, that is, the point cloud data obtained from the two adjacent detections respectively represent non-adjacent parts of the stack to be detected, resulting in the presence of undetected parts in the stack to be detected, which will also result in incomplete point cloud data to be used.

[0106] In order to avoid this situation and ensure the integrity of the point cloud data to be used, the lifting speed of the fork tine can be preset so that the height of the fork tine lifted within a preset detection cycle is not greater than the detection height of the laser radar during a single detection. For example, the ratio of the detection height of each laser radar detection to the duration of the preset detection cycle can be calculated, and the lifting speed of the fork tine can be set not greater than the calculated ratio. Accordingly, it can be ensured that the posture information of the stack to be detected can be obtained, and further ensure that the subsequent fork-picking mobile robot can accurately remove the goods from the stack according to the posture information of the stack.

[0107] In one embodiment, step 1 includes:

[0108] Step 11: During the process of fork lifting, the laser radar detects the direction of the stack to be detected according to a preset detection cycle until the fork stops lifting, and obtains the initial point cloud data detected by the laser radar each time the preset detection cycle is reached.

[0109] The initial point cloud data obtained in each detection includes: the position of each detected three-dimensional point projected on the horizontal plane.

[0110] Step 12: For the initial point cloud data obtained from each detection, combined with the height of the fork tines during the detection, calculate the height of each three-dimensional point detected during the detection.

[0111] Step 13: Combine the position and height of the projection of each three-dimensional point detected this time on the horizontal plane to obtain the point cloud data to be processed for this detection.

[0112] In the embodiment of the present application, the initial point cloud data detected by the laser radar each time when the preset detection cycle is reached can represent: the distance and angle of each three-dimensional point detected this time relative to the laser radar, and the relative position of each three-dimensional point detected this time and the laser radar can be obtained. Correspondingly, according to the relative position of each three-dimensional point detected this time and the laser radar, and the position of the laser radar projected on the horizontal plane, the position of each three-dimensional point detected this time projected on the horizontal plane can be obtained. And because the relative position of the fork tine and the laser radar is fixed, therefore, for the initial point cloud data obtained each time, the relative position of each three-dimensional point detected this time and the fork tine can be determined according to the relative position of the fork tine and the laser radar, and the relative position of each three-dimensional point detected this time and the laser radar. Correspondingly, according to the height of the fork tine during this detection, and the relative position of each three-dimensional point detected this time and the fork tine, the height of each three-dimensional point detected this time can also be calculated.

[0113] Furthermore, the point cloud data to be processed of this detection can be obtained by combining the position and height of the projection of each three-dimensional point detected this time on the horizontal plane. In this way, the point cloud data to be processed can include: the position and height of the projection of each three-dimensional point detected on the horizontal plane. Correspondingly, multiple point cloud data to be processed are fused and processed, and the point cloud data to be used of the stack to be detected can include: the position and height of the projection of the three-dimensional point belonging to the stack to be detected on the horizontal plane. The information to be used determined based on the point cloud data to be used can also include the height information of each layer of goods, for example, it can include the height of the fork hole corresponding to any layer of goods (also known as the fork-accessible height of the layer of goods) and the height of any layer of goods. That is, it can further ensure that the posture information of the stack to be detected can be obtained, and further ensure that the subsequent fork-picking mobile robot can accurately remove the goods from the stack according to the posture information of the stack.

[0114] In one embodiment, see Figure 3 , Figure 3 A schematic diagram of point cloud data to be used in the stack detection method provided in the embodiment of the present application. The stack to be detected has 3 layers of goods, 31 represents the point cloud data of the front facade of the stack to be detected, 32 represents the fork hole in the front facade of the stack to be detected, that is, the part of 31 where there are no three-dimensional points. There are two fork holes corresponding to each layer of goods in the stack to be detected. 33 represents the point cloud data of the upper surface of the stack to be detected, and 34 represents the partial point cloud data in the vertical surface adjacent to the front facade of the stack to be detected. Figure 4 In the stacking detection method provided in the embodiment of the present application Figure 3The physical object schematic diagram corresponding to the point cloud data to be used is shown. There are 3 layers of goods in the stack to be detected, and a loading platform is set at the bottom of each layer of goods. For example, a loading platform 42 is set at the bottom of the top layer of goods 41. Two fork holes are set on either side of the loading platform corresponding to each layer of goods. Each layer of goods can include one cargo, or it can also be assembled from multiple cargoes.

[0115] In one embodiment, see Figure 5 , Figure 5 A schematic diagram of the state transition of the stacking detection state machine in the stacking detection method provided in the embodiment of the present application. The state machine has a total of six states, namely, idle state, inspection state, scan to low-level operation state, scan to high-level operation state, completion state and abnormal state. When the fork-picking mobile robot arrives at the designated detection position, the start recognition signal is triggered. The state machine can monitor the start recognition signal in real time. After receiving the start recognition signal, the state machine first enters the inspection state to check and verify the actuator state, laser radar state, recognition parameters, etc. to determine whether the actuator and laser radar can operate normally. The recognition parameters may include the recognition parameters of the DLR library in the above-mentioned embodiment, and may also include the lifting speed of the fork teeth (i.e., the actuator) and the detection height and preset detection cycle of the laser radar for one detection.

[0116] After the verification is passed, the state machine enters the scanning to low position operation state. When scanning to the low position operation state, the fork picking mobile robot will control the fork tine to move to the preset low position (i.e., the first height) and clear the historical recognition results in the DLR library. In the process of adjusting the fork tine to the preset low position, it is possible to monitor whether the adjustment is completed, that is, to determine whether the fork tine has reached the preset low position. After the adjustment is completed, the state machine enters the scanning to high position operation state, the DLR library starts recognition, and completes the recognition parameter setting of the DLR library. Furthermore, the lifting mechanism is controlled to the preset high position, and the lifting mechanism can control the fork tine to lift upward, that is, step S101 and step S102 in the above embodiment. The state machine can monitor whether the adjustment is completed and whether the recognition result has been obtained, that is, to determine whether the stop lifting condition of the fork tine has been met, and obtain the point cloud data to be used.

[0117] When the lifting mechanism has been adjusted, a flag is set to force the accumulated results to be obtained from the DLR library, that is, when the fork tine is lifted to the maximum lifting height, the point cloud data to be used can be obtained from the DLR library; or, if the projection of the highest three-dimensional point currently detected by the laser radar on the horizontal plane is outside the specified range, the point cloud data to be used can also be obtained from the DLR library. When the recognition result (i.e., the point cloud data to be used) is obtained from the DLR library, it means that the scanning and recognition of the laser radar is completed, and the lifting task of the current lifting mechanism will be canceled, that is, the lifting of the fork tine is stopped, and the laser radar detection can be stopped, and the state machine switches to the completion state. In the completion state, the state machine can perform operation parameter verification, for example, the operation parameters can include the current state of the fork-picking mobile robot, and the state machine can also output a completion signal to enter the idle state. In the idle state, the operation parameters can be initialized to restore the operation parameters to the original state, and then the state machine stop flag can be set to mark the completion of the detection task of the stack to be detected and exit the state machine. If an abnormality occurs during the operation of the state machine, for example, the fork tines of the fork picking mobile robot cannot move or the laser radar fails, the state machine enters the abnormal state and can output an abnormal signal. After the fault is eliminated, the state machine can enter the idle state from the abnormal state.

[0118] In one embodiment, after obtaining the information to be used, when it is necessary to remove the goods in the stack to be detected (which can be called the goods to be removed), the position of the fork hole corresponding to the layer where the goods to be removed are located can be determined according to the information to be used, so as to perform path planning and control the fork picking mobile robot to move according to the planned path and carry the goods to be removed. Figure 6 , Figure 6 A second flow chart of the stacking detection method provided in the embodiment of the present application. The position of the fork hole corresponding to any layer of goods includes: the height of the fork hole corresponding to the layer of goods, the position of the projection of the fork hole corresponding to the layer of goods on the horizontal plane, and the deflection angle of the fork hole corresponding to the layer of goods; wherein the deflection angle of the fork hole corresponding to the layer of goods represents: the angle between the projection of the surface of the stack to be detected to which the fork hole corresponding to the layer of goods belongs on the horizontal plane and the projection of the specified plane on the horizontal plane; the specified plane is perpendicular to the direction in which the fork teeth point. After step S103, the method further includes:

[0119] Step S104: When a disassembly instruction is received, the position and posture of the fork hole corresponding to the layer where the goods to be disassembled indicated by the disassembly instruction are located is determined from the information to be used.

[0120] Step S105: Based on the posture of the fork hole corresponding to the layer where the goods to be removed are located and the posture of the fork-picking mobile robot, path planning is performed, and the fork teeth are controlled to move to the height of the fork hole corresponding to the layer where the goods to be removed are located.

[0121] Step S106: Control the steering wheel of the fork-picking mobile robot to move according to the planned path so that the fork tines are inserted into the fork holes corresponding to the layer where the goods to be removed are located, and control the fork tines to be lifted to transport the goods to be removed.

[0122] In an embodiment of the present application, when it is necessary to remove the goods to be removed, the RCS sends a removal instruction to the fork picking mobile robot. When receiving the removal instruction, the fork picking mobile robot can determine the position and posture of the fork hole corresponding to the layer where the goods to be removed are located indicated by the removal instruction from the information to be used. For example, the removal instruction can carry information indicating the layer where the goods to be removed are located (such as the identifier of the layer where the goods to be removed are located), and accordingly, the fork picking mobile robot can determine the position and posture of the fork hole corresponding to the layer where the goods to be removed are located from the information to be used; or, the removal instruction can also carry the height of the goods to be removed, and accordingly, the fork picking mobile robot can determine the layer where the goods to be removed are located based on the height of the fork holes corresponding to each layer of goods recorded in the information to be used and the height of the goods to be removed, and then, can determine the position and posture of the fork hole corresponding to the layer where the goods to be removed are located.

[0123] The posture of the fork hole corresponding to any layer of goods may include: the height of the fork hole corresponding to the layer of goods, the position of the projection of the fork hole corresponding to the layer of goods on the horizontal plane, and the deflection angle of the fork hole corresponding to the layer of goods. Accordingly, the fork picking mobile robot can perform path planning according to the preset path planning algorithm and the pre-set depth of the fork tine inserted into the fork hole (which can be called the preset depth), combined with the position of the projection of the fork hole corresponding to the layer where the goods to be removed are located on the horizontal plane, the deflection angle of the fork hole corresponding to the layer where the goods to be removed are located, and the posture of the fork picking mobile robot. The preset path planning algorithm can also be called DAP (Decision And Planning, decision algorithm), such as Dijkstra algorithm (a path planning algorithm), or genetic algorithm. And the fork tine can be controlled to move to the height of the fork hole corresponding to the layer where the goods to be removed are located. For example, any height can be selected in the height range with the height of the center position of the fork hole corresponding to the layer where the goods to be removed are located and the height of the highest position of the fork hole corresponding to the layer where the goods to be removed are located as the endpoints, as the height of the fork hole corresponding to the layer where the goods to be removed are located. It can be ensured that the fork tine with a certain thickness can be safely inserted into the fork hole corresponding to the layer where the goods to be removed are located.

[0124] After the planned path is obtained and the fork tine moves to the height of the fork hole corresponding to the layer where the goods to be removed are located, the fork-picking mobile robot can control its own steering wheel to move according to the planned path so that the fork tine is inserted into the fork hole corresponding to the layer where the goods to be removed are located, and after the fork tine is inserted, that is, the depth of the fork tine inserted into the fork hole reaches the preset depth, the fork tine is controlled to be lifted to carry the goods to be removed. For example, the fork-picking mobile robot can control its own steering wheel to move through CC (Chassis Control, chassis control algorithm).

[0125] Based on the above processing, when it is necessary to remove the goods in the stack to be detected (which can be called the goods to be removed), the fork-picking mobile robot can determine the position and posture of the fork hole corresponding to the layer where the goods to be removed are located according to the information to be used, and perform path planning in combination with the position and posture of the fork-picking mobile robot to control the fork-picking mobile robot to move and carry the goods to be removed. That is, it can be ensured that the fork-picking mobile robot can accurately remove the goods from the stack.

[0126] In one embodiment, see Figure 7 , Figure 7 The third flow chart of the stacking detection method provided in the embodiment of the present application. Before step S105, the method further includes:

[0127] Step S107: Based on the position of the fork hole corresponding to the layer where the goods to be removed are located, and the current position of the fork tine projected on the horizontal plane, determine the current distance between the fork tine and the fork hole corresponding to the layer where the goods to be removed are located in the specified horizontal direction to obtain the position deviation.

[0128] The specified horizontal direction is perpendicular to the direction of the fork tines.

[0129] Step S108: When the deflection angle of the fork hole corresponding to the layer where the goods to be removed are located is greater than a preset angle, and / or the position deviation is greater than a preset distance, an alarm is issued.

[0130] Step S105 includes:

[0131] Step S1051: When the deflection angle of the fork hole corresponding to the layer where the goods to be picked are located is not greater than the preset angle, and the position deviation is not greater than the preset distance, path planning is performed based on the posture of the fork hole corresponding to the layer where the goods to be picked are located and the posture of the fork mobile robot, and the fork teeth are controlled to move to the height of the fork hole corresponding to the layer where the goods to be picked are located.

[0132] In the embodiment of the present application, the position of the fork hole corresponding to the layer where the goods to be removed are projected on the horizontal plane may be: the position of the center position of the fork hole corresponding to the layer where the goods to be removed are projected on the horizontal plane, such as Figure 3The position of the center position of the middle fork hole 32 projected on the horizontal plane. The current position of the fork tine projected on the horizontal plane can be: the current position of the center position of the front end of the fork tine projected on the horizontal plane, the front end of the fork tine is the end of the fork tine close to the stack to be detected, such as Figure 2 The upper ends of the tines 221 are shown.

[0133] The horizontal direction perpendicular to the direction of the fork tine can be called the designated horizontal direction. According to the position of the projection of the fork hole corresponding to the layer where the goods to be removed are located on the horizontal plane, and the current projection of the fork tine on the horizontal plane, the fork picking mobile robot can determine the current distance between the fork tine and the fork hole corresponding to the layer where the goods to be removed are located in the designated horizontal direction, and obtain the position deviation. For example, there are two fork holes corresponding to the layer where the goods to be removed are located, and the fork picking mobile robot also has two fork tines. The fork picking mobile robot can determine the position deviation according to the position of the projection of any fork hole corresponding to the layer where the goods to be removed are located on the horizontal plane, and the current projection of the fork tine on the same side as the fork hole on the horizontal plane.

[0134] When the position deviation is greater than the preset distance, it means that the current fork tine is far away from the fork hole corresponding to the layer where the goods to be removed are located in the specified horizontal direction. For example, the preset distance can be 20 cm or 25 cm. In this case, it is impossible to plan an effective safe path, that is, when the fork-picking mobile robot controls the steering wheel to move according to the planned path, it will collide with objects in the warehouse.

[0135] The posture of the goods to be removed and the posture of the fork picking mobile robot when it reaches the designated detection position will affect the deflection angle of the fork hole corresponding to the layer where the goods to be removed are located. When the deflection angle of the fork hole corresponding to the layer where the goods to be removed are located is greater than the preset angle, it means that the projection of the surface of the stack to be detected to which the fork hole corresponding to the layer where the goods to be removed belong on the horizontal plane and the projection of the designated plane on the horizontal plane are large. For example, the preset angle can be 15° or 20°. In this case, it is impossible to plan an effective safe path.

[0136] Therefore, in order to avoid the collision between the fork picking mobile robot and the objects in the warehouse during the process of forking the goods to be removed, an alarm can be issued when the deflection angle of the fork hole corresponding to the layer where the goods to be removed are located is greater than the preset angle, and / or the position deviation is greater than the preset distance. Correspondingly, the management personnel can adjust the placement of the goods to be removed, or manually control the fork picking mobile robot to remove the goods to be removed. When the deflection angle of the fork hole corresponding to the layer where the goods to be removed are located is not greater than the preset angle, and the position deviation is not greater than the preset distance, an effective safe path can be planned, and when the fork picking mobile robot controls the steering wheel to move along the planned path, the probability of collision with objects in the warehouse is low. In this case, the fork picking mobile robot can perform path planning, and subsequently, the steering wheel can be controlled to move along the planned path.

[0137] Based on the above processing, it can be determined whether an effective safe path can be planned according to the position deviation and the size of the deflection angle of the fork hole corresponding to the layer where the goods to be removed are located. If it is determined that an effective safe path can be planned, the fork-picking mobile robot can perform path planning. If it is determined that an effective safe path cannot be planned, an alarm can be issued. In this way, the safety of forking the goods to be removed can be improved, and it can be further ensured that the fork-picking mobile robot can accurately remove the goods from the stack. And before the fork-picking mobile robot performs path planning, it can be determined whether an effective safe path can be planned. If an effective safe path cannot be planned, there is no need to perform path planning, which can avoid invalid path planning and save computing resources.

[0138] Based on the same inventive concept, the embodiment of the present application also provides a fork picking mobile robot, comprising: a laser radar and a processor arranged on the fork tine;

[0139] Laser radar, used to detect in front of the fork picking mobile robot during the process of fork tine lifting;

[0140] A processor is used to execute any of the above-mentioned stacking detection methods.

[0141] Based on the same inventive concept, the present application also provides a stacking detection device, see Figure 8 , Figure 8 A schematic diagram of the structure of a stacking detection device provided in an embodiment of the present application, which is applied to a fork-picking mobile robot, wherein a laser radar is provided on the fork tines of the fork-picking mobile robot, and the device comprises:

[0142] The fork tine control module 801 is used to control the fork tine to be lifted upward from a first height when the fork picking mobile robot reaches a designated detection position near the stack to be detected; wherein, when the fork tine is at the first height, the fork hole corresponding to the bottom layer of the goods in the stack to be detected is within the detection range of the laser radar;

[0143] The radar detection module 802 is used to detect the direction of the stack to be detected by the laser radar during the lifting process of the fork teeth until the lifting stop condition of the fork teeth is reached, so as to obtain the point cloud data to be used of the stack to be detected; wherein, when the lifting stop condition is reached, the range detected by the laser radar includes the fork hole corresponding to the highest layer of goods in the stack to be detected;

[0144] The posture determination module 803 is used to determine the posture of the fork holes corresponding to each layer of goods in the stack to be detected based on the point cloud data to be used, and obtain the information to be used as a basis for removing each layer of goods in the stack to be detected.

[0145] Based on the stack detection device provided by the embodiment of the present application, the laser radar deployed on the fork tine of the fork picking mobile robot can detect the stack to be detected during the process of the fork tine being lifted upward from the first height. During the process of the fork tine being lifted, the detection range of the laser radar can include the fork holes corresponding to the highest layer of goods and the lowest layer of goods in the stack to be detected, so that the detected point cloud data to be used can include the posture information of all fork holes in the stack to be detected. And because the fork holes in the stack to be detected will not reflect the detection signal emitted by the laser radar, the laser radar will not receive the signal reflected by the fork holes. Accordingly, it can be determined that the part of the point cloud data to be used of the stack to be detected that does not contain three-dimensional points is the fork hole. Accordingly, according to the point cloud data to be used, the posture of the fork holes corresponding to each layer of goods in the stack to be detected can be determined to obtain the information to be used. That is, the laser radar set on the fork tine of the fork picking mobile robot can be used to obtain the posture information of the stack to be detected. Subsequently, the fork picking mobile robot can accurately remove the goods from the stack to be detected according to the information to be used.

[0146] In one embodiment, the position of the fork hole corresponding to any layer of goods includes: the height of the fork hole corresponding to the layer of goods, the position of the projection of the fork hole corresponding to the layer of goods on the horizontal plane, and the deflection angle of the fork hole corresponding to the layer of goods; wherein the deflection angle of the fork hole corresponding to the layer of goods represents: the angle between the projection of the surface of the stack to be detected to which the fork hole corresponding to the layer of goods belongs on the horizontal plane and the projection of the designated plane on the horizontal plane; the designated plane is perpendicular to the direction in which the fork teeth point; the device also includes:

[0147] a posture acquisition module, for determining the posture of the fork hole corresponding to each layer of the goods in the to-be-detected stack based on the to-be-utilized point cloud data, and obtaining the to-be-utilized information as a basis for removing the goods in each layer of the to-be-detected stack, and when receiving a removal instruction, determining the posture of the fork hole corresponding to the layer where the goods to be removed are located indicated by the removal instruction from the to-be-utilized information;

[0148] A path planning module, used for performing path planning based on the posture of the fork hole corresponding to the layer where the goods to be removed are located and the posture of the fork-picking mobile robot, and controlling the fork teeth to move to the height of the fork hole corresponding to the layer where the goods to be removed are located;

[0149] The mobile control module is used to control the steering wheel of the fork-picking mobile robot to move according to the planned path, so that the fork teeth are inserted into the fork holes corresponding to the layer where the goods to be removed are located, and control the lifting of the fork teeth to carry the goods to be removed.

[0150] In one embodiment, the apparatus further comprises:

[0151] A position deviation determination module is used to perform path planning based on the position of the fork hole corresponding to the layer where the goods to be removed are located and the position of the fork-picking mobile robot, and before controlling the fork tine to move to the height of the fork hole corresponding to the layer where the goods to be removed are located, determine the distance between the fork tine and the fork hole corresponding to the layer where the goods to be removed are located in a specified horizontal direction based on the position of the fork hole corresponding to the layer where the goods to be removed are located on the horizontal plane and the current position of the fork tine projected on the horizontal plane, so as to obtain the position deviation; wherein the specified horizontal direction is perpendicular to the direction of the fork tine;

[0152] An alarm module, used for giving an alarm when the deflection angle of the fork hole corresponding to the layer where the goods to be removed are located is greater than a preset angle, and / or the position deviation is greater than a preset distance;

[0153] The path planning module is specifically used to perform path planning based on the posture of the fork hole corresponding to the layer where the goods to be picked up are located and the posture of the fork-picking mobile robot, when the deflection angle of the fork hole corresponding to the layer where the goods to be picked up are located is not greater than a preset angle, and the position deviation is not greater than the preset distance, and control the fork teeth to move to the height of the fork hole corresponding to the layer where the goods to be picked up are located.

[0154] In one embodiment, the lifting stop condition includes:

[0155] The fork tines are lifted to a maximum liftable height;

[0156] or,

[0157] The projection of the highest three-dimensional point currently detected by the laser radar on the horizontal plane is outside a specified range; wherein the specified range represents: the portion of the projection of the stack to be detected on the horizontal plane, close to one side of the fork mobile robot.

[0158] In one embodiment, the radar detection module 802 includes:

[0159] A periodic detection submodule is used to detect the direction of the stack to be detected by the laser radar according to a preset detection cycle during the lifting process of the fork tine until the lifting stop condition of the fork tine is reached, and obtain the point cloud data to be processed detected by the laser radar each time the preset detection cycle is reached;

[0160] The point cloud data fusion submodule is used to fuse the multiple point cloud data to be processed to obtain the point cloud data to be used of the stack to be detected.

[0161] In one embodiment, the periodic detection submodule is specifically used to detect the direction of the stack to be detected by the laser radar according to a preset detection cycle during the lifting process of the fork tine until the lifting condition of the fork tine is reached, and obtain the initial point cloud data detected by the laser radar each time the preset detection cycle is reached; wherein the initial point cloud data obtained by each detection includes: the position of each detected three-dimensional point projected on the horizontal plane;

[0162] For the initial point cloud data obtained in each detection, combined with the height of the fork tine during the detection, the height of each three-dimensional point detected in the detection is calculated;

[0163] The point cloud data to be processed of this detection is obtained by combining the position and height of the projection of each three-dimensional point detected this time on the horizontal plane.

[0164] In one embodiment, the laser radar is a three-dimensional laser radar; the height to which the fork tine is lifted within a preset detection cycle is not greater than the detection height of the laser radar during a single detection.

[0165] The present application also provides an electronic device, such as Fig. 9 As shown, including:

[0166] Memory 901, used for storing computer programs;

[0167] The processor 902 is used to execute the program stored in the memory 901, and implements the following steps:

[0168] When the fork picking mobile robot reaches a designated detection position near the stack to be detected, the fork tine is controlled to be lifted upward from a first height; wherein, when the fork tine is at the first height, the fork hole corresponding to the bottom layer of the goods in the stack to be detected is within the detection range of the laser radar;

[0169] During the lifting process of the fork teeth, the laser radar is used to detect the direction of the stack to be detected until the lifting condition of the fork teeth is reached, and the point cloud data to be used of the stack to be detected is obtained; wherein, when the lifting condition is reached, the range detected by the laser radar includes the fork hole corresponding to the highest layer of goods in the stack to be detected;

[0170] Based on the point cloud data to be utilized, the positions and postures of the fork holes corresponding to each layer of goods in the stack to be detected are determined, and the information to be utilized as a basis for removing each layer of goods in the stack to be detected is obtained.

[0171] Furthermore, the electronic device may further include a communication bus and / or a communication interface, and the processor 902, the communication interface, and the memory 901 communicate with each other via the communication bus.

[0172] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0173] The communication interface is used for communication between the above electronic device and other devices.

[0174] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0175] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0176] In another embodiment provided in the present application, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned stacking detection methods are implemented.

[0177] In another embodiment provided by the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any of the stacking detection methods in the above embodiments.

[0178] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a solid-state hard disk (SSD), etc.

[0179] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0180] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the fork-picking mobile robot, device, electronic device, computer-readable storage medium, and computer program product embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0181] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A stacking detection method, characterized in that: Applied to a fork picking mobile robot, the fork tines of the fork picking mobile robot are provided with a laser radar, and the method comprises: When the fork picking mobile robot reaches a designated detection position near the stack to be detected, the fork tine is controlled to be lifted upward from a first height; wherein, when the fork tine is at the first height, the fork hole corresponding to the bottom layer of the goods in the stack to be detected is within the detection range of the laser radar; During the lifting process of the fork teeth, the laser radar is used to detect the direction of the stack to be detected until the lifting condition of the fork teeth is reached, and the point cloud data to be used of the stack to be detected is obtained; wherein, when the lifting condition is reached, the range detected by the laser radar includes the fork hole corresponding to the highest layer of goods in the stack to be detected; Based on the point cloud data to be utilized, the positions and postures of the fork holes corresponding to each layer of goods in the stack to be detected are determined, and the information to be utilized as a basis for removing each layer of goods in the stack to be detected is obtained.

2. The method according to claim 1, characterized in that The position and posture of the fork hole corresponding to any layer of goods include: the height of the fork hole corresponding to the layer of goods, the position of the projection of the fork hole corresponding to the layer of goods on the horizontal plane, and the deflection angle of the fork hole corresponding to the layer of goods; wherein the deflection angle of the fork hole corresponding to the layer of goods represents: the angle between the projection of the surface of the stack to be detected to which the fork hole corresponding to the layer of goods belongs on the horizontal plane and the projection of the designated plane on the horizontal plane; the designated plane is perpendicular to the direction in which the fork teeth point; After determining the positions of the fork holes corresponding to each layer of goods in the to-be-detected stack based on the to-be-utilized point cloud data and obtaining the to-be-utilized information as a basis for removing each layer of goods in the to-be-detected stack, the method further includes: When receiving a removal instruction, determining the position of the fork hole corresponding to the layer where the goods to be removed indicated by the removal instruction are located from the information to be used; Based on the position of the fork hole corresponding to the layer where the goods to be removed are located and the position of the fork-picking mobile robot, path planning is performed, and the fork teeth are controlled to move to the height of the fork hole corresponding to the layer where the goods to be removed are located; The steering wheel of the fork-picking mobile robot is controlled to move according to the planned path so that the fork teeth are inserted into the fork holes corresponding to the layer where the goods to be removed are located, and the fork teeth are controlled to be lifted to carry the goods to be removed.

3. The method according to claim 2, characterized in that Before performing path planning based on the posture of the fork hole corresponding to the layer where the goods to be removed are located and the posture of the fork-picking mobile robot, and controlling the fork teeth to move to the height of the fork hole corresponding to the layer where the goods to be removed are located, the method further includes: Based on the position of the fork hole corresponding to the layer where the goods to be removed are projected on the horizontal plane and the current position of the fork tine projected on the horizontal plane, determine the current distance between the fork tine and the fork hole corresponding to the layer where the goods to be removed are located in a specified horizontal direction to obtain the position deviation; wherein the specified horizontal direction is perpendicular to the direction of the fork tine; When the deflection angle of the fork hole corresponding to the layer where the goods to be removed are located is greater than a preset angle, and / or the position deviation is greater than a preset distance, an alarm is issued; Based on the posture of the fork hole corresponding to the layer where the goods to be removed are located and the posture of the fork-picking mobile robot, path planning is performed, and the fork teeth are controlled to move to the height of the fork hole corresponding to the layer where the goods to be removed are located, including: When the deflection angle of the fork hole corresponding to the layer where the goods to be picked are located is not greater than the preset angle, and the position deviation is not greater than the preset distance, path planning is performed based on the posture of the fork hole corresponding to the layer where the goods to be picked are located and the posture of the fork mobile robot, and the fork teeth are controlled to move to the height of the fork hole corresponding to the layer where the goods to be picked are located.

4. The method according to claim 1, characterized in that The lifting stop conditions include: The fork tines are lifted to a maximum liftable height; or, The projection of the highest three-dimensional point currently detected by the laser radar on the horizontal plane is outside a specified range; wherein the specified range represents: the portion of the projection of the stack to be detected on the horizontal plane, close to one side of the fork mobile robot.

5. The method according to any one of claims 1 to 4, characterized in that: During the lifting process of the fork tine, the laser radar is used to detect the direction of the stack to be detected until the lifting condition of the fork tine is reached, and the point cloud data to be used of the stack to be detected is obtained, including: During the lifting process of the fork tine, the laser radar detects the direction of the stack to be detected according to a preset detection cycle until the lifting condition of the fork tine is reached, and the point cloud data to be processed detected by the laser radar each time the preset detection cycle is reached; The obtained multiple point cloud data to be processed are fused to obtain the point cloud data to be used of the stack to be detected.

6. The method according to claim 5, characterized in that During the lifting process of the fork tine, the laser radar detects the direction of the stack to be detected according to a preset detection cycle until the lifting condition of the fork tine is reached, and the point cloud data to be processed detected by the laser radar each time the preset detection cycle is reached is obtained, including: During the lifting process of the fork tine, the laser radar detects the direction of the stack to be detected according to a preset detection cycle until the lifting condition of the fork tine is reached, and the initial point cloud data detected by the laser radar each time the preset detection cycle is reached is obtained; wherein the initial point cloud data obtained by each detection includes: the position of each detected three-dimensional point projected on the horizontal plane; For the initial point cloud data obtained in each detection, combined with the height of the fork tine during the detection, the height of each three-dimensional point detected in the detection is calculated; The point cloud data to be processed of this detection is obtained by combining the position and height of the projection of each three-dimensional point detected this time on the horizontal plane.

7. The method according to claim 5, characterized in that The laser radar is a three-dimensional laser radar; the height to which the fork tine is lifted within a preset detection cycle is not greater than the detection height of the laser radar during one detection.

8. A fork-picking mobile robot, characterized in that: include: A laser radar and a processor mounted on the tines of the fork; The laser radar is used to detect the front of the fork mobile robot during the lifting process of the fork tine; The processor is used to execute the method according to any one of claims 1 to 7.

9. A stacking detection device, characterized in that: Applied to a fork-picking mobile robot, the fork tines of the fork-picking mobile robot are provided with a laser radar, and the device comprises: a fork tine control module, configured to control the fork tine to be lifted upward from a first height when the fork picking mobile robot reaches a designated detection position near the stack to be detected; wherein, when the fork tine is at the first height, the fork hole corresponding to the bottom layer of goods in the stack to be detected is within the detection range of the laser radar; A radar detection module, used to detect the direction of the stack to be detected by the laser radar during the lifting process of the fork teeth, until the lifting stop condition of the fork teeth is reached, and obtain the point cloud data to be used of the stack to be detected; wherein, when the lifting stop condition is reached, the range detected by the laser radar includes the fork hole corresponding to the highest layer of goods in the stack to be detected; The posture determination module is used to determine the posture of the fork holes corresponding to each layer of goods in the stack to be detected based on the point cloud data to be used, and obtain the information to be used as a basis for removing each layer of goods in the stack to be detected.

10. The device according to claim 9, characterized in that The position and posture of the fork hole corresponding to any layer of goods include: the height of the fork hole corresponding to the layer of goods, the position of the projection of the fork hole corresponding to the layer of goods on the horizontal plane, and the deflection angle of the fork hole corresponding to the layer of goods; wherein the deflection angle of the fork hole corresponding to the layer of goods represents: the angle between the projection of the surface of the stack to be detected to which the fork hole corresponding to the layer of goods belongs on the horizontal plane and the projection of the designated plane on the horizontal plane; the designated plane is perpendicular to the direction in which the fork teeth point; the device also includes: a posture acquisition module, for determining the posture of the fork hole corresponding to each layer of the goods in the to-be-detected stack based on the to-be-utilized point cloud data, and obtaining the to-be-utilized information as a basis for removing the goods in each layer of the to-be-detected stack, and when receiving a removal instruction, determining the posture of the fork hole corresponding to the layer where the goods to be removed are located indicated by the removal instruction from the to-be-utilized information; A path planning module, used for performing path planning based on the posture of the fork hole corresponding to the layer where the goods to be removed are located and the posture of the fork-picking mobile robot, and controlling the fork teeth to move to the height of the fork hole corresponding to the layer where the goods to be removed are located; A mobile control module, used to control the steering wheel of the fork-picking mobile robot to move according to the planned path, so that the fork tine is inserted into the fork hole corresponding to the layer where the goods to be removed are located, and control the fork tine to be lifted to carry the goods to be removed; and / or, The device also includes: A position deviation determination module is used to perform path planning based on the position of the fork hole corresponding to the layer where the goods to be removed are located and the position of the fork-picking mobile robot, and before controlling the fork tine to move to the height of the fork hole corresponding to the layer where the goods to be removed are located, determine the distance between the fork tine and the fork hole corresponding to the layer where the goods to be removed are located in a specified horizontal direction based on the position of the fork hole corresponding to the layer where the goods to be removed are located on the horizontal plane and the current position of the fork tine projected on the horizontal plane, so as to obtain the position deviation; wherein the specified horizontal direction is perpendicular to the direction of the fork tine; An alarm module, used for giving an alarm when the deflection angle of the fork hole corresponding to the layer where the goods to be removed are located is greater than a preset angle, and / or the position deviation is greater than a preset distance; The path planning module is specifically used to plan the path based on the posture of the fork hole corresponding to the layer where the goods to be removed are located and the posture of the fork-picking mobile robot, and control the fork teeth to move to the height of the fork hole corresponding to the layer where the goods to be removed are located, when the deflection angle of the fork hole corresponding to the layer where the goods to be removed are located is not greater than the preset angle and the position deviation is not greater than the preset distance; and / or, The lifting stop conditions include: The fork tines are lifted to a maximum liftable height; or, The projection of the highest three-dimensional point currently detected by the laser radar on the horizontal plane is outside the specified range; wherein the specified range represents: the portion of the projection of the stack to be detected on the horizontal plane close to one side of the fork-picking mobile robot; and / or, Radar detection module, including: A periodic detection submodule is used to detect the direction of the stack to be detected by the laser radar according to a preset detection cycle during the lifting process of the fork tine until the lifting stop condition of the fork tine is reached, and obtain the point cloud data to be processed detected by the laser radar each time the preset detection cycle is reached; A point cloud data fusion submodule is used to fuse the obtained multiple point cloud data to be processed to obtain the point cloud data to be used of the stack to be detected; and / or, The periodic detection submodule is specifically used to detect the direction of the stack to be detected by the laser radar according to a preset detection cycle during the process of the fork lifting, until the lifting condition of the fork is stopped, and obtain the initial point cloud data detected by the laser radar each time the preset detection cycle is reached; wherein the initial point cloud data obtained by each detection includes: the position of each detected three-dimensional point projected on the horizontal plane; for the initial point cloud data obtained by each detection, combined with the height of the fork during the detection, calculate the height of each three-dimensional point detected at this time; combined with the position and height of each three-dimensional point detected at this time on the horizontal plane, obtain the point cloud data to be processed of this detection; and / or, The laser radar is a three-dimensional laser radar; the height to which the fork tine is lifted within a preset detection cycle is not greater than the detection height of the laser radar during one detection.

11. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-7 when executing a program stored in a memory.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

13. A computer program product, characterized in that When the computer program product is run on a computer, the computer is enabled to execute the method according to any one of claims 1 to 7.

Images