Pallet recognition device

Abstract

To provide a pallet recognition device capable of improving the estimation accuracy of the position and orientation of a pallet regardless of the type of the pallet.SOLUTION: A pallet recognition device 10 comprises a memory unit 13 that stores correct point cloud data, which is data indicating a plurality of types of pallets 3 that has been acquired in advance, a camera 11 and a pallet detection unit 20 that detect the type of the pallet 3, a correct point cloud selection unit 21 that selects correct point cloud data corresponding to the type of the pallet 3 detected by the camera 11 and the pallet detection unit 20 from the correct point cloud data of the plurality of types of the pallets 3 stored in the memory unit 13, a laser sensor 12 that acquires detected point cloud data of the pallet 3, and a matching processing unit 22 that matches the detected point cloud data acquired by the laser sensor 12 with the correct point cloud data selected by the correct point cloud selection unit 21 to calculate an estimate of the position and orientation of the pallet 3.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a palette recognition device. [Background technology]

[0002] Known as a conventional pallet recognition device is, for example, the technology described in Patent Document 1. The pallet recognition device described in Patent Document 1 includes a laser sensor that detects the distance to the pallet to be handled, a camera that captures the image of the pallet to be handled, and a controller that calculates a plane equation for the front surface of the pallet based on image data from the camera and measurement point data from the laser sensor, and estimates the position and orientation of the pallet using the plane equation. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-42070 Summary of the Invention [Problem to be solved by the invention]

[0004] Incidentally, pallets used in loading and unloading operations include flat pallets as described in the above Patent Document 1, as well as post pallets that can be stacked with cargo placed on them. Depending on how the cargo is placed on the pallet and where the pallet is placed, it may be difficult to detect the front of the pallet, making it difficult to estimate the position and orientation of the pallet.

[0005] An object of the present invention is to provide a pallet recognition device that can improve the accuracy of estimating the position and orientation of a pallet regardless of the type of pallet. [Means for solving the problem]

[0006] (1) One aspect of the present invention is a pallet recognition device that recognizes the position and orientation of a pallet, and includes a memory unit that stores correct answer data, which is data indicating multiple types of pallets that have been previously acquired, a detection unit that detects the type of pallet, a selection unit that selects correct answer data corresponding to the type of pallet detected by the detection unit from the correct answer data for the multiple types of pallets stored in the memory unit, a data acquisition unit that acquires detection data for the pallet, and a matching processing unit that matches the detection data acquired by the data acquisition unit with the correct answer data selected by the selection unit to calculate an estimate of the position and orientation of the pallet.

[0007] In such a pallet recognition device, correct answer data for multiple types of pallets is acquired in advance and stored in a memory unit. When recognizing a pallet, the type of pallet is first detected by the detection unit. Then, correct answer data corresponding to the detected type of pallet is selected from the correct answer data for multiple types of pallets stored in the memory unit. Then, the detection data of the pallet acquired by the data acquisition unit is matched with the selected correct answer data to calculate an estimate of the position and orientation of the pallet. By matching the detection data of the pallet with the correct answer data corresponding to the type of pallet in this manner, the device becomes more resistant to disturbances, and therefore erroneous estimation of the position and orientation of the pallet is less likely to occur even if the type of pallet used changes. This improves the estimation accuracy of the position and orientation of the pallet regardless of the type of pallet.

[0008] (2) In the above (1), the pallet recognition device may further include an evaluation unit that evaluates the estimated values ​​of the position and orientation of the pallet calculated by the matching processing unit, and a position and orientation determination unit that determines the position and orientation of the pallet based on the evaluation result by the evaluation unit.

[0009] In such a configuration, the estimated values ​​of the position and attitude of the pallet obtained by matching the detection data acquired by the data acquisition unit with the selected correct data are evaluated, and the position and attitude of the pallet are determined in accordance with the evaluation results, thereby further improving the estimation accuracy of the position and attitude of the pallet.

[0010] (3) In (2) above, the matching processing unit may match the detection data with the correct data to calculate an estimate of the position and orientation of the pallet, and may calculate the degree of matching between the detection data and the correct data. The evaluation unit may evaluate the estimate of the position and orientation of the pallet by determining whether the degree of matching between the detection data and the correct data is greater than or equal to a predetermined threshold.

[0011] In such a configuration, the degree of matching between the detection data acquired by the data acquisition unit and the selected correct data is calculated, and it is determined whether the degree of matching between the detection data and the correct data is above a threshold value, thereby making it possible to easily and accurately evaluate the estimated values ​​for the position and orientation of the pallet.

[0012] (4) In (3) above, when the evaluation unit determines that the degree of matching between the detection data and the correct data is equal to or greater than a threshold, the position and orientation determination unit may determine the estimated values ​​of the position and orientation of the pallet calculated by the matching processing unit as the position and orientation of the pallet; and when the evaluation unit determines that the degree of matching between the detection data and the correct data is not equal to or greater than a threshold, the position and orientation determination unit may estimate the position and orientation of the pallet based on the detection data of the pallet acquired by the data acquisition unit.

[0013] In such a configuration, when the degree of match between the detection data acquired by the data acquisition unit and the selected correct data is equal to or greater than a threshold value, the estimated values ​​of the position and attitude of the pallet obtained by matching the detection data with the correct data are determined as the position and attitude of the pallet, thereby more reliably improving the estimation accuracy of the position and attitude of the pallet.

[0014] (5) In (4) above, when the position and orientation determination unit determines that the degree of matching between the detection data and the correct data is not greater than or equal to a threshold, it may extract detection data that matches the correct data from the detection data acquired by the data acquisition unit, and estimate the position and orientation of the pallet based on the detection data that matches the correct data.

[0015] In such a configuration, when the degree of match between the detection data acquired by the data acquisition unit and the selected correct data is not above a threshold, detection data that matches the correct data is extracted from the detection data, and the position and attitude of the pallet are estimated based on the detection data that matches the correct data, thereby making it possible to confirm whether the matching between the detection data and the correct data has truly failed.

[0016] (6) In any of (1) to (5) above, the detection unit has a camera that images the pallet and detects the type of pallet based on image data from the camera, the data acquisition unit irradiates the pallet with a laser and receives the reflected laser light to detect the distance to the pallet and acquire detection point cloud data, and the memory unit may store correct point cloud data for multiple types of pallets.

[0017] In this configuration, the type of pallet can be easily and inexpensively detected by using a camera to capture an image of the pallet. Also, by irradiating a laser onto the pallet to obtain detection point cloud data, it is easy to match the detection data with the correct data.

[0018] (7) In any of (1) to (6) above, one of the correct answer data for the plurality of types of pallets may be correct answer data for a state in which a part of the pallet is hidden by another member.

[0019] In this configuration, for example, when a pallet is placed on a conveyor equipped with a guard to prevent the pallet from falling, even if part of the pallet is hidden by the guard, the accuracy of estimating the position and orientation of the pallet is improved by pre-storing correct data for the state in which part of the pallet is hidden by the guard in the memory unit. Effect of the Invention

[0020] According to the present invention, it is possible to improve the accuracy of estimating the position and orientation of a pallet, regardless of the type of pallet. [Brief description of the drawings]

[0021] [Figure 1] 1 is a schematic configuration diagram showing a driving control device equipped with a pallet recognition device according to an embodiment of the present invention; [Diagram 2] FIG. 1 is a schematic plan view showing a state in which a pallet is unloaded by a forklift. [Diagram 3] FIG. 2 is a front view showing a flat pallet and a post pallet as pallets. [Figure 4] 4 is a flowchart showing a procedure for pallet recognition and travel control executed by a controller shown in FIG. 1. [Diagram 5] 13 is a diagram showing a state in which detection point cloud data acquired by a laser sensor is matched with selected correct point cloud data. FIG. [Figure 6] FIG. 13 is a diagram showing a flat pallet placed on a conveyor equipped with a guard to prevent the pallet from falling. [Figure 7] 13 is a diagram showing how, when a package is placed on the pallet portion of the post pallet and the front of the package protrudes from the post pallet, both side edges of the package are mistakenly detected as pillars of the post pallet. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

[0023] Fig. 1 is a schematic diagram showing a travel control device equipped with a pallet recognition device according to one embodiment of the present invention. As shown in Fig. 2, the travel control device 1 is a device that automatically drives a forklift 2 to a loading position when the forklift 2 loads a pallet 3. The forklift 2 has a pair of forks 4 on the left and right that hold the pallet 3. The loading position is a position where each fork 4 of the forklift 2 is inserted into a fork hole (described later) of the pallet 3.

[0024] The pallets 3 are loaded, for example, on the bed of a truck (not shown). As the pallets 3, as shown in FIG.

[0025] The flat pallet 3A shown in Fig. 3(a) is made of plastic or wood. The flat pallet 3A has a square shape in a plan view. An item M is placed on the flat pallet 3A. The flat pallet 3A has two fork holes 5 into which the forks 4 of the forklift 2 are inserted. The fork holes 5 extend from the front surface 3a of the flat pallet 3A to the rear side. The shape of the fork holes 5 is rectangular in a front view.

[0026] The post pallet 3B shown in FIG. 3(b) has a pallet portion 6 that is square in plan view, and four pillar portions 7 erected at each of the four corners, front, back, left and right, of the pallet portion 6. Baggage M is placed on the pallet portion 6. The pallet portion 6 is provided with two fork holes 8 into which the forks 4 of a forklift 2 are inserted. The fork holes 8 extend from the front surface 3a of the post pallet 3B to the rear side. The fork holes 8 are rectangular in front view. The pillar portions 7 are in the shape of a quadrangular prism. The pillar portions 7 are foldably connected to the corners of the pallet portion 6.

[0027] 1, a driving control device 1 is mounted on a forklift 2. The driving control device 1 includes a camera 11, a laser sensor 12, a storage unit 13, a drive unit 14, and a controller 15.

[0028] The camera 11 captures an image of the front surface 3a of the pallet 3 to be picked up and acquires image data. As the camera 11, for example, a monocular camera or the like is used.

[0029] The laser sensor 12 irradiates a laser toward the front surface 3a of the pallet 3 to be removed, and receives the reflected light of the laser to detect the distance to the pallet 3 and acquire point cloud data. A point cloud is a collection of laser reflection points on an object including the pallet 3. The laser sensor 12 constitutes a data acquisition unit that acquires detection point cloud data (detection data) of the pallet 3. The laser sensor 12 irradiates a 2D or 3D laser with a high point density. For example, a LIDAR or a laser range finder may be used as the laser sensor 12.

[0030] The storage unit 13 stores correct point cloud data (correct data) of the front faces 3a of multiple types of pallets 3. The correct point cloud data is acquired in advance using the laser sensor 12. The correct point cloud data is data indicating multiple types of pallets 3 acquired in advance. Here, different types of pallets 3 include not only different types of pallets 3 themselves, but also cases where the type of pallet 3 itself is the same but the appearance of the pallet 3 is different, such as when part of the pallet 3 is hidden by another member due to changes in the way the cargo M is loaded onto the pallet 3 or the location of the pallet 3.

[0031] Specifically, the correct point cloud data for the front faces 3a of the multiple types of pallets 3 includes correct point cloud data for the front face 3a of flat pallet 3A (see FIG. 3(a)) and correct point cloud data for the front face 3a of post pallet 3B (see FIG. 3(b)). The correct point cloud data for the front faces 3a of the multiple types of pallets 3 also includes correct point cloud data for a state in which part of the front face 3a of flat pallet 3A is hidden by other members (see FIG. 6(b)).

[0032] Although not specifically shown, the drive unit 14 has, for example, a travel motor that rotates the drive wheels of the forklift 2 and a steering motor that steers the steering wheels of the forklift 2.

[0033] The controller 15 is composed of a CPU, a RAM, a ROM, an input / output interface, etc. The controller 15 has a palette detection unit 20, a correct point group selection unit 21 (selection unit), a matching processing unit 22, a matching evaluation unit 23 (evaluation unit), a position and orientation determination unit 24, and a driving control unit 25.

[0034] Here, the pallet detection unit 20, the correct point group selection unit 21, the matching processing unit 22, the matching evaluation unit 23 and the position and orientation determination unit 24 cooperate with the camera 11, the laser sensor 12 and the storage unit 13 to configure the pallet recognition device 10 of this embodiment. The pallet recognition device 10 is a device that recognizes the position and orientation of a pallet 3.

[0035] The pallet detection unit 20 detects the type of pallet 3 based on the image data of the camera 11. The pallet detection unit 20 cooperates with the camera 11 to configure a detection unit that detects the type of pallet 3.

[0036] The correct point cloud selection unit 21 selects correct point cloud data corresponding to the type of pallet 3 detected by the pallet detection unit 20 from the correct point cloud data of multiple types of pallets 3 stored in the storage unit 13.

[0037] The matching processing unit 22 matches the detected point cloud data of the pallet 3 acquired by the laser sensor 12 with the correct point cloud data selected by the correct point cloud selection unit 21 (see FIG. 5), calculates estimated values ​​for the position and orientation of the pallet 3, and calculates the degree of match between the detected point cloud data and the correct point cloud data. The function of the matching processing unit 22 will be described in detail later.

[0038] The matching evaluation unit 23 evaluates the estimated values ​​of the position and orientation of the pallet 3 calculated by the matching processing unit 22. The matching evaluation unit 23 evaluates the position and orientation of the pallet 3 by determining whether the degree of match between the detected point cloud data of the pallet 3 and the ground truth point cloud data is equal to or greater than a predetermined threshold.

[0039] The position and orientation determination unit 24 determines the position and orientation of the pallet 3 according to the evaluation result by the matching evaluation unit 23 .

[0040] When the matching evaluation unit 23 determines that the degree of matching between the detected point cloud data and the correct point cloud data is equal to or greater than a threshold, the position and orientation determination unit 24 determines the estimated values ​​of the position and orientation of the pallet 3 calculated by the matching processing unit 22 as the position and orientation of the pallet 3. When the matching evaluation unit 23 determines that the degree of matching between the detected point cloud data and the correct point cloud data is not equal to or greater than a threshold, the position and orientation determination unit 24 estimates the position and orientation of the pallet 3 based on the detected point cloud data of the pallet 3 acquired by the laser sensor 12.

[0041] When the position and orientation determination unit 24 determines that the degree of matching between the detected point cloud data and the correct point cloud data is not above a threshold, it extracts the detected point cloud data of the pallet 3 acquired by the laser sensor 12 that matches the correct point cloud data, and estimates the position and orientation of the pallet 3 based on the detected point cloud data that matches the correct point cloud data.

[0042] The travel control unit 25 controls the drive unit 14 to travel the forklift 2 to the pickup position based on the position and attitude of the pallet 3 determined by the position and attitude determination unit 24.

[0043] Fig. 4 is a flowchart showing the procedure for pallet recognition and travel control executed by the controller 15. This process is executed when an instruction to start loading is given when the forklift 2 has reached the front surface 3a of the pallet 3 to be loaded, as shown in Fig. 2. Note that the angle θ in Fig. 2 indicates the detection range of the pallet 3.

[0044] 4, the controller 15 first detects the type, number and approximate location of the pallets 3 to be picked up based on image data from the camera 11 (step S101). At this time, the controller 15 detects the type, etc. of the pallets 3 by machine learning such as deep learning.

[0045] Next, the controller 15 selects correct point cloud data that matches the type of pallet 3 detected in step S101 from the correct point cloud data of multiple types of pallets 3 stored in the memory unit 13 (step S102). When the type of pallet 3 is detected to be flat pallet 3A (see FIG. 3(a)), the correct point cloud data for flat pallet 3A is selected. When the type of pallet 3 is detected to be post pallet 3B (see FIG. 3(b)), the correct point cloud data for post pallet 3B is selected.

[0046] Next, as shown in Fig. 5(a), the controller 15 matches the detection point cloud data of the pallet 3 acquired by the laser sensor 12 with the answer point cloud data selected in step S102 (step S103). In Fig. 5(a), the detection point cloud data Dp acquired when the laser is irradiated toward the post pallet 3B is matched with the answer point cloud data Dp0 for the post pallet 3B. As a matching method, for example, ICP (iteretive closest point) matching or the like is adopted.

[0047] The controller 15 then calculates an estimate of the position and attitude of the pallet 3 to be removed from the matching result between the detected point cloud data and the correct point cloud data (step S104). At this time, the controller 15 performs an estimation calculation of the position and attitude of the pallet 3 by translating the detected point cloud data Dp with respect to the correct point cloud data Dp0 as shown in Fig. 5(b) or by rotating the detected point cloud data Dp with respect to the correct point cloud data Dp0 as shown in Fig. 5(c). Note that, for the sake of convenience, the amount of deviation between the detected point cloud data Dp and the correct point cloud data Dp0 is shown to be larger than it actually is in Fig. 5(b) and Fig. 5(c).

[0048] The controller 15 also calculates the degree of matching between the detected point cloud data and the correct point cloud data from the matching result between the detected point cloud data and the correct point cloud data (step S105). The calculation of the degree of matching is performed as an evaluation of the estimated values ​​of the position and orientation of the pallet 3 calculated in step S104.

[0049] Next, the controller 15 determines whether the degree of matching between the detected point cloud data and the answer point cloud data is equal to or greater than a predetermined threshold (step S106). The degree of matching is expressed, for example, as a percentage (%).

[0050] When the controller 15 determines that the degree of matching between the detected point cloud data and the correct point cloud data is equal to or greater than a threshold value, it determines the estimated values ​​of the position and attitude of the pallet 3 calculated in step S104 as the position and attitude of the pallet 3 to be picked up (step S107).

[0051] When the controller 15 determines that the degree of matching between the detected point cloud data and the correct point cloud data is not equal to or greater than the threshold, the controller 15 extracts the detected point cloud data that matches the correct point cloud data selected in step S102 from the detected point cloud data acquired by the laser sensor 12 (step S108).Then, the controller 15 estimates the position and attitude of the pallet 3 to be unloaded based on the detected point cloud data extracted in step S108 (step S109).

[0052] Specifically, as shown in Figure 3(a), when the pallet 3 is a flat pallet 3A, the controller 15 extracts, for example, two fork holes 5 of the flat pallet 3A based on the detection point cloud data, and estimates the position and posture of the flat pallet 3A from the positional relationship of the two fork holes 5.

[0053] As shown in Figure 3(b), when the pallet 3 is a post pallet 3B, the controller 15 extracts, for example, a horizontal line corresponding to the pallet section 6 and a vertical line corresponding to the column section 7 based on the detection point cloud data, and calculates the position and posture of the post pallet 3B from the positional relationship between the horizontal and vertical lines.

[0054] In addition, when matching between the detected point cloud data and the correct point cloud data fails, the controller 15 utilizes the fact that the variation in the detected point cloud data that matches the correct point cloud data becomes large, and determines that the estimation of the position and orientation of the pallet 3 has failed due to the variation in the detected point cloud data that matches the correct point cloud data.

[0055] After executing step S107 or step S109, the controller 15 generates a travel route from the forklift 2 to the unloading position based on the position and posture of the pallet 3 to be unloaded (step S110). Then, the controller 15 controls the drive unit 14 to make the forklift 2 travel to the unloading position according to the travel route (step S111).

[0056] Here, the pallet detection unit 20 executes step S101. The correct point group selection unit 21 executes step S102. The matching processing unit 22 executes steps S103 to S105. The matching evaluation unit 23 executes step S106. The position and orientation determination unit 24 executes steps S107 to S109. The travel control unit 25 executes steps S110 and S111.

[0057] Incidentally, as shown in FIG. 6(a), in some cases, fall prevention guards 51 are provided on both the left and right sides of the tip of the conveyor 50 to prevent the flat pallet 3A placed on the inclined conveyor 50 from falling off the conveyor 50. In this case, as shown in FIG. 6(b), a part of the front surface 3a of the flat pallet 3A is hidden by the guard 51. Therefore, when the detection point cloud data is acquired by the laser sensor 12, the front surface 3a of the flat pallet 3A is not accurately detected, and the estimation accuracy of the position and attitude of the flat pallet 3A is reduced. Note that the conveyor 50 is omitted in FIG. 6(b) for convenience.

[0058] Furthermore, as shown in Fig. 7(a), when a large piece of luggage M placed on the pallet portion 6 of the post pallet 3B protrudes from the front surface 3a of the post pallet 3B (see A in the figure), the piece of luggage M is closer to the laser sensor 12 than the post pallet 3B. For this reason, as shown in Fig. 7(b), when the detection point cloud data Dp is acquired by the laser sensor 12, both side edge portions E of the piece of luggage M may be erroneously detected as the pillar portions 7 of the post pallet 3B. In this case, the estimation accuracy of the position and attitude of the post pallet 3B decreases.

[0059] To address such issues, in this embodiment, correct point cloud data for multiple types of pallets 3 is acquired in advance and stored in storage unit 13. When recognizing a pallet 3, first the type of pallet 3 is detected using camera 11. Then, from the correct point cloud data for multiple types of pallets 3 stored in storage unit 13, correct point cloud data corresponding to the type of pallet 3 detected is selected.

[0060] For example, if the pallet 3 is a flat pallet 3A and the flat pallet 3A is placed on the bed of a normal truck (see FIG. 3(a)), the correct point cloud data of the front surface 3a of the flat pallet 3A is selected. If the pallet 3 is a flat pallet 3A and the flat pallet 3A is placed on a conveyor 50 equipped with guards 51 to prevent the pallet 3A from falling (see FIG. 6), the correct point cloud data of the front surface 3a of the flat pallet 3A including the two guards 51 is selected. If the pallet 3 is a post pallet 3B (see FIGS. 3(b) and 7(a)), the correct point cloud data of the front surface 3a of the post pallet 3B is selected.

[0061] Then, the detection point cloud data of the pallet 3 acquired by the laser sensor 12 is matched with the selected correct point cloud data to calculate an estimate of the position and orientation of the pallet 3. By matching the detection point cloud data of the pallet 3 with the correct point cloud data suitable for the type of pallet 3 in this way, it becomes more resistant to disturbances, and therefore erroneous estimation of the position and orientation of the pallet 3 is less likely to occur even if the type of pallet 3 used changes. This improves the accuracy of estimating the position and orientation of the pallet 3 regardless of the type of pallet 3. As a result, the forklift 2 can be moved to the loading position with high accuracy.

[0062] In addition, in this embodiment, the estimated values ​​of the position and attitude of the pallet 3 obtained by matching the detection point cloud data acquired by the laser sensor 12 with the selected correct point cloud data are evaluated, and the position and attitude of the pallet 3 are determined based on the evaluation results, thereby further improving the estimation accuracy of the position and attitude of the pallet 3.

[0063] In addition, in this embodiment, the degree of matching between the detection point cloud data acquired by the laser sensor 12 and the selected correct point cloud data is calculated, and it is determined whether the degree of matching between the detection point cloud data and the correct point cloud data is equal to or greater than a threshold value, thereby enabling the estimated values ​​of the position and orientation of the pallet 3 to be evaluated easily and accurately.

[0064] Furthermore, in this embodiment, when the degree of matching between the detection point cloud data acquired by the laser sensor 12 and the selected correct point cloud data is equal to or greater than a threshold value, the estimated values ​​of the position and attitude of the pallet 3 obtained by matching the detection point cloud data with the correct point cloud data are determined as the position and attitude of the pallet 3, thereby more reliably improving the estimation accuracy of the position and attitude of the pallet 3.

[0065] Furthermore, in this embodiment, when the degree of matching between the detection point cloud data acquired by the laser sensor 12 and the selected correct point cloud data is not above a threshold, detection point cloud data that matches the correct point cloud data is extracted from the detection point cloud data, and the position and attitude of the pallet 3 are estimated based on the detection point cloud data that matches the correct point cloud data, thereby making it possible to confirm whether the matching between the detection point cloud data and the correct point cloud data has truly failed.

[0066] Furthermore, in this embodiment, the type of pallet 3 can be easily and inexpensively detected by using the camera 11 that captures the image of the pallet 3. Furthermore, by irradiating a laser onto the pallet 3 to obtain detection point cloud data, matching between the detection point cloud data and the correct answer point cloud data can be easily performed.

[0067] Furthermore, in this embodiment, when a pallet 3 is placed on a conveyor 50 provided with a guard 51 to prevent the pallet 3 from falling, even if part of the pallet 3 is hidden by the guard 51, the accuracy of estimating the position and orientation of the pallet 3 is improved by pre-storing in the memory unit 13 the correct point cloud data for the state in which part of the pallet 3 is hidden by the guard 51.

[0068] The present invention is not limited to the above embodiment. For example, in the above embodiment, when the degree of matching between the detection point cloud data acquired by the laser sensor 12 and the selected correct answer point cloud data is not equal to or greater than a threshold, the position and orientation of the pallet 3 are estimated based on the detection point cloud data that matches the correct answer point cloud data among the detection point cloud data acquired by the laser sensor 12, but the present invention is not particularly limited to such a form. For example, the position and orientation of the pallet 3 may be estimated using the detection point cloud data acquired by the laser sensor 12 as it is.

[0069] In the above embodiment, the estimated values ​​of the position and orientation of the pallet 3 are evaluated by determining whether the degree of matching between the detection point cloud data acquired by the laser sensor 12 and the selected ground truth point cloud data is equal to or greater than a predetermined threshold, but the present invention is not limited to such an embodiment. For example, the estimated values ​​of the position and orientation of the pallet 3 may be evaluated using the variation (variance) of matching between the detection point cloud data and the ground truth point cloud data.

[0070] In addition, in the above embodiment, the position and posture of the pallet 3 are recognized when the pallet 3 is unloaded by the forklift 2, but the present invention is also applicable to cases where the position and posture of the pallet 3 are recognized in work environments other than loading and unloading by the forklift 2. [Explanation of symbols]

[0071] 3...pallet, 10...pallet recognition device, 11...camera (detection unit), 12...laser sensor (data acquisition unit), 13...memory unit, 20...pallet detection unit (detection unit), 21...correct point cloud selection unit (selection unit), 22...matching processing unit, 23...matching evaluation unit (evaluation unit), 24...position and posture determination unit, 51...guard (other component), Dp...detected point cloud data, Dp0...correct point cloud data.

Claims

1. A pallet recognition device that recognizes the position and orientation of a pallet, A storage unit that stores previously acquired correct answer data that indicates a plurality of types of palettes; A detection unit that detects the type of the pallet; a selection unit that selects correct answer data corresponding to the type of pallet detected by the detection unit from the correct answer data of the plurality of types of pallets stored in the storage unit; A data acquisition unit that acquires detection data of the pallet; a matching processing unit that matches the detection data acquired by the data acquisition unit with the correct answer data selected by the selection unit to calculate estimated values ​​of the position and orientation of the pallet.

2. an evaluation unit that evaluates the estimated values ​​of the position and orientation of the pallet calculated by the matching processing unit; 2. The pallet recognition device according to claim 1, further comprising a position and orientation determination unit that determines a position and orientation of the pallet in accordance with a result of the evaluation by the evaluation unit.

3. the matching processing unit matches the detection data with the ground truth data to calculate an estimate of the position and orientation of the pallet, and calculates a degree of matching between the detection data and the ground truth data; The pallet recognition device according to claim 2 , wherein the evaluation unit evaluates the estimated values ​​of the position and orientation of the pallet by determining whether the degree of matching between the detection data and the ground truth data is equal to or greater than a predetermined threshold.

4. The pallet recognition device of claim 3, wherein when the evaluation unit determines that the degree of matching between the detection data and the correct data is equal to or greater than the threshold, the position and orientation determination unit determines the estimated values ​​of the position and orientation of the pallet calculated by the matching processing unit as the position and orientation of the pallet, and when the evaluation unit determines that the degree of matching between the detection data and the correct data is not equal to or greater than the threshold, estimates the position and orientation of the pallet based on the detection data acquired by the data acquisition unit.

5. The pallet recognition device of claim 4, wherein when the position and orientation determination unit determines that the degree of matching between the detection data and the correct data is not greater than or equal to the threshold, the position and orientation determination unit extracts detection data that matches the correct data from the detection data acquired by the data acquisition unit, and estimates the position and orientation of the pallet based on the detection data that matches the correct data.

6. The detection unit has a camera that captures an image of the pallet, and detects the type of the pallet based on image data of the camera. the data acquisition unit irradiates a laser toward the pallet and receives reflected light of the laser to detect a distance to the pallet and acquire detection point cloud data; 2. The pallet recognition device according to claim 1, wherein the storage unit stores the correct point cloud data of the plurality of types of pallets.

7. 2. A pallet recognition device according to claim 1, wherein one of the correct answer data for the plurality of types of pallets is correct answer data for a state in which the pallet is partially hidden by another member.

Images