PALLET DETECTION DEVICE

Abstract

Pallet detection device (10), which detects a position and state of a pallet (3), comprises a memory (13) configured to store elements of reference data that were obtained in advance, wherein the elements of reference data correspond to a plurality of types of pallets (3), a detection unit (11, 20) configured to determine a type of pallet (3), a selection unit (21) configured to select reference data corresponding to the type of pallet (3) detected by the detection unit (11, 20) from elements of reference data of the plurality of types of pallets (3) stored in the memory (13), a data reference unit (12) configured to obtain detection data of the pallet (3), and a matching processor (22) configured toto calculate estimated values ​​of a position and state of the pallet (3) by comparing the acquisition data obtained from the data reference unit (12) and the reference data selected by the selection unit (21).

Description

STATE OF THE ART

[0001] The present invention relates to a device for pallet detection.

[0002] As a conventional pallet detection device, a technique disclosed in Japanese Patent Application No. 2021-042070 is known. The pallet detection device of the above publication includes a laser sensor that detects a distance to a pallet corresponding to a load handling target, a camera that captures an image of the pallet corresponding to the load handling target, and a controller that calculates a plane equation for a front surface of the pallet based on image data from the camera and measurement point data from the laser sensor, and estimates a position and state of the pallet using the plane equation.

[0003] In cargo handling operations, a flat pallet, as described in the above publication, a stake pallet, or a stacking pallet that can be stacked with a load placed on it, and the like are used as pallets. Depending on how the load is loaded onto the pallet and where the pallet is placed, detecting the front surface of the pallet becomes difficult. As a result, estimating the position and condition of the pallet may become difficult.

[0004] The present invention is directed to providing a pallet detection device that can improve the accuracy in estimating the position and condition of a pallet regardless of the pallet type. SUMMARY

[0005] According to one aspect of the present invention, there is provided a pallet recognition device that recognizes a position and a state of a pallet, comprising a memory configured to store reference data acquired in advance, the reference data corresponding to a plurality of types of pallets, a detection unit configured to determine a type of the pallet, a selection unit configured to select reference data corresponding to the type of the pallet acquired by the detection unit from the reference data of the plurality of types of pallets stored in the memory, a data acquisition unit configured to acquire detection data of the pallet, and a matching processor configured toto calculate estimated values ​​of a position and a state of the pallet by comparing the detection data acquired by the data acquisition unit and the reference data selected by the selection unit.

[0006] Further aspects and advantages of the invention will become apparent from the following description taken in conjunction with the accompanying drawings, which illustrate by way of example the principles of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The invention, together with its objects and advantages, will be best understood by reference to the following description of the embodiments together with the accompanying drawings, in which: Fig. 1 is a schematic diagram showing the configuration of a travel control device including a pallet detection device according to an embodiment of the present invention; Fig. Figure 2 is a schematic plan view illustrating a pallet being unloaded by a forklift; Fig. 3A and Fig. 3B are front views illustrating a flat pallet and a stacking pallet as the pallet, respectively; Fig. 4 is a flowchart of a pallet detection and travel control process performed by a Fig. 1 illustrated control is executed; The Fig. 5A, Fig. 5B and Fig. 5C are views in which acquisition point cloud data obtained from a laser sensor is compared with selected reference point cloud data; Fig. 6A and Fig. 6B are views in which a flat pallet is placed on a conveyor belt provided with a guard to prevent the pallet from falling; and Fig. 7A and Fig. 7B are views in which opposite edges of a load are incorrectly detected as posts of the stacking pallet when the load placed on a pallet section of the stacking pallet extends beyond a front surface of the stacking pallet. DETAILED DESCRIPTION OF EMBODIMENTS

[0008] An embodiment of the present invention will be described in detail below with reference to the accompanying drawings.

[0009] Fig. 1 is a schematic diagram of the configuration of a travel control device including a pallet detection device according to an embodiment of the present invention. A travel control device 1 is a device that causes a forklift 2 to automatically travel to an unloading position when the forklift 2 unloads a pallet 3, as shown in FIG. Fig. 2. The forklift 2 has a pair of left and right forks 4 for picking up the pallet 3. The unloading position is a position in which the forks 4 of the forklift 2 are inserted into the fork holes of the pallet 3, which will be described later.

[0010] The pallet 3 is loaded, for example, on a platform of a truck (not illustrated). Pallet 3 is a flat pallet 3A and a stacking pallet 3B, as shown in the Fig. 3A and Fig. 3B illustrates.

[0011] For the flat pallet 3A in Fig. 3A, for example, is a plastic flat pallet or a wooden flat pallet. The flat pallet 3A has a substantially square shape when viewed from above. A load M is placed on the flat pallet 3A. The flat pallet 3A has two fork holes 5 into which the forks 4 of the forklift truck 2 are inserted. The fork holes 5 extend rearward from the front surface 3a of the flat pallet 3A. The fork holes 5 each have a rectangular shape when viewed from the front.

[0012] The Fig. The stacking pallet 3B illustrated in FIG. 3B has a pallet section 6 which has a square shape in plan view and four posts 7 which stand at four corners of the pallet section 6. The load M is placed on the pallet section 6. The pallet section 6 has two fork holes 8 into which the forks 4 of the forklift 2 are inserted. The fork holes 8 extend rearward from the front surface 3a of the stacking pallet 3B. The fork holes 8 each have a rectangular shape in front view. The posts 7 each have a square column shape. The posts 7 are each connected to the corners of the pallet section 6 so that they are foldable.

[0013] In Fig. 1, the driving control device 1 is mounted on the forklift 2. The driving control device 1 includes a camera 11, a laser sensor 12, a memory 13, a drive unit 14, and a controller 15.

[0014] Camera 11 captures an image of the front surface 3a of the pallet 3 to be unloaded to obtain image data. A monocular camera, for example, is used as camera 11.

[0015] The laser sensor 12 detects a distance to the pallet 3 to be unloaded by irradiating the front surface 3a of the pallet 3 with a laser and receiving the laser's reflected light to obtain point cloud data. The point cloud is a set of laser reflected points in objects encompassing the pallet 3. The laser sensor 12 serves as a data acquisition unit that acquires detection point cloud data (detection data) of the pallet 3. The laser sensor 12 emits a 2D or 3D laser with a high point density. For example, a LIDAR or a laser rangefinder is used as the laser sensor 12.

[0016] The memory 13 stores pieces of reference point cloud data (reference data) of the front surfaces 3a of a plurality of types of pallets 3. The pieces of reference point cloud data are acquired in advance using the laser sensor 12. The pieces of reference point cloud data are data corresponding to the plurality of types of pallets 3 acquired in advance. The plurality of types of pallets 3 includes not only different types of pallets 3, but also pallets 3 of the same type with different appearances. For example, the appearance of one of the pallets 3 of the same type may differ from the others if part of the pallet 3 is obscured by another object because the placement of the load M or the location of the pallet 3 has changed.

[0017] Specifically, the elements of the reference point cloud data of the front surfaces 3a of the plurality of types of pallets 3 include reference point cloud data of the front surface 3a of the flat pallet 3A (see Fig. 3A) and reference point cloud data of the front surface 3a of the stacking pallet 3B (see Fig. 3B). In addition, the items of the reference point cloud data of the front surfaces 3a of the plurality of types of pallets 3 also include reference point cloud data of the front surface 3a of the flat pallet 3A in a state where the front surface 3a is partially covered by another object (see Fig. 6B).

[0018] The drive unit 14 includes a drive motor (not illustrated) that rotates the drive wheels of the forklift 2, and the steering motor (not illustrated) that steers, for example, a steering wheel of the forklift 2.

[0019] The controller 15 includes a CPU, a RAM, a ROM, and input / output interfaces. The controller 15 has a pallet detection unit 20, a reference point cloud selection unit 21 (selection unit), a matching processor 22, a matching evaluation unit 23 (evaluation unit), a position state determination unit 24, and a travel control unit 25.

[0020] The pallet recognition unit 20, the reference point cloud selection unit 21, the matching processor 22, the matching evaluation unit 23, and the position / state determination unit 24, together with the camera 11, the laser sensor 12, and the memory 13, constitute the pallet recognition device 10 of the above embodiment. The pallet recognition device 10 is a device that recognizes a position and state of the pallet 3.

[0021] The pallet detection unit 20 detects the type of pallet 3 based on the image data from the camera 11. The pallet detection unit 20 and the camera 11 cooperate to form a detection unit that determines the pallet type 3.

[0022] The reference point cloud selecting unit 21 selects, from items of the detection point cloud data of the plurality of types of pallets 3 stored in the memory 13, reference point cloud data corresponding to the type of pallet 3 detected by the pallet detection unit 20.

[0023] The matching processor 22 calculates estimated values ​​of the position and state of the pallet 3 by matching the detection point cloud data of the pallet 3 obtained from the laser sensor 12 with the reference point cloud data selected by the reference point cloud selection unit 21 (see Fig. 5A, Fig. 5B, Fig. 5C) and calculates the degree of agreement between the acquisition point cloud data and the reference point cloud data. The functions of the matching processor 22 will be described in detail later.

[0024] The matching evaluation unit 23 estimates the estimated values ​​for the position and state of the pallet 3 calculated by the matching processor 22. The estimation evaluation unit 23 evaluates the estimated values ​​of the position and state of the pallet 3 by determining whether the degree of matching between the detection point cloud data of the pallet 3 and the reference point cloud data is equal to or greater than a predetermined threshold.

[0025] The position / state determination unit 24 determines the position and state of the pallet 3 according to a result of estimation by the conformity evaluation unit 23.

[0026] When the evaluation unit 23 determines that the degree of agreement between the detection point cloud data and the reference point cloud data is equal to or greater than the threshold value, the position / state determination unit 24 determines the estimated values ​​of the position and state of the pallet 3 calculated by the matching processor 22 as the position and state of the pallet 3. When the matching evaluation unit 23 determines that the degree of agreement between the detection point cloud data and the reference point cloud data is less than the threshold value, the position / state determination unit 24 estimates the position and state of the pallet 3 based on the detection point cloud data of the pallet 3 acquired by the laser sensor 12.

[0027] When it is determined that the degree of agreement between the detection point cloud data and the reference point cloud data is below the threshold, the position / state determination unit 24 extracts items of the detection point cloud data of the pallet 3 acquired from the laser sensor 12 that match the reference point cloud data, and estimates the position and state of the pallet 3 based on the items of the detection point cloud data that match the reference point cloud data.

[0028] The travel control unit 25 controls the drive unit 14 so that the forklift 2 travels to the unloading position depending on the position determined by the position / state determination unit 24 and the state of the pallet 3.

[0029] Fig. Fig. 4 is a flowchart of a pallet detection and travel control process executed by the controller 15. This process is executed when the start of unloading is instructed in a state where the forklift 2 reaches the front surface 3a of the pallet 3 to be unloaded, as shown in Fig. 2. It is noted that an angle θ in Fig. 2 represents a detection area in which the pallet 3 is detected.

[0030] In Fig. 4, the controller 15 first detects the type, number of detections, and approximate location of the pallets 3 to be unloaded based on image data from the camera 11 (step S101). At this time, the controller 15 detects the type, number of detections, and approximate position of the pallets 3 based on machine learning such as deep learning.

[0031] Next, the controller 15 selects, from the pieces of the reference point cloud data of the plurality of types of pallets 3 stored in the memory 13, the reference point cloud data corresponding to the type of pallets 3 detected in step S101 (step S102). If it is detected that the pallet type 3 is the flat pallet 3A (see Fig. 3A), the reference point cloud data corresponding to the flat palette 3A is selected. If it is detected that the palette type 3 is the stack palette 3B (see Fig. 3B), the reference point cloud data corresponding to the stacking palette 3B is selected.

[0032] Then the control 15, as in Fig. 5A illustrates the detection point cloud data of pallet 3 obtained from the laser sensor 12 with the reference point cloud data selected in step S102 (step S103). In Fig. 5A, detection point cloud data Dp obtained at the time of laser irradiation of pallet 3B is compared with reference point cloud data Dp0 corresponding to pallet 3B. For example, ICP (iterative closest point) is used as the comparison method.

[0033] Then, the controller 15 calculates the estimated values ​​of the position and state of the pallet 3 to be unloaded from a result of the comparison between the acquisition point cloud data and the reference point cloud data (step S104). At this time, the controller 15 performs an estimated calculation of the position and state of the pallet 3 by shifting the acquisition point cloud data Dp relative to the reference point cloud data Dp0, as shown in Fig. 5B, and by rotating the acquisition point cloud data Dp relative to the reference point cloud data Dp0, as shown in Fig. 5C. It should be noted that, for the sake of description, a deviation between the acquisition point cloud data Dp and the reference point cloud data Dp0 is illustrated which is larger than that shown in Fig. 5B and Fig. 5C actually is.

[0034] Furthermore, the controller 15 calculates the degree of agreement between the acquired point cloud data and the reference point cloud data from the result of the comparison between the acquired point cloud data and the reference point cloud data (step S105). The degree of agreement is calculated by evaluating the estimated values ​​of the position and state of pallet 3 calculated in step S104.

[0035] Next, the controller 15 determines whether the degree of agreement between the acquisition point cloud data and the reference point cloud data is equal to or greater than the previously set threshold (step S106). The degree of agreement is expressed, for example, as a percentage (%).

[0036] When the controller 15 determines that the degree of agreement between the detection point cloud data and the reference point cloud data is equal to or greater than the threshold value, the controller 15 determines the estimated values ​​of the position and state of the pallet 3 calculated in step S104 as the position and state of the pallet t 3 to be unloaded (step S107).

[0037] If the controller 15 determines that the degree of agreement between the acquisition point cloud data and the reference point cloud data is below the threshold, the controller 15 extracts items of the acquisition point cloud data acquired from the laser sensor 12 that match the reference point cloud data selected in step S102 (step S108). Subsequently, the controller 15 estimates the position and state of the pallet 3 to be unloaded based on the items of the acquisition point cloud data extracted in step S108 (step S109).

[0038] If pallet 3 is the one in Fig. 3A, the controller 15 extracts, for example, the two fork holes 5 of the flat pallet 3A based on the detection point cloud data and estimates the position and state of the flat pallet 3A from the positional relationship between the two fork holes 5.

[0039] If pallet 3 is the one in Fig. 3B, the controller 15 extracts, for example, a horizontal line corresponding to the pallet portion 6 and vertical lines corresponding to the posts 7 based on the detection point cloud data, and calculates the position and state of the stacking pallet 3B from the positional relationship between the horizontal line and the vertical lines.

[0040] In addition, the controller 15 determines that the estimation of the position and state of the pallet 3 is unsuccessful depending on the deviation of the acquisition point cloud data matching the reference point cloud data by using features that the deviation of the acquisition point cloud data matching the reference point cloud data increases when the matching between the acquisition point cloud data and the reference point cloud data is unsuccessful.

[0041] After executing step S107 or step S109, the controller 15 establishes a travel route from the forklift 2 to the unloading position based on the position and state of the pallet 3 to be unloaded (step S110). Subsequently, the controller 15 controls the drive unit 14 so that the forklift 2 travels along the travel route to the unloaded position (step S111).

[0042] The pallet detection unit 20 executes step S101. The reference point cloud selection unit 21 executes step S102. The matching processor 22 executes steps S103 to S105. The matching evaluation unit 23 executes step S106. The position / state determination unit 24 executes steps S107 to S109. The travel control unit 25 executes steps S110 and S111.

[0043] In some cases, fall arresters 51 are provided on both the left and right sides of the tip of a conveyor 50 to prevent the flat pallet 3A placed on the inclined conveyor 50 from falling, as shown in Fig. 6A. In this case, the front surface 3a of the flat pallet 3A is partially covered by the protective devices 51, as shown in Fig. 6B. As a result, the front surface 3a of the flat pallet 3A is not accurately detected when the detection point cloud data is obtained from the laser sensor 12, so that the accuracy in estimating the position and state of the flat pallet 3A decreases. For the sake of simplicity, the conveyor belt 50 is shown in Fig. 6B is not illustrated.

[0044] As in Fig. 7A, the load M is positioned closer to the laser sensor 12 than the stacking pallet 3B when a large load M placed on the pallet section 6 of the stacking pallet 3B protrudes beyond the front surface 3a of the stacking pallet 3B (see A in the figure). As shown in Fig. Therefore, as illustrated in FIG. 7B, when the detection point cloud data Dp is obtained from the laser sensor 12, the side edges E of the load M may be mistakenly detected as the posts 7 of the stacking pallet 3B. In this case, the accuracy in estimating the position and state of the pallet 3B decreases.

[0045] To solve this problem, in the above embodiment, the reference point cloud data for a plurality of types of pallets 3 are acquired in advance and stored in the memory 13. Upon detecting the pallet 3, the pallet type 3 is identified using the camera 11. Then, from items of the reference point cloud data of the plurality of types of pallets 3 stored in the memory 13, the reference point cloud data corresponding to the identified type of pallet 3 is selected.

[0046] For example, if pallet 3 is flat pallet 3A and flat pallet 3A is placed on the platform of a normal truck (see Fig. 3A), the reference point cloud data of the front surface 3a of the flat pallet 3A is selected. If the pallet 3 is a flat pallet 3A and the flat pallet 3A is placed on the conveyor 50, which is equipped with protective devices 51 to prevent the pallet 3A from falling (see Fig. 6A and Fig. 6B), the reference point cloud data for the front surface 3a of the flat pallet 3A including the two protective devices 51 are selected. If the pallet 3 is the stacking pallet 3B (see Fig. 3B and Fig. 7A), the reference point cloud data of the front surface 3a of the stacking palette 3B are selected.

[0047] Subsequently, the detection point cloud data of the pallet 3 obtained by the laser sensor 12 is compared with the selected reference point cloud data to calculate the estimated values ​​for the position and state of the pallet 3. By thus matching the detection point cloud data of the pallet 3 with the reference point cloud data corresponding to the type of the pallet 3, the pallet detection device 10 becomes more resistant to external disturbances, and erroneous estimation of the position and state of the pallet 3 is less likely even if the type of the pallet 3 used is changed. This improves the accuracy of estimating the position and state of the pallet 3 regardless of the types of pallets 3. As a result, the forklift 2 can be moved to the unloading position with high accuracy.

[0048] In the above embodiment, the estimated values ​​of the position and state of the pallet 3, which are obtained by comparing the detection point cloud data obtained from the laser sensor 12 with the selected reference point cloud data, are estimated, and the position and state of the pallet 3 are determined based on the estimation result. This further improves the accuracy of estimating the position and state of the pallet 3.

[0049] In the above embodiment, the degree of agreement between the detection point cloud data acquired by the laser sensor 12 and the selected reference point cloud data is calculated, and it is determined whether the degree of agreement between the detection point cloud data and the reference point cloud data is equal to or greater than a threshold value. This allows the estimated values ​​of the position and state of the pallet 3 to be easily and accurately estimated.

[0050] In the above embodiment, when the degree of agreement between the detection point cloud data acquired by the laser sensor 12 and the selected reference point cloud data is equal to or greater than the threshold value, the estimated values ​​of the position and state of the pallet 3 obtained by matching the detection point cloud data with the reference point cloud data are determined as the position and state of the pallet 3. This more reliably improves the accuracy of estimating the position and state of the pallet 3.

[0051] In the above embodiment, when the matching degree between the detection point cloud data acquired by the laser sensor 12 and the selected reference point cloud data is below the threshold, the elements of the detection point cloud data that match the reference point cloud data are extracted from the detection point cloud data, and the position and state of the pallet 3 are estimated based on the elements of the detection point cloud data that match the reference point cloud data. Thus, it can be verified whether the matching between the detection point cloud data and the reference point cloud data was indeed unsuccessful.

[0052] In the above embodiment, the pallet type 3 can be easily and inexpensively identified using the camera 11, which captures an image of the pallet 3. Furthermore, the detection point cloud data and the reference point cloud data can be easily aligned by irradiating the pallet 3 with a laser to obtain the detection point cloud data.

[0053] In the above embodiment, the reference point cloud data for a state where the pallet 3 is partially covered by the protective devices 51 is stored in advance in the memory 13. This improves the accuracy in estimating the position and state of the pallet 3 even when the pallet 3 is partially covered by the protective devices 51 when the pallet 3 is placed on the conveyor 50 provided with the protective devices 51 to prevent the pallet 3 from falling.

[0054] The present invention is not limited to the above-described embodiment. For example, in the above embodiment, when the degree of agreement between the detection point cloud data acquired by the laser sensor 12 and the selected reference point cloud data is below the threshold, the position and state of the pallet 3 are estimated based on the elements of the detection point cloud data acquired by the laser sensor 12 that match the reference point cloud data, but the present invention is not limited to this. For example, the position and state of the pallet 3 can be estimated by directly using the detection point cloud data acquired by the laser sensor 12.

[0055] In the above embodiment, the estimated values ​​of the position and state of the pallet 3 are estimated by determining whether the degree of agreement between the detection point cloud data acquired by the laser sensor 12 and the selected reference point cloud data is equal to or greater than the predetermined threshold; however, the present invention is not limited to this. For example, the estimated values ​​of the position and state of the pallet 3 can be estimated by matching the detection point cloud data with the reference point cloud data.

[0056] In the above embodiment, the position and state of the pallet 3 are detected when the pallet 3 is to be unloaded by the forklift 2, but the present invention is also applicable to a case where the position and state of the pallet 3 are detected in working environments other than the cargo handling by the forklift 2. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] JP 2021-042070

[0002]

Claims

[1] A pallet detection device (10) that detects a position and a state of a pallet (3), the pallet detection device (10) comprising: a memory (13) configured to store items of pre-obtained reference data, the items of reference data corresponding to a plurality of types of pallets (3); a detection unit (11, 20) configured to determine a pallet type (3); a selection unit (21) configured to select reference data corresponding to the pallet type (3) detected by the detection unit (11, 20) from the items of reference data of the plurality of types of pallets (3) stored in the memory (13); a data acquisition unit (12) configured to acquire detection data of the pallet (3); and a comparison processor (22) configured to calculate estimated values for a position and a state of the pallet (3) by comparing the detection data obtained by the data acquisition unit (12) and the reference data selected by the selection unit (21). [2] Pallet detection device (10) according to claim 1, further comprising: an evaluation unit (23) configured to evaluate the estimated values of the position and state of the pallet (3) calculated by the comparison processor (22); and a position / state determination unit (24) configured to determine the position and state of the pallet (3) based on a result of the evaluation by the evaluation unit (23). [3] Pallet detection device (10) according to claim 2, wherein the comparison processor (22) calculates the estimated values of the position and state of the pallet (3) by comparing the detection data and the reference data, and calculates a degree of agreement between the detection data and the reference data, and the evaluation unit (23) evaluates the estimated values of the position and the state of the pallet (3) by determining whether the degree of agreement between the detection data and the reference data is equal to or greater than a predetermined threshold value. [4] The pallet recognition device (10) according to claim 3, wherein the position / state determination unit (24) determines the estimated values of the position and state of the pallet (3) calculated by the matching processor (22) when the evaluation unit (23) determines that the degree of agreement between the detection data and the reference data is equal to or greater than the threshold value, and estimates the position and state of the pallet (3) based on the detection data acquired by the data acquisition unit (12) when the evaluation unit (23) determines that the degree of agreement between the detection data and the reference data is less than the threshold value. [5] The pallet recognition device (10) according to claim 4, wherein the position / state determining unit (24) extracts pieces of the detection data that match the reference data from the detection data acquired by the data acquisition unit (12), and estimates the position and state of the pallet (3) based on the pieces of the detection data that match the reference data when it is determined that the degree of match between the detection data and the reference data is less than the threshold. [6] Pallet detection device (10) according to one of claims 1 to 5, wherein the detection unit (11, 20) comprises a camera (11) which takes an image of the pallet (3) and determines the type of the pallet (3) on the basis of image data from the camera (11), the data acquisition unit (12) irradiates the pallet (3) with a laser and receives reflected light from the laser to detect a distance to the pallet (3) and acquire detection point cloud data, and the memory (13) stores elements of the reference point cloud data corresponding to the plurality of types of palettes (3). [7] A pallet recognition device (10) according to any one of claims 1 to 6, wherein one of the items of reference data of the plurality of types of pallets (3) is reference data for a state in which the pallet (3) is partially covered by another object.

Images