Forklift and forklift fleet

Abstract

To provide a forklift and a forklift fleet that can improve work efficiency by recognizing the arrangement of each pallet and shortening the time required for processing even when there is a plurality of pallets.SOLUTION: A forklift comprises a depth distance detection unit 17 mounted on the side of a vehicle body 11 to detect the depth distance of a pallet 30 located on the side of the vehicle body 11, and a calculation unit 23 that acquires placement information of the pallet 30 based on a group of depth distance data consisting of a plurality of depth distances detected while traveling. The calculation unit 23 detects the type of the pallet 30 by comparing the depth distance data set with a pre-set template 41, and acquires a relative position and relative orientation of the pallet 30 with respect to the vehicle body 11 as placement information.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to forklift trucks and fleets of forklift trucks. [Background technology]

[0002] In recent years, many forklifts that travel unmanned to transport loads have been proposed. For example, Patent Document 1 proposes an autonomous forklift equipped with an object detection unit consisting of a sensor unit and a shape recognition unit, in which the sensor unit is attached to the vehicle body and acquires a three-dimensional point cloud ahead, and the shape recognition unit recognizes the shape and position of an object located ahead from the three-dimensional point cloud acquired by the sensor unit. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2022-34408 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional self-driving forklifts recognize the shape and position of objects such as pallets by driving the vehicle so that the objects are positioned in front of the vehicle when picking up an object. This required the forklift to move forward to recognize each object, which took time to recognize and process multiple pallets and other objects.

[0005] Therefore, an object of the present invention is to provide a forklift and a group of forklifts that can reduce the time required to recognize and process the arrangement of each three-dimensional object even when there are multiple three-dimensional objects. [Means for solving the problem]

[0006] The forklift of the present invention, which solves the above-mentioned problems, comprises a depth distance detection unit attached to the side of the vehicle body to detect the depth distance of a three-dimensional object present to the side of the vehicle body, and a calculation unit that acquires position information of the three-dimensional object based on a group of depth distance data consisting of multiple depth distances detected while traveling, and the calculation unit is configured to acquire the relative position and relative orientation of the three-dimensional object with respect to the vehicle body as position information based on the group of depth distance data.

[0007] With such a forklift, the depth distance of a three-dimensional object located to the side of the vehicle body can be detected while the forklift is moving using a depth distance detection unit, and the calculation unit can obtain the relative position and orientation of the three-dimensional object with respect to the vehicle body as placement information based on a group of depth distance data containing multiple depth distances.

[0008] According to the present invention, a forklift can acquire the relative position and orientation of a three-dimensional object, such as a cargo pallet, simply by passing near the object while traveling, and can continuously acquire and simultaneously recognize the position information of each object, whether there is one or multiple objects. Therefore, there is no need to travel to a position directly opposite each three-dimensional object to recognize its position and orientation, and the time required to recognize and process each three-dimensional object can be significantly reduced. Furthermore, since the forklift can recognize the position and orientation of each three-dimensional object as long as it can travel, various processes such as transportation and cleanup can be efficiently performed without any limit on the number of three-dimensional objects, even if they are arranged in a disorderly manner.

[0009] Furthermore, since the depth distance detection unit is attached to the side of the forklift, rather than the front, the depth distance can be detected even when a load is placed on the front fork. Therefore, the position and orientation of a three-dimensional object can be recognized even while the load is being transported, improving processing efficiency. Therefore, according to the present invention, a forklift can be provided that can recognize the position of each three-dimensional object and reduce the time required to process it, even when there are multiple three-dimensional objects.

[0010] The forklift of the present invention includes a vehicle body detection unit that detects the position and orientation of the vehicle body within a predetermined area when detecting the depth distance, and the calculation unit can obtain the absolute position and orientation of the three-dimensional object within the predetermined area as location information based on the relative position and relative orientation as well as the vehicle body position and orientation. In this way, the absolute position and absolute orientation of the three-dimensional object within the predetermined area can be obtained as location information, making it easy to grasp the location of the three-dimensional object.

[0011] The forklift of the present invention preferably includes a memory unit that stores the position and orientation of the vehicle body along with the relative position and orientation, a travel drive unit that can drive the vehicle body forward, backward, turning, and rotating, and a travel control unit that controls the travel drive unit, and after traveling, the travel control unit generates a travel path from the current position to a predetermined proximity position relative to the three-dimensional object based on the position and orientation of the vehicle body in the memory unit and the placement information of the three-dimensional object, and controls the travel drive unit to travel along the travel path to the predetermined proximity position.

[0012] This allows the vehicle to acquire location information for three-dimensional objects while driving, and store the vehicle body position and orientation along with the location information in a memory unit.After driving, a travel route from the current position to a predetermined proximity position to the three-dimensional object is generated based on this information, and the driving unit is controlled to drive from the current position to the predetermined proximity position.This allows the vehicle to detect multiple three-dimensional objects at once, and then drive sequentially to the predetermined proximity position of each three-dimensional object, efficiently processing each three-dimensional object.

[0013] The travel control unit preferably controls the travel drive unit so that the vehicle body travels along a preset reference path during the detection period of the depth distance detection unit, and controls the travel control unit so that after the detection period, the vehicle body travels along the reference path from its current position to a detection position where the depth distance data group was detected, turns at the detection position, and travels to a predetermined approach position. This improves the movement accuracy of the vehicle body, allowing it to accurately travel to the predetermined approach position of each three-dimensional shape part.

[0014] In the present invention, the three-dimensional object may be a loading pallet, and the predetermined approach position is preferably the insertion position of the fork relative to the pallet insertion opening. Preferably, a shape detection unit is provided on a side of the vehicle body to detect the two-dimensional planar shape of the three-dimensional object present on that side, and the calculation unit is configured to determine a candidate range for the pallet insertion opening from the two-dimensional planar shape and to extract the depth distance data group from the depth distance of each part in the candidate range.

[0015] In this way, the shape detection unit detects the two-dimensional planar shape of a three-dimensional object located on the side of the vehicle body to determine a candidate range for the pallet insertion port, and a group of depth distance data is extracted from the depth distance within this candidate range, thereby making it possible to efficiently and accurately obtain the relative position and relative orientation of the pallet insertion port.

[0016] The calculation unit can detect the type of pallet by comparing the group of depth distance data with a preset template, and can also obtain the relative position and relative orientation of the pallet facing the end face and insertion slot. This makes it possible to obtain the type of pallet by comparing the group of depth distance data with the preset template, and further obtain the relative position and relative orientation of the pallet facing the end face and insertion slot based on the group of depth distance data, making it easy to recognize the insertion slot of the pallet and to properly insert the forks of a forklift into the pallet insertion slot.

[0017] The calculation unit may fit a virtual line to the depth distance data group, select three approximate portions along the virtual line, and acquire the type of pallet from the template that best matches the three approximate portions when the three approximate portions are the end faces of the pallet beams and the space between the approximate portions is the insertion slot. This makes it possible to more accurately acquire the relative position and orientation of the pallet with respect to the end face and insertion slot.

[0018] The calculation unit may obtain information about the pallet or the cargo on the pallet from a code included in the two-dimensional planar shape detected by the shape detection unit. In this way, information about the pallet or the cargo on the pallet is obtained from the code included in the two-dimensional planar shape detected by the shape detection unit, so that a large amount of information can be obtained to perform processes such as transporting and putting away the pallet, thereby further improving work efficiency.

[0019] The depth distance detection unit can detect groups of depth distance data at multiple height positions, and the calculation unit can obtain the relative position and relative orientation of the pallet end face and insertion slot based on the multiple groups of depth distance data at multiple height positions. This allows for more accurate determination of the relative position and relative orientation. In this case, the calculation unit may obtain the relative position and relative orientation for each of multiple pallets stacked one above the other based on the multiple groups of depth distance data. In this way, multiple pallets can be recognized efficiently.

[0020] The shape detection unit and depth distance detection unit are composed of a common depth camera, and the calculation unit determines a candidate range for the pallet insertion port from the two-dimensional planar shape acquired by the depth camera, acquires depth distance data groups within the candidate range at multiple heights using the depth camera, fits horizontal virtual straight lines to each of the multiple depth distance data groups, determines positions that match multiple pre-created template pallets and scores them, recognizes the pallet as a template pallet if there are a predetermined number or more depth distance data groups with scores above a threshold, and can calculate the coordinates of the pallet from the position of the forklift at the time the depth distance data groups were acquired by the depth camera.

[0021] The travel control unit may determine a transport route from the insertion position to a preset pallet destination position and control the travel drive unit to travel along the transport route. The calculation unit may detect multiple pallets during a detection period by the depth distance detection unit, store the relative position and relative orientation for each pallet together with the position and orientation of the vehicle body, and, after the detection period, move sequentially to the insertion positions of the multiple pallets based on these to transport each pallet to the pallet destination position.

[0022] The calculation unit may detect a group of two-dimensional planar shape and depth distance data to the side of a predetermined pallet transport destination position while traveling, and determine whether a pallet can be placed at the predetermined pallet transport destination position based on the group of two-dimensional planar shape and depth distance data.

[0023] The forklift fleet of the present invention includes a plurality of forklifts as described above, and is configured so that the relative position and orientation of one of the forklifts to a pallet, which are stored in the memory unit of the other forklifts along with the vehicle body position and orientation, can be used by the travel control units of the other forklifts via a network to transport the pallet. In this way, multiple pallets can be efficiently transported to their destination locations by multiple forklifts. [Effects of the Invention]

[0024] According to the present invention, it is possible to provide a forklift and a group of forklifts that can reduce the time required to recognize and process the arrangement of each three-dimensional object even when there are multiple three-dimensional objects. [Brief explanation of the drawings]

[0025] [Figure 1] 1 is a schematic side view of a forklift according to an embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram showing the configuration of the forklift shown in FIG. 1. [Figure 3] FIG. 10 is a diagram illustrating a travel path of a forklift during a detection period according to an embodiment of the present invention. [Figure 4A] 1 is a first half of a flowchart illustrating the operation of a forklift according to an embodiment of the present invention. [Figure 4B] FIG. 4B shows the second half of the flowchart shown in FIG. 4A. [Figure 5] 10A and 10B are diagrams illustrating a state in which a pallet is detected by a forklift according to an embodiment of the present invention. [Figure 6] 10(a) and 10(b) are diagrams illustrating the traveling state when a forklift according to an embodiment of the present invention detects a plurality of pallets. [Figure 7] 1A and 1B are diagrams illustrating a process for detecting a pallet using a forklift according to an embodiment of the present invention, in which (a) shows a candidate range and (b) shows a slice area. [Figure 8] FIG. 10 is a diagram illustrating the process of detecting a pallet using a forklift according to an embodiment of the present invention, where (a) shows the state in which a group of depth distances is fitted to a virtual straight line, and (b) shows the state in which it is matched with a template. [Figure 9] 10A and 10B are diagrams illustrating a traveling state in which the forklift according to the embodiment of the present invention moves to a predetermined approach position. [Figure 10] 1 is a diagram illustrating a traveling state in which a forklift truck according to an embodiment of the present invention transports a pallet to a destination position. FIG. [Figure 11] 10A and 10B are schematic diagrams illustrating a modified example of the embodiment of the present invention, illustrating the operation when the end face of the pallet having the insertion opening does not face the reference path side. [Figure 12] FIG. 10 is a perspective view showing a modified example of the embodiment of the present invention, illustrating the operation when the depth distance detection unit is a 2D laser scanner. [Figure 13] FIG. 10 is a schematic diagram illustrating a modified example of the embodiment of the present invention, illustrating the operation when a plurality of pallets are arranged so as to overlap one another on the side of the reference path. DETAILED DESCRIPTION OF THE INVENTION

[0026] An embodiment of the present invention will be described in detail below with reference to the drawings. In this embodiment, an example will be described in which an autonomous forklift detects and transports a pallet as a three-dimensional object within a predetermined area such as a warehouse.

[0027] FIG. 1 is a schematic perspective view showing a forklift truck according to this embodiment, and FIG. 2 is a block diagram showing the configuration thereof. The forklift 10 comprises a body 11, an operation panel 12 for various inputs, a travel drive unit 13 capable of driving the body 11 forward, backward, turning and rotating, a depth camera 17 as a depth distance detection unit attached to the side of the body 11, a color camera 18 as a shape detection unit, a laser sensor 19 as a body detection unit that detects the position and orientation of the body 11 when detected by the depth camera 17, a calculation unit 23 that processes the detection information of the depth camera 17, the color camera 18 and the laser sensor 19 and generates control commands for each unit, and a memory unit 25 that stores the detection information and processing results of the depth camera 17 as well as the detection information and processing results of the laser sensor 19.

[0028] The calculation unit 23 shown in FIG. 2 is a so-called CPU and includes a microcomputer, etc. The storage unit 25 is composed of a volatile memory such as a DRAM and a non-volatile memory such as a flash memory and / or an HDD. The travel drive unit 13, the fork drive unit 11d, the operation panel 12, and monitoring sensors such as the depth camera 17, the color camera 18, the laser sensor 19, and an obstacle sensor, all connected to the calculation unit 23, are connected via an interface circuit (I / O circuit) not shown. The calculation unit 23, the storage unit 25, the I / O circuit, etc. may be integrated into a single board or computer. In the calculation unit 23, the travel control unit 11, the fork drive unit 11d, and programs and applications (hereinafter referred to as "programs") for candidate area extraction, pallet position detection, vehicle position estimation, obstacle detection, etc., which will be described in the flowcharts below, are stored in the storage unit 25. The program stored in the memory unit 25 is executed by the calculation unit 23 and the memory of the memory unit 25, and the traveling control unit 15, traveling drive unit 13, fork control unit 11e, fork drive unit 11d, etc. are controlled based on signals detected by the monitoring sensors and control signals input via the operation panel.

[0029] The vehicle body 11 has a vehicle body main body 11a, a mast 11b provided vertically at the front of the vehicle body main body 11a, a fork 11c supported on the mast 11b so as to be movable up and down and protruding forward, and a fork drive unit 11d that drives the fork 11c up and down. The fork 11c is configured to be capable of loading and unloading operations by being controlled by a fork control unit 11e based on the processing results of each detection information.

[0030] The travel drive unit 13 is provided at the bottom of the vehicle body 11a, has a plurality of wheels 13a that are partially or entirely driven by travel motors 13b, and is configured to be steerable by a steering motor 13c. By adjusting the rotation direction and rotation speed of the wheels 13a and the steering direction and steering speed by the steering motor 13c, the vehicle body 11 can move forward, backward, and turn, and can also turn on the spot, rotating in the same position.

[0031] The traveling control unit 15 can control the traveling drive unit 13 to make the vehicle body 11 travel along a predetermined route, a short-distance or long-distance route calculated by calculation, etc., and can make the vehicle body 11 travel along various routes by controlling the rotation direction and rotation speed of the wheels 13a, the steering direction and steering speed, etc.

[0032] In the travel control unit 15 of this embodiment, for example, during a detection period in which a pallet 30 is detected and its location information is acquired, the travel drive unit 13 may be controlled so that the vehicle body 11 travels along a predetermined reference path 33 set in advance within a predetermined area 31, as shown in Fig. 3. Furthermore, after the detection period, i.e., after pallet detection has been interrupted or terminated at an appropriate point in time, the travel control unit 15 generates a travel path 34 from the current position, i.e., the position and orientation of the vehicle body 11 at the end of the detection period, to a predetermined proximity position 32 relative to the pallet 30 based on the detected location information of the pallet 30, and controls the travel drive unit 13 to travel along the travel path 34.

[0033] This movement path 34 may be, for example, a path that travels the shortest distance from the current position to the predetermined approach position 32, but in this embodiment, as shown in Figure 9, a path is set that travels along the reference path 33 from the current position to the detection position 36 where the placement information of the pallet 30 is detected, and then turns around on the spot at the detection position 36 to travel to the predetermined approach position 32.

[0034] Furthermore, after picking up the pallet 30 at the predetermined approach position 32, the travel control unit 15 determines a transfer route 35 from the predetermined approach position 32 to a preset pallet destination position 38, and controls the travel drive unit 13 to travel along the transfer route 35. This transfer route 35 may be, for example, the shortest route from the predetermined approach position 32 to the pallet destination position 38, but in this embodiment, as shown in Figure 10, a route is set that returns from the predetermined approach position 32 to the detection position 36 and travels along the reference route 33 to the pallet destination position 38.

[0035] Here, the predetermined approach position 32 is a position where the forklift 10 faces the end face of the pallet 30 where the insertion openings 30a are provided, and where each fork 11c faces directly toward each insertion opening 30a of the pallet 30, and is a position where the protruding direction of the forks 11c is arranged substantially along the axial direction of the insertion openings 30a, or an insertion position 37 where the forks 11c are inserted into the insertion openings 30a. When the predetermined approach position 32 is a position where the forks 11c face directly toward the insertion openings 30a, the forks 11c move forward to the insertion position 37 when picking up the load. The driving control unit 15 controls the driving drive unit 13 to stop the vehicle body 11 or to make a detour based on detection information from various sensors such as an obstacle sensor 16 provided on the vehicle body 11, an IMU such as a 6-axis gyro sensor (not shown), and a distance sensor.

[0036] The depth camera 17 detects the depth distance of each part and is attached to the side of the vehicle body 11, specifically to the side of the mast 11b in this embodiment, facing laterally in a direction perpendicular to the fore-and-aft direction of the vehicle body 11. The depth camera 17 may be installed on one or both sides of the vehicle body 11 and may be installed so that it can be raised and lowered along the mast, or multiple cameras may be installed on the side at intervals in the vertical direction. By being able to detect depth distance data at multiple height positions, it is possible to improve recognition accuracy by detecting the depth distance of, for example, a single pallet 30 at multiple heights. It is also possible to detect multiple pallets 30 stacked one on top of the other. In this embodiment, the depth camera 17 detects depth distances at multiple locations on the pallet 30, specifically at multiple locations on the pallet 30 in the horizontal direction at a predetermined height, and transmits depth distance data consisting of the detected multiple depth distances to the calculation unit 23.

[0037] The color camera 18 detects the two-dimensional planar shape, and in this embodiment, is provided integrally with the depth camera 17 and, like the depth camera 17, is attached to the side of the mast 11b on one or both sides of the vehicle body 11, facing laterally in a direction perpendicular to the fore-and-aft direction of the vehicle body 11. The color camera 18 may be provided so that it can be raised and lowered, like the depth camera, or multiple color cameras 18 may be provided on the side, spaced apart in the vertical direction. When the color camera 18 in this embodiment detects a pallet 30 present to the side of the vehicle body 11 while traveling, it captures a color image of the pallet 30 and transmits this color image data to the calculation unit 23 as a two-dimensional planar shape.

[0038] The color camera 18 captures color image data as a two-dimensional planar shape of the pallet 30 located on the side of the vehicle body 11, either continuously or periodically, or when it detects a pallet 30 on the side of the vehicle body 11, as shown in Figure 7(a), and transmits this to the calculation unit 23, which then determines a candidate range 40 for the insertion port of the pallet 30 from the two-dimensional planar shape. The socket candidate range 40 of this embodiment is a range that includes the two sockets 30a and the end faces 30b of the three digits arranged on both sides of each socket 30a. The depth camera 17 may detect a plurality of depth distances within the candidate range 40 of the insertion slot of the pallet 30 .

[0039] The laser sensor 19 detects a detection position 36 where depth distance data is detected by the depth camera 17, i.e., the position and orientation of the vehicle body 11 at the time of detection. In this embodiment, a position sensor is provided on top of the vehicle body 11, and detects the position and orientation of the vehicle body 11 within a predetermined area 31, such as a logistics warehouse. The position sensor is composed of, for example, a laser scanner and a light receiving unit, and detects the position and orientation of the vehicle body 11 by emitting a laser beam from the laser scanner and receiving the laser beam reflected by reflectors arranged at various locations within the predetermined area 31 with the light receiving unit.

[0040] This laser sensor 19 detects the position and orientation of the forklift 10 within the specified area 31 when it acquires the depth distance of each part within the candidate range 40 of the insertion port of the pallet 30, and transmits this to the calculation unit 23.

[0041] The calculation unit 23 is configured to process the detection information from the operation panel 12, the depth camera 17, the color camera 18, the obstacle detection sensor 16, and the laser sensor 19 to determine an action and generate control commands for each unit based on the determined action. Specifically, the determination of an action here means that the calculation unit 23 executes a program related to determining an action regarding the control of the traveling drive unit 13 and the fork drive unit 11d in the body 11 of the forklift 10 based on the control signal from the operation panel 12, the pallet position recognition signal from the depth camera 17 and / or the candidate range extraction signal from the color camera 18, the obstacle detection signal from the obstacle detection sensor 16, and the vehicle position estimation signal from the laser sensor 19. The calculation unit 23 first recognizes the pallet position from information detected by the depth camera 17 and color camera 18 while traveling. As shown in Fig. 7(a), a candidate range 40 for the insertion slot of the pallet 30 is extracted based on color image data from the color camera 18. From multiple depth distances obtained by the depth camera 17 in the candidate range 40 for the insertion slot of the pallet 30, a group of depth distance data consisting of the depth distances of each part on a virtual plane obtained by slicing the candidate range 40 horizontally at a certain height is obtained, as shown in Fig. 7(b). At this time, multiple groups of depth distance data at multiple height positions may be obtained.

[0042] The type of pallet 30 is detected by comparing the obtained group of depth distance data with a preset template 41. For example, as shown in Figures 8(a) and 8(b), a horizontal virtual line 42 is fitted to the group of depth distance data, three approximate portions 43 along the virtual line 42 are selected, and the three approximate portions 43 are set as the end faces 30b of the beams of the pallet 30 and the gaps between the approximate portions 43 are set as the insertion slots 30a, and the type of pallet 30 can be identified from the template 41 that best matches.

[0043] At the same time, the calculation unit 23 can obtain placement information including the relative position and orientation of the pallet 30 with respect to the vehicle body 11 based on a depth distance data group having a plurality of depth distances. For example, the relative position and orientation of the end face 30b and insertion port 30a of the pallet 30 with respect to the vehicle body 11 may be obtained from the orientation and distance of the depth distance data group fitted to the virtual straight line 42. This allows the calculation unit 23 to recognize the pallet position.

[0044] The calculation unit 23 also estimates the vehicle position from the information detected by the laser sensor 19. The position and orientation of the forklift 10 at the time when the placement information of the pallet 30 is acquired, i.e., at the time when the depth distance data group is acquired, are estimated, and by combining this position and orientation of the forklift 10 with the placement information of the pallet 30, the absolute position and orientation of the pallet 30 within the specified area 31, world coordinates, etc. are acquired.

[0045] The calculation unit 23 of this embodiment recognizes the presence of an obstacle or the like that may hinder the operation of the forklift 10 based on detection information from the obstacle sensor 16 and other sensors. Furthermore, various types of information about the pallet 30 or the cargo on the pallet 30 can be obtained from the code contained in the two-dimensional planar shape detected by the color camera 18. Methods for obtaining such various types of information include QR Code (registered trademark) and recognition of surface printing using image processing. The calculation unit 23 of this embodiment then determines the behavior of the forklift 10 based on the results of recognizing the pallet position, estimating the vehicle position, recognizing obstacles, and the like.

[0046] Furthermore, the calculation unit 23 of this embodiment is provided with a travel control unit 15 that controls the travel drive unit 13 and a fork control unit 11e that controls the fork drive unit 11d. The travel control unit 15 generates travel commands and steering commands based on the action-determined information, and can control the travel motor 13b and the steering motor 13c. Furthermore, the fork control unit 11e generates a fork control command based on the action-determined information, and can control the fork drive unit 11d.

[0047] The memory unit 25 stores pallet arrangement information including the relative position and relative orientation of the pallet 30, vehicle body arrangement including the position and orientation of the vehicle body 11 at the time of detection, arrangement information such as the absolute position and absolute orientation of the pallet 30 or world coordinates, pre-set templates, various pre-set information regarding the pallet 30 and the specified area 31, etc., which can be used by the calculation unit 23 as needed.

[0048] Next, we will explain the operation of using the forklift 10 described above to detect multiple pallets 30 arranged in a predetermined area 31 such as a logistics warehouse and transport them to a predetermined pallet destination position 38. Figures 4A and 4B are flowcharts showing the flow of the operation of detecting and transporting a pallet 30. In Figures 4A and 4B, the display positions of symbol A are consecutive and the display positions of symbol B are consecutive. First, as shown in Fig. 3, the travel control unit 15 controls the travel drive unit 13 so that the forklift 10 travels along a preset reference path 33 within a predetermined area 31 (S11). This period is a detection period for detecting pallets 30 present within the predetermined area 31 and acquiring information such as the placement of the pallets 30.

[0049] As shown in Figure 5, when a pallet 30 is present on the side of the vehicle body 11 while the vehicle is traveling, the color camera 18 captures an image of the pallet 30 on the side of the vehicle body 11 while the vehicle is traveling, and color image data is obtained as a two-dimensional planar shape (S12). The color image data detected during the detection period is transmitted to the calculation unit 23. At this time, as shown in Figure 5, a single pallet 30 may be detected, but multiple pallets 30 may also be detected successively or collectively while the vehicle is traveling.

[0050] The multiple pallets 30 do not need to be arranged side by side in the running direction of the vehicle body 11; for example, as shown in Figure 6(a), the multiple pallets 30 may be arranged diagonally with respect to the running direction of the vehicle body 11 on the reference path 33, or as shown in Figure 6(b), each pallet 30 may be arranged at an arbitrary incline with respect to the running direction of the reference path 33.

[0051] 7(a), calculation unit 23 sets candidate range 40 for the insertion slot of pallet 30 from the color image data acquired by color camera 18 (S13). In this embodiment, candidate range 40 for the insertion slot of pallet 30 may be determined by machine learning based on the color image data. For example, by preparing a recognition model that has learned the shape of the target pallet 30 in advance and inputting an image captured during detection into the learning model, a circumscribed rectangle for the surface including the insertion slot of pallet 30 can be obtained, and this rectangle can be used as candidate range 40.

[0052] If multiple palettes 30 exist in the color image data, multiple candidate ranges 40 are determined. In this embodiment, multiple palettes 30 are arranged horizontally on the left and right, and multiple palettes 30 are stacked vertically. Therefore, multiple candidate ranges 40 are determined in the horizontal and vertical directions for the color image data.

[0053] Meanwhile, the depth camera 17 captures depth images and detects the depth distance of each part. The depth images detected during the detection period are transmitted to the calculation unit 23 (S14). The calculation unit 23 extracts a depth distance data group (candidate point group) consisting of depth distances of multiple positions from the depth distances of each part of the pallet 30 in the candidate range 40 (S15).

[0054] As shown in FIG. 7(b), the calculation unit 23 sets slice areas at a certain height for each candidate range 40 for the insertion slot of each pallet 30, based on the depth distances of each part in the candidate range 40 for the insertion slot of each pallet 30 detected by the depth camera 17, and acquires a group of depth distance data for each slice area. In this embodiment, fine slice areas are set for each candidate range 40, and groups of depth distance data at multiple heights are acquired. The multiple groups of horizontal depth distance data acquired for each slice area are each projected onto a virtual horizontal plane, as shown in FIG. 8(a), and virtual line fitting is performed to fit a virtual line 42 on the virtual horizontal plane (S16).

[0055] Furthermore, the depth distance data group to which the virtual line 42 has been fitted is compared with various pre-created palette templates as shown in Figure 8(b) to find the matching position, score it, and perform template matching to score all slice areas in the same way (S16).

[0056] Meanwhile, the laser sensor 19 detects the position and orientation of the forklift 10 within the specified area 31 at the time the depth distance is acquired by the depth camera 17, and transmits this to the calculation unit 23, thereby acquiring vehicle body positioning information (S17).

[0057] If template matching finds that there are a predetermined number or more positions within candidate range 40 where the score is equal to or greater than the threshold, calculation unit 23 determines that the detected pallet 30 is a pallet of template 41, and acquires the type of pallet. At the same time, the relative orientation of end face 30b of pallet 30 where insertion slot 30a is provided with respect to vehicle body 11 is acquired from virtual straight line 42. Furthermore, the relative positions of end face 30b of the girders of pallet 30 and insertion slot 30a with respect to vehicle body 11 are acquired from a depth distance data group fitted to virtual straight line 42. Then, calculation unit 23 combines the relative orientation and relative position with vehicle body arrangement information to acquire pallet arrangement information (S18).

[0058] In this way, the detection of the relative orientation and relative position of the pallet 30 with respect to the vehicle body 11 is performed sequentially for each pallet 30 detected during the detection period for detecting the pallet 30, and the pallet placement information including the relative position and relative orientation for each pallet 30 obtained is recorded in the memory unit 25 together with vehicle placement information including the position and orientation of the vehicle body 11 (S19).

[0059] The detection period continues, and it is determined whether the preset pallet detection period has been interrupted or ended (S20). If it has been determined that the period should be continued, the pallet travels along the reference path and the detection period continues.

[0060] On the other hand, after the pallet detection period is interrupted or ended, each pallet 30 is sequentially unloaded based on the pallet location information of each pallet 30 recorded in the memory unit 25 and the vehicle body location information of the vehicle body 11 recorded together with this pallet location information, and the pallet is transported to a predetermined pallet destination position.

[0061] First, at the position where detection of the pallet 30 was interrupted or ended, the laser sensor 19 acquires the current position, including the current position and orientation of the vehicle body, and transmits this to the travel control unit 15 (S21). The travel control unit 15 generates a travel path 34 from the current position to a predetermined approach position 32 relative to the pallet 30 (S21). As shown in Fig. 9, by controlling the travel drive unit 13, the forklift 10 travels from the current position to a detection position 36 along a reference path 33, and then turns around on the spot at the detection position 36 to travel to the predetermined approach position 32 (S22).

[0062] The forklift 10 is turned around on the spot at the detection position to correspond to the relative orientation of the pallet 30, and is brought directly face-to-face with the end face of the pallet 30 where the insertion opening 30a is provided, thereby reaching the predetermined approach position 32 (S23). Then, each fork 11c is placed in a position corresponding to the insertion opening 30a, and is moved forward to insert position 37, where the pallet 30 is picked up (S24).

[0063] Thereafter, the travel control unit 15 generates a transfer path 35 from the insertion position 37 to a preset pallet destination position 38 (S25), and the travel drive unit 13 is controlled to transfer the pallet 30 along the transfer path 35 (S26). The transfer path 35 is set so that the forklift 10 travels to the pallet destination position 38, for example, by traveling along the reference path 33 from the predetermined approach position 32 to near the pallet destination position 38, and then turning around on the spot at that position. The forklift 10 travels until it reaches the pallet destination position 38 (S27), and the pallet 30 is unloaded at the pallet destination position 38 (S28).

[0064] When the forklift 10 travels along the transport path 35 and passes by the side of the pallet destination position 38, the calculation unit 23 detects the two-dimensional planar shape and depth distance data to determine whether a pallet 30 can be placed at the pallet destination position 38, and determines whether the next pallet 30 can be transported.

[0065] The same operation is repeated for the multiple pallets 30 detected during the detection period, and the pallets 30 are transported sequentially. When the transport of all pallets 30 detected during the travel period has been completed (S29), a determination is made as to whether to continue the operation of detecting the pallets 30 and transporting them to the pallet transport destination position 38 (S30). If the determination is to continue, the detection period is restarted, and the forklift 10 begins traveling along the reference path 33 (S11).

[0066] According to the forklift 10 of this embodiment as described above, the depth distance of the pallet 30 located to the side of the vehicle body 11 is detected while traveling using the depth camera 17, and the calculation unit 23 obtains the relative position and relative orientation of the pallet 30 with respect to the vehicle body 11 as placement information based on a group of depth distance data having multiple depth distances.

[0067] The forklift 10 can acquire the location information of the pallet 30 simply by passing near the pallet 30 while traveling, and whether there is one or multiple pallets 30, the location information of each pallet 30 is continuously acquired and recognized all at once while traveling. Here, there is no need to travel to a position directly opposite each pallet 30 to recognize the position and orientation of the pallet 30, and it is possible to significantly reduce the time required to recognize and process each pallet 30. Furthermore, since the forklift 10 can recognize the position and orientation of each pallet 30 as long as it can travel, various processes such as transportation and clearing can be performed efficiently without any limit on the number of pallets 30, even if they are arranged in a disorderly manner.

[0068] Furthermore, because the depth camera 17 is attached to the side of the forklift 10, rather than the front, the depth distance can be detected by the depth camera 17 even when a load is placed on the front fork 11c. Therefore, the position and orientation of other pallets 30 can be recognized even while a load is being transported, improving processing efficiency. For example, it is possible to check whether the pallet destination position 38 is empty while a load is being transported, or to transport a pallet 30 stacked on top of another pallet 30. Even when there are multiple pallets 30, the time required to recognize the position of each pallet 30 and process it can be reduced.

[0069] The forklift 10 of this embodiment is equipped with a laser sensor 19 that detects the position and orientation of the vehicle body 11 within the specified area 31 when detecting the depth distance, and the calculation unit 23 can obtain the absolute position and orientation of the pallet 30 within the specified area 31 and world coordinates as placement information based on the relative position and relative orientation as well as the position and orientation of the vehicle body 11, so the placement of the pallet 30 can be easily grasped.

[0070] The forklift 10 of this embodiment is equipped with a memory unit 25 that stores the position and orientation of the vehicle body 11 along with the placement information of the pallets 30, a travel drive unit 13, and a travel control unit 15. After traveling during the detection period, the travel control unit 15 generates a travel path 34 from the current position to a predetermined approach position 32 based on the position and orientation of the vehicle body 11 and the placement information of the pallets 30 stored in the memory unit 25, and the travel drive unit 13 and the travel control unit 15 travel along the travel path 34 to the predetermined approach position 32. Therefore, during traveling during the detection period, multiple pallets 30 can be detected at once, and then the forklift 10 can travel sequentially to the predetermined approach position 32 of each three-dimensional object, allowing efficient processing of each pallet 30.

[0071] In the forklift 10 of this embodiment, the travel control unit 15 controls the travel drive unit 13 to travel along a preset reference path 33 during the detection period of the depth camera 17, and after the detection period, controls the travel drive unit 13 to travel along the reference path 33 from the current position to a detection position 36 where a group of depth distance data is detected, turn at the detection position 36, and travel to the predetermined approach position 32. This improves the movement accuracy of the vehicle body 11, allowing it to accurately travel to the predetermined approach position 32 of each three-dimensional part. Furthermore, generation of a travel path from the current position to the predetermined approach position 32 and control of the travel control unit 15 can be simplified.

[0072] The forklift 10 of this embodiment is equipped with a color camera 18 as a shape detection unit that detects the two-dimensional planar shape of the pallet 30 located to the side of the vehicle body 11, and determines a candidate range 40 for the insertion opening of the pallet 30 from the two-dimensional planar shape, while the depth camera 17 extracts a group of depth distance data for each part within the candidate range 40. The color camera 18 detects the two-dimensional planar shape of the pallet 30 located to the side of the vehicle body 11 to determine the candidate range 40 for the insertion opening of the pallet 30, and depth distances within this candidate range 40 can be detected intensively by setting multiple slice regions, etc., so that the relative position and relative orientation of the insertion opening 30a of the pallet 30 can be obtained efficiently and accurately.

[0073] The forklift 10 of this embodiment acquires the type of pallet 30 by comparing the depth distance data group with a pre-set template 41, and further acquires the relative position and relative orientation of the end face 30b and insertion port 30a of that type of pallet 30 based on the depth distance data group, thereby accurately recognizing the insertion port 30a of the pallet 30 and being able to properly insert the forks 11c of the forklift 10 into the insertion port 30a of the pallet 30.

[0074] In the forklift 10 of this embodiment, the calculation unit 23 can obtain information about the pallet 30 or the cargo on the pallet 30 from the code contained in the two-dimensional planar shape detected by the color camera 18. This allows for obtaining a large amount of information and processing such as transporting and clearing away the pallet, allowing for more appropriate work to be performed and improving efficiency.

[0075] The above embodiment can be modified as appropriate within the scope of the present invention. For example, in the above embodiment, an example was described in which a single forklift 10 transports multiple pallets 30 in a specified area 31 sequentially to the pallet destination position, but it is also possible to transport multiple pallets 30 to the pallet destination position 38 using a group of forklifts including multiple forklifts 10.

[0076] In this case, the location information of the relative position and orientation with respect to the pallet 30 stored in the memory unit 25 of one of the forklifts 10 together with the position and orientation of the vehicle body 11 may be made available to the travel control unit 15 of the other forklifts 10 via the network. The other forklifts 10 can use this location information to generate a movement route 34 from their current position to a predetermined proximity position 32, move along this route, and pick up the pallet 30. In this way, multiple forklifts 10 can efficiently transport multiple pallets 30 to a pallet destination position 38.

[0077] In the above embodiment, an example has been described in which each pallet 30 in the predetermined area 31 is arranged with its insertion port 30a facing the reference path 33. However, as shown in Fig. 11, for example, the present invention can also be applied to cases in which the end face of the pallet 30 having the insertion port 30a does not face the reference path 33. In that case, the forklift 10 may travel along each side of the pallet 30 while detecting the end face 30b having the insertion port 30a of the pallet 30 on the side of the vehicle body 11.

[0078] In the above embodiment, an example has been described in which a group of horizontal depth distance data is acquired by detecting the horizontal depth distance using the depth camera 17. However, instead of a depth camera, a 2D laser scanner may be installed on the side of the vehicle body 11 so that it can scan vertically, as shown in FIG. 12. In this case, 3D position information can be acquired by scanning the depth distance of each part vertically while traveling. This allows for improved accuracy and resolution compared to when a depth camera is used.

[0079] In the above embodiment, an example was described in which pallets 30 were arranged in a single layer along the reference path 33, but as shown in Figures 13(a) to (c), it may also be a group of pallets arranged in multiple layers on the side of the reference path 33. In this case, as the forklift 10 travels along the reference path 33, as shown in Figure 13(a), it may detect multiple pallets 30 arranged in positions closest to the reference path 33, transport each of them to the pallet transport destination position 38, and then, as shown in Figures 13(b) and 13(c), it may travel along the reference path again while detecting the pallets 30 and transporting the pallets, repeating this process. This allows multiple pallets 30 to be transported sequentially in parallel, making it easy to transport a large number of pallets 30.

[0080] In the above embodiment, a pallet 30 was used as an example of a three-dimensional object, but the present invention can also be applied to various three-dimensional objects that can be transported by a forklift 10, such as nesters, mesh pallets, shelves, etc.

[0081] In the above embodiment, an example was described in which the depth camera 17 is used in common for the depth distance detection unit and the shape detection unit. However, for example, a two-dimensional laser scanner, a stereo camera, a TOF camera, or a monocular camera may also be used. In the case of a two-dimensional laser scanner, three-dimensional information can be obtained by scanning vertically as the vehicle body 11 travels, which can provide improved accuracy and resolution compared to a depth camera. It is also possible to provide the depth distance detection unit and the shape detection unit separately on the side of the vehicle body. In this case, a stereo camera, TOF camera, monocular camera, etc. may be used as the depth distance detection unit, and a monocular camera, three-dimensional laser scanner, etc. may be used as the shape detection unit.

[0082] Furthermore, in the above embodiment, an example was described in which the forklift 10 travels within a predetermined area 31 and detects pallets 30. However, the present invention can also be applied to cases in which the forklift 10 travels within an undefined area and detects pallets. For example, this may be the case when pallets 30 are to be removed from the bed of a truck parked in an open space. In this case, the forklift 10 first travels along an appropriately set travel path, and the depth camera 17 on the side of the vehicle body 11 detects multiple pallets 30 on the bed all at once, captures color images, and obtains the depth distance of each part, thereby allowing each pallet 30 to be removed in the same manner as in the above embodiment. The present invention is not limited to the example in which the forklift 10 detects the pallet 30 while traveling along a predetermined reference path 33, but can also be applied to a case in which the forklift 10 detects the pallet 30 while traveling along an arbitrary traveling path determined by various processes. [Explanation of symbols]

[0083] 10. Forklift 11 Body 11a Body 11b Mast 11c fork 11d Fork drive unit 11e Fork control section 12 Operation panel 13 Travel drive unit 13a wheels 13b Travel motor 13c Steering motor 15 Travel control unit 16 Obstacle Sensor 17 Depth Camera 18 Color Camera 19 Laser Sensor 23 Arithmetic section 25 Memory section 30 palettes 30a outlet 30b end face 31 Predetermined area 32 Predetermined proximity position 33 Reference Path 34 Travel Route 35 Transport Route 36 Detection position 37 Insertion position 38 Pallet destination position 40 candidate ranges 41 templates 42 Imaginary Line 43 Approximation part

Claims

1. A travel drive unit that can drive a vehicle body forward, backward, turning, and rotating; a travel control unit that controls the travel drive unit; a depth distance detection unit attached to a side of the vehicle body and configured to detect a depth distance of a three-dimensional object present on the side of the vehicle body; a calculation unit that detects the depth distance of the three-dimensional object at a plurality of locations while traveling to acquire a group of depth distance data, and acquires, as position information, a relative position and a relative orientation of the three-dimensional object with respect to the vehicle body based on the acquired group of depth distance data; a vehicle body detection unit that detects the position and orientation of the vehicle body when the depth distance is detected; a storage unit that stores the detected position and orientation of the vehicle body together with the location information; Equipped with the calculation unit acquires a relative position and a relative orientation facing the three-dimensional object based on the depth distance data group; When loading or unloading the three-dimensional object, The travel control unit controls the travel drive unit to travel from the current position to the relative position where the vehicle body faces the three-dimensional object based on the position and orientation of the vehicle body and the placement information of the three-dimensional object stored in the memory unit, and turns the vehicle on the spot to face the three-dimensional object.

2. A forklift truck as described in Claim 1, wherein the calculation unit obtains the absolute position and absolute orientation of the three-dimensional object within the specified area as the placement information based on the relative position and relative orientation as well as the position and orientation of the vehicle body.

3. The forklift described in claim 1, wherein the travel control unit generates a travel path from the current position to the relative position that faces the three-dimensional object after travel, and controls the travel drive unit to travel along the travel path to the relative position.

4. the traveling control unit controls the traveling drive unit so that the vehicle body travels along a preset reference path during a detection period of the depth distance detection unit; 4. The forklift according to claim 3, wherein the travel control unit is controlled so that, after the detection period, the forklift travels along the reference path from the current position to a detection position where the depth distance data group is detected, turns around on the spot at the detection position, and travels to the predetermined approach position.

5. 4. The forklift according to claim 3, wherein the three-dimensional object is a loading pallet, and the travel control unit controls the travel drive unit so that the forks are positioned directly opposite an insertion opening of the pallet.

6. a shape detection unit that detects a two-dimensional planar shape of the three-dimensional object present on the side of the vehicle body, The forklift according to claim 5, wherein the calculation unit determines a candidate range for the insertion opening of the pallet from the two-dimensional planar shape, and extracts the depth distance data group from the depth distance of each part in the candidate range.

7. 6. The forklift according to claim 5, wherein the calculation unit detects the type of the pallet by comparing the depth distance data group with a preset template, and acquires the relative position and the relative orientation of the end face of the pallet and the insertion opening of the pallet.

8. 8. The forklift according to claim 7, wherein the calculation unit fits a virtual straight line to the depth distance data group, selects three approximate portions along the virtual straight line, and acquires the type of the pallet from the template that best matches when the three approximate portions are set as end faces of a beam of the pallet and the space between the approximate portions is set as the insertion port.

9. The forklift according to claim 6, wherein the calculation unit acquires information about the pallet or the cargo on the pallet from a code included in the two-dimensional planar shape detected by the shape detection unit.

10. 7. The forklift according to claim 6, wherein the depth distance detection unit is capable of detecting the depth distance data groups at a plurality of height positions, and the calculation unit acquires the relative position and the relative orientation of the end face of the pallet and the insertion opening of the pallet based on the plurality of depth distance data groups at the plurality of height positions.

11. 8. The forklift according to claim 7, wherein the depth distance detection unit is capable of detecting the depth distance data groups at a plurality of height positions, and the calculation unit acquires the relative position and the relative orientation for each of a plurality of pallets stacked vertically based on the plurality of depth distance data groups.

12. the shape detection unit and the depth distance detection unit are formed by a common depth camera, The calculation unit determining a candidate range for the insertion slot of the pallet from the two-dimensional planar shape acquired by the depth camera; acquiring the depth distance data group within the candidate range at a plurality of heights using the depth camera; Fitting a horizontal virtual straight line to each of the plurality of depth distance data groups, and obtaining a score by finding a position that matches a palette of the plurality of templates that have been created in advance; If the palette has a predetermined number or more of the depth distance data groups whose scores are equal to or greater than a threshold, the palette is recognized as a palette of the template; The forklift according to claim 6, wherein the coordinates of the pallet are calculated from the position of the forklift at the time when the depth distance data group is acquired by the depth camera.

13. The forklift according to claim 7, wherein the travel control unit determines a transport route from the insertion position to a preset pallet transport destination position, and controls the travel drive unit to travel along the transport route.

14. 14. The forklift according to claim 13, wherein the calculation unit detects the plurality of pallets during the detection period of the depth distance detection unit, stores the relative position and the relative orientation for each pallet together with the position and orientation of the vehicle body, and, after the detection period, sequentially moves to the insertion positions of the plurality of pallets based on the stored information to transport each pallet to the pallet destination position.

15. 7. The forklift according to claim 6, wherein the calculation unit detects the two-dimensional planar shape and the depth distance data group on the side of a predetermined pallet destination position while traveling, and determines whether the predetermined pallet destination position can accommodate the pallet based on the two-dimensional planar shape and the depth distance data group.

16. A system including a plurality of forklifts according to any one of claims 5 to 15, A group of forklifts in which the relative position and relative orientation with respect to the pallet, which are stored in the memory unit of any one of the forklifts together with the position and orientation of the vehicle body, are used by the travel control unit of another of the forklifts via a network, thereby enabling the other forklifts to transport the pallet.

Images