Cargo Handling System

Abstract

To enable a control device to determine a direction of cargo-handling operation.SOLUTION: A cargo handling vehicle is equipped with a cargo handling device, an external sensor and a control device. The external sensor detects a position of an object using coordinates of a three-dimensional coordinate system. The control device determines whether the cargo handling vehicle is positioned at a left side or a right side of a conveying vehicle, from point cloud data that is an assembly of points representing the position of the object. The control device determines a direction of cargo handling operation on the basis of the determined result of whether the cargo handling vehicle is positioned at the left side or the right side of the conveying vehicle. The direction of cargo handling operation is a direction in which the cargo handling device is operated, which differs depending on whether the cargo handling vehicle is positioned at the left side or the right side of the conveying vehicle.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present disclosure relates to a material handling system. [Background technology]

[0002] The cargo handling vehicle disclosed in Patent Document 1 is equipped with a cargo handling device. The cargo handling vehicle performs cargo handling operations. The cargo handling operations include loading and unloading. In loading, cargo is loaded onto the transport vehicle. In unloading, the cargo loaded onto the transport vehicle is unloaded by the cargo handling vehicle. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-160860 Summary of the Invention [Problem to be solved by the invention]

[0004] The direction in which the loading device is operated may differ depending on whether the loading vehicle is located on the left or right side of the transport vehicle. [Means for solving the problem]

[0005] A loading and unloading system that solves the above problem is a loading and unloading system that includes an external sensor that detects the position of an object in coordinates of a three-dimensional coordinate system, a loading and unloading vehicle equipped with a loading and unloading device, and a control device, wherein the control device determines whether the loading and unloading vehicle is located on the left or right side of a transport vehicle from point cloud data, which is a collection of points representing the position of the object, and determines the loading and unloading operation direction, which is the direction in which the loading and unloading device is operated and which differs depending on whether the loading and unloading vehicle is located on the left or right side of the transport vehicle, depending on whether the loading and unloading vehicle is located on the left or right side of the transport vehicle.

[0006] The control device determines whether the cargo handling vehicle is located on the left or right side of the transport vehicle from the point cloud data. The direction of the cargo handling operation differs depending on whether the cargo handling vehicle is located on the left or right side of the transport vehicle. By determining whether the cargo handling vehicle is located on the left or right side of the transport vehicle, the control device can determine the direction of the cargo handling operation.

[0007] In the above-mentioned cargo handling system, the control device may create a side view of the transport vehicle from the point cloud data, extract the driver's seat from the side view using image recognition, and determine whether the cargo handling vehicle is located on the left or right side of the transport vehicle based on the position of the driver's seat in the side view.

[0008] In the above-mentioned loading and unloading system, the control device may estimate the self-position of the loading and unloading vehicle and determine whether the loading and unloading vehicle is located to the left or right of the transport vehicle based on the positional relationship between the self-position and the stopping position of the transport vehicle. [Effects of the Invention]

[0009] According to the present invention, the control device can determine the direction of the loading and unloading operation. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a schematic diagram of an area in which a loading vehicle operates; [Figure 2] FIG. 2 is a perspective view of a transport vehicle and a loading vehicle. [Figure 3] FIG. 2 is a perspective view of a side shift device. [Figure 4] FIG. 1 is a schematic configuration diagram of a cargo handling vehicle. [Figure 5] 10 is a flowchart showing a cargo handling operation direction determination control. [Figure 6] FIG. 1 is a schematic diagram of a point cloud map. [Figure 7] FIG. [Figure 8] FIG. 1 is a schematic diagram of a work area. [Figure 9] 10 is a flowchart showing a modified example of cargo handling operation direction determination control. [Figure 10] 10 is a diagram showing the positional relationship between a transport vehicle and a cargo handling vehicle when the area of ​​the stopping position is large. FIG. DETAILED DESCRIPTION OF THE INVENTION

[0011] An embodiment of the cargo handling system will be described below. As shown in FIG. 1, a stopping position PS1 is set in the area A1. A plurality of stopping positions PS1 are set. The stopping positions PS1 are arranged at intervals from each other. A transport vehicle 10 stops at the stopping position PS1. The area of ​​the stopping position PS1 is larger than the area of ​​the transport vehicle 10 when viewed from above when stopped at the stopping position PS1. No structures such as pillars are present at the stopping position PS1. The area A1 is, for example, the entire area or part of a place such as a factory, port, airport, commercial facility, or public facility. A cargo handling vehicle 20 is operated in the area A1. The cargo handling vehicle 20 performs loading and unloading. Loading is the work of loading a load C1 placed on a pallet PA1 onto the transport vehicle 10. Unloading is the work of removing the load C1 loaded on the transport vehicle 10 from the transport vehicle 10. In the following description, the front, back, left, right, top and bottom of the transport vehicle 10 refer to the front, back, left, right, top and bottom when the transport vehicle 10 is used as the reference. The left, right, top and bottom of the cargo handling vehicle 20 refer to the left, right, top and bottom when the cargo handling vehicle 20 is used as the reference.

[0012] 2, the transport vehicle 10 is a wing truck. The transport vehicle 10 may be any type of truck, such as a flatbed truck. The transport vehicle 10 includes a driver's seat 11, a front panel 12, a rear door 13, a loading platform 14, wing side panels 16, and a tailgate 17.

[0013] The driver's seat 11 is a position where the driver of the transport vehicle 10 sits. The front panel 12 is located further rearward of the driver's seat 11 on the transport vehicle 10. The front panel 12 is located adjacent to the driver's seat 11. The rear door 13 is located further rearward of the front panel 12 on the transport vehicle 10. The front panel 12 and the rear door 13 are located at a distance from each other in the fore-and-aft direction of the transport vehicle 10. The loading platform 14 extends in the fore-and-aft direction of the transport vehicle 10 between the front panel 12 and the rear door 13. The loading platform 14 has a loading surface 15. The loading surface 15 is the upper surface of the loading platform 14. A load C1 placed on a pallet PA1 is loaded on the loading surface 15. The wing side panel 16 is located between the front panel 12 and the rear door 13. The wing side panel 16 is located so as to be rotatable in the up-and-down direction of the transport vehicle 10 around the center position of the transport vehicle 10 in the vehicle width direction. The wing side panels 16 are provided one on each side in the vehicle width direction of the transport vehicle 10. The tailgates 17 are provided so as to extend in the front-to-rear direction of the transport vehicle 10. The tailgates 17 are provided along the edges of the loading platform 14, which are the edges that extend in the front-to-rear direction of the transport vehicle 10. The tailgates 17 are provided one on each side in the vehicle width direction of the transport vehicle 10.

[0014] <Loading vehicle> The cargo handling vehicle 20 includes a vehicle body 21, drive wheels 22, steering wheels 23, and a cargo handling device 24. The cargo handling device 24 is provided at the front of the vehicle body 21. The cargo handling device 24 includes a mast 25, a lift cylinder 28, a lift bracket 29, two forks 30, and a side shift device 40.

[0015] The mast 25 includes an outer mast 26 and an inner mast 27. The inner mast 27 is provided so as to be able to move up and down relative to the outer mast 26. The lift bracket 29 and the fork 30 move up and down together with the inner mast 27. The lift cylinder 28 moves up and down the inner mast 27. The lift cylinder 28 is a hydraulic cylinder.

[0016] The side shift device 40 moves the forks 30 in the left-right direction of the cargo handling vehicle 20. The side shift device 40 moves the two forks 30 in the left-right direction of the cargo handling vehicle 20 while maintaining the distance between the two forks 30.

[0017] 3, the side shift device 40 is attached to the lift bracket 29. The lift bracket 29 includes two finger bars 31, 32. The two finger bars 31, 32 are provided at an interval in the vertical direction of the cargo handling vehicle 20.

[0018] The side shift device 40 includes a shifter 41 and a shift cylinder 46 . The shifter 41 is provided so as to be movable to the left and right of the cargo handling vehicle 20 relative to the two finger bars 31, 32. The shifter 41 includes two shifter bars 42, 43 and two connecting members 44, 45 that connect the two shifter bars 42, 43. The two shifter bars 42, 43 are provided at an interval in the up-down direction of the cargo handling vehicle 20. The two connecting members 44, 45 are provided at an interval in the left-right direction of the cargo handling vehicle 20. The connecting members 44, 45 connect the ends of the shifter bars 42, 43 to each other. The shifter bars 42, 43 are connected to the forks 30.

[0019] The shift cylinder 46 is a hydraulic cylinder. The shifter 41 moves in the left-right direction of the cargo handling vehicle 20 by supplying and discharging hydraulic oil to the shift cylinder 46. The forks 30 also move in the left-right direction of the cargo handling vehicle 20 together with the shifter 41.

[0020] 4, the cargo handling vehicle 20 includes an external sensor 51, a control device 52, an auxiliary storage device 55, a vehicle control device 56, a travel actuator 59, a cargo handling actuator 60, and a communication device 61. The cargo handling vehicle 20 is a cargo handling system.

[0021] The external sensor 51 detects the position of an object using coordinates in a three-dimensional coordinate system. The external sensor 51 is provided on the upper part of the cargo handling vehicle 20. For example, the external sensor 51 is provided on a head guard of the cargo handling vehicle 20.

[0022] Examples of the external sensor 51 include a millimeter-wave radar, a stereo camera, a ToF (Time of Flight) camera, and a LIDAR (Laser Imaging Detection and Ranging). In this embodiment, a LIDAR is used as the external sensor 51. The external sensor 51 irradiates the surroundings with a laser and receives light reflected from the point where the laser hits, thereby deriving the distance to the point. The point where the laser hits represents a part of the surface of the object. The position of the point can be expressed by coordinates in a polar coordinate system. The coordinates of the point in the polar coordinate system are converted into coordinates in a Cartesian coordinate system. The conversion from the polar coordinate system to the Cartesian coordinate system may be performed by the external sensor 51 or by the control device 52. In this embodiment, it is assumed that the conversion from the polar coordinate system to the Cartesian coordinate system is performed by the external sensor 51. The external sensor 51 derives the coordinates of the point in the sensor coordinate system. The sensor coordinate system is a three-axis Cartesian coordinate system with the external sensor 51 as the origin. The external sensor 51 outputs the coordinates of a plurality of points obtained by irradiating the laser to the control device 52 as point cloud data.

[0023] The control device 52 includes a processor 53 and a storage unit 54. The storage unit 54 includes a random access memory (RAM) and a read-only memory (ROM). The storage unit 54 stores program code or instructions configured to cause the processor 53 to execute processes. The storage unit 54, i.e., a computer-readable medium, includes any available medium accessible by a general-purpose or special-purpose computer. The control device 52 may be configured with a hardware circuit such as an application-specific integrated circuit (ASIC) or a field-programmable gate array (FPGA). The control device 52, which is a processing circuit, may include one or more processors operating according to a computer program, one or more hardware circuits such as an ASIC or FPGA, or a combination thereof.

[0024] The auxiliary storage device 55 stores information that can be read by the control device 52. Examples of the auxiliary storage device 55 include a hard disk drive and a solid state drive. The auxiliary storage device 55 stores an environmental map D1. The auxiliary storage device 55 stores an image recognition model D2.

[0025] The environmental map D1 is information about the physical structure of area A1, such as the shapes of objects present in area A1 and the size of area A1. In this embodiment, the environmental map D1 is data that represents the structure of area A1 using coordinates in a map coordinate system. The map coordinate system is a three-axis Cartesian coordinate system. The map coordinate system is a coordinate system that has an arbitrary point in area A1 as its origin. In the map coordinate system, the horizontal direction is defined by the X-axis and Y-axis, which are perpendicular to each other. The XY plane defined by the X-axis and Y-axis can be said to represent the horizontal plane. In the map coordinate system, the up-down direction is defined by the Z-axis, which is perpendicular to the X-axis and Y-axis. Coordinates in the map coordinate system will be referred to as map coordinates where appropriate. The map coordinate system is a three-dimensional coordinate system that represents three-dimensional positions.

[0026] The control device 52 estimates the self-position of the cargo handling vehicle 20. The control device 52 outputs the self-position of the cargo handling vehicle 20 to the vehicle control device 56. The self-position is the position of the cargo handling vehicle 20 on the environmental map D1. The self-position is a coordinate indicating a point of the cargo handling vehicle 20 in the map coordinate system. The point of the cargo handling vehicle 20 is arbitrary, but can be, for example, the center position of the cargo handling vehicle 20 in the horizontal direction.

[0027] The self-location is estimated by comparing the detection results of the external sensor 51 with the environmental map D1. The control device 52 extracts landmarks from the environmental map D1 that have the same shape as the landmarks obtained from the point cloud data. The control device 52 recognizes the positions of the landmarks from the environmental map D1. The positional relationship between the landmark positions and the cargo handling vehicle 20 can be determined from the detection results of the external sensor 51. Therefore, the control device 52 can estimate the self-location by recognizing the positions of the landmarks. A landmark is an object that has characteristics that can be identified by the external sensor 51. A landmark is a physical structure whose position is unlikely to change. Examples of landmarks include walls and pillars. The self-location may be estimated by combining the self-location estimation using the external sensor 51 with dead reckoning using an internal sensor. The self-location may be estimated by combining the self-location estimation using the external sensor 51 with satellite signals transmitted from GNSS (Global Navigation Satellite System) satellites.

[0028] The vehicle control device 56 has, for example, the same hardware configuration as the control device 52. The vehicle control device 56 has a processor 57 and a storage unit 58. The travel actuator 59 is an actuator that causes the cargo handling vehicle 20 to travel. The travel actuator 59 includes, for example, a motor that rotates the drive wheels 22 and a steering mechanism. The vehicle control device 56 controls the travel actuator 59 while determining its own position to cause the cargo handling vehicle 20 to travel.

[0029] The cargo handling actuator 60 is an actuator that causes the cargo handling vehicle 20 to handle cargo. The cargo handling actuator 60 includes, for example, a motor that drives a pump that supplies hydraulic oil to hydraulic equipment, and a control valve that controls the supply of hydraulic oil. The hydraulic equipment includes the lift cylinder 28 and the shift cylinder 46. The vehicle control device 56 controls the cargo handling actuator 60 to raise and lower the forks 30 and to side-shift the forks 30.

[0030] The cargo handling vehicle 20 travels automatically by controlling a travel actuator 59 by a vehicle control device 56. The cargo handling vehicle 20 automatically handles cargo by controlling a cargo handling actuator 60 by a vehicle control device 56. The cargo handling vehicle 20 is an automatically operated forklift.

[0031] The communication device 61 is a communication device capable of communicating using any wireless communication method, such as wireless LAN, Zigbee (registered trademark), LPWA (Low Power Wide Area), or a mobile communication system. The communication device 61 is capable of transmitting and receiving wireless signals. The communication device 61 demodulates the received wireless signals and outputs the demodulated data to the vehicle control device 56.

[0032] The communication device 61 receives commands from the host control device 71. The host control device 71 is installed in, for example, area A1. The host control device 71 transmits operation data input by a person to the communication device 61. The operation data includes commands for the cargo handling vehicle 20. Examples of commands include a command to move the cargo handling vehicle 20 forward, a command to start loading the cargo handling vehicle 20, and a command to start unloading the cargo handling vehicle 20. The vehicle control device 56 controls the cargo handling vehicle 20 in accordance with the commands from the host control device 71.

[0033] The cargo handling operation direction determination control performed by the control device 52 will be described. The cargo handling operation direction determination control is control for determining the cargo handling operation direction. The cargo handling operation direction is the direction in which the cargo handling device 24 is operated and differs depending on whether the cargo handling vehicle 20 is located on the left or right side of the transport vehicle 10. One example of the cargo handling operation direction is the side shift direction of the forks 30. When the cargo handling vehicle 20 loads cargo, cargoes C1 are loaded sequentially onto the loading platform 14 of the transport vehicle 10 from the front to the rear of the transport vehicle 10. The cargoes C1 are loaded starting from a position closest to the driver's seat 11. At this time, in order to reduce the gap between the cargoes C1 loaded on the loading platform 14, the forks 30 are side-shifted by the side shift device 40. When loading cargo, if the cargo handling vehicle 20 is located on the right side of the transport vehicle 10, the side shift direction is to the right of the cargo handling vehicle 20. When loading cargo and the cargo handling vehicle 20 is located on the left side of the transport vehicle 10, the side shift direction is toward the left of the cargo handling vehicle 20. When the cargo handling vehicle 20 retrieves cargo, it sequentially retrieves cargoes C1 loaded on the loading platform 14 of the transport vehicle 10 from the rear to the front of the transport vehicle 10. If the gap between the cargoes C1 is small, there is a risk of adjacent cargoes C1 interfering with each other. Therefore, to increase the gap between the cargoes C1, the forks 30 are side-shifted by the side shift device 40. When retrieving cargo and the cargo handling vehicle 20 is located on the right side of the transport vehicle 10, the side shift direction is toward the left of the cargo handling vehicle 20. When retrieving cargo and the cargo handling vehicle 20 is located on the left side of the transport vehicle 10, the side shift direction is toward the right of the cargo handling vehicle 20.

[0034] The cargo handling vehicle 20 approaches the transport vehicle 10, for example, based on a command from the upper control device 71. Then, when the distance between the cargo handling vehicle 20 and the transport vehicle 10 becomes less than a predetermined distance, the cargo handling operation direction determination control is started. The predetermined distance is, for example, a distance that can be detected by the external sensor 51 of the transport vehicle 10.

[0035] <Load handling direction determination control> As shown in FIGS. 5 and 6, in step S1, the control device 52 creates a point cloud map PM1. The point cloud map PM1 is created by overlaying point cloud data obtained from the detection results of the external sensor 51. The control device 52 acquires point cloud data from the external sensor 51 multiple times while the loading vehicle 20 is moving. The control device 52 converts the coordinates of each point P1 in the point cloud data from the coordinates of the sensor coordinate system into map coordinates based on its own position. The control device 52 can recognize the origin of the sensor coordinate system in the map coordinate system based on its own position estimated by the control device 52. The control device 52 can recognize the deviation between the coordinate axes of the map coordinate system and the coordinate axes of the sensor coordinate system based on the origin of the sensor coordinate system in the map coordinate system and the deviation between the coordinate axes of the map coordinate system and the coordinate axes of the sensor coordinate system. The control device 52 creates the point cloud map PM1 by overlaying each point P1 converted into map coordinates each time point cloud data is acquired. The point cloud map PM1 is a collection of point cloud data. Compared to the points P1 of the point cloud data, the points P1 of the point cloud map PM1 are denser.

[0036] FIG. 6 shows the point cloud map PM1 obtained by the processing of step S1. Each point P1 included in the point cloud map PM1 represents the map coordinates of an object. For ease of explanation, the points P1 included in the point cloud map PM1 will be classified into a first point P11, a second point P12, a third point P13, and a fourth point P14. The first point P11 is a point P1 obtained by irradiating the gate 17 with a laser. The second point P12 is a point P1 obtained by irradiating the driver's seat 11 with a laser. The third point P13 is a point P1 obtained by irradiating the pallet PA1 and the load C1 loaded on the loading surface 15 with a laser. The fourth point P14 is a point P1 that does not fall into any of the first point P11, the second point P12, and the third point P13.

[0037] As shown in Figures 5 and 7, in step S2, the control device 52 creates a side view IM1 from the point cloud map PM1. Since the point cloud map PM1 is a collection of point cloud data, it can be said that the side view IM1 is created from the point cloud data. The side view IM1 is a view in which point P1 is projected in the vehicle width direction of the transfer vehicle 10. This makes it possible to make the point cloud map PM1 two-dimensional, so that the side view IM1 can be treated as image data. An example of processing performed by the control device 52 when creating the side view IM1 will be described.

[0038] The control device 52 calculates the normal vector of each point P1. A normal vector is a vector oriented in a direction perpendicular to a plane surrounded by multiple points P1. Methods for deriving a normal vector include a method of determining a curved surface from each point P1 and deriving the normal vector of each point P1 from the curved surface, and a method using the cross product of vectors. For example, when the control device 52 determines the normal vector of one point P1, it determines the cross product of vectors directed from this point P1 to each of two points P1 located within a predetermined range. This cross product is the normal vector.

[0039] The control device 52 extracts a point P1 where the normal vector is oriented horizontally. For example, the control device 52 determines whether the angle of the normal vector with respect to the XY plane of the map coordinate system is within a predetermined range. If the normal vector is oriented horizontally, the normal vector is parallel to the XY plane of the map coordinate system. Taking into consideration the inclination of the transfer vehicle 10 and measurement errors, the predetermined range is set so that the point P1 where the normal vector can be considered to be oriented horizontally can be extracted.

[0040] The control device 52 derives a plane equation from point P1, whose normal vector is oriented horizontally. The plane equation can be derived using, for example, a robust estimation method such as RANSAC (Random Sample Consensus) or the least squares method. The plane represented by the plane equation is a surface that extends in the vertical direction. In the case of the transport vehicle 10 of this embodiment, the plane can be obtained from a first point P11 defined by the gate 17. Furthermore, when loading an item, since the transport vehicle 10 is loaded with a pallet PA1 and an item C1, the plane can be obtained from a third point P13 defined by the pallet PA1 and the item C1. The control device 52 creates a side view IM1 by projecting point P1 in a direction perpendicular to the plane represented by the plane equation.

[0041] Next, in step S3, the control device 52 extracts the driver's seat 11 from the side view IM1 to determine the position of the driver's seat 11 in the side view IM1. The position of the driver's seat 11 in the side view IM1 can be represented by an image coordinate system. The image coordinate system is a coordinate system in which the horizontal direction of the side view IM1 is the X axis and the vertical direction is the Y axis. The position of the driver's seat 11 is determined using image recognition. In this embodiment, a case will be described in which image recognition is performed using the image recognition model D2, but image recognition may also be performed by pattern matching.

[0042] The image recognition model D2 is a trained model generated by machine learning. The image recognition model D2 uses an algorithm capable of determining the class of an object on a region-by-region basis. The class is set to "driver's seat." Examples of machine learning algorithms include SSD (Single Shot Multibox Detector), R-CNN (Regional Convolutional Neural Network), fast R-CNN, faster R-CNN, and YOLO (You Only Look Once). The image recognition model D2 is generated by supervised learning or semi-supervised learning using training data. The training data includes image data containing an object corresponding to the class, and data including the position and label of the object in the image data. The training data can be generated, for example, by surrounding an object in the image data with a frame and labeling the image data. In this embodiment, the training data may be obtained by surrounding the driver's seat 11 in the image data with a frame and labeling the image data with "driver's seat." The image data used as training data may be obtained from the point cloud map PM1, as with the side view IM1, or may be obtained by capturing an image using an imaging device.

[0043] The image recognition model D2 identifies an area including the driver's seat 11 from the input side view IM1. The area including the driver's seat 11 is represented by a bounding box B1. In the example shown in Fig. 7, the driver's seat 11 represented by the second point P12 is surrounded by the bounding box B1.

[0044] As shown in FIGS. 5 and 8, in step S4, the control device 52 determines working areas A11 and A12. Specifically, the control device 52 generates a boundary line BL1 that passes through the transfer vehicle 10 in the forward / backward direction of the transfer vehicle 10 in the map coordinate system. The control device 52 determines both sides of the boundary line BL1 as working areas A11 and A12. The working areas A11 and A12 are, for example, areas of a predetermined size. The working areas A11 and A12 include a right working area and a left working area. The right working area is the working areas A11 and A12 on the right of the transfer vehicle 10. The left working area is the working areas A11 and A12 on the left of the transfer vehicle 10.

[0045] When the cargo handling vehicle 20 is located to the right of the transport vehicle 10, the driver's seat 11 is located on the right side of the side view IM1. When the driver's seat 11 is located to the right of the center position in the X-axis direction of the image coordinate system in the side view IM1, the control device 52 determines that the cargo handling vehicle 20 is located to the right of the transport vehicle 10. In this case, the control device 52 can determine that the work areas A11, A12 in which the cargo handling vehicle 20 is located are the right work areas. The control device 52 can determine that the work areas A11, A12 in which the cargo handling vehicle 20 is not located are the left work areas.

[0046] When the cargo handling vehicle 20 is located to the left of the transport vehicle 10, the driver's seat 11 is located on the left side of the side view IM1. When the driver's seat 11 is located to the left of the center position in the X-axis direction of the image coordinate system in the side view IM1, the control device 52 determines that the cargo handling vehicle 20 is located to the left of the transport vehicle 10. In this case, the control device 52 can determine that the work areas A11, A12 in which the cargo handling vehicle 20 is located are the left work areas. The control device 52 can determine that the work areas A11, A12 in which the cargo handling vehicle 20 is not located are the right work areas.

[0047] Next, in step S5, the control device 52 determines the direction of the loading / unloading operation. An example will be described in which the side shift direction is determined as the direction of the loading / unloading operation. As mentioned above, the direction of the loading / unloading operation differs depending on whether the loading / unloading vehicle 20 is loading or unloading. When the loading / unloading vehicle 20 is loading in the right working area, the control device 52 determines the direction of the loading / unloading operation to be the right direction of the loading / unloading vehicle 20. When the loading / unloading vehicle 20 is loading in the left working area, the control device 52 determines the direction of the loading / unloading operation to be the left direction of the loading / unloading vehicle 20. When the loading / unloading vehicle 20 is unloading in the right working area, the control device 52 determines the direction of the loading / unloading operation to be the left direction of the loading / unloading vehicle 20. When the loading / unloading vehicle 20 is unloading in the left working area, the control device 52 determines the direction of the loading / unloading operation to be the right direction of the loading / unloading vehicle 20.

[0048] After completing the processing of step S5, the control device 52 ends the cargo handling operation direction determination control. The control device 52 outputs the cargo handling operation direction determined by the cargo handling operation direction determination control to the vehicle control device 56. The vehicle control device 56 performs control according to the cargo handling operation direction.

[0049] [Effects of this embodiment] (1) The control device 52 determines whether the cargo handling vehicle 20 is located on the left or right side of the transport vehicle 10 from the point cloud data. The control device 52 can determine the direction of the cargo handling operation depending on whether the cargo handling vehicle 20 is located on the left or right side of the transport vehicle 10.

[0050] (2) The control device 52 extracts the driver's seat 11 from the side view IM1 by image recognition. The position of the driver's seat 11 differs depending on whether the cargo handling vehicle 20 is located on the left or right side of the transport vehicle 10. By extracting the driver's seat 11 from the side view IM1, it is possible to determine whether the cargo handling vehicle 20 is located on the left or right side of the transport vehicle 10.

[0051] (3) When the stopping position PS1 of the transport vehicle 10 is determined and the stopping direction of the transport vehicle 10 is determined, the control device 52 can determine whether the cargo handling vehicle 20 is located on the left or right side of the transport vehicle 10 from its own position. However, as shown in FIG. 1 , when the cargo handling vehicle 20 is located between two stopping positions PS1, the cargo handling vehicle 20 is located on the right side when one of the two transport vehicles 10 is used as the reference, and on the left side when the other is used as the reference. Therefore, when the cargo handling vehicle 20 is located between two stopping positions PS1, it is not possible to determine whether the cargo handling vehicle 20 is located on the left or right side of the transport vehicle 10. In contrast, when the driver's seat 11 is extracted from the side view IM1 by image recognition, it is possible to determine whether the cargo handling vehicle 20 is located on the left or right side of the transport vehicle 10 even when the cargo handling vehicle 20 is located between two stopping positions PS1.

[0052] (4) The vehicle control device 56 controls the loading vehicle 20 in accordance with commands from the higher-level control device 71. The commands are included in the operation data input by a person. If the loading operation direction is not determined by the control device 52, a person needs to specify the loading operation direction when inputting the operation data. At this time, there is a risk that the person may input the wrong loading operation direction. In contrast, by having the control device 52 determine the loading operation direction, it is possible to prevent a person from inputting the wrong loading operation direction. This can prevent the loading vehicle 20 from malfunctioning due to a person inputting the wrong loading operation direction. It can also reduce the amount of work required to create operation data.

[0053] (5) The external sensor 51 used to estimate the self-position can be used to determine whether the cargo handling vehicle 20 is located to the left or right of the transport vehicle 10. Compared to using a dedicated sensor to determine whether the cargo handling vehicle 20 is located to the left or right of the transport vehicle 10, the number of parts can be reduced.

[0054] [Example of change] The embodiment can be modified as follows: The embodiment and the following modifications can be combined with each other to the extent that they are not technically inconsistent.

[0055] The control for determining the direction of loading and unloading operation may be a control for determining the direction of loading and unloading operation based on the positional relationship between the vehicle's own position and the stopping position PS1. 9, in step S11, the control device 52 estimates its own position. The estimation of the own position is the same process as in the embodiment.

[0056] Next, in step S12, the control device 52 determines the direction of the loading / unloading operation from the positional relationship between its own position and the stopping position PS1. When determining the direction of the loading / unloading operation from the positional relationship between its own position and the stopping position PS1, it is necessary to determine the stopping direction of the transporting vehicle 10. This makes it possible to determine whether the own position corresponds to the left or right side of the transporting vehicle 10 when the transporting vehicle 10 is stopped at the stopping position PS1. It can be said that the control device 52 can determine whether the loading / unloading vehicle 20 is located on the left or right side of the transporting vehicle 10 from its own position. Then, the control device 52 can determine the direction of the loading / unloading operation depending on whether the loading / unloading vehicle 20 is located on the left or right side of the transporting vehicle 10.

[0057] When determining the direction of the cargo handling operation based on the positional relationship between the vehicle's own position and the stopping position PS1, it is preferable to make the area of ​​the stopping position PS1 small. If the area of ​​the stopping position PS1 is large, even if the position of the cargo handling vehicle 20 is the same, the cargo handling vehicle 20 may be located on the left or right side of the transport vehicle 10 depending on the position of the transport vehicle 10. For example, as shown in FIG. 10, assume that the transport vehicle 10 is stopped at a first position A2 of the stopping positions PS1, and the cargo handling vehicle 20 is located to the right of the transport vehicle 10. When the position of the cargo handling vehicle 20 is the same, assume that the transport vehicle 10 is stopped at a second position A3 of the stopping positions PS1. The first position A2 and the second position A3 are located on either side of the cargo handling vehicle 20. In this case, the cargo handling vehicle 20 is located to the left of the transport vehicle 10 stopped at the second position A3. It is preferable to set the area of ​​the stopping position PS1 so that the cargo handling vehicle 20 is positioned on either the left or right side of the transporting vehicle 10 depending on the position of the transporting vehicle 10.

[0058] The image recognition model D2 may be an algorithm capable of determining the class of an object for each side view IM1. In this case, "right-facing" and "left-facing" may be set as classes. An example of the machine learning algorithm is a convolution neural network (CNN). As training data, data in which image data showing the transfer vehicle 10 facing right is labeled as "right-facing" and data in which image data showing the transfer vehicle 10 facing left is labeled as "left-facing" may be used.

[0059] The control device 52 inputs the side view IM1 into the image recognition model D2 to determine whether the transport vehicle 10 shown in the side view IM1 is facing right or left. When the cargo vehicle 20 is located to the right of the transport vehicle 10, the transport vehicle 10 appears facing right in the side view IM1. When the cargo vehicle 20 is located to the left of the transport vehicle 10, the transport vehicle 10 appears facing left in the side view IM1. Therefore, the control device 52 can determine whether the cargo vehicle 20 is located to the right or left of the transport vehicle 10 from the output of the image recognition model D2.

[0060] The control device 52 does not have to create the point cloud map PM1. In this case, the control device 52 converts the point cloud data acquired from the external sensor 51 into map coordinates and performs the processes from step S2 onwards. That is, the control device 52 may determine whether the cargo handling vehicle 20 is located on the left or right side of the transport vehicle 10 from a single point cloud data, without creating the point cloud map PM1 which is a collection of multiple point cloud data.

[0061] The cargo handling device 24 may be equipped with, for example, a robot arm. In this case, the cargo handling operation direction may be the direction in which the robot arm is operated. For example, the cargo handling operation direction may be the direction in which the robot arm loads the cargo C1.

[0062] The transport vehicle 10 may be any vehicle capable of transporting goods, such as an AGV (Automatic Guided Vehicle) or an AMR (Autonomous Mobile Robot). In this case, the control device 52 may use the image recognition model D2 to determine whether the transport vehicle 10 is facing right or left, thereby determining whether the cargo handling vehicle 20 is located to the right or left of the transport vehicle 10. The control device 52 may also determine whether the cargo handling vehicle 20 is located to the right or left of the transport vehicle 10 from its own position.

[0063] Part of the processing for determining the direction of cargo handling operations may be performed by the vehicle control device 56. In this case, the vehicle control device 56 can also be considered a control device. The upper control device 71 may perform part of the processing for determining the direction of cargo handling operations. In this case, the upper control device 71 can also be considered a control device. The upper control device 71 may perform all of the processing for determining the direction of cargo handling operations. In this case, the upper control device 71 is the control device. In this way, when the upper control device 71 is made to perform at least part of the processing for determining the direction of cargo handling operations, the cargo handling vehicle 20 and the upper control device 71 form a cargo handling system. [Explanation of symbols]

[0064] P1...point, 10...transport vehicle, 11...driver's seat, 20...loading vehicle which is a loading system, 24...loading device, 51...external sensor, 52...control device.

Claims

1. a cargo handling vehicle equipped with an external sensor that detects the position of an object in coordinates of a three-dimensional coordinate system and a cargo handling device; A cargo handling system comprising: The control device determining whether the cargo handling vehicle is located on the left or right side of the transport vehicle from point cloud data, which is a set of points representing the positions of the object; A loading and unloading system in which the loading and unloading operation direction, which differs depending on whether the loading and unloading vehicle is located on the left or right side of the transport vehicle, is determined based on whether the loading and unloading vehicle is located on the left or right side of the transport vehicle.

2. The control device creating a side view of the transport vehicle from the point cloud data; extracting a driver's seat from the side view by image recognition; The cargo handling system according to claim 1 , wherein it is determined whether the cargo handling vehicle is located on the left or right side of the transport vehicle from the position of the driver's seat in the side view.

3. The transport vehicle stops at a stopping position with a predetermined stopping direction, The control device estimates a self-position of the cargo handling vehicle, 2. The cargo handling system according to claim 1, wherein it is determined whether the cargo handling vehicle is located to the left or right of the transport vehicle based on the positional relationship between the vehicle's own position and the stopping position.

Images