Remote operating system

Abstract

The present invention addresses the problem of enabling the operation of a forklift to be intuitively and easily performed and the loading and unloading to be efficiently performed when remotely operating the forklift. The present invention provides a remote operation system comprising: a camera provided in a forklift; and a display unit that displays an image captured by the camera. In addition, the remote operation system includes a loading / unloading object position determination unit that determines the positions of a first loading / unloading object, which is the next object to be loaded / unloaded, and a second loading / unloading object, which is the next object to be loaded / unloaded. The remote control system is configured to display a first guide route for guiding the horizontal position of a first object to be loaded and unloaded, a second guide route for guiding the vertical position of the first object to be loaded and unloaded, and a third guide route for guiding the horizontal position of a second object to be loaded and unloaded. And a fourth guide route for guiding the vertical position of the second loading / unloading object.

Description

TECHNICAL FIELD

[0001] The present application relates to a remote operation system for remotely operating a forklift truck. BACKGROUND

[0002] A forklift truck is used for an operator to get on and perform a cargo handling and loading / unloading work. Therefore, to operate the forklift truck quickly and accurately, the operator needs many years of experience. On the other hand, the number of skilled operators is limited, and therefore, in a region with a small population, there is a case where the number of skilled operators is insufficient. Therefore, it is strongly desired that a forklift truck in a region with a small population is remotely operated by a skilled operator in a region with a large population.

[0003] In addition, in a case where a tank filled with a gas that has an adverse effect on the human body or an explosive that explodes when dropped is handled, it is preferable that the operator remotely operates the forklift truck while being away from a warehouse as a handling place.

[0004] Therefore, a remote operation system for remotely operating a forklift truck is known (for example, refer to Patent Literature 1). The remote operation system is configured such that an operation device is provided in a remote base station away from a warehouse where the forklift truck performs a loading / unloading work, and the operator operates the forklift truck from the remote base station using the operation device.

[0005] The forklift truck includes a control device, a wireless communication device, a camera, a sensor, a driving device that performs a traveling and a lifting work of the vehicle, and the like. The operation device of the base station includes a steering wheel, an operation lever, a pedal, a control section, a display section, and the like. For example, the display section is provided with two display monitors, one of which displays an image of the front of the vehicle taken by the camera mounted on the forklift truck, and the other of which displays information detected by the sensor, and the like.

[0006] Further, a remote operation system in which even when a handling object is stored in a storage portion of a shelf, the position of the handling object can be recognized without being lost when a forklift truck is remotely operated is known (refer to Patent Literature 2). In the remote operation system, a first guide line that guides a horizontal position of a handling object in a horizontal direction on an image displayed on a display section is displayed, and further, a second guide line that guides a vertical position of a handling object in a vertical direction on the image displayed on the display section is displayed.

[0007] In the remote operation system, the operator can recognize the position of the handling object without being lost. In the remote operation system, the operator performs a plurality of handling, and therefore, by recognizing the position of the handling object in advance, the handling can be performed efficiently, but since only the position of a handling object that should be handled next is displayed, the position of a handling object that should be handled next next cannot be recognized unless the handling of the handling object that should be handled next is completed, and therefore, a remote operation system in which the handling can be performed more efficiently is desired.

[0008] [Related Art Documents]

[0009] [Patent Documents]

[0010] [Patent Document 1] Japanese Patent Application Laid-Open (JP A) No. 2014-11518

[0011] [Patent Document 2] Japanese Patent Application Laid-Open (JP A) No. 2023-85019 SUMMARY

[0012] [Problems to be Solved by the Invention]

[0013] Therefore, the problem to be solved by the present application is to provide a remote operation system in which loading and unloading can be performed more efficiently when a forklift is remotely operated.

[0014] [Means of Solving the Problems]

[0015] To solve the problem, the present application is a remote operation system for a forklift, including: a facility provided with a rack having a plurality of storage portions in a vertical direction; and a forklift that travels and performs loading and unloading work in the facility.

[0016] The remote operation system includes: a camera provided to the forklift; a display portion that displays an image captured by the camera; and a loading and unloading object position determination portion that determines positions of a first loading and unloading object that is a next object to be loaded and unloaded and a second loading and unloading object that is a next next object to be loaded and unloaded.

[0017] In addition, the remote operation system includes: a first guide route display portion that displays a first guide route for guiding a horizontal position of the first loading and unloading object in a horizontal direction on the image displayed on the display portion; a second guide route display portion that displays a second guide route for guiding a vertical position of the first loading and unloading object in a vertical direction on the image displayed on the display portion; a third guide route display portion that displays a third guide route for guiding a horizontal position of the second loading and unloading object in the horizontal direction on the image displayed on the display portion; and a fourth guide route display portion that displays a fourth guide route for guiding a vertical position of the second loading and unloading object in the vertical direction on the image displayed on the display portion.

[0018] Preferably:

[0019] The second guide route display portion to the fourth guide route display portion display the second guide route to the fourth guide route when the forklift enters an inside of a proximity area that is a prescribed range from the first loading and unloading object.

[0020] Preferably:

[0021] The approach area is composed of a first approach area to a third approach area,

[0022] The first approach area is disposed away from the first load and unload object,

[0023] The second approach area is disposed closer to the first load and unload object than the first approach area,

[0024] The third approach area is disposed closer to the first load and unload object than the second approach area,

[0025] The second guide route display portion displays the second guide route when the forklift enters inside the first approach area,

[0026] The third guide route display portion displays the third guide route when the forklift enters inside the second approach area,

[0027] The fourth guide route display portion displays the fourth guide route when the forklift enters inside the third approach area.

[0028] Preferably,

[0029] The second guide route display portion and the fourth guide route display portion determine the second guide route and the fourth guide route displayed on the display portion from a plurality of second guide routes and fourth guide routes according to the vertical position of the load and unload object.

[0030] Preferably,

[0031] The plurality of second guide routes and fourth guide routes include any one or combination of different shapes, patterns, colors.

[0032] Preferably,

[0033] The first guide route display portion and the third guide route display portion display the first guide route and the third guide route extending in a horizontal direction on the image,

[0034] The second guide route display portion and the fourth guide route display portion display the second guide route and the fourth guide route extending in a vertical direction on the image from the front end of the first guide route and the third guide route.

[0035] Preferably,

[0036] A passageway for the forklift to travel and perform load and unload work is formed between adjacent shelves in the facility,

[0037] The first guide route display portion to the fourth guide route display portion display the first guide route to the fourth guide route on the passageway in the image.

[0038] Preferably,

[0039] The first to fourth guide route display sections display the first to fourth guide routes each having a width corresponding to the width of the forklift truck.

[0040] [Effects of the Invention]

[0041] The remote operation system according to the present application, when remotely operating the forklift truck, displays not only the position of the next load-unload object to be loaded or unloaded but also the position of the load-unload object to be loaded or unloaded after the next one, thereby enabling the loading and unloading to be performed more efficiently. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 is a plan view showing the remote operation system of the first embodiment.

[0043] Figure 2 is a side view showing the forklift truck.

[0044] Figure 3 is a block diagram showing the structure of the forklift truck.

[0045] Figure 4 is a block diagram showing the structure of the operation device.

[0046] Figure 5 is a flowchart showing the control flow of the remote operation system of the first embodiment.

[0047] Figure 6 is an image diagram showing the display section of the first guide route of the first embodiment.

[0048] Figure 7 is an image diagram showing the display sections of the first to fourth guide routes of the first embodiment.

[0049] Figure 8 is a plan view showing the remote operation system of the second embodiment.

[0050] Figure 9 is a flowchart showing the control flow of the remote operation system of the second embodiment.

[0051] Figure 10 is an image diagram showing the display sections of the first and second guide routes of the second embodiment.

[0052] Figure 11 is an image diagram showing the display sections of the first to third guide routes of the second embodiment.

[0053] Figure 12 is an image diagram showing the display sections of the first to fourth guide routes of the second embodiment.

[0054] [Explanation of Symbols]

[0055] 1: fork truck

[0056] 10: camera

[0057] 14: control section

[0058] 141: image processing section

[0059] 142: fork truck position acquisition section

[0060] 2: operation device

[0061] 20: display section

[0062] 21: operation section

[0063] 23: control section

[0064] 231: image processing section

[0065] 232: loading / unloading object decision section

[0066] 233: loading / unloading object position determination section

[0067] 234: first guide route display section

[0068] 235: second guide route display section

[0069] 237: third guide route display section

[0070] 238: fourth guide route display section

[0071] 100: facility

[0072] 102: shelf

[0073] 103: passage

[0074] 110: first loading / unloading object

[0075] 111: second loading / unloading object

[0076] AR: approach region

[0077] AR1: first approach region

[0078] AR2: second approach region

[0079] AR3: third approach region

[0080] GR1: first guide route

[0081] GR2: second guide route

[0082] GR3: third guide route

[0083] GR4: fourth guide route

[0084] E1: front end of the first guide route

[0085] E2: front end of the second guide route DETAILED DESCRIPTION

[0086] Hereinafter, an embodiment of a remote operation system of the present application will be described based on the drawings.

[0087] <First Embodiment>

[0088] A first embodiment of a remote operation system will be described.

[0089] [Remote Operation System]

[0090] As Figure 1 , the remote operation system includes a forklift 1 for traveling and performing loading and unloading work within a facility 100. Also, the remote operation system is provided with a base station 200 provided at a remote place from the facility 100. An operation device 2 is provided in the base station 200, and an operator can remotely operate a prescribed forklift 1 using the operation device 2.

[0091] In the present embodiment, the facility 100 is a warehouse, but can also be a factory or the like. Also, in the present embodiment, the forklift 1 is a forward-moving forklift, but can also be a counterbalance forklift or the like. For example, by operating the forklift 1 from the remote base station 200 by an operator, a canister filled with a gas that has an adverse effect on the human body or an explosive that explodes when dropped can be loaded and unloaded within the facility 100.

[0092] A plurality of shelves 102 are provided within the facility 100. A plurality of objects (goods) are placed on the shelves 102, and the forklift 1 travels within the facility 100 to perform loading and unloading work of prescribed objects (goods) from and to prescribed shelves 102.

[0093] [Forklift]

[0094] As Figure 2 , the forklift 1 includes a camera 10, a laser scanner 12, a drive device 13, a control section 14, a wireless communication section 15, and the like. As Figure 3That way, the control section 14 includes an image processing section 141, a forklift position acquisition section 142, a drive control section 143, and the like. The control section 14 contains a central processing unit (CPU), an input / output interface, a read only memory (ROM), a random access memory (RAM), and the like, and stores a program for processing information. The camera 10, the laser scanner 12, the drive device 13, and the wireless communication section 15 are connected to the control section 14.

[0095] The drive device 13 contains a travel motor for driving a drive wheel 16 provided at the rear of the vehicle body, and a plurality of hydraulic cylinders for raising and lowering, tilting, and advancing and retreating a fork 17 provided at the front of the vehicle body. An operation signal from the operation device 2 provided in the base station 200 is sent to the control section 14 via the wireless communication section 15, and the operation signal is processed in the drive control section 143, so that the travel motor and the hydraulic cylinders, that is, the drive device 13, are driven in accordance with the operation of the operation device 2. Thus, the drive device 13 including the travel motor and the hydraulic cylinders can be driven in conjunction with the operation of the operation device 2 by the operator to operate the forklift 1.

[0096] The camera 10 is disposed at a position of the operator's line of sight when the operator rides on the forklift 1 to operate, and captures the front of the forklift 1 from the position of the operator's line of sight. The camera 10 includes, for example, a charge coupled device (CCD) image sensor or a complementary metal oxide semiconductor (C-MOS) image sensor. An image captured by the camera 10 is subjected to image processing in the image processing section 141 of the control section 14, and is displayed on a display section 20 provided in the operation device 2. Figure 4 ) Thus, the operator who remotely operates the forklift 1 using the operation device 2 in the base station 200 can confirm the front of the forklift 1 by the display section 20 in the same line of sight as when the operator rides on the forklift 1 to operate.

[0097] The forklift 1 is equipped with a laser scanner 12, and a plurality of reflecting sections 101 are provided in the facility 100. The laser scanner 12 transmits and receives laser light to and from the reflecting sections 101 in a manner of horizontally rotating 360 degrees. As a result, the forklift 1 can recognize the plurality of reflecting sections 101 arranged along a travel path in the facility 100 by the laser scanner 12. Here, the reflecting sections 101 are fixed to wall surfaces and shelves 102 in the facility 100, and positional information thereof is stored in a map of a forklift position acquisition section 142 of a control section 14. The forklift 1 recognizes the plurality of reflecting sections 101 by the laser scanner 12, and thus the forklift position acquisition section 142 can measure and acquire the position coordinates of the forklift 1 according to the principle of triangulation.

[0098] [Operation device]

[0099] As Figure 4 , the operation device 2 includes a display section 20, an operation section 21, a control section 23, a wireless communication section 24, and the like. The control section 23 includes an image processing section 231, a loading / unloading object decision section 232, a loading / unloading object position determination section 233, a first guide route display section 234, a second guide route display section 235, a third guide route display section 237, a fourth guide route display section 238, a facility environment storage section 236, and the like. The control section 23 includes a CPU (Central Processing Unit), an input / output interface, a ROM, a RAM, and the like, and stores a program for processing information. The display section 20, the operation section 21, and the wireless communication section 24 are connected to the control section 23.

[0100] The display section 20 includes, for example, two display monitors, and an image of a front of the vehicle captured by the camera 10 mounted on the forklift 1 is displayed on one of the display monitors, and information detected by various sensors, information required for operation, and the like are displayed on the other display monitor.

[0101] The operation section 21 includes a steering wheel, an operation lever, a pedal, and the like, and an operator can operate the forklift 1 in the same manner as in the case of actually riding on the forklift 1. An operation signal from the operation section 21 is transmitted by the wireless communication section 24 via the control section 23, and is received by the wireless communication section 15 of the forklift 1. Thus, the operator can remotely operate the drive device 13 and the like of the forklift 1 by operating the operation section 21, that is, the steering wheel, the operation lever, and the like, as described above.

[0102] The loading / unloading object determination unit 232 stores a plurality of objects (goods) arranged in the facility 100, and determines, in order, the first loading / unloading object 110 and the second loading / unloading object 111 to be loaded / unloaded by the forklift 1, based on the loading / unloading status, the delivery time, the purchase time, the goods placement time, and the like of each object (goods) by a program or machine learning, or the like. In this way, the objects (goods) to be loaded / unloaded by the forklift 1 can be set as the first loading / unloading object 110 and the second loading / unloading object 111 in order. As the program, for example, a schedule is set and stored in advance, and when the purchase time is reached, the first loading / unloading object 110 and the second loading / unloading object 111 to be purchased are determined, and when the delivery time is reached, the first loading / unloading object 110 and the second loading / unloading object 111 to be delivered are determined. Alternatively, the first loading / unloading object 110 and the second loading / unloading object 111 can be determined in order based on the loading / unloading status of other forklifts.

[0103] The loading / unloading object position determination unit 233 stores the positions of each object (goods), and determines the positions of the first loading / unloading object 110 and the second loading / unloading object 111 when the first loading / unloading object 110 and the second loading / unloading object 111 are determined by the loading / unloading object determination unit 232. The objects (goods) are stored in the shelves 102 having a plurality of storage layers in the vertical direction, and thus the positions of the first loading / unloading object 110 and the second loading / unloading object 111 are determined based on the horizontal coordinates (XY coordinates) and the vertical coordinates (Z coordinates).

[0104] The first guide route display unit 234 displays, by the stored program, the first guide route GR1 for guiding the horizontal position of the first loading / unloading object 110 in the horizontal direction (XY direction) on the image displayed on the display 20. Also, the second guide route display unit 235 displays, by the stored program, the second guide route GR2 for guiding the vertical position of the first loading / unloading object 110 in the vertical direction (Z direction) on the image displayed on the display 20.

[0105] Also, the third guide route display unit 237 displays, by the stored program, the third guide route GR3 for guiding the horizontal position of the second loading / unloading object 111 in the horizontal direction (XY direction) on the image displayed on the display 20. Also, the fourth guide route display unit 238 displays, by the stored program, the fourth guide route GR4 for guiding the vertical position of the second loading / unloading object 111 in the vertical direction (Z direction) on the image displayed on the display 20.

[0106] In the present embodiment, as Figure 6 and Figure 7In this way, the first guide route display unit to the fourth guide route display unit 234, 235, 237, and 238 display the first guide route to the fourth guide route GR1, GR2, GR3, and GR4, which have a width equivalent to the width of the forklift 1. As a result, the operator can intuitively visualize the actual travel route of the forklift 1 and the lifting route of the forks 17.

[0107] The facility environment storage unit 236 stores the horizontal position of the wall of facility 100, the horizontal position and vertical position of the shelf 102 and its various storage sections. In this embodiment, as... Figure 1 Thus, a passageway 103 for forklift 1 to travel and perform loading and unloading operations is formed between adjacent racks 102. The horizontal position of the passageway 103 is stored by the facility environment storage unit 236. Forklift 1 can travel along the passageway 103 to the desired position on the rack 102 and can load and unload the first loading / unloading item 110 and the second loading / unloading item 111 relative to the rack 102. Therefore, the first guide route display units to the fourth guide route display units 234, 235, 237, and 238 display the first guide route to the fourth guide route GR1, GR2, GR3, and GR4 on the passageway 103 displayed in the image on the display unit 20. Thus, the operator can appropriately operate the forklift 1 to perform driving and loading / unloading operations on the passageway 103 even remotely.

[0108] In addition, the first to fourth guide route display units 234, 235, 237, and 238 set proximity zones (AR) on the passageway 103 within a specified range from the first loading / unloading object 110. Figure 1 That is, an approach area AR is set on the passage 103 that the forklift 1 passes through when it travels toward the first loading / unloading object 110. Moreover, when the forklift 1 enters the inside of the approach area AR in the first loading / unloading object 110, the second guide route display unit to the fourth guide route display unit 235, 237, 238 displays the second guide route to the fourth guide route GR2, GR3, GR4.

[0109] Therefore, as Figure 6 Therefore, when the forklift 1 moves away from the first loading / unloading object 110, the second to fourth guide routes GR2, GR3, and GR4 are not displayed on the display unit 20, but the first guide route GR1 is displayed instead. Figure 7 Thus, when the forklift 1 approaches the object to be loaded / unloaded 110, the second to fourth guide routes GR2, GR3, and GR4 are displayed on the display unit. Therefore, the operator can visually identify that they have approached the first object to be loaded / unloaded 110 by displaying the second to fourth guide routes GR2, GR3, and GR4 on the display unit 20.

[0110] Further, the operator can visually recognize the position or distance sense to the second object 111 to be handled next after the first object 110 is handled by displaying the third guide route GR3 and the fourth guide route GR4 on the display 20, and thus, the operation for the second object 111 can be taken immediately after the handling of the first object 110 is completed.

[0111] Further, as Figure 6 and Figure 7 The first guide route display 234 displays the first guide route GR1 extending in the horizontal direction to the first object 110, and further, the second guide route display 235 displays the second guide route GR2 extending in the vertical direction from the front end El of the first guide route GR1. Also, the third guide route display 237 displays the third guide route GR3 extending in the horizontal direction to the second object 111, and further, the fourth guide route display 238 displays the fourth guide route GR4 extending in the vertical direction from the front end E2 of the fourth guide route GR4.

[0112] Further, the second guide route display 235 and the fourth guide route display 238 are configured to display an arrow A toward the right or left on the second guide route GR2 and the fourth guide route GR4, and show the shelf 102 in which the first object 110 and the second object 111 are stored. Thus, the operator can visually recognize that the first object 110 and the second object 111 are located at the front ends El, E2 of the first guide route GR1 and the third guide route GR3 and the vertical positions of the second guide route GR2 and the fourth guide route GR4.

[0113] Further, the second guide route display section 235 and the fourth guide route display section 238 store a plurality of second guide routes GR2 and fourth guide routes GR4. For example, in the case of the shelf 102 having three tiers of storage sections, as in the present embodiment, the second guide routes GR2 and the fourth guide routes GR4 extending in the vertical direction to the height of the first tier of the shelf 102, the second guide routes GR2 and the fourth guide routes GR4 extending in the vertical direction to the height of the second tier, and the second guide routes GR2 and the fourth guide routes GR4 extending in the vertical direction to the height of the third tier are stored. Further, the second guide route display section 235 and the fourth guide route display section 238 determine the second guide routes GR2 and the fourth guide routes GR4 displayed on the display section 20 from among the plurality of second guide routes GR2 and the fourth guide routes GR4 in accordance with the vertical positions of the first and second objects 110 and 111. For example, in the case where the first and second objects 110 and 111 are stored in the storage section of the first tier of the shelf 102, the second guide routes GR2 and the fourth guide routes GR4 extending in the vertical direction to the height of the first tier of the shelf 102 are displayed, and, as in the present embodiment, in the case where the first object 110 is stored in the storage section of the third tier of the shelf 102, the second guide routes GR2 extending in the vertical direction to the height of the third tier of the shelf 102 are displayed, and, in the case where the second object 111 is stored in the storage section of the second tier of the shelf 102, the fourth guide routes GR4 extending in the vertical direction to the height of the second tier of the shelf 102 are displayed. Thus, the operator can visually and easily recognize that the first and second objects 110 and 111 are stored in the storage section of which tier of the shelf 102 by confirming the second guide routes GR2 and the fourth guide routes GR4. Figure 7

[0114] The plurality of second guide routes GR2 and the fourth guide routes GR4 can include any one or combination of different shapes, patterns, colors. For example, the second guide routes GR2 and the fourth guide routes GR4 on which the words "first tier", "second tier", "third tier" are displayed, or the second guide routes GR2 and the fourth guide routes GR4 on which one, two, or three arrows pointing upward are displayed, or the second guide routes GR2 and the fourth guide routes GR4 including any one or combination of shapes, patterns, colors different in accordance with the height of the storage section of the shelf 102 in which the first and second objects 110 and 111 are stored are stored. Thus, the operator can visually recognize that the first and second objects 110 and 111 are stored in the storage section of which tier of the shelf 102 by confirming the second guide routes GR2 and the fourth guide routes GR4.

[0115] ​By the first guide route GR1 and the second guide route GR2, the operator can travel and stop along the first guide route GR1 displayed on the display portion 20 when traveling to the first loading / unloading object 110 using the forklift 1, and then lift and stop the forks 17 along the second guide route GR2 when loading / unloading the loading / unloading object 110 using the forklift 1, and thus can intuitively and easily perform the operation of the forklift 1 without losing the loading / unloading object 110.

[0116] Further, the operator can intuitively recognize the position up to the second loading / unloading object 111 by displaying the third guide route GR3 and the fourth guide route GR4 on the display portion 20, and thus can immediately take action for loading / unloading the second loading / unloading object 111 after the loading / unloading of the first loading / unloading object 110 is completed.

[0117] [Control flow]

[0118] Based on Figure 5 The control flow performed by the control portion 14, the control portion 23 in the remote operation system will be described.

[0119] The absolute coordinates (Fx, Fy) of the forklift 1 are always measured and acquired by the forklift position acquisition portion 142 (step Sll). Also, the image captured by the camera 10 is always displayed on the display portion 20. The operator remotely operates the forklift 1 by operating the operation portion 21 based on the image displayed on the display portion 20.

[0120] A plurality of objects (goods) are stored by the loading / unloading object decision portion 232, and the next object (the first loading / unloading object) 110 to be loaded / unloaded and the object (the second loading / unloading object) 111 to be loaded / unloaded next to the next object are decided in order based on a program or the like (step S12). Then, the positions of the first loading / unloading object 110 and the second loading / unloading object 111 are determined by the loading / unloading object position determination portion 233 (step S13).

[0121] The first guide route GR1 up to the first loading / unloading object 110 is displayed on the passage 103 in the screen of the display portion 20 by the first guide route display portion 234 (step S14). Then, the operator operates the forklift 1 to travel along the first guide route GR1. Also, the second guide route display portion to the fourth guide route display portion 235, 237, 238 determine whether the forklift 1 enters inside the approach area AR (step S15).

[0122] Then, when the forklift 1 approaches the first load-unload object 110 and it is determined by the second guide route display section ~ fourth guide route display section 235, 237, 238 that the forklift 1 has entered the inside of the approach area AR (step S15), the second guide route ~ fourth guide route GR2, GR3, GR4 is displayed on the passage 103 in the screen of the display section 20 (step S16).

[0123] Then, the operator notifies the second guide route display section 235 that the load-unload of the first load-unload object 110 is ended by the end button (not shown) or the like provided to the operating device 2, and the second guide route GR2 is not displayed from the screen of the display section 20.

[0124] By displaying the second guide route GR2 on the display section 20, the operator can recognize the vertical position of the first load-unload object 110 which should be next subjected to the load-unload, and thus, the first load-unload object 110 is subjected to the load-unload (load-unload of the cargo) by the forklift 1. In addition, by displaying the third guide route GR3 and the fourth guide route GR4 on the display section 20, the operator can recognize the horizontal position and the vertical position of the second load-unload object 111 which should be next subjected to the load-unload, and thus, can visually recognize the position up to the second load-unload object 111, and thus, can immediately take an action for the load-unload of the second load-unload object 111 after the load-unload of the first load-unload object 110 is ended.

[0125] <Second Embodiment>

[0126] The second embodiment of the remote operation system will be described. Further, for the portions repeated with the first embodiment, the description will be sometimes omitted in order to avoid the repetition.

[0127] [Operating Device]

[0128] In the present embodiment, as Figure 8 the approach area AR is constituted by the first approach area ~ third approach area AR1, AR2, AR3. The first approach area AR1 is disposed away from the first load-unload object 110. The second approach area AR2 is disposed in the vicinity of the first load-unload object 110 more than the first approach area AR1. The third approach area AR3 is disposed in the vicinity of the first load-unload object 110 more than the second approach area AR2. That is, the first approach area AR1, the second approach area AR2, the third approach area AR3 are disposed in the order away from the first load-unload object 110, and the forklift 1 passes through in the order of the first approach area AR1, the second approach area AR2, the third approach area AR3, and approaches the first load-unload object 110.

[0129] Further, as Figure 10Thus, the second guidance route display unit 235 displays the second guidance route GR2 when the forklift 1 enters the inner side of the first approach area AR1. Furthermore, as... Figure 11 In this way, the third guidance route display unit 237 displays the third guidance route GR3 when the forklift 1 enters the inner side of the second approach area AR2. Furthermore, as... Figure 12 In this way, the fourth guide route display unit 238 displays the fourth guide route GR4 when the forklift 1 enters the inner side of the third approach area AR3.

[0130] [Control Flow]

[0131] based on Figure 9 The control flow performed by the control unit 14 and control unit 23 in the remote operating system is explained.

[0132] The forklift position acquisition unit 142 continuously measures and acquires the absolute coordinates (Fx, Fy) of the forklift 1 (step S21). The loading / unloading object determination unit 232 stores multiple objects (goods) and, based on a program, sequentially determines the next object to be loaded / unloaded (first loading / unloading object) 110 and the next next (second loading / unloading object) 111 using the forklift 1 (step S22). Then, the loading / unloading object position determination unit 233 determines the positions of the first loading / unloading object 110 and the second loading / unloading object 111 (step S23).

[0133] The first guide route GR1, extending to the first loading / unloading object 110, is displayed on the path 103 in the screen of the display unit 20 via the first guide route display unit 234 (step S24). Then, the operator operates the forklift 1 to travel along the first guide route GR1. In addition, the second to fourth guide route display units 235, 237, and 238 determine whether the forklift 1 has entered the inner side of the first approach area to the third approach area AR1, AR2, and AR3 (steps S25, S27, and S29).

[0134] Then, when the forklift 1 approaches the first loading / unloading object 110, and the second guide route display unit to the fourth guide route display unit 235, 237, 238 determines that the forklift 1 has entered the inner side of the first approach area AR1 (step S25), the second guide route GR2 is displayed on the path 103 in the screen of the display unit 20 (step S26).

[0135] Further, when the forklift 1 approaches the first loading-unloading object 110 and it is determined by the second to fourth guide route display sections 235, 237, 238 that the forklift 1 has entered the inside of the second approach area AR2 (step S27), the third guide route GR3 is displayed on the passageway 103 in the screen of the display section 20 (step S28).

[0136] Further, when the forklift 1 approaches the first loading-unloading object 110 and it is determined by the second to fourth guide route display sections 235, 237, 238 that the forklift 1 has entered the inside of the third approach area AR3 (step S29), the fourth guide route GR4 is displayed on the passageway 103 in the screen of the display section 20 (step S30).

[0137] By sequentially displaying the second to fourth guide routes GR2, GR3, GR4 on the screen of the display section 20, the operator can quickly and easily grasp the order of operations for loading and unloading the first and second loading-unloading objects 110, 111.

[0138] Further, if the third and fourth guide routes GR3, GR4 are displayed on the screen of the display section 20 at a position where the forklift 1 is away from the first loading-unloading object 110, the second to fourth guide routes GR2, GR3, GR4 are displayed overlapping in the early stage, so there is a possibility that the operator erroneously recognizes the fourth guide route GR4 as the position of the first loading-unloading object 110 or gets lost at the position of the first loading-unloading object 110, but by displaying the third and fourth guide routes GR3, GR4 after the forklift 1 sufficiently approaches the first loading-unloading object 110, it is possible to prevent the recognition error or the getting lost of the operator with respect to the position of the first loading-unloading object 110. Therefore, the first to third approach areas AR1, AR2, AR3 are configured so that the third guide route GR3 is displayed after the second guide route GR2 is displayed, and the fourth guide route GR4 is displayed after the third guide route GR3 is displayed.

[0139] The above describes the preferred embodiments of the present application, but the configuration of the present application is not limited to these embodiments. For example, it can be changed in the following manner.

[0140] • In the above embodiment, the first to fourth guide routes GR1, GR2, GR3, GR4 are in a band shape having a width corresponding to the width W of the forklift 1, but can be an arrow, a line flickering at a certain cycle, a thin line, or the like.

[0141] • Since there is also a case where there is no passageway 103, the approach area AR can be set on the floor of the facility 100, and need not be set on the passageway 103.

[0142] In the present embodiment, the first and second objects 110 and 111 are determined by the attachment / detachment object determination section 232, but the first and second objects 110 and 111 can be determined by the operator designating the first and second objects 110 and 111 displayed on the display section 20 using a mouse or a touch panel or the like.

[0143] The effects of the present application will be described.

[0144] The present application is a remote operation system including a facility 100 provided with a rack 102 having a plurality of storage portions in a vertical direction, and a forklift 1 that travels and performs attachment / detachment work in the facility 100 and is remotely operated. In addition, the remote operation system includes a camera 10 provided to the forklift 1, and a display section 20 that displays an image captured by the camera 10. In addition, the remote operation system includes an attachment / detachment object position determination section 233 that determines the positions of a first attachment / detachment object 110 that is a next object to be attached / detached and a second attachment / detachment object 111 that is a next next object to be attached / detached. Further, the remote operation system includes a first guide route display section 234 configured to display a first guide route GR1 for guiding a horizontal position of the first attachment / detachment object 110 in a horizontal direction on an image displayed on the display section 20. Further, the remote operation system includes a second guide route display section 235 configured to display a second guide route GR2 for guiding a vertical position of the first attachment / detachment object 110 in a vertical direction on the image displayed on the display section 20. Further, the remote operation system includes a third guide route display section 237 configured to display a third guide route GR3 for guiding a horizontal position of the second attachment / detachment object 111 in the horizontal direction on the image displayed on the display section 20. Further, the remote operation system includes a fourth guide route display section 238 configured to display a fourth guide route GR4 for guiding a vertical position of the second attachment / detachment object 111 in the vertical direction on the image displayed on the display section 20.

[0145] The operator can remotely operate the forklift 1 based on the image displayed on the display section 20. Also, the operator can travel the forklift 1 to the first attachment / detachment object 110 and stop along the first guide route GR1 displayed on the image on the display section 20. Also, the operator can raise and stop the forks 17 along the second guide route GR2 displayed on the image on the display section 20, and thus can recognize without losing the position of the first attachment / detachment object 110.

[0146] Further, the operator can visually recognize the position up to the second load-unload object 111 by displaying the third guide route GR3 and the fourth guide route GR4 on the display portion 20, and thus can immediately take action for the load-unload of the second load-unload object 111 after the load-unload of the first load-unload object 110 is completed.

[0147] Further, the second to fourth guide route display portions 235, 237, 238 display the second to fourth guide routes GR2, GR3, GR4 when the forklift 1 enters inside the approach area AR to a prescribed range from the first load-unload object 110.

[0148] Therefore, the second to fourth guide routes GR2, GR3, GR4 are not displayed on the display portion 20 when the forklift 1 is away from the first load-unload object 110, and the second to fourth guide routes GR2, GR3, GR4 are displayed on the display portion when the forklift 1 approaches the load-unload object 110. Thus, the operator can visually recognize that the first load-unload object 110 has been approached by displaying the second to fourth guide routes GR2, GR3, GR4 on the display portion 20.

[0149] Further, the approach area AR is composed of the first to third approach areas AR1, AR2, AR3. The first approach area AR1 is disposed away from the first load-unload object 110, the second approach area AR2 is disposed closer to the first load-unload object 110 than the first approach area AR1, and the third approach area AR3 is disposed closer to the first load-unload object 110 than the second approach area AR2. Further, the second guide route display portion 235 displays the second guide route GR2 when the forklift 1 enters inside the first approach area AR1. The third guide route display portion 237 displays the third guide route GR3 when the forklift 1 enters inside the second approach area AR2. Further, the fourth guide route display portion 238 displays the fourth guide route GR4 when the forklift 1 enters inside the third approach area AR3.

[0150] Therefore, the operator can quickly and easily grasp the order of operations for the load-unload of the first load-unload object 110 and the second load-unload object 111 by sequentially displaying the second guide route GR2, the third guide route GR3, and the fourth guide route GR4 on the screen of the display portion 20.

[0151] Further, if the third guide route GR3 and the fourth guide route GR4 are displayed on the screen of the display section 20 at a position where the fork truck 1 is away from the first loading and unloading object 110, the second to fourth guide routes GR2, GR3, GR4 are displayed in superimposition at an early stage, and thus there is a possibility that the operator erroneously recognizes the fourth guide route GR4 as the position of the first loading and unloading object 110 or gets lost at the position of the first loading and unloading object 110, but by displaying the third guide route GR3 and the fourth guide route GR4 after the fork truck 1 sufficiently approaches the first loading and unloading object 110, it is possible to prevent the recognition error or the getting lost of the operator with respect to the position of the first loading and unloading object 110. Therefore, the first to third approach regions AR1, AR2, AR3 are set so that the third guide route GR3 is displayed after the second guide route GR2 is displayed, and the fourth guide route GR4 is displayed after the third guide route GR3 is displayed.

[0152] Further, the second guide route display section 235 and the fourth guide route display section 238 are configured to determine the second guide route GR2 and the fourth guide route GR4 displayed on the display section 20 from a plurality of second guide routes GR2 and fourth guide routes GR4 in accordance with the vertical positions of the first loading and unloading object 110 and the second loading and unloading object 111.

[0153] Further, the plurality of second guide routes GR2 and fourth guide routes GR4 can include any one or a combination of different shapes, patterns, colors.

[0154] Thus, the operator can visually and easily recognize that the first loading and unloading object 110 and the second loading and unloading object 111 are stored in the storage section of which floor of the shelf 102 by confirming the second guide route GR2 and the fourth guide route GR4.

[0155] Further, the first guide route display section 234 and the third guide route display section 237 are configured to display the first guide route GR1 and the third guide route GR3 extending in the horizontal direction on the image displayed on the display section 20, and the second guide route display section 235 and the fourth guide route display section 238 are configured to display the second guide route GR2 and the fourth guide route GR4 extending in the vertical direction from the front ends E1, E2 of the first guide route GR1 and the third guide route GR3 on the image.

[0156] Thus, the operator can visually recognize that the first loading and unloading object 110 and the second loading and unloading object 111 are located at the front ends E1, E2 of the first guide route GR1 and the third guide route GR3 and in the vertical direction of the second guide route GR2 and the fourth guide route GR4.

[0157] Further, a passage 103 for the forklift 1 to travel and perform loading and unloading work is formed between the adjacent racks 102 in the facility 100, and the first to fourth guide route display sections 234, 235, 237, 238 display the first to fourth guide routes GR1, GR2, GR3, GR4 on the passage 103 in the image.

[0158] By setting the passage 103 in the facility 100, the forklift 1 travels along the passage 103 to a desired position of the rack 102, and loads and unloads the first and second loading and unloading objects 110, 111 on the rack 102 on the passage 103. Therefore, by displaying the first to fourth guide routes GR1, GR2, GR3, GR4 on the passage 103 displayed on the image of the display section 20 by the first to fourth guide route display sections 234, 235, 237, 238, the operator can appropriately operate the forklift 1 traveling and performing loading and unloading work on the passage 103 even remotely.

[0159] Further, it is desirable that the first to fourth guide route display sections 234, 235, 237, 238 are configured to display the first to fourth guide routes GR1, GR2, GR3, GR4 having a width corresponding to the width of the forklift 1.

[0160] Thus, the operator can intuitively conceive the actual travel route of the forklift 1 and the actual lifting route of the forks 17 by the first to fourth guide routes GR1, GR2, GR3, GR4 displayed on the image of the display section 20.

Claims

1. A teleoperation system, comprising: A facility provided with a shelf having a plurality of storage portions in a vertical direction; and a forklift that travels and performs loading and unloading work in the facility, and is remotely operated, the remote operation system characterized by comprising: a camera provided to the forklift; a display portion that displays an image captured by the camera; a loading and unloading object position determination portion that determines a position of a first loading and unloading object that is a next object to be loaded and unloaded and a position of a second loading and unloading object that is a next next object to be loaded and unloaded; a first guide route display portion that displays a first guide route for guiding a horizontal position of the first loading and unloading object in a horizontal direction on the image displayed on the display portion; a second guide route display portion that displays a second guide route for guiding a vertical position of the first loading and unloading object in a vertical direction on the image displayed on the display portion; a third guide route display portion that displays a third guide route for guiding a horizontal position of the second loading and unloading object in the horizontal direction on the image displayed on the display portion; and a fourth guide route display portion that displays a fourth guide route for guiding a vertical position of the second loading and unloading object in the vertical direction on the image displayed on the display portion.

2. The remote operation system according to claim 1, wherein the second guide route display portion to the fourth guide route display portion display the second guide route to the fourth guide route when the forklift enters an inside of a proximity area that is a prescribed range from the first loading and unloading object.

3. The remote operation system according to claim 2, wherein the proximity area is constituted by a first proximity area to a third proximity area, the first proximity area is disposed away from the first loading and unloading object, the second proximity area is disposed closer to the first loading and unloading object than the first proximity area, the third proximity area is disposed closer to the first loading and unloading object than the second proximity area, the second guide route display portion displays the second guide route when the forklift enters the inside of the first proximity area, the third guide route display portion displays the third guide route when the forklift enters the inside of the second proximity area, the fourth guide route display portion displays the fourth guide route when the forklift enters the inside of the third proximity area.

4. The remote operation system according to claim 1, wherein the second guide route display portion and the fourth guide route display portion determine the second guide route and the fourth guide route displayed on the display portion from a plurality of the second guide routes and the fourth guide routes according to a vertical position of the loading and unloading object.

5. The remote operation system according to claim 4, wherein the plurality of the second guide routes and the fourth guide routes include any one or a combination of different shapes, patterns, colors.

6. The remote operation system according to claim 1, wherein The first guide route display and the third guide route display display the first guide route and the third guide route extending in a horizontal direction on the image, The second guide route display and the fourth guide route display display the second guide route and the fourth guide route extending in a vertical direction on the image from the front ends of the first guide route and the third guide route.

7. The remote operation system according to claim 1, wherein A passage for travel and loading and unloading work of the forklift truck is formed between the adjacent racks in the facility, The first guide route display to the fourth guide route display display the first guide route to the fourth guide route on the passage in the image.

8. The remote operation system according to claim 1, wherein The first guide route display to the fourth guide route display display the first guide route to the fourth guide route having a width equivalent to the width of the forklift truck.

Images