Support assistance system

Abstract

To provide a support assistance system that enables support personnel to more accurately grasp states of mobile devices in a transport device.SOLUTION: A support assistance system 100 in a transport device 1 having mobile devices 13, 21, 23, and 50 moving along a predetermined route includes: a monitoring device 40 having an acquisition unit 41 that acquires information about the states and positions of the mobile devices 13, 21, 23, and 50; and a virtual reality device 60 that generates a virtual transport device 1V having a virtual mobile device 13V capable of moving in a virtual space VC based on the information about the route of the transport device 1 and the mobile devices 13, 21, 23, and 50, and makes a wearer experience the virtual space VS, in which a virtual mobile device 13V moves, based on information about the states and positions of the mobile devices 13, 21, 23, and 50 acquired through the monitoring device 40.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] One aspect of the present invention relates to a support assistance system. [Background technology]

[0002] For example, in an automated warehouse (conveyance device) with mobile equipment, if a problem occurs with the mobile equipment, a support person at a remote support center or the like would understand the current situation based on explanations from on-site workers or photos sent from the site. Also, from a remote location, for example, a system that displays the operating status of the automated warehouse, such as that shown in Patent Document 1, would be accessed and the current situation would be understood based on the numerical information displayed on the system. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2000-309409 Summary of the Invention [Problem to be solved by the invention]

[0004] However, with this method, although the support person can recognize that a problem has occurred in the mobile device, it is difficult for the support person to understand the situation (state) in which the problem has occurred in the mobile device.

[0005] Therefore, an object of one aspect of the present invention is to provide a support assistance system that enables a support person to more accurately grasp the status of a mobile device in a transport device. [Means for solving the problem]

[0006] (1) A support support system according to one aspect of the present invention is a support support system for a transport device having a mobile device that moves along a predetermined route, and includes a monitoring device having an acquisition unit that acquires information regarding the status and position of the mobile device, and a virtual reality device that generates a virtual transport device having a virtual mobile device that can move in a virtual space based on information regarding the transport device and the route of the mobile device, and allows the wearer to experience the virtual space in which the virtual mobile device moves based on information regarding the status and position of the mobile device acquired via the monitoring device.

[0007] The support support system with this configuration allows the wearer, who is the support person, to experience a virtual space in which a virtual transport device that replicates a real transport device travels. This makes it easier for the support person to understand the configuration of the real transport device. Furthermore, the support support system with this configuration can move the virtual mobile device generated in the virtual space in almost real time based on information about the status and position of the mobile device acquired by the monitoring device. This allows the support person to understand the current status of the real transport device. As a result, the support person can more accurately understand the status of the mobile device on the transport device.

[0008] (2) In the support assistance system described in (1) above, the virtual reality device may generate a virtual operation terminal in the virtual space that can receive operations via an input device, and the transport device may control the operation of the mobile device based on the operation content received via the virtual operation terminal. In this configuration, the support person can operate the virtual mobile device while viewing the virtual mobile device generated in the virtual space.

[0009] (3) In the support assistance system described in (2) above, the virtual reality device may be configured to be switchable between a first mode in which only the operation of the virtual mobile device in the virtual space is controlled based on the operation content received at the virtual operation terminal, and a second mode in which both the operation of the virtual mobile device in the virtual space and the operation of the mobile device are controlled based on the operation content received. This configuration can accommodate, for example, both cases where a user wants to operate the virtual mobile device only in the virtual space and cases where a user wants to operate the virtual mobile device in the virtual space and the real mobile device simultaneously. This allows a user to try to recover from an abnormal state by moving only the virtual mobile device in the virtual space. As a result, a tried and found effective operation can be executed on the real mobile device.

[0010] (4) In the support assistance system described in any one of (1) to (3) above, the monitoring device may have a memory unit that stores information about the state and position of the mobile device acquired by the acquisition unit, and the virtual reality device may allow the wearer to experience a virtual space in which the virtual mobile device moves based on the information about the state and position of the mobile device read from the memory unit. In this configuration, even if the timing at which the monitoring device acquires the information about the state and position of the mobile device varies or there is a delay in communication, the virtual mobile device can be moved based on information about the state and position of the mobile device that is thought to have been acquired at the same time. As a result, it is possible to reproduce movements that are closer to those of a real mobile device.

[0011] (5) In the support assistance system described in any one of (1) to (4) above, the mobile device may be equipped with a camera, the acquisition unit of the monitoring device may acquire images or videos captured by the camera, and the virtual reality device may display the images or videos in the virtual space as information about the virtual mobile device. In this configuration, real camera images or videos are displayed in the virtual space, so that the situation can be grasped by referring to both the virtual display and the real images or videos.

[0012] (6) In the support assistance system described in (5) above, the virtual reality device may generate a second virtual operation terminal in the virtual space that can receive operations via an input device, and may switch on or off the display of images or videos in the virtual space based on the operation content received via the second virtual operation terminal. In this configuration, real images or videos are displayed by performing operations in the virtual space similar to those of the real device, so that images or videos can be viewed only when necessary. As a result, the display of images or videos can be prevented from interfering with the display of the virtual mobile device in the virtual space.

[0013] (7) In the support support system described in (6) above, the second virtual operation terminal may have a display unit for displaying images or videos. In this configuration, by displaying images or videos on the display unit of the virtual operation terminal in the same way as on the real device, it is possible to prevent the support person from getting confused about how to operate the device. [Effects of the Invention]

[0014] According to one aspect of the present invention, a support person can more accurately grasp the status of a mobile device in a transport device. [Brief explanation of the drawings]

[0015] [Figure 1] FIG. 1 is a plan view of an automated warehouse system supported by a support assistance system according to one embodiment. [Figure 2] FIG. 2 is a block diagram showing the functional configuration of the support assistance system. [Figure 3] FIG. 3 is a schematic diagram showing an example of use of the VR system of FIG. [Figure 4] FIG. 4 is a diagram showing an example of a display of the virtual space. [Figure 5] FIG. 5 is a diagram showing another display example of the virtual space. [Figure 6] FIG. 6 is a diagram showing a display example when a virtual operation terminal is displayed in a virtual space. [Figure 7]FIG. 7 is a perspective view of a stacker crane included in the automated warehouse system. [Figure 8] FIG. 8 is a diagram showing an example of a display of a virtual space in a support support system according to a modified example. [Figure 9] FIG. 9 is a diagram showing an example of a virtual operation terminal displayed in a virtual space in a support system according to a modified example. DETAILED DESCRIPTION OF THE INVENTION

[0016] Hereinafter, a support assistance system 100 for an automated warehouse system (transportation device) 1 according to one embodiment will be described with reference to the drawings. In the description of the drawings, the same elements are given the same reference numerals, and duplicated explanations will be omitted.

[0017] First, we will explain the automated warehouse system 1 that is the support target of the support assistance system 100. As shown in Fig. 1, the automated warehouse system 1 is a system that takes out items of the type and quantity that match the order of the shipping destination from each of a plurality of item box warehouses 10 that are provided for each type of item.

[0018] The automated warehouse system 1 includes a plurality of article box warehouses 10, a plurality of loading / unloading conveyors 20, a plurality of automatic guided vehicles 30, and a system controller 3.

[0019] The item box warehouse 10 is an automated warehouse that stores a plurality of item boxes B, each containing different types of items, and can deliver the item boxes B based on an order, and can also re-store the item boxes B that have returned after delivery. The type of item box warehouse 10 is not particularly limited, but the item box warehouse 10 of this embodiment includes a first rack 11, a second rack 12, and a stacker crane (mobile equipment) 13.

[0020] The first rack 11 is a shelf for storing item boxes B. In this modified example, the first rack 11 is configured so that multiple item boxes B can be placed on it along one horizontal direction. The first rack 11 is arranged in multiple tiers along the vertical direction. The second rack 12 is arranged opposite the first rack 11 with the stacker crane 13 in between. The automated warehouse system 1 may be equipped with multiple item box warehouses 10 for storing the same type of item boxes B. The number of item box warehouses 10 equipped in the automated warehouse system 1 is not particularly limited. Furthermore, the type and size of the item box warehouse 10 (the number of item boxes B stored) are not particularly limited, and the types and sizes of the multiple item box warehouses 10 may be different from each other.

[0021] The stacker crane 13 transports the item boxes B within the item box warehouse 10. As shown in Figures 1 and 7, the stacker crane 13 has a running section 13A that can run on rails (predetermined paths) 14 that extend along the extension direction of the first rack 11 and the second rack 12, a mast 13B that stands upright in the vertical direction from the running section 13A, and a transfer device 13C that can rise and fall along the mast (predetermined paths) 13B and transfer the item boxes B to each of the placement sections of the first rack 11, the second rack 12, the incoming conveyor 21, and the outgoing conveyor 23.

[0022] Returning to FIG. 1 , each of the multiple inbound and outbound conveyors 20 has an inbound conveyor (moving equipment) 21 and an outbound conveyor (moving equipment) 23. The inbound conveyor 21 and the outbound conveyor 23 are arranged opposite each other with the travel path of the stacker crane 13 in between. The inbound conveyor 21 and the outbound conveyor 23 are, for example, belt conveyors. The inbound conveyor 21 transports the item box B handed over from the automated guided vehicle 30 and hands it over to the stacker crane 13. The outbound conveyor 23 transports the item box B handed over from the stacker crane 13 and hands it over to the automated guided vehicle 30.

[0023] The automated guided vehicle 30 is supplied with power from a power storage device and travels along a predetermined route. The predetermined route may be, for example, magnetic tape or magnetic markers provided on the floor of a building to define a travel route for the automated guided vehicle 30. In this case, the automated guided vehicle 30 travels while detecting the magnetic tape or magnetic markers. The automated guided vehicle 30 may travel while detecting its own position by generating light from a laser or the like and detecting the light reflected by a mirror attached to the wall of the building, or it may travel while detecting its own position using known SLAM (Simultaneous Localization and Mapping) technology or the like.

[0024] The system controller 3 is an electronic control unit including a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), etc. The system controller 3 controls the operations of the stacker crane 13, the infeed conveyor 21, the outfeed conveyor 23, and the automated guided vehicle 30. The system controller 3 can be configured as software in which a program stored in the ROM is loaded onto the RAM and executed by the CPU, for example. The system controller 3 may also be configured as hardware including electronic circuits, etc.

[0025] The system controller 3 controls the operations of the stacker crane 13, the storage conveyor 21, the delivery conveyor 23, and the automated guided vehicle 30 based on transport commands transmitted from a higher-level controller (not shown). The system controller 3 also controls the operations of the stacker crane 13, the storage conveyor 21, the delivery conveyor 23, and the automated guided vehicle 30 based on operation details received via a virtual operation terminal (first virtual operation terminal) 70V, which will be described in detail later.

[0026] 2, the support support system 100 for the automated warehouse system 1 as described above includes a monitoring device 40 and a virtual reality device 60. For example, when a problem occurs in the automated warehouse system 1, the support support system 100 is a system that supports a support person at a support center or the like located remotely from the location where the automated warehouse system 1 is installed, to grasp the status of the automated warehouse system 1.

[0027] The monitoring device 40 is provided so as to be able to communicate with the system controller 3 of the automated warehouse system 1 and the virtual reality device 60. The monitoring device 40 monitors the automated warehouse system 1. The monitoring device 40 includes an acquisition unit 41, a machine information storage unit 42, a status information storage unit (storage unit) 43, and a monitoring controller 44. The monitoring device 40 may include a monitor or the like that displays the status of the automated warehouse system 1.

[0028] The acquisition unit 41 acquires information regarding the status and position of the stacker crane 13, the storage conveyor 21, the outgoing conveyor 23, and the automated guided vehicle 30. The acquisition unit 41 may acquire the status and position of the stacker crane 13, the storage conveyor 21, the outgoing conveyor 23, and the automated guided vehicle 30 via the system controller 3, or may acquire the information directly from the stacker crane 13, the storage conveyor 21, the outgoing conveyor 23, and the automated guided vehicle 30. Examples of the status of the stacker crane 13, the storage conveyor 21, the outgoing conveyor 23, and the automated guided vehicle 30 include information such as whether they are traveling, whether they are transporting an item box B, whether they are moving based on a transport command, whether there is an abnormality, measurement values of various sensors used for position recognition, traveling speed, traveling direction of the vehicle body, steering angle, detection status by obstacle sensors, and operating status of the vehicle transfer device. Examples of the positions of the stacker crane 13, the inlet conveyor 21, the outlet conveyor 23, and the automated guided vehicle 30 include information on their relative positions with respect to the rails 14, etc., and their own positions as described above.

[0029] The machine information storage unit 42 is configured with an SSD (Solid State Drive), HDD (Hard Disk Drive), etc. The machine information storage unit 42 stores information regarding the shape, size, material, etc. of each part that configures the item box warehouse 10, the loading / unloading conveyor 20, and the automated guided vehicle 30. The machine information storage unit 42 also stores information regarding the movement paths of the stacker crane 13, the loading / unloading conveyor 21, the unloading conveyor 23, and the automated guided vehicle 30. The machine information storage unit 42 also stores the shape and size of the operation terminals that operate the item box warehouse 10, the loading / unloading conveyor 20, and the automated guided vehicle 30, and the screen configuration (display content) displayed on the screen of the operation terminal. The various pieces of information stored in the machine information storage unit 42 are used by the virtual reality device 60, which will be described in detail later, to generate a virtual space VS.

[0030] The status information storage unit 43 is configured with an SSD, HDD, etc. The status information storage unit 43 stores information regarding the status and positions of the stacker crane 13, the storage conveyor 21, the delivery conveyor 23, and the automated guided vehicle 30 acquired by the acquisition unit 41. The various pieces of information stored in the status information storage unit 43 are used by the virtual reality device 60, which will be described in detail later, to move the virtual stacker crane 13V, the virtual storage conveyor (not shown), the virtual delivery conveyor (not shown), and the virtual automated guided vehicle (not shown) generated in the virtual space VS as shown in FIG.

[0031] The monitoring controller 44 controls the exchange of various information between the monitoring device 40 and the system controller 3, between the monitoring device 40 and the virtual reality device 60, and within the monitoring device 40.

[0032] The virtual reality device 60 is a system that uses VR (Virtual Reality) technology to generate a virtual space VS of a logistics facility and allows a person wearing a head-mounted display 65 to experience the virtual space VS. The virtual reality device 60 is provided so as to be able to communicate with the system controller 3 and the monitoring device 40 of the automated warehouse system 1. The virtual reality device 60 includes a VR controller 61, a head-mounted display 65, and an input device 67. The VR controller 61 is built into the head-mounted display 65.

[0033] The VR controller 61 may be configured by a personal computer having a ROM (Read Only Memory) storing programs and the like, a RAM (Random Access Memory) for temporarily storing data, a storage medium such as an HDD (Hard Disk Drive), a CPU (Central Processing Unit), and a communication circuit such as a wireless LAN. The VR controller 61 includes software in which a program stored in the ROM is loaded onto the RAM and executed by the CPU. The VR controller 61 may be configured as hardware such as electronic circuits. The VR controller 61 may be configured as a single device or multiple devices. When configured as multiple devices, these devices are connected via a communication network such as the Internet or an intranet to logically configure a single VR controller 61.

[0034] The VR controller 61 generates a virtual space VS of the logistics facility. Based on information regarding the shape, size, material, etc. of each part constituting the goods box warehouse 10, the loading / unloading conveyor 20, and the automated guided vehicle 30, as well as information regarding the movement paths of the stacker crane 13, the loading / unloading conveyor 21, the unloading conveyor 23, and the automated guided vehicle 30, the VR controller 61 generates a virtual automated warehouse system (virtual transport device) 1V, as shown in FIGS. 4 and 5, that includes a virtual stacker crane (virtual mobile device) 13V that can move in the virtual space VS, a virtual loading conveyor (virtual mobile device), a virtual unloading conveyor (virtual mobile device), and a virtual automated guided vehicle (virtual mobile device). The virtual space VS is a space that can be viewed by a support person wearing a head-mounted display 65, which will be described in detail later.

[0035] The method for generating the virtual space VS is not particularly limited, and various known methods can be used. For example, information regarding the shape, size, material, etc. of each part constituting the item box warehouse 10, the loading / unloading conveyor 20, and the automated guided vehicle 30 may be an image of the automated warehouse system 1 taken with a 360° camera, or may be 3D CAD data of the automated warehouse system 1.

[0036] The VR controller 61 moves the virtual stacker crane 13V, the virtual inbound conveyor, the virtual outbound conveyor, and the virtual automated guided vehicle, which are generated in the virtual space VS, based on information relating to the status and positions of the stacker crane 13, the inbound conveyor 21, the outbound conveyor 23, and the automated guided vehicle 30 acquired via the monitoring device 40. Note that the VR controller 61 of this embodiment does not acquire information relating to the status and positions of the stacker crane 13, the inbound conveyor 21, the outbound conveyor 23, and the automated guided vehicle 30 directly from the acquisition unit 41 of the monitoring device 40, but acquires it from the status information storage unit 43.

[0037] The head-mounted display 65 is a wearable terminal that is worn on the head of the support person so as to cover the support person's eyes. The head-mounted display 65 outputs a virtual space VS to the support person wearing it. Specifically, the head-mounted display 65 receives information about the virtual space VS from the VR controller 61, and displays a field-of-view image of the support person within the virtual space VS. The head-mounted display 65 is not particularly limited, and various types of head-mounted displays may be used. The head-mounted display 65 may be either a goggle type or a glasses type.

[0038] The head mounted display 65 employs an inside-out system, and acquires information about the support person (for example, information about at least one of the support person's posture, position, movement, gesture, voice, biometric data, line of sight, and direction) based on the detection results of various sensors mounted on the head mounted display 65 itself. That is, the head mounted display 65 has a function of accepting various inputs from the support person wearing it. The head mounted display 65 outputs the accepted inputs to the VR controller 61.

[0039] The input device 67 is a terminal that accepts various inputs from the support person. The input device 67 is, for example, a device that is held by the support person. The input device 67 here can accept, from the support person, operations regarding the operation of the virtual stacker crane 13V, the virtual inbound conveyor, the virtual outbound conveyor, and the virtual automated guided vehicle that travel in the virtual space VS. The input device 67 outputs the accepted inputs to the VR controller 61. The input device 67 is not particularly limited, and various types of devices may be used.

[0040] The VR controller 61 of this embodiment generates a virtual operation terminal 70V in the virtual space that can receive operations via an input device 67. The virtual operation terminal 70V is displayed in the virtual space VS with the same shape, size, and screen configuration (display content) as a real operation terminal (not shown). As shown in FIG. 6 , a support person wearing a head-mounted display 65 can view the virtual operation terminal 70V through the head-mounted display 65. A support person holding the input device 67 can operate the virtual operation terminal 70V that can be viewed through the head-mounted display 65. That is, the VR controller 61 of this embodiment can receive operations from the support person regarding the operation of the virtual stacker crane 13V, the virtual inbound conveyor, the virtual outbound conveyor, and the virtual automated guided vehicle that travel within the virtual space VS.

[0041] The VR controller 61 transmits the operation content received via the virtual operation terminal 70V to the system controller 3. The system controller 3, which receives the operation content transmitted from the VR controller 61, controls the operations of the stacker crane 13, the receiving conveyor 21, the discharging conveyor 23, and the automatic guided vehicle 30 based on the content.

[0042] The VR controller 61 is configured to be switchable between a first mode and a second mode. The first mode is a mode for controlling the operation of only the virtual stacker crane 13V, virtual inbound conveyor, virtual outbound conveyor, and virtual automated guided vehicle generated in the virtual space VS based on the operation content received at the virtual operation terminal 70V. The second mode is a mode for controlling both the operation of the virtual stacker crane 13V, virtual inbound conveyor, virtual outbound conveyor, and virtual automated guided vehicle generated in the virtual space VS, and the operation of the real stacker crane 13, inbound conveyor 21, outbound conveyor 23, and automated guided vehicle 30 based on the operation content received at the virtual operation terminal 70V.

[0043] In addition to the virtual operation terminal 70V, the VR controller 61 also displays messages regarding operation methods to a support person wearing a head-mounted display 65, and notifies the support person by surrounding the mobile device to be operated (e.g., a virtual stacker crane 13V) with a frame W.

[0044] The effects of the support support system 100 of the above embodiment will be described. The support support system 100 of the above embodiment can show a virtual stacker crane 13V, which is a reproduction of the actual stacker crane 13, storage conveyor 21, delivery conveyor 23, and automated guided vehicle 30, as well as a virtual space VS in which the virtual storage conveyor, virtual delivery conveyor, and virtual automated guided vehicle travel, to a support person wearing a head-mounted display 65. This makes it easier for the support person to understand the configuration of the actual stacker crane 13, storage conveyor 21, delivery conveyor 23, and automated guided vehicle 30.

[0045] Furthermore, in the virtual space VS, the support person can easily grasp the status of each part from their own perspective (the relative positions of the problem area and its surrounding devices) and the status of the area around the problem area (the location of fences, racks, space, etc.). Furthermore, in the virtual space VS, the support person can enter areas that are structurally difficult to access, and can grasp the status of each part from that perspective. This intuitive understanding cannot be obtained from drawings or photographs, and is an effect unique to the use of the virtual reality device 60. Furthermore, the support assistance system 100 configured as described above has the effect of enabling such understanding, for example, when a problem occurs at a high altitude, by checking in advance whether a ladder or other equipment needs to be prepared and where the work can be carried out before proceeding to the actual work on site.

[0046] Furthermore, in the support assistance system 100 configured as described above, the virtual stacker crane 13V, virtual inbound conveyor 21, virtual outbound conveyor 23, and virtual automated guided vehicle 30 generated in the virtual space VS can be moved in almost real time based on information about the status and position of the stacker crane 13, inbound conveyor 21, outbound conveyor 23, and automated guided vehicle 30 acquired by the monitoring device 40. This allows the support person to grasp the current status of the real stacker crane 13, inbound conveyor 21, outbound conveyor 23, and automated guided vehicle 30. As a result, the support person can more accurately grasp the status of the stacker crane 13, inbound conveyor 21, outbound conveyor 23, and automated guided vehicle 30 in the automated warehouse system 1.

[0047] In the support assistance system 100 of the above embodiment, the virtual reality device 60 generates a virtual operation terminal 70V in the virtual space VS that can receive operations via the input device 67. The system controller 3 controls the operations of the stacker crane 13, the storage conveyor 21, the outgoing conveyor 23, and the automated guided vehicle 30 based on the operation content received via the virtual operation terminal 70V. In this configuration, the support person can operate the virtual stacker crane 13V, the virtual storage conveyor, the virtual outgoing conveyor, and the virtual automated guided vehicle while viewing the virtual stacker crane 13V, the virtual storage conveyor, the virtual outgoing conveyor, and the virtual automated guided vehicle generated in the virtual space VS right in front of them. This allows the support personnel to feel as if they are operating the stacker crane 13, the storage conveyor 21, the delivery conveyor 23 and the unmanned transport vehicle 30 while seeing the actual stacker crane 13, the storage conveyor 21, the delivery conveyor 23 and the unmanned transport vehicle 30 right in front of them.

[0048] In the support assistance system 100 of the above embodiment, the virtual reality device 60 is configured to be switchable between a first mode in which the operation of only the virtual stacker crane 13V, virtual inbound conveyor, virtual outbound conveyor, and virtual unmanned guided vehicle in the virtual space VS is controlled based on the operation content received at the virtual operation terminal, and a second mode in which the operation of both the virtual stacker crane 13V, virtual inbound conveyor, virtual outbound conveyor, and virtual unmanned guided vehicle in the virtual space VS and the operation of the stacker crane 13, inbound conveyor 21, outbound conveyor 23, and unmanned guided vehicle 30 are controlled based on the operation content received. This allows for both cases, for example, where one wishes to operate the virtual stacker crane 13V, virtual inbound conveyor, virtual outbound conveyor, and virtual unmanned guided vehicle only in the virtual space VS, and where one wishes to simultaneously operate the virtual stacker crane 13V, virtual inbound conveyor, virtual outbound conveyor, and virtual unmanned guided vehicle in the virtual space VS and the real stacker crane 13, inbound conveyor 21, outbound conveyor 23, and unmanned guided vehicle 30.

[0049] In the support assistance system 100 of the above embodiment, the virtual reality device 60 may move the virtual stacker crane 13V, the virtual inbound conveyor, the virtual outbound conveyor, and the virtual automated guided vehicle based on information about the status and position of the stacker crane 13, the inbound conveyor 21, the outbound conveyor 23, and the automated guided vehicle 30 read from the status information storage unit 43. In this configuration, even if the timing at which the monitoring device 40 acquires information about the status and position of the stacker crane 13, the inbound conveyor 21, the outbound conveyor 23, and the automated guided vehicle 30 differs or a communication delay occurs, the virtual stacker crane 13V, the virtual inbound conveyor, the virtual outbound conveyor, and the virtual automated guided vehicle can be moved based on information about the status and position of the stacker crane 13, the inbound conveyor 21, the outbound conveyor 23, and the automated guided vehicle 30 that is thought to have been acquired at the same time. As a result, it becomes possible to reproduce movements of the stacker crane 13, the inlet conveyor 21, the outlet conveyor 23 and the automatic guided vehicle 30 that are closer to the real movements.

[0050] Although one embodiment has been described above, one aspect of the present invention is not limited to the above embodiment, and various modifications are possible without departing from the spirit of the invention.

[0051] In a support assistance system 100 according to a modified example, the automated warehouse system 1 to be supported is equipped with a camera 13D in addition to the configuration of the stacker crane 13 of the above embodiment, as shown in Fig. 7. As shown in Fig. 7, the camera 13D is provided on a transfer device 13C that can rise and fall along a mast 13B. The transfer device 13C has an arm 13Ca that pulls in and pushes out item boxes B from each of the placement sections of the first rack 11, the second rack 12, the incoming conveyor 21, and the outgoing conveyor 23, and a placement table 13Cb on which the item boxes B are placed.

[0052] The camera 13D is disposed so as to be able to capture an image of the placement table 13Cb. More specifically, the camera 13D is provided so as to be able to capture an image of the state of the item box B placed on the placement table 13Cb. The video or image (still image) captured by the camera 13D is acquired by the acquisition unit 41. The camera 13D is equipped with, for example, a fisheye lens, and can capture a video or image such as that shown in FIG. 8. In the following, a description will be given assuming that a video is captured by the camera 13D.

[0053] The VR controller 61 displays a video (hereinafter simply referred to as a "captured video") captured by the camera 13D via the monitoring device 40 in the virtual space VS. As shown in FIG. 8, the VR controller 61 can display the captured video within a frame W1, for example, and move the frame W (i.e., the captured video) to any position in the virtual space VS. Instead of displaying the captured video within such a frame W1, the VR controller 61 may display the captured video at a fixed position on the right side of the virtual space VS, for example. In the configuration of the support support system 100 according to this modified example, a video of the real space is displayed within the virtual space VS, so that the situation can be grasped by referring to both the virtual display and the real captured video.

[0054] The VR controller 61 provides the support person wearing the head-mounted display 65 with a menu (command) to select whether or not to display the captured video. The support person can switch the display of the captured video ON / OFF by operating the menu via the input device 67. The VR controller 61 may, for example, display menu buttons or the like that can be operated via the input device 67 at a predetermined position in the virtual space VS, or may display a virtual operation terminal (first virtual operation terminal) 70V as shown in the above-mentioned FIG. 6 and display menu buttons or the like that can be operated via the input device 67 on the virtual operation terminal 70V.

[0055] In the configuration of the support assistance system 100 relating to such a modified example, a real captured video is displayed by performing the same operations as with real equipment within the virtual space VS, so that the captured video can be viewed only when necessary, and the display of the captured video can be prevented from interfering with the display of virtual moving equipment in the virtual space VS (virtual stacker crane 13V, virtual inbound conveyor, virtual outbound conveyor, virtual unmanned guided vehicle, etc.).

[0056] The VR controller 61 may generate a virtual operation terminal (second virtual operation terminal) 170V including a display unit 172V that displays a captured video, as shown in Fig. 9, and display the captured video on the display unit 172V. Switching between displaying on the display unit 172V may be realized by an image ON button 173V and an image OFF button 174V that are generated on the virtual operation terminal 170V and can be operated via the input device 67. In the configuration of the support support system 100 according to such a modification, displaying on the display unit 172V of the operation terminal in the same way as a real device can prevent the support person from getting confused about how to operate it.

[0057] In the above-described modified example, the camera 13D is positioned so as to be able to capture an image of the platform 13Cb of the transfer device 13C. However, the present invention is not limited to this. For example, the camera 13D may be positioned at the front or rear end of the travel section 13A so as to capture an image of the travel section of the stacker crane 13. The camera 13D may also be provided at multiple locations on the stacker crane 13, or may be provided on other mobile equipment such as an inbound conveyor, an outbound conveyor, or an automated guided vehicle. When multiple cameras 13D are provided, the captured videos from all the cameras 13D may be displayed in the virtual space VS, or may be displayed in one location while being switched sequentially. Alternatively, a menu for selecting which camera 13D's captured video to display in the virtual space VS may be displayed in the virtual space VS or in the virtual operation terminal 170V displayed in the virtual space VS (e.g., an image selection button 75V).

[0058] In the above embodiment and modified example, the support assistance system 100 has been described with reference to an example in which the support target is an automated warehouse system 1 having a stacker crane 13, an inlet conveyor 21, an outlet conveyor 23, and an automated guided vehicle 30, but the present invention is not limited to this. For example, the support assistance system 100 may support only an automated warehouse (conveyance device) equipped with a stacker crane (mobile equipment) 13, or may support only a conveyance system (conveyance device) equipped with a plurality of automated guided vehicles (mobile equipment) 30. Furthermore, the support assistance system 100 may also support a system that includes a rail-guided vehicle system.

[0059] The item box warehouse 10 to be supported by the support assistance system 100 in the above embodiment and modified example may be configured to include, instead of the stacker crane 13, a lift device (moving equipment) that is provided so as to be able to move up and down in the vertical direction, and a plurality of horizontal conveying devices (moving equipment) that are provided so as to be able to travel horizontally for each rack level between the first rack 11 and the second rack 12. Even an automated warehouse with such a configuration can be supported by the support assistance system 100.

[0060] The support assistance system 100 in the above embodiment and modified example has been explained using an example in which the virtual stacker crane 13V, virtual inbound conveyor, virtual outbound conveyor, and virtual unmanned guided vehicle are moved based on information about the status and position of the stacker crane 13, inbound conveyor 21, outbound conveyor 23, and unmanned guided vehicle 30 read from the status information memory unit 43, but the virtual stacker crane 13V, virtual inbound conveyor, virtual outbound conveyor, and virtual unmanned guided vehicle may also be moved based on information about the status and position transmitted directly from the system controller 3 or the stacker crane 13, inbound conveyor 21, outbound conveyor 23, and unmanned guided vehicle 30.

[0061] In the support support system 100 of the above embodiment and the above modified example, the virtual operation terminal (first virtual operation terminal) 70V and the virtual operation terminal (second virtual operation terminal) 170V are described as separate virtual operation terminals, but it is also possible to use the configuration of either virtual operation terminal in combination. [Explanation of symbols]

[0062] 1...Automated warehouse system (conveying device), 3...System controller, 10...Goods box warehouse, 13...Stacker crane (mobile equipment), 13V...Virtual stacker crane, 20...Input / output conveyor, 21...Input conveyor (mobile equipment), 23...Output conveyor (mobile equipment), 30...Automated guided vehicle (mobile equipment), 40...Monitoring device, 41...Acquisition unit, 42...Machine information memory unit, 43...Status information memory unit (memory unit), 44...Monitoring controller, 60...Virtual reality device, 61...VR controller, 65...Head-mounted display, 67...Input device, 70V...Virtual operation terminal (first virtual operation terminal), 100...Support assistance system, 170V...Virtual operation terminal (second virtual operation terminal), B...Goods box, VS...Virtual space.

Claims

1. A support assistance system for a transport device having a mobile device that moves along a predetermined path, comprising: a monitoring device having an acquisition unit that acquires information about the state and location of the mobile device; A support assistance system comprising: a virtual reality device that generates a virtual transport device having a virtual mobile device that can move in a virtual space based on information about the transport device and the route of the mobile device, and allows a wearer to experience the virtual space in which the virtual mobile device moves based on information about the status and position of the mobile device obtained via the monitoring device.

2. the virtual reality apparatus generates a first virtual operation terminal in the virtual space that can receive operations via an input device; The support assistance system according to claim 1 , wherein the transporting device controls the operation of the mobile device based on the operation content received via the first virtual operation terminal.

3. The support support system of claim 2, wherein the virtual reality device is configured to be switchable between a first mode in which the operation of only the virtual mobile device in the virtual space is controlled based on the operation content received at the first virtual operation terminal, and a second mode in which the operation of both the virtual mobile device in the virtual space and the operation of the mobile device are controlled based on the operation content received.

4. the monitoring device has a storage unit that stores information about the state and location of the mobile device acquired by the acquisition unit; The support support system according to any one of claims 1 to 3, wherein the virtual reality device allows the wearer to experience the virtual space in which the virtual mobile device moves based on information regarding the state and position of the mobile device read from the memory unit.

5. the mobile device comprises a camera; an acquisition unit of the monitoring device acquires images or videos captured by the camera; 4. The support assistance system according to claim 1, wherein the virtual reality device displays the image or the video in the virtual space as information about the virtual mobile device.

6. The support support system described in claim 5, wherein the virtual reality device generates a second virtual operation terminal in the virtual space that can accept operations via an input device, and switches whether or not to display the image or the video in the virtual space based on the operation content accepted via the second virtual operation terminal.

7. 7. The support system according to claim 6, wherein the second virtual operation terminal has a display unit that displays the image or the video.

Images