Maintenance methods and maintenance servers

Abstract

The method for maintaining equipment in an automatic warehouse system (1) using a maintenance terminal (21a, 21b) installed at a maintenance base (2) comprises: a step (S11) of creating a field image (FM) representing the automatic warehouse system (1); a step (S21 to S23) of acquiring access information (AC); a step (S41 to S44) of generating a composite image (SI) by associating the access information (AC) for accessing the corresponding equipment with an object (O) in the field image (FM); a step (S37) of outputting the composite image (SI) to the maintenance terminal (21a, 21b); a step (S3) of accepting selection of an object (O); and a step (S51 to S53) of establishing a communication session between the corresponding equipment and the maintenance terminal (21a, 21b) using the access information (AC) associated with the selected object (O).

Description

Technical Field

[0001] The present invention relates to a maintenance method for equipment in a maintenance work site, a maintenance server used for the maintenance work, and a program for causing a computer to execute the maintenance method. Background Art

[0002] Conventionally, there is a known technique for accessing a server or terminal in a LAN and a terminal outside the LAN via a VPN (Virtual Private Network) (for example, see Patent Document 1). In the system of Patent Document 1, when accessing a server or terminal in a certain LAN from an external terminal via VPN, information for connecting to a gateway of the VPN is sent from a server that manages the information to the external terminal. The information for connecting to the gateway includes the address of the gateway, etc.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent No. 5254909 Summary of the invention

[0006] Problems to be solved by the invention

[0007] It is conceivable to apply the above-mentioned technique of VPN connection between an external terminal and a terminal in a specific LAN to a work site having a plurality of equipment such as an automated warehouse, etc. Thus, the equipment in the work site can be accessed from the external terminal via the network.

[0008] However, conventional VPN connection techniques do not envision the use of multiple devices such as those at a work site. For example, conventional VPN connection techniques provide access information (address) to a gateway connected to a VPN, but do not provide information related to the configuration of terminals and servers within a LAN.

[0009] In a work site with multiple devices, the target device is sometimes identified based on its location in the work site. However, if information related to the device's location is not provided as in conventional VPN connections, it is sometimes difficult to quickly access the target device from an external terminal.

[0010] An object of the present invention is to facilitate and speed up access to equipment from an external terminal in a system in which equipment within a work site is accessed from a terminal outside the work site.

[0011] Means for solving problems

[0012] Hereinafter, a plurality of methods will be described as means for solving the problems. These methods can be arbitrarily combined as needed.

[0013] A maintenance method according to one aspect of the present invention is a method executed by a maintenance server for maintaining equipment in a work site using a maintenance terminal installed at a maintenance base outside the work site. The maintenance method includes the following steps.

[0014] ◎Steps for creating a site image showing the work site.

[0015] ◎Steps for obtaining access information for accessing a device.

[0016] ◎A step of generating a composite image by associating an object representing a device in a live image with access information for accessing the device.

[0017] ◎Steps for outputting the composite image to the maintenance terminal.

[0018] ◎A step of accepting selection of an object in a composite image using a maintenance terminal.

[0019] ◎A step of establishing a communication session between a device corresponding to the selected object and a maintenance terminal using access information associated with the selected object.

[0020] In the above maintenance method for maintaining equipment at a work site using an external maintenance terminal, a synthetic image including objects representing the arrangement and form of equipment at the work site is generated, and the generated synthetic image is output to the external maintenance terminal.

[0021] In addition, when the access information for accessing the corresponding device is associated with the object of the composite image and a certain object within the composite image is selected through the maintenance terminal, a communication session is established between the device corresponding to the object and the maintenance terminal using the access information associated with the selected object.

[0022] Thus, the user of the maintenance terminal can easily identify the device that he wants to access by referring to the synthesized image, and can quickly access the identified device.

[0023] There may be a plurality of work sites. In this case, the on-site image, the access information, and the synthesized image are generated for each of the plurality of work sites.

[0024] Thus, the user of the maintenance terminal can easily identify the equipment present in a plurality of work sites and can quickly access the identified equipment.

[0025] The above maintenance method may also include the following steps.

[0026] ◎A step of receiving, from a device, abnormality information indicating an abnormality occurring in the device.

[0027] ◎A step of generating a composite image by associating abnormality information with an object corresponding to a device in which the abnormality occurred.

[0028] This allows the user of the maintenance terminal to easily and quickly access information related to an abnormality occurring in the device.

[0029] The above maintenance method may also include the following steps.

[0030] ◎When there is a change in equipment in the work site after the synthetic image is generated, a step of generating a new synthetic image using new access information corresponding to the change in the equipment.

[0031] ◎Steps for outputting a new composite image to the maintenance terminal.

[0032] Thus, when there is a change in equipment at the work site, the latest synthesized image reflecting the change can be provided to the maintenance terminal.

[0033] The above maintenance method may further include, when a new composite image is generated, a step of notifying the maintenance terminal that the composite image has been updated.

[0034] This makes it possible to notify the user of the maintenance terminal of the update of the synthesized image and to notify the user of the change in the equipment at the work site.

[0035] The maintenance method may further include a step of displaying a dialog box together with the composite image. The dialog box provides a dialog function between a maintenance worker using the maintenance terminal and a worker using a worker terminal provided at the work site. In this case, a notification that the composite image has been updated is displayed in the dialog box.

[0036] This allows workers at the work site to be notified of updates to the synthetic image and to be notified of changes to equipment at the work site.

[0037] A maintenance server according to another aspect of the present invention is a server for maintaining equipment in a work site using a maintenance terminal installed at a maintenance base outside the work site. The maintenance server includes a control unit.

[0038] The control unit produces a field image representing the work site, obtains access information for accessing the equipment, associates the object representing the equipment in the field image with the access information for accessing the equipment to generate a composite image, outputs the above-mentioned composite image to the maintenance terminal, accepts the selection of the object in the composite image using the maintenance terminal, and uses the access information associated with the selected object to establish a communication session between the equipment corresponding to the selected object and the maintenance terminal.

[0039] In the above-mentioned maintenance server, a composite image including objects representing the arrangement and form of equipment at a work site is generated, and the generated composite image is output to an external maintenance terminal.

[0040] In addition, when the access information for accessing the corresponding device is associated with the object of the composite image and a certain object in the composite image is selected through the maintenance terminal, a communication session is established between the device corresponding to the object and the maintenance terminal using the access information associated with the selected object.

[0041] Thus, the user of the maintenance terminal can easily identify the device that he wants to access by referring to the synthesized image, and can quickly access the identified device.

[0042] There may be a plurality of work sites. In this case, the control unit generates a site image, access information, and a composite image for each of the plurality of work sites.

[0043] Thus, the user of the maintenance terminal can easily identify the equipment present in a plurality of work sites and can quickly access the identified equipment.

[0044] The control unit may receive, from the device, abnormality information indicating an abnormality occurring in the device, and associate the abnormality information with an object corresponding to the device in which the abnormality occurred to generate the composite image.

[0045] This allows the user of the maintenance terminal to easily and quickly access information related to an abnormality occurring in the device.

[0046] When a device in the work site is changed after the synthetic image is generated, the control unit may generate a new synthetic image using new access information corresponding to the change in the device, and output the new synthetic image to the maintenance terminal.

[0047] Thus, when there is a change in equipment at the work site, the latest synthesized image reflecting the change can be provided to the maintenance terminal.

[0048] When a new composite image is generated, the control unit may notify the maintenance terminal that the composite image has been updated.

[0049] This allows the user of the maintenance terminal to be notified of the update of the synthetic image and to be notified of the change in the equipment at the work site.

[0050] The control unit may also display a dialog box together with the composite image. The dialog box provides a dialog function between the maintenance personnel using the maintenance terminal and the operator using the operator terminal installed at the work site. In this case, the control unit displays a notification that the composite image has been updated in the dialog box.

[0051] This allows workers at the work site to be notified of updates to the synthetic image and to be notified of changes to equipment at the work site.

[0052] A program according to another aspect of the present invention is a program for causing a computer to execute the above-mentioned maintenance method.

[0053] Effects of the Invention

[0054] By generating the above-mentioned composite image and outputting it to the external maintenance terminal, the external maintenance terminal can easily identify the device to be accessed by referring to the composite image, and can quickly access the identified device. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 This is a diagram showing an example of the overall configuration of a communication system.

[0056] Figure 2 This is a diagram showing an example of the configuration of an automatic warehouse system.

[0057] Figure 3 This is a diagram showing an example of image data.

[0058] Figure 4 This is a diagram showing an example of measurement data of a sensor.

[0059] Figure 5 This is a diagram showing an example of the configuration of a maintenance base.

[0060] Figure 6 This is a diagram showing an example of maintenance information (part 1).

[0061] Figure 7 This is a diagram showing an example of maintenance information (part 2).

[0062] Figure 8 This is a diagram showing an example of the hardware configuration of the maintenance server.

[0063] Fig. 9 This is a diagram showing the hardware configuration of the relay server.

[0064] Fig. 10A This is a diagram showing an example of a gateway list.

[0065] Fig. 10B This is a diagram showing the structure of the synthetic image creation information.

[0066] Fig.11A This is a diagram showing an example of the data structure of the first device list.

[0067] Fig. 11B This is a diagram showing an example of the data structure of the second device list.

[0068] Fig.12 This is a flowchart showing the general operation of the maintenance server.

[0069] Fig.13 This is a flowchart showing the initial setting operation.

[0070] Fig.14 This is a flowchart showing the operation of obtaining access information.

[0071] Fig.15 This is a diagram showing the transmission and reception of information in the acquisition of access information.

[0072] Fig.16 This is a flowchart showing the operation of the maintenance server in the operation until the synthesized image is output.

[0073] Fig.17 This is a diagram showing the transmission and reception of information during the operation until the synthesized image is output.

[0074] Fig.18 This is a diagram showing an example of a login screen.

[0075] Fig.19 This is a flowchart showing the operation of generating a synthetic image.

[0076] Fig. 20 This is a flowchart showing the operation of establishing a communication session of the maintenance server.

[0077] Fig.21 This is a diagram showing the transmission and reception of information in the communication session establishment operation.

[0078] Fig. 22 This is a flowchart showing the operation of the maintenance server when there is a change in equipment.

[0079] Fig.23 This is a diagram showing the transmission and reception of information when there is a change in equipment.

[0080] Fig.24 This is a diagram showing an example of a notification that a composite image has been updated.

[0081] Fig.25 This is a diagram showing an example of an initial screen.

[0082] Fig.26A This is a diagram showing an example of a synthesized image.

[0083] Fig.26B is an enlargement of a portion of the composite image.

[0084] Fig. 27 This is a diagram showing an example of a display screen on which a composite image and a dialog box are displayed simultaneously.

[0085] Fig.28 This is a diagram showing an example of a display screen that displays access information as a list of characters. DETAILED DESCRIPTION

[0086] 1. First Implementation

[0087] (1) Overall structure of the communication system

[0088] The maintenance server involved in the present invention is described below. The maintenance server is a server for maintaining the equipment installed at the work site using a maintenance terminal installed at a maintenance base located at a place separate from the work site. Therefore, the maintenance server is installed in a communication system including the equipment and terminals inside the work site and the terminals located at the maintenance base.

[0089] Below, use Figure 1 A communication system 100 according to the first embodiment will be described as an example of the communication system. Figure 1 This is a diagram showing an example of the overall configuration of a communication system.

[0090] The communication system 100 according to the first embodiment includes an automatic warehouse system 1 as an example of the above-mentioned work site, a maintenance base 2 , a maintenance server 3 , and a relay server 4 .

[0091] The automatic warehouse system 1 is a system for storing and unloading goods, for example, in a logistics center. The automatic warehouse system 1 has an automatic warehouse 11 (an example of a device) for storing and unloading goods. Figure 2 ) The communication system 100 of this embodiment includes a plurality of automatic warehouse systems 1 .

[0092] The maintenance base 2 is a base where maintenance personnel who perform maintenance on the automatic warehouse 11 in the automatic warehouse system 1 stay, and includes terminals (maintenance terminals 21a and 21b) for maintaining the automatic warehouse system 1. The maintenance terminals 21a and 21b of the maintenance base 2 can access the devices of the plurality of automatic warehouse systems 1.

[0093] The maintenance server 3 is connected to the above-mentioned automatic warehouse system 1 and (gateway device of) the maintenance base 2 via a network NW (such as WAN, etc.). The maintenance server 3 stores a database that stores information related to (each device of) the automatic warehouse 11 stored in the automatic warehouse system 1 (referred to as operation information) and information used for maintaining the equipment managed at the maintenance base 2 (referred to as maintenance information).

[0094] In addition, the maintenance server 3 provides the stored operation information and / or maintenance information to the operator terminal used by the operator who uses the automatic warehouse 11 in the automatic warehouse system 1 and the maintenance terminals 21a and 21b used by the maintenance personnel in the maintenance base 2 as needed. Thus, in the communication system 100, the operation information and maintenance information can be shared between the operator terminal and the maintenance terminals 21a and 21b via the maintenance server 3.

[0095] Furthermore, when the maintenance terminals 21a and 21b are used to maintain the equipment in the automated warehouse system 1, the maintenance server 3 mediates a communication session between the maintenance terminals 21a and 21b and the equipment. How the mediation is performed will be described in detail later.

[0096] The relay server 4 is connected to the automatic warehouse system 1 and (the gateway device of) the maintenance base 2 via the network NW. The relay server 4 is a server that relays the communication between the automatic warehouse system 1 and the maintenance base 2.

[0097] Furthermore, the relay server 4 is connected to the maintenance server 3 via the network NW, and provides various information stored in the relay server 4 to the maintenance server 3 in response to a request from the maintenance server 3 .

[0098] In the communication system 100 of the present embodiment, transmission and reception of commands and the like between each device described later (gateway device, camera and sensor of the automated warehouse system 1 ) and the maintenance server 3 are performed using, for example, the WebSocket protocol.

[0099] In addition, Figure 1 In the communication system 100 shown, there is only one maintenance base 2, but the present invention is not limited thereto, and there may be multiple maintenance bases 2 in the communication system 100. For example, not only can multiple automatic warehouse systems 1 be maintained at one maintenance base 2, but one automatic warehouse system 1 can also be maintained from multiple maintenance bases 2.

[0100] (2)Automatic warehouse system

[0101] (2-1) Automatic warehouse

[0102] Below, use Figure 2 The specific configuration of the automated warehouse system 1 included in the communication system 100 will be described. Figure 2 1 is a diagram showing an example of the configuration of an automatic warehouse system. The plurality of automatic warehouse systems 1 differ only in the number of devices they have, and have the same basic configuration. Therefore, the basic configuration of a representative automatic warehouse system 1 will be described below.

[0103] The automated warehouse system 1 includes an automated warehouse 11. The automated warehouse 11 includes shelves, a stacker crane, a conveyor, and the like, and performs loading and unloading, transfer, and storage of goods.

[0104] In this embodiment, the stacking crane and conveyor of the automatic warehouse 11 are provided with cameras CA1 to CA4 capable of capturing static images and dynamic images of a predetermined length of time, so that the status of the stacking crane and conveyor can be obtained as visual operation information FI. The cameras CA1 to CA4 are, for example, fisheye cameras. Figure 3 The image data (moving image) shown is the operation information FI. Figure 3 This is a diagram showing an example of image data.

[0105] In addition, a sensor SE (for example, a temperature and humidity sensor) for measuring the ambient temperature and humidity is provided near the automatic warehouse 11, and the ambient temperature and humidity can be obtained as the operation information FI. Figure 4 As shown, the data of the measured temperature and humidity are displayed in a line graph or a bar graph as measurement data (operation information FI). Figure 4 This is a diagram showing an example of measurement data of a sensor.

[0106] Furthermore, the stacker crane is provided with a sensor SE for measuring a predetermined physical quantity. When cargo collapse occurs in the stacker crane, the magnitude of vibration and / or sound generated by the cargo collapse can be measured as operation information FI. Examples of such sensors SE include acceleration sensors and sound sensors.

[0107] In addition, Figure 2 In the automated warehouse system 1 shown, four cameras CA1 to CA4 and one sensor SE are installed. However, the number of cameras and sensors installed in the automated warehouse 11 is not limited to the above, and can be set to any number according to the scale of the automated warehouse system 1 and the like.

[0108] (2-2) Network structure of automatic warehouse system

[0109] use Figure 2 The network structure for mutual communication between devices in the automatic warehouse system 1 is further described. The network structure of the automatic warehouse system 1 includes a first switching hub SH1. Figure 2 As shown, the first switching hub SH1 connects the cameras CA1 and CA2 to the first external storage device 13a. The first external storage device 13a is, for example, a NAS (Network Attached Storage) device, and stores image data acquired by the cameras CA1 and CA2.

[0110] Furthermore, control devices (not shown) such as a stacker crane and a conveyor of the automated warehouse 11 are connected to the first switching hub SH1.

[0111] The network structure of the automated warehouse system 1 includes a second switching hub SH2. The second switching hub SH2 connects the cameras CA3 and CA4, the sensor SE, and the second external storage device 13b. The second external storage device 13b is, for example, a NAS device, and stores image data acquired by the cameras CA3 and CA4 and measurement data acquired by the sensor SE.

[0112] The second switching hub SH2 is also connected to the first switching hub SH1, and the image data obtained by the cameras CA1 and CA2 can also be stored in the second external storage device 13b, and the image data obtained by the cameras CA3 and CA4 and the data obtained by the sensor SE can also be stored in the first external storage device 13a.

[0113] The network configuration of the automatic warehouse system 1 includes a server for managing the automatic warehouse system 1. Specifically, the network configuration includes a transport management server 15a and an information management server 15b.

[0114] The transport management server 15a is a computer system having various interfaces such as a CPU, a storage device (RAM, ROM, SSD, hard disk, etc.), and a network interface. The transport management server 15a can communicate with the stacking cranes, conveyors, etc. of the automatic warehouse 11, and control / manage the entry and exit of goods in the automatic warehouse 11 by controlling these stacking cranes and conveyors.

[0115] In addition, in this embodiment, the transport management server 15a is connected to the first switching hub SH1.

[0116] The information management server 15b is a computer system having various interfaces such as a CPU, a storage device (RAM, ROM, SSD, hard disk, etc.), and a network interface. The information management server 15b manages various data (for example, data acquired by the cameras CA1 to CA4 and the sensor SE) stored in the first external storage device 13a and the second external storage device 13b. The information management server 15b acquires information managed by the transmission management server 15a as needed.

[0117] Furthermore, in this embodiment, the information management server 15b is connected to the second switching hub SH2.

[0118] The network structure of the automatic warehouse system 1 has an operator terminal 17. The operator terminal 17 is, for example, a personal computer, a tablet terminal, etc. The operator terminal 17 is a terminal used by operators who perform operations in the automatic warehouse 11, and can communicate with the maintenance server 3 via the network NW. Thus, the operator can use the operator terminal 17 to browse the operation information FI and maintenance information MI stored in the maintenance server 3. In the case where the operator terminal 17 is a tablet terminal, the operator can carry the operator terminal to the location where the equipment (automatic warehouse, camera, sensor) of the maintenance object is set up to perform maintenance work. In addition, in the case where a terminal other than a tablet terminal (for example, a personal computer) is the operator terminal 17, the operator terminal 17 can also access the cameras CA1~CA4, the sensor SE, etc.

[0119] In addition, Figure 2 In the illustrated automatic warehouse system 1 , only one operator terminal 17 is provided, but the present invention is not limited to this, and a plurality of operator terminals 17 may be provided.

[0120] The network structure of the automatic warehouse system 1 has a first gateway device 19. The first gateway device 19 is a computer system having various interfaces such as a CPU, a storage device (RAM, ROM, SSD, hard disk, etc.), and a network interface. In addition, the first gateway device 19 can also be hardware that implements part or all of its functions through SoC, etc.

[0121] The first gateway device 19 is connected to the second switching hub SH2 and the external network NW, and can relay the communication between each device in the network structure of the automatic warehouse system 1 and the outside. Therefore, the first gateway device 19 stores the first device list EL1 (described later) that lists the information related to the devices connected to the first gateway device 19 in the storage device.

[0122] When the first gateway device 19 receives an instruction from the relay server 4 that a communication session should be established between the various devices of the automatic warehouse system 1 (cameras CA1 to CA4, sensor SE, stacker crane and conveyor control device, etc.) and the maintenance terminals 21a, 21b, it executes control to establish a VPN session with the second gateway device 23 (described later) of the maintenance base 2.

[0123] In addition, the first gateway device 19 has a function of uploading the image data of the cameras CA1 to CA4, the measurement data of the sensor SE, and the abnormality information indicating the abnormality occurring in each device, etc. stored in the first external storage device 13a and the second external storage device 13b as operation information FI to the maintenance server 3 at a specified time. In addition, the first gateway device 19 can upload information related to the maintenance of each device, operation information of each device, photos of the device taken by the operator, etc., and information related to the ordering of parts for each device, etc., as operation information FI to the maintenance server 3.

[0124] The above-mentioned specified time can be appropriately determined as, for example, every certain time period (for example, every hour, every day, etc.), the time when new operation information FI is saved, the time when the existing operation information FI is updated, the time when the capacity of the saved operation information FI reaches a determined size, etc.

[0125] (3) Maintenance base

[0126] Below, use Figure 5 The specific configuration of the maintenance base 2 included in the communication system 100 will be described. Figure 5 This is a diagram showing an example of the configuration of a maintenance base.

[0127] The maintenance base 2 has maintenance terminals 21a and 21b. The maintenance terminals 21a and 21b are connected to the third switching hub SH3. The maintenance terminals 21a and 21b are, for example, personal computers, and are used by maintenance personnel stationed at the maintenance base 2 when performing maintenance on the equipment and the like of the automatic warehouse system 1.

[0128] The maintenance base 2 has a second gateway device 23. The second gateway device 23 is a computer system having various interfaces such as a CPU, a storage device (RAM, ROM, SSD, hard disk, etc.), and a network interface. In addition, the second gateway device 23 can also be hardware that implements part or all of its functions through SoC, etc.

[0129] The second gateway device 23 is connected to the third switching hub SH3 and the external network, and relays the communication between the maintenance terminals 21a and 21b and the outside. Therefore, the second gateway device 23 stores the second device list EL2 (described later) that lists the information related to the maintenance terminals 21a and 21b connected to the second gateway device 23 in the storage device.

[0130] The second gateway device 23 executes control to establish a VPN session with the first gateway device 19 after a VPN session establishment request is issued from the first gateway device 19 of the automated warehouse system 1 .

[0131] In addition, the second gateway device 23 has a function of uploading information stored in the backbone system (not shown) provided in the maintenance terminals 21a, 21b and / or the maintenance base 2 as maintenance information MI to the maintenance server 3 at a specified time. The specified time can be appropriately determined as, for example, every certain time (for example, every hour, every day, etc.), when new maintenance information MI is stored, when the existing maintenance information MI is updated, when the capacity of the stored maintenance information MI reaches a specified size, etc.

[0132] Examples of the maintenance information MI include: Figure 6 A list of components of the equipment shown is a parts list, Figure 7 The layout diagram etc. shown in the figure show the arrangement of the equipment (automatic warehouse 11) in the automatic warehouse system 1. Figure 6 as well as Figure 7 2 is a diagram showing an example of maintenance information. The second gateway device 23 can upload the maintenance history of each device of the automated warehouse system 1 and the maintenance report of each device to the maintenance server 3 as maintenance information MI.

[0133] The maintenance server 3 can determine which device in the automatic warehouse system 1 the maintenance information MI received from the second gateway device 23 (maintenance base 2) is related to based on, for example, the file name.

[0134] (4) Server maintenance

[0135] Below, use Figure 8 The specific hardware configuration of the maintenance server 3 included in the communication system 100 will be described. Figure 8 This is a diagram showing an example of the hardware configuration of the maintenance server.

[0136] like Figure 8 As shown, the maintenance server 3 is a computer system including a processor 31 , a RAM 32 , a ROM 33 , a network interface 34 , and a storage device 35 .

[0137] The processor 31 is, for example, a CPU or a SoC in which a CPU and various interfaces are integrated into one chip. The processor 31 executes a program stored in the storage device 35 or the like, and performs various information processing performed in the maintenance server 3 .

[0138] The RAM 32 stores temporarily required information such as the programs stored in the storage device 35. The ROM 33 stores programs and settings for controlling the maintenance server 3.

[0139] Thus, the processor 31, RAM 32, and ROM 33 are hardware components that perform control to realize the functions of the maintenance server 3. Therefore, these three hardware components are hereinafter referred to as the "control unit" of the maintenance server 3. That is, the control unit executes the program stored in the storage device 35 to realize the functions (actions) of the maintenance server 3 described later.

[0140] The network interface 34 performs communication with external terminals and / or devices, etc. The network interface 34 is, for example, an Ethernet (registered trademark) card that communicates with external terminals and / or devices through wired communication, and / or a wireless LAN interface that communicates with external terminals and / or devices through wireless communication, etc.

[0141] When the maintenance server 3 sends various information and data to the outside, the control unit outputs the information and data to the outside via the network interface 34. When the maintenance server 3 receives various information and data from the outside, the control unit inputs the information and data from the outside via the network interface 34.

[0142] The storage device 35 stores programs executed by the maintenance server 3, various information stored in the maintenance server 3, and various setting information. The various information stored in the maintenance server 3 involved in this embodiment is operation information FI related to the equipment (automatic warehouse 11) of the automatic warehouse system 1, and maintenance information MI related to the maintenance of the equipment managed at the maintenance base 2.

[0143] On the other hand, the setting information stored in the storage device 35 of the maintenance server 3 according to the present embodiment is the user information UI, the gateway list GL, and the composite image creation information GI.

[0144] The user information UI is information for managing users who can log in to the maintenance server 3. The gateway list GL is a list for identifying the first gateway device 19 of each automatic warehouse system 1 and the second gateway device 23 of each maintenance base 2. The synthetic image creation information GI is created for each automatic warehouse system 1 and each maintenance base 2 and is used to generate the synthetic image SI of each automatic warehouse system 1 ( Fig.26A ) and information used to determine from which maintenance terminal 21a, 21b of the maintenance base 2 there is access.

[0145] With the above-described configuration, the maintenance server 3 can receive and store the operation information FI on the equipment of the automatic warehouse system 1 and the maintenance information MI on the maintenance of the equipment from the automatic warehouse system 1 and the maintenance base 2 .

[0146] Furthermore, the maintenance server 3 having such a configuration can generate an image (referred to as a composite image SI) by associating an object O (described later) of the field image FM with access information AC to a device corresponding to the object O, as described later. Furthermore, in the composite image SI, operation information FI and maintenance information MI related to the corresponding device can also be associated with each object O.

[0147] In addition to the above configuration, the maintenance server 3 may further include a display 36. The display 36 is a display such as a liquid crystal display or an organic EL display that displays various information on the maintenance server 3. Thus, the user can manage the maintenance server 3 while referring to the information displayed on the display 36, for example.

[0148] The maintenance server 3 may further include an input interface 37 for receiving input from a user. The input interface 37 is, for example, an input device such as a keyboard, a mouse, or a touch panel. Thus, the user can directly manage the maintenance server 3 using the input interface 37 .

[0149] Even without the input interface 37 and / or the display 36 , for example, the maintenance server 3 can be managed by using a terminal that can access the maintenance server 3 via the network interface 34 , while referring to information related to the maintenance server 3 .

[0150] (5) Relay Server

[0151] Below, use Fig. 9 The specific hardware configuration of the relay server 4 included in the communication system 100 will be described. Fig. 9 This is a diagram showing the hardware configuration of the relay server.

[0152] like Fig. 9 As shown, the relay server 4 is a computer system including a processor 41 , a RAM 42 , a ROM 43 , a network interface 44 , and a storage device 45 .

[0153] The processor 41 is, for example, a CPU or a SoC in which a CPU and various interfaces are integrated into one chip. The processor 41 executes a program stored in the storage device 45 or the like, and performs various information processing performed in the relay server 4 .

[0154] The RAM 42 stores temporarily required information such as the programs stored in the storage device 45. The ROM 43 stores programs and settings for controlling the relay server 4.

[0155] The network interface 44 performs communication with external terminals and / or devices, etc. The network interface 44 is, for example, an Ethernet (registered trademark) card that communicates with external terminals and / or devices through wired communication, and / or a wireless LAN interface that communicates with external terminals and / or devices through wireless communication, etc.

[0156] The storage device 45 is a storage device such as a hard disk or SSD that stores the program executed by the relay server 4 and the login information LI. The storage device 45 also stores a setting file describing settings required for operating the relay server 4.

[0157] In other embodiments, the relay server 4 may also include a display 46. The display 46 is a display such as a liquid crystal display or an organic EL display that displays various information about the relay server 4. Thus, the user can manage the relay server 4 while referring to the information displayed on the display 46, for example.

[0158] In another embodiment, the relay server 4 may also include an input interface 47 for receiving input from the user. The input interface 47 is, for example, an input device such as a keyboard, a mouse, or a touch panel. Thus, the user can directly manage the relay server 4 using the input interface 47 .

[0159] Even without the input interface 47 and / or the display 46 , for example, the relay server 4 can be managed by referring to information related to the relay server 4 using a terminal that can access the relay server 4 via the network interface 44 .

[0160] (6) Information managed in the communication system

[0161] (6-1) Information managed by the maintenance server

[0162] The following describes information stored in the maintenance server 3, the relay server 4, the first gateway device 19, and the second gateway device 23. First, the information managed by the maintenance server 3 is described. As described above, the maintenance server 3 stores the user information UI, the composite image creation information GI, and the gateway list GL in the storage device 35.

[0163] like Fig. 10A As shown, the gateway list GL is, for example, a list that associates the identification information of each automatic warehouse system 1 with the identification information of the first gateway device 19 possessed by the automatic warehouse system 1. In addition, the gateway list GL stores the identification information of each maintenance base 2 in association with the identification information of the second gateway device 23 possessed by the maintenance base 2. Fig. 10A This is a diagram showing an example of a gateway list.

[0164] The identification information of each automated warehouse system 1 and each maintenance base 2 is, for example, the identification number of the automated warehouse system 1 and the maintenance base 2. The identification number of the automated warehouse system 1 in the identification information is also included in the composite image creation information GI described later. Fig. 10A In the example shown, the identification numbers of the automatic warehouse system 1 are set to 100 to 199, and the identification numbers of the maintenance base 2 are set to 200 to 299. The identification numbers of the automatic warehouse system 1 and the maintenance base 2 are not limited to Fig. 10A The examples shown can be set arbitrarily.

[0165] On the other hand, the identification information of the first gateway device 19 and the second gateway device 23 is, for example, a login ID used by the first gateway device 19 and the second gateway device 23 to log in to the relay server 4. In addition, the network address (IP address) of the first gateway device 19 and the second gateway device 23 can also be set as the identification information of the gateway device.

[0166] The user information UI stores a user ID and a password for logging into the maintenance server 3. In addition, in the user information UI, a user type is associated with each user ID. The user category indicates whether the user ID is a maintenance personnel category or a category other than maintenance personnel such as an operator. In addition, the information of each user (for example, the user's affiliation, contact information, authority level, etc.) can also be stored in the user information UI.

[0167] like Fig. 10B As shown, the composite image creation information GI includes identification information IN, live image FM, related information AI, and a first device list EL1. Fig. 10B 2 is a diagram showing the structure of the composite image creation information. The identification information IN is information for identifying the automatic warehouse system 1, and is the same as the identification information of the gateway list GL.

[0168] The field image FM is an image showing the automated warehouse system 1. Specifically, the field image FM is a real-shot image or a digital model of the automated warehouse system 1. The field image FM shows each device of the automated warehouse system 1 in an image. The device shown in an image is referred to as an object O.

[0169] The related information AI is information storing the association between the operation information FI and the maintenance information MI and the equipment of the automated warehouse system 1. The related information AI is associated with the equipment of the automated warehouse system 1 and access information AC (described later) for accessing the equipment from the maintenance terminals 21a and 21b.

[0170] The first device list EL1 is a list of information related to devices connected to the first gateway apparatus 19. The specific configuration will be described in detail later.

[0171] (6-2) Information managed by the relay server

[0172] Next, the information managed by the relay server 4 is described. The login information LI managed by the relay server 4 stores a user ID for logging into the relay server 4. In addition, the login information LI includes user IDs for the first gateway device 19 and the second gateway device 23. That is, the first gateway device 19 and the second gateway device 23 log in to the relay server 4.

[0173] In addition, in the login information LI, the user ID for the gateway device is associated with the network address (eg, IP address) of the gateway device having the user ID. Thus, the relay server 4 can obtain the network address of the corresponding gateway device according to the user ID of the gateway device and can access the gateway device.

[0174] (6-3) Information managed by the gateway device

[0175] Finally, the information managed by the gateway device is described. Each gateway device manages a list of information related to devices connected to its own LAN. The first gateway device 19 manages Fig.11A On the other hand, the second gateway device 23 manages Fig. 11B A second device list EL2 is shown. Fig.11A This is a diagram showing an example of the data structure of the first device list. Fig. 11B This is a diagram showing an example of the data structure of the second device list.

[0176] in addition, Fig.11A as well as Fig. 11B The equipment list shown is an example, and the types and number of equipment listed will vary depending on the scale of the automated warehouse system 1 and the maintenance base 2 .

[0177] (6-3-1) First device list

[0178] like Fig.11A As shown, the first device list EL1 managed by the first gateway apparatus 19 includes a device name record part R1, an address record part R2, a sub-ID record part R3, a sub-type record part R4, and a status record part R5.

[0179] The device name recording part R1 records the device name. In the first device list EL1, if Fig.11AAs shown, the names of cameras CA1 to CA4 (crane #1 camera to conveyor #2 camera), the name of sensor SE (crane #1 sensor), the name of the first external storage device 13a (external storage device #1), the name of the second external storage device 13b (external storage device #2), the name of the information management server 15b (information management server), and the name of the conveyance management server 15a (conveyance management server) are recorded.

[0180] The address record part R2 records the network address assigned to the device that can be connected to the gateway device. In the first device list EL1, the rows with the device names of cameras CA1 to CA4 respectively record the IP addresses of "192.168.0.1" to "192.168.0.4". In addition, the row with the device name of the sensor SE records the IP address of "192.168.0.11".

[0181] Furthermore, in the rows with the device names of external storage device #1 to external storage device #2, the IP addresses "192.168.0.21" and "192.168.0.22" are recorded respectively, and in the rows with the device names of the information management server and the delivery management server, the IP addresses "192.168.0.31" and "192.168.0.32" are recorded respectively.

[0182] In addition, IP addresses "192.168.0.111" and "192.168.0.112" are recorded in the rows with the names of the control devices of the stacker cranes (control panels), and IP addresses "192.168.0.41" and "192.168.0.42" are recorded in the rows with the names of the control devices of the conveyors (PLC).

[0183] As described later, the IP address recorded in the address record portion R2 is used when a communication session is established between gateway devices and when devices connected to the gateway device communicate with each other via the gateway device.

[0184] The subID record part R3 records the login ID (subID) used for the device to log in to the relay server 4. In the first device list EL1, the subIDs "cam 001" to "cam 004" are recorded in the rows with the device names of cameras CA1 to CA4, respectively, and the subID "sen 001" is recorded in the row with the device name of the sensor SE.

[0185] Furthermore, in the row having the device name of “control panel”, sub-IDs “ctrl 001” and “ctrl 002” are recorded, respectively, and in the row having the device name of “PLC”, sub-IDs “PLC001” and “PLC002” are recorded, respectively.

[0186] The subtype record part R4 records the user type (subtype) representing the category of the device connected to the gateway device. In the first device list EL1, the row with the device name of camera CA1 to camera CA4 records the subtype of "camera", and the row with the device name of sensor SE records the subtype of "sensor".

[0187] Furthermore, in the row having the device name of “control panel”, the subtype of “control panel” is recorded, and in the row having the device name of “PLC”, the subtype of “PLC” is recorded.

[0188] The status record part R5 records whether each device recorded in the device list is in a state where it can communicate with the gateway device. In the example of the first device list EL1, for example, "NG" is recorded in the rows of the device names of cameras CA3 and CA4, and "OK" is recorded in other rows. That is, cameras CA3 and CA4 cannot communicate with the first gateway device 19.

[0189] In this way, the status recording section R5 records "OK" for the devices (devices) connected to the gateway device that can communicate with the gateway device, and records "NG" for the devices that cannot communicate.

[0190] like Fig.11A As shown, the access information AC includes at least one line of information of the first device list EL1, namely, the IP address, device name, sub-ID, sub-type and status of one device, and is associated with the object O of the scene image FM.

[0191] (6-3-2) Second device list

[0192] Next, use Fig. 11B The second device list EL2 managed by the second gateway device 23 is described below. The second device list EL2 has a device name record part R1, an address record part R2, a sub-ID record part R3, a sub-type record part R4, and a status record part R5, similarly to the first device list EL1.

[0193] In the second device list EL2, the name of the maintenance terminal 21a (maintenance terminal #1) and the name of the maintenance terminal 21b (maintenance terminal #2) are recorded in the device name record part R1. In addition, in the address record part R2, the IP addresses "172.28.0.2" and "172.28.0.3" are recorded in the rows having the device names of the maintenance terminals 21a and 21b, respectively.

[0194] On the other hand, no information is recorded in the subID record part R3, the subtype record part R4, and the status record part R5 of the second device list EL2. This means that the maintenance terminals 21a and 21b cannot log in to the relay server 4.

[0195] (7) Summary of maintenance server operations

[0196] Next, the operation of the maintenance server 3 of the communication system 100 according to the present embodiment will be described. Fig.12 The general operation of the maintenance server 3 will be described. Fig.12 This is a flowchart showing the general operation of the maintenance server.

[0197] When the maintenance server 3 uses the remote maintenance terminals 21a and 21b to access the various devices of the automatic warehouse system 1, it is possible to easily identify the desired device and quickly access it. Fig.12 In addition, Fig.12 The operation shown is for one automatic warehouse system 1. The maintenance server 3 executes operations for each of the plurality of automatic warehouse systems 1 included in the communication system 100. Fig.12 Action shown.

[0198] Specifically, first, in step S1, the maintenance server 3 performs an initial setting operation for generating a composite image SI. After performing the initial setting operation, the maintenance server 3 generates a composite image SI in step S2 using information created and acquired in the initial setting operation. The maintenance server 3 outputs the generated composite image SI to the corresponding maintenance terminals 21a and 21b. The maintenance terminals 21a and 21b that receive the composite image SI display the composite image SI on a display (not shown).

[0199] After outputting the synthesized image SI, the maintenance server 3 receives selection of an object O in the synthesized image SI using the maintenance terminals 21 a and 21 b in step S3 .

[0200] When the selection of the object O is accepted in step S3 ("Yes" in step S3), the maintenance server 3 uses the access information AC associated with the selected object O in step S4 to issue an instruction for establishing a communication session between the device corresponding to the selected object O and the maintenance terminals 21a, 21b.

[0201] (8) Specific actions to maintain the server

[0202] (8-1) Initial setting operation

[0203] Below, use Fig.13The initial setting operation in step S1 described above will be described in detail. Fig.13 This is a flowchart showing the initial setting operation.

[0204] In the initial setting operation, first, in step S11 , a live image FM is created and stored in the storage device 35 of the maintenance server 3 .

[0205] When the scene image FM is set as a real shot image of the automatic warehouse system 1, the real shot images are taken at multiple locations of the automatic warehouse system 1. Next, the multiple real shot images are connected to create the scene image FM, considering at which location of the automatic warehouse system 1 the images are taken. Alternatively, the real shot images may be taken as a panoramic image and set as the scene image FM.

[0206] On the other hand, when the on-site image FM is a digital model of the automated warehouse system 1 , for example, a three-dimensional image of the automated warehouse system 1 drawn using CAD or the like is used as the on-site image FM.

[0207] Next, in step S12, the maintenance server 3 obtains access information AC for accessing each device of the automatic warehouse system 1 from the maintenance terminals 21a and 21b. In the present embodiment, the access information AC is obtained from the relay server 4. The operation of obtaining the access information AC will be described in detail later.

[0208] After acquiring the access information AC, in step S13, the maintenance server 3 associates the acquired access information AC with the object O in the live image FM. This association can be performed, for example, as follows.

[0209] First, in the scene image FM, images representing the equipment of the automated warehouse system 1 are identified as objects O. Then, coordinate values ​​are obtained for each identified object O. The coordinate values ​​of the object O may be, for example, the configuration positions (coordinate values) of the pixels representing the object O in the scene image FM.

[0210] Next, the coordinate value of the object O and the access information AC of the device corresponding to the object O are associated with each other and stored in the associated information AI.

[0211] By executing the above steps S11 to S13, the maintenance server 3 obtains the information required to generate the composite image SI. That is, the maintenance server 3 associates the field image FM, the associated information AI, the first equipment list EL1 including the access information AC, and the identification information IN of the automated warehouse system 1 to obtain the composite image creation information GI.

[0212] In addition, in the above-mentioned step S13, the maintenance server 3 receives the operation information FI and maintenance information MI possessed by the automatic warehouse system 1 and the maintenance base 2. After that, the maintenance server 3 further associates the operation information FI and maintenance information MI of the corresponding equipment with each object O of the field image FM and stores them in the associated information AI. At this time, the maintenance server 3 associates the link to the operation information FI and the maintenance information MI with each object O.

[0213] As described above, the operation information FI includes abnormality information indicating abnormalities occurring in the equipment of the automated warehouse system 1. Therefore, by associating the operation information FI with each object O of the field image FM, the abnormality information can be associated with the object O corresponding to the equipment in which the abnormality occurred, thereby generating a synthetic image SI.

[0214] Furthermore, as the initial setting operation, the maintenance server 3 may perform setting operations such as user registration for logging into the maintenance server 3 and access restriction to information for each user.

[0215] (8-2) Access information acquisition action

[0216] Next, use Fig.14 as well as Fig.15 The operation of step S12 of the above-mentioned initial setting operation, that is, the operation of acquiring the access information AC, will be described in more detail. Fig.14 This is a flowchart showing the operation of obtaining access information. Fig.15 This is a diagram showing the transmission and reception of information in the acquisition of access information.

[0217] In order to obtain the access information AC, the maintenance server 3 transmits a device list request signal to the relay server 4 in step S21 .

[0218] Specifically, the maintenance server 3 first identifies the identification number of the automated warehouse system 1 in which the desired access information AC exists, and then identifies the login ID of the first gateway device 19 associated with the identified identification number in the gateway list GL.

[0219] After determining the login ID, the maintenance server 3 transmits the determined login ID together with a device list request signal to the relay server 4 (step S201 ).

[0220] The relay server 4 that has received the request signal identifies the first gateway device 19 of which automated warehouse system 1 the login ID sent along with the request signal belongs. Then, the relay server 4 sends a device list request signal to the identified first gateway device 19 (step S202).

[0221] When receiving the device list request signal from the relay server 4 (step S202 ), the first gateway device 19 transmits the first device list EL1 held by itself to the relay server 4 (step S203 ).

[0222] After that, when the desired first device list EL1 is acquired from the first gateway device 19 (step S203 ), the relay server 4 transmits the first device list EL1 to the maintenance server 3 (step S204 ).

[0223] When receiving the first device list EL1 from the relay server 4 (Yes in step S22), the maintenance server 3 stores the acquired first device list EL1 as the composite image creation information GI in the storage device 35. Thereafter, the maintenance server 3 extracts the access information AC of each device from the stored first device list EL1.

[0224] As described above, the maintenance server 3 can receive the desired first device list EL1 from the corresponding first gateway device 19 via the relay server 4 , and extract and acquire the access information AC from the received first device list EL1 .

[0225] (8-3) Synthetic Image Generation / Output Operation

[0226] Next, use Fig.16 as well as Fig.17 The specific operation of generating and outputting the synthesized image SI in the above-mentioned step S2 will be described. Fig.16 This is a flowchart showing the operation of the maintenance server in the operation until the synthesized image is output. Fig.17 2 is a diagram showing information transmission and reception during the operation until the synthesized image is output. Hereinafter, a case where a maintenance worker at the maintenance base 2 accesses the maintenance server 3 using the maintenance terminal 21a will be described as an example.

[0227] The generation and output of the composite image SI is started by issuing a connection request to the maintenance server 3 from the maintenance terminal 21a to the maintenance server 3 that has completed the initial setting operation and is in operation (step S301). Specifically, for example, the address of the maintenance server 3 is specified in the web browser of the maintenance terminal 21a, and the web browser outputs a connection request to the maintenance server 3.

[0228] When receiving the connection request ("Yes" in step S31), the maintenance server 3 outputs the login screen I1 to the maintenance terminal 21a that has made the connection request in step S32 (step S302). Fig.18 The login screen I1 is shown. Fig.18 This is a diagram showing an example of a login screen.

[0229] The maintenance personnel enters their own user ID in the user ID input field T1 and their own password in the password input field T2 on the login screen I1 displayed on the maintenance terminal 21a, and then presses the login button B1 to output the entered user ID and password as authentication information to the maintenance server 3 (step S303).

[0230] When receiving authentication information from the maintenance terminal 21a (YES in step S33), the maintenance server 3 performs authentication in step S34. Specifically, the maintenance server 3 compares the received authentication information, namely the user ID and password, with the user ID and password stored in the user information UI.

[0231] As a result of the comparison, if authentication fails because the received authentication information does not exist in the user information UI or the received authentication information does not match the user ID and password stored in the user information UI (No in step S34), the maintenance server 3 ends the composite image output processing.

[0232] When the authentication fails, the maintenance server 3 notifies the maintenance terminal 21a that attempts to log in of the fact that the authentication has failed. Alternatively, the maintenance server 3 may terminate the synthesized image output process after the authentication has failed a predetermined number of times.

[0233] On the other hand, if the authentication is successful based on the received authentication information ("Yes" in step S34), the maintenance server 3 outputs the initial screen I2 ( Fig.25 ) (Step S304). The structure of the initial screen I2 will be described later.

[0234] After the maintenance terminal 21a displays the initial screen I2, the maintenance worker uses the initial screen I2 to select a room to enter. The worker terminal 17 or the maintenance terminal 21a transmits a request to join a conversation in the selected room as a conversation entry request to the maintenance server 3 (step S305).

[0235] When receiving the conversation entry request from the maintenance terminal 21a, the maintenance server 3 generates a composite image SI in step S36. Fig.19 The operation of generating the composite image SI executed in step S36 will be described in further detail. Fig.19 This is a flowchart showing the operation of generating a synthetic image.

[0236] First, in step S41 , the maintenance server 3 selects, for each device corresponding to each object O of the field image FM, operating information FI and maintenance information MI that can be browsed by a maintenance worker as a target, and devices that can be accessed by the maintenance worker.

[0237] Specifically, the maintenance server 3 determines the operation information FI and maintenance information MI that can be viewed by the maintenance personnel identified by the user ID, and also determines the devices that can be accessed by the maintenance personnel.

[0238] Next, the maintenance server 3 refers to the related information AI to extract the operating information FI and maintenance information MI that are viewable by the target maintenance personnel from among the operating information FI, maintenance information MI, and access information AC associated with each object O. In addition, the maintenance personnel can access the device access information AC.

[0239] Furthermore, the maintenance server 3 associates each object O with the operation information FI, maintenance information MI, and access information AC extracted for the corresponding device, and generates unique association information for each maintenance worker as an object.

[0240] After generating the unique related information, the maintenance server 3 determines in step S42 whether the operation information FI, the maintenance information MI, or the access information AC is related to each object O of the field image FM in the unique related information.

[0241] When the operation information FI, the maintenance information MI, and the access information AC are not associated with the object O in the independent related information (No in step S42 ), the generation process of the composite image SI proceeds to step S44 .

[0242] On the other hand, in the independent association information, when any one of the operation information FI, the maintenance information MI, and the access information AC is associated with the object O ("Yes" in step S42), the maintenance server 3 arranges a predetermined icon at the coordinate position of the field image FM where the object O is located in step S43. The predetermined icon indicates that the operation information FI and / or the maintenance information MI is associated with the object O.

[0243] If the device corresponding to the object O is accessible using the maintenance terminal 21a, the access information AC of the corresponding device is displayed in the icon of the object O. As described later, the type (shape) of the arranged icon varies depending on the type of operation information FI and / or maintenance information MI to be associated.

[0244] After arranging the icon, the maintenance server 3 generates a link that allows reference to the operation information FI and the maintenance information MI selected in step S41 by clicking the icon based on the unique related information.

[0245] At the same time, by clicking the icon displaying the access information AC, a command is generated to output an instruction to establish a communication session (referred to as a session establishment instruction) using the access information AC selected in step S41.

[0246] Here, the link to the operating information FI and the maintenance information MI is information indicating the storage location of the operating information FI and / or the maintenance information MI in the maintenance server 3. The link is, for example, a URL for accessing the operating information FI and / or the maintenance information MI.

[0247] In an automatic warehouse system 1 having a large number of devices, there are usually a plurality of operation information FI and maintenance information MI associated with one object O (device). Therefore, in the present embodiment, links to the operation information FI and maintenance information MI associated with each object O are tabulated and managed as a list. In addition, in the present embodiment, links to the list are associated with each object O.

[0248] On the other hand, since the access information AC is different for each device, the command to output a communication session establishment instruction is directly associated with each object O.

[0249] Through the above processing, in the field image FM, an icon can be arranged on the object O associated with any of the operation information FI, maintenance information MI, and access information AC, while no icon can be arranged on the object O not associated with any of these information.

[0250] After placing icons on objects O as needed, in step S44, it is checked whether steps S41 to S43 are executed for all objects O in the scene image FM. In other words, it is checked whether operation information FI, maintenance information MI, and access information AC to be associated with all objects O are determined.

[0251] When the association of information is not determined for all the objects O (No in step S44 ), the above-mentioned S41 to S43 are further executed for other objects O.

[0252] On the other hand, when the information association is determined for all the objects O (“Yes” in step S44), the synthesis image SI ( Fig.26A ) is generated.

[0253] The specific configuration will be described later. However, in the composite image SI, if any of the operation information FI, the maintenance information MI, and the access information AC is associated with the object O of the field image FM, an icon is placed on the object O.

[0254] Furthermore, the shape of the icon differs depending on the type of information associated with the object O, and in the object O associated with the access information AC, the access information AC is displayed together with the icon.

[0255] Such a composite image SI can show maintenance personnel which type of operation information FI, maintenance information, and access information AC is associated with which object O in a visually organized state.

[0256] Return to Fig.16 as well as Fig.17 After generating the composite image SI as described above, the maintenance server 3 sets the dialog box A4 ( Fig. 27 ) is sent together with the synthesized image SI to the maintenance terminal 21a used by the maintenance personnel of the user ID used to generate the synthesized image SI (step S306).

[0257] For example, a dialog box A4 is provided outside the area where the synthesized image SI is displayed.

[0258] In addition, when the operator of the automatic warehouse system 1 accesses the maintenance server 3 using the operator terminal 17, the above steps S31 to S37 and steps S41 to S44 are also executed. That is, the synthetic image SI is generated, and the display screen I3 is output to the operator terminal 17. However, the access information AC is not associated with the object O of the synthetic image SI in this case. That is, the operator using the operator terminal 17 can refer to the synthetic image SI and browse only the operation information FI and the maintenance information MI.

[0259] Furthermore, the maintenance server 3 associates the identification information IN of the automatic warehouse system 1 represented by the generated synthetic image SI with the synthetic image SI. When the object O in the synthetic image SI is selected (clicked), the identification information IN is included in the selection result of the object O (described later) and transmitted to the maintenance server 3.

[0260] (8-4) Summary of the Operations Until the Output of the Synthesized Image

[0261] About Usage Figures 13 to 19 The operations described until the output of the synthesized image SI by the maintenance server 3 can be summarized as follows.

[0262] In this embodiment, the maintenance server 3 creates a field image FM representing the automatic warehouse system 1 (step S11), obtains access information AC for accessing the device (step S12, steps S21 to S23), associates the access information AC for accessing the device with the object O representing the device in the field image FM to generate a composite image SI (step S13, steps S41 to S44), and outputs the composite image SI to the maintenance terminals 21a, 21b (step S37).

[0263] The access information AC for accessing the corresponding device is associated with the object O of the synthetic image SI generated as described above. Thus, the user (maintenance personnel) of the maintenance terminals 21a and 21b can easily identify the device they want to access by referring to the synthetic image SI.

[0264] Furthermore, in the present embodiment, the maintenance server 3 obtains the operation information FI from the automatic warehouse system 1, and associates the operation information FI with the corresponding object O to generate a composite image SI. As described above, the operation information FI includes abnormality information indicating abnormalities occurring in the equipment of the automatic warehouse system 1, so that maintenance personnel can easily and quickly access information related to the abnormalities occurring in the equipment.

[0265] In addition, in the present embodiment, the maintenance server 3 outputs the display screen I3 displaying the dialog box A4 together with the synthesized image SI. In addition, the maintenance server 3 provides the dialog box A4 together with the synthesized image SI generated using the user ID of the operator.

[0266] Thus, by using the dialogue field A4 , a dialogue function can be used between the maintenance personnel using the maintenance terminals 21 a and 21 b and the operator using the operator terminal 17 .

[0267] (9) Communication session establishment action

[0268] (9-1) Communication session establishment action

[0269] As described above, when the composite image SI is generated and output, the display screen I3 showing the composite image SI and the dialog box A4 at the same time is displayed on the maintenance terminal 21a, 21b. By displaying the display screen I3, the composite image SI that can easily identify the accessible device begins to be provided. At the same time, the operation information FI and maintenance information MI stored in the maintenance server 3 begin to be provided.

[0270] That is, the maintenance personnel can refer to the synthesized image SI and click the icon arranged on the desired object O, thereby easily and quickly accessing the equipment corresponding to the object O. In addition, the operation information and maintenance information MI associated with the object O can be accessed at the same time.

[0271] Below, use Fig. 20 as well as Fig.21 The operation until a communication session is established between the maintenance terminals 21a and 21b and the device corresponding to the object O specified in the synthesized image SI will be described. Fig. 20 This is a flowchart showing the operation of establishing a communication session of the maintenance server. Fig.21 This is a diagram showing the transmission and reception of information in the communication session establishment operation.

[0272] The following describes, by way of example, the establishment of a communication session between the maintenance terminal 21 a logged into the maintenance server 3 and the control panel of the stacker crane of the automated warehouse 11 (device name: crane #1 control panel).

[0273] In the composite image SI ( Fig.26A ), if you click the icon with access information AC (in Fig.12 If the answer is "Yes" in step S3 of the control unit 21, the information of the clicked icon is sent from the maintenance terminal 21a to the maintenance server 3 as the selection result of the object O (step S501).

[0274] For example, when the icon of the access information AC with "Ctrl001 192.168.0.111" is clicked, the maintenance terminal 21a sends its own IP address (within the LAN), the identification number of the maintenance base 2 where the maintenance terminal 21a is located, the access information AC, and the identification information IN associated with the synthetic image SI as the selection result of the object O to the maintenance server 3.

[0275] Furthermore, when an icon that does not indicate that the access information AC is associated with the object O is clicked, the maintenance server 3 performs an operation to associate the icon.

[0276] When receiving the selection result of the object O from the maintenance terminal 21 a , the maintenance server 3 determines in step S51 which first gateway device 19 the equipment that the maintenance server 3 wants to access is connected to.

[0277] Specifically, the maintenance server 3 determines in the gateway list GL which first gateway device 19 identification information is associated with the received identification information IN of the selection result. For example, when the identification number "101" is received as the identification information IN, the maintenance server 3 refers to Fig. 10A The gateway list GL determines the first gateway device 19 with the login ID "vpn001".

[0278] Furthermore, in step S51, the maintenance server 3 also identifies the second gateway device 23 connected to the maintenance terminal 21a that has transmitted the selection result of the object O. For example, upon receiving the identification number "201", the maintenance server 3 refers to Fig. 10A The gateway list GL determines the second gateway device 23 with the login ID "vpn002".

[0279] Next, in step S52, the maintenance server 3 identifies the access information AC of the device that the maintenance server 3 wants to access. Specifically, the maintenance server 3 identifies the access information AC transmitted as a result of the selection of the object O as the access information AC of the device that the maintenance server 3 wants to access.

[0280] Furthermore, in step S52 , the maintenance server 3 specifies the IP address of the maintenance terminal 21 a based on the address of the maintenance terminal 21 a received as a result of the selection of the object O.

[0281] Afterwards, in step S53, the maintenance server 3 sends the login ID (vpn001) of the first gateway device 19 determined by the above steps S51 to S52, the login ID (vpn002) of the second gateway device 23, the determined access information AC and the IP address of the maintenance terminal 21a as a connection request to the first gateway device 19 to the relay server 4 (step S502).

[0282] By executing the above-described steps S51 to S53 , the maintenance server 3 can transmit, to the relay server 4 in the communication system 100 , a command for establishing a communication session between the maintenance terminal 21 a and the equipment of the automated warehouse system 1 .

[0283] After sending the instruction for establishing a communication session between the maintenance terminal 21a and the equipment of the automatic warehouse system 1, the communication session establishment operation is mainly performed by the relay server 4, the first gateway device 19 and the second gateway device 23. Fig.21 The establishment action is described.

[0284] The relay server 4 that has received the connection request specifies the network address of the first gateway device 19 based on the user ID "vpn001" of the connection request destination, and specifies the network address of the second gateway device 23 to which the maintenance terminal 21a is connected based on the user ID "vpn002".

[0285] Thereafter, the determined network address of the second gateway device 23, the access information AC, and the IP address of the maintenance terminal 21a are sent as a connection request to the determined first gateway device 19 (step S503).

[0286] The first gateway device 19 that receives the connection request decides whether to accept the connection request. If it decides to accept the connection request, the first gateway device 19 sends a notification indicating acceptance of the connection request to the second gateway device 23 having the network address included in the received connection request (step S504).

[0287] The second gateway device 23 that has received the notification transmits the second device list EL2 that it has to the first gateway device 19 that is the transmission source of the notification (step S505 ).

[0288] The first gateway device 19 that has received the second device list EL2 transmits the first device list EL1 that it has to the second gateway device 23 that is the source of the received device list (step S506 ).

[0289] Through the above steps S505 and S506 , after exchanging the first device list EL1 and the second device list EL2 , the first gateway device 19 and the second gateway device 23 establish a VPN session between each other.

[0290] As described above, by establishing a VPN session between the first gateway device 19 and the second gateway device 23 , the maintenance terminal 21 a can access the equipment connected to the first gateway device 19 via the second gateway device 23 and the first gateway device 19 .

[0291] In addition, when a VPN session is established between the first gateway device 19 and the second gateway device 23, the maintenance terminal 21a and each device of the automatic warehouse system 1 can communicate with each other using the IP addresses recorded in the replaced first device list EL1 and the address record part R2 of the second device list EL2.

[0292] For example, after establishing a VPN session, the device of the automated warehouse system 1 (device name: control panel of crane #1 control panel) notifies the maintenance terminal 21a's IP address "172.28.0.2" and the device's IP address "192.168.0.111" that a VPN connection to itself is possible.

[0293] The maintenance terminal 21a that has received the notification starts, for example, a maintenance application for the device, thereby allowing the user (maintenance worker) of the maintenance terminal 21a to perform maintenance on the device (control panel) using the application.

[0294] Through the above-mentioned VPN session, the maintenance terminal 21a and the equipment of the automatic warehouse system 1 can use the IP addresses used in the LAN (for example, IP addresses such as "192.168.*.*", "172.28.*.*" (*: wildcard)) via the first gateway device 19 and the second gateway device 23 to send and receive data to each other while ensuring security.

[0295] (9-2) Summary of actions related to establishing a communication session

[0296] About Usage Fig. 20 as well as Fig.21 The operation of establishing a communication session between the maintenance terminals 21a and 21b and the devices of the automatic warehouse system 1 described above can be summarized as follows.

[0297] As described above, when a certain object O (object O with access information AC) in the composite image SI is selected by the maintenance terminals 21a and 21b (step S3, step S501), the maintenance server 3 uses the access information AC associated with the selected object O to establish a communication session between the device corresponding to the object O and the maintenance terminals 21a and 21b (steps S51 to S53, step S502).

[0298] Thus, the users (maintenance personnel) of the maintenance terminals 21a and 21b can easily identify the device they want to access by referring to the synthesized image SI, and can quickly access the identified device.

[0299] (10) Functions that can be executed during display of the composite image SI

[0300] (10-1) Overview of Functions Executable During Display of Synthesized Image SI

[0301] In the display of the above description Fig. 27 After the screen I3 shown is displayed, the maintenance personnel can have a conversation using the conversation box A4 in the maintenance terminals 21a and 21b. In addition to the conversation, various functions using the synthesized image SI can be executed.

[0302] For example, the conversation content of the conversation field A4 can be associated with the composite image SI to generate a new composite image SI. In addition, when the conversation content of the conversation field A4 is clicked, the position (object O) of the composite image SI associated with the conversation content can be "jumped" (accessed).

[0303] Furthermore, it is also possible to generate new operating information FI and maintenance information MI, or edit existing operating information FI and maintenance information MI, and associate them with the synthesized image SI.

[0304] The maintenance server 3 not only updates the operation information FI and the maintenance information MI, but also can reflect these changes in the composite image SI when the connection status of the equipment in the automatic warehouse system 1 changes, when there is a change in the equipment of the automatic warehouse system 1, and when the number of equipment increases or decreases.

[0305] (10-2) Updating of the composite image accompanying device change

[0306] Below, use Fig. 22 as well as Fig.23 The operation of the communication system 100 when there is a change in the device will be described. Fig. 22 This is a flowchart showing the operation of the maintenance server when there is a change in equipment. Fig.23 This is a diagram showing the transmission and reception of information when there is a change in equipment.

[0307] For example, when a new gateway device is added to the automatic warehouse system 1 and / or equipment is removed and / or added to the existing automatic warehouse system 1, the first gateway device 19 of the automatic warehouse system 1 with equipment changes generates (updates) a first equipment list EL1 corresponding to the new equipment changes. The first gateway device 19 sends the new first equipment list EL1 to the relay server 4 (step S601).

[0308] The relay server 4 receives the new first device list EL1 and sends it to the maintenance server 3 (step S602). At this time, the relay server 4 sends the identification information IN (eg, login ID) of the first gateway device 19 that sent the new first device list EL1 together with the new first device list EL1.

[0309] The maintenance server 3 that has received the new first device list EL1 updates the synthesized image SI so as to correspond to the new first device list EL1. The updating of the synthesized image SI is performed as follows.

[0310] First, in step S61 , the maintenance server 3 performs a new association between each access information AC of the new first device list EL1 and the object O in the synthetic image SI.

[0311] Specifically, the maintenance server 3 first determines which synthetic image production information GI should be changed based on the identification information IN sent together with the new first device list EL1. Then, the first device list EL1 of the corresponding synthetic image production information GI is replaced with the new first device list EL1. Furthermore, the maintenance server 3 performs the association between the object O and the new access information AC included in the new first device list EL1, and generates new association information AI.

[0312] In addition, the association between the object O in the above step S61 and the new access information AC can be performed by executing the above-described steps S11 to S13 based on the new first device list EL1 ( Fig.13 ) and steps S21 to S23 ( Fig.14 Therefore, the detailed description is omitted here.

[0313] After executing the new association between the object O and the access information AC, the maintenance server 3 generates a new synthetic image SI based on the new synthetic image creation information GI in step S62, and outputs it to the maintenance terminals 21a and 21b (step S603).

[0314] In addition, the generation and output of the new synthetic image SI in step S52 can be performed by executing steps S31 to S37 ( Fig.16 ) and steps S41 to S44 ( Fig.19 Therefore, the detailed description is omitted here.

[0315] In this way, when there is a change in the equipment of the automatic warehouse system 1 after the synthetic image SI is generated, a new synthetic image SI is generated and output by using new access information corresponding to the change in the equipment. Thus, when there is a change in the equipment in the automatic warehouse system 1, the latest synthetic image SI reflecting the change can be provided to the maintenance terminals 21a, 21b.

[0316] After generating the new composite image SI as described above, the maintenance server 3 notifies the maintenance terminals 21 a and 21 b that the composite image SI has been updated in step S63 .

[0317] For example, Fig.24 As shown, the above notification is in the specified area of ​​the dialogue box A4 (in Fig.24 In the example, it is the lower area) and an instruction to display a pop-up window PU having a display of “Updated synthetic image” on the display screen I3. Fig.24 2 is a diagram showing an example of a notification that a composite image has been updated. The notification may be a command to cause the maintenance terminals 21a and 21b to generate a predetermined sound.

[0318] Alternatively, a message such as “The composite image has been updated” may be posted to the dialog box A4. Furthermore, a link to the updated data may be included in the message, and the updated data in the composite image SI may be accessed from the link in the message.

[0319] In this way, when a new synthetic image SI is generated, the maintenance server 3 notifies the maintenance terminals 21a and 21b of this fact, so that the maintenance personnel using the maintenance terminals 21a and 21b can easily recognize that a new synthetic image SI (the latest synthetic image SI) is generated, that is, there is an update of the synthetic image SI. In addition, by recognizing that there is an update of the synthetic image SI, the maintenance personnel can recognize that there is a change in the equipment in the automatic warehouse system 1.

[0320] Furthermore, when the maintenance personnel using the maintenance terminals 21a and 21b are communicating with the workers using the worker terminal 17, the pop-up window is also displayed in the dialogue field A4 of the worker terminal 17. Thus, the workers of the automatic warehouse system 1 can be informed of changes in the equipment in the automatic warehouse system 1 together with the update of the synthetic image SI.

[0321] (11) Screen display configuration

[0322] (11-1) Initial screen

[0323] Hereinafter, the configuration of the display screen displayed on the display of the maintenance terminals 21a and 21b will be briefly described. In addition, the display screen shown below is an example, and the configuration and / or appearance of the display screen can be changed as appropriate.

[0324] First, use Fig.25 The configuration of the initial screen I2 displayed on the operator terminal 17 and the maintenance terminals 21 a and 21 b after logging into the maintenance server 3 will be described. Fig.25 1 is a diagram showing an example of an initial screen. The initial screen I2 has a usage history display field A1. The usage history display field A1 displays a usage history list L1 that lists usage history, an update button B2 for updating usage history, and a memo field T3 for recording a memo.

[0325] The initial screen I2 has a room selection field A2. In the room selection field A2, selection buttons B3 for selecting a room to enter into a conversation are displayed. When any selection button B3 is pressed, a request is made to enter the room of the conversation of the selected button.

[0326] In addition, the buttons of rooms that the user cannot enter or exit are set to be hidden or invalid (cannot be selected).

[0327] The initial screen I2 has a user information change column A3. The user information change column A3 displays an information edit column T4 for editing user information (user ID, password, name of the user associated with the user ID), an image change button B4 for changing the user image (avatar), a category change column M1 for changing the account category, and an update button B5 for updating the user information when pressed.

[0328] (11-2) Synthetic Image

[0329] Next, use Fig.26A as well as Fig.26B The structure of the synthesized image SI will be described. Fig.26A This is a diagram showing an example of a synthesized image. Fig.26B It is an enlarged view of a part of the composite image. The composite image SI shows a plurality of objects O representing the equipment of the automated warehouse system 1. Specifically, the objects O show a plurality of types of automated warehouses 11 included in the automated warehouse system 1.

[0330] In addition, in the composite image SI, an icon is arranged on each object O. Fig.26A In the example of the composite image SI shown, eight types of icons (ICO1 to ICO8) are arranged. The types of icons displayed in the composite image SI can be arbitrarily determined according to the type of information associated with the device, etc.

[0331] The first icon ICO1 has a shape of an "i" character surrounded by a circle. The first icon ICO1 indicates that various information related to the device (manuals, etc.) is associated as operation information FI and / or maintenance information MI in the device configured with the icon. In addition, the characters "***INFO" ("***" is the name of the device. The same applies below) are marked below the first icon ICO1, specifically indicating which object the first icon ICO1 is configured for.

[0332] The second icon ICO2 has a shape imitating a small camera. The second icon ICO2 indicates that the image data obtained by the camera CA installed in the device is associated as the operation information FI in the device configured with the icon. In addition, the characters "*** camera" are marked below the second icon ICO2, which specifically indicates which object the second icon ICO2 is configured for.

[0333] The third icon ICO3 has a shape imitating a thermometer and a hygrometer. The third icon ICO3 indicates that the data (temperature, humidity) obtained by the sensor SE provided in the device is associated with the device marked with the icon as the operation information FI. In addition, the characters "***humidity temperature" are marked below the third icon ICO3, which specifically indicates which object the third icon ICO3 is configured for.

[0334] The fourth icon ICO4 has a document shape. The fourth icon ICO4 indicates that the parts list of the device is associated with the device marked with the icon as maintenance information MI. In addition, the characters "*** parts list" are marked below the fourth icon ICO4, specifically indicating which object the fourth icon ICO4 is configured for.

[0335] The fifth icon ICO5 has a shape imitating a large camera. The fifth icon ICO5 indicates that the image data obtained by the large camera marked with the icon is associated as the operation information FI. In addition, the character "camera" is marked below the fifth icon ICO5, indicating that the fifth icon ICO5 is configured for the object of the large camera.

[0336] The sixth icon ICO6 has an inverted triangle shape with an arrow next to it. The sixth icon ICO6 is provided at a predetermined position of the live image FM (synthetic image SI) and indicates that by clicking the icon, the field of view of the synthetic image SI moves in the direction of the icon.

[0337] When the sixth icon ICO6 is pressed or operated, the center of the field of view of the synthetic image SI becomes the position of the selected sixth icon ICO6, and the direction of the field of view becomes the front direction of the sixth icon ICO6. Thus, in the synthetic image SI, virtual reality such as the mobile automatic warehouse system 1 can be realized.

[0338] The seventh icon ICO7 is set at a predetermined position (in Fig.26A The seventh icon ICO7 has a switch shape. In addition, the characters "View Mode" are marked below it. The seventh icon ICO7 has a function of switching the display and non-display of the first icon ICO1 to the sixth icon ICO6 by clicking.

[0339] The eighth icon ICO8 is set at a predetermined position (at Fig.26A The eighth icon ICO8 has a switch shape. In addition, the characters "VPN" are marked on it. The eighth icon ICO8 has a function of switching the display and non-display of the access information AC in the composite image SI by clicking.

[0340] like Fig.26B ( Fig.26A As shown in the enlarged view of the portion surrounded by the dotted line, several icons included in the composite image SI show access information AC. Fig.26BIn the example shown, "Ctrl001 192.168.0.111" and the sub-ID and IP address of the device (the control panel of the stacking crane) are displayed as access information AC between the description "PL1_INFO" and the first icon ICO1. In addition, "Cam001 192.168.0.1" and the sub-ID and IP address of the device (camera CA) are displayed as access information AC between the description "PL1_Crane Camera" and the second icon ICO2.

[0341] (11-3) Composite image and dialog box display screen

[0342] Below, use Fig. 27 The configuration of the display screen I3 that simultaneously displays the synthesized image SI and the dialog box A4 will be described. Fig. 27 This is a diagram showing an example of a display screen on which a composite image and a dialog box are displayed simultaneously.

[0343] like Fig. 27 As shown, in the display screen I3, the room selection bar A2 and the dialogue bar A4 described above are displayed outside the area where the composite image SI is displayed. Fig. 27 In the example shown, the room selection bar A2 is displayed on the left side of the composite image SI, and the dialogue bar A4 is displayed on the right side.

[0344] In the conversation field A4, the conversation contents at a predetermined time in the conversation are arranged in time series as speech bubbles CH. In addition, a file selection field FS is arranged in the conversation field A4. The file selection field FS selects a file stored in the operator terminal 17 or the maintenance terminal 21a, etc., and adds it to the conversation contents (speech bubbles CH).

[0345] By dragging and dropping the file added to the conversation content in this way onto the synthesized image SI, it is possible to associate the file with a predetermined position or object O in the synthesized image SI as the operation information FI or the maintenance information MI.

[0346] The file associated based on the conversation content can be associated as operation information FI when the above operation is performed by the operator terminal 17, and can be associated as maintenance information MI when the above operation is performed by the maintenance terminals 21a and 21b.

[0347] In addition, if Fig.28 As shown, by switching the display state on the display screen I3, the access information AC associated with the object O of the synthetic image SI can also be displayed in a list by characters. Fig.28 This is a diagram showing an example of a display screen that displays access information as a list LST of characters.

[0348] 2. Other Implementation Methods

[0349] An embodiment of the present invention has been described above, but the present invention is not limited to the above embodiment, and various modifications can be made without departing from the gist of the invention. In particular, the multiple embodiments and modifications described in this specification can be arbitrarily combined as needed.

[0350] The order of the processes in the flowcharts described above and the process contents of each step can be changed without departing from the gist of the invention.

[0351] (A) The synthetic image SI may not have an icon indicating that the access information AC is associated with the object O. In this case, for example, when a cursor (e.g., a mouse cursor) is placed on the synthetic image SI, if the access information AC is associated with the object O pointed to by the cursor, the access information AC of the object O may be displayed. In addition, the access information AC itself may be displayed without being associated with the object O, for example, a message asking whether to access the device corresponding to the object O may be displayed. That is, as long as the access information AC is associated with the object O in the synthetic image SI, the specific display method is not limited.

[0352] (B) In addition to the user ID and password, user authentication using the human body information (eg, fingerprint, blood vessel (vein) pattern, iris pattern, facial pattern, etc.) of the operator and maintenance personnel can be used.

[0353] Furthermore, for example, user authentication using identification information (e.g., MAC address, IP address, etc.) of the operator terminal 17 and the maintenance terminals 21a, 21b can also be used. In this case, reading authority can be set for the operator terminal 17, the maintenance terminals 21a, 21b, the LAN group, the automatic warehouse system 1, and the maintenance base 2.

[0354] (C) The communication system 100 described above can be used for information sharing between the automated warehouse system 1 and the maintenance base 2 for maintaining the equipment of the automated warehouse system 1, and can also be used in a system for sharing information in isolated locations.

[0355] (D) The dialog box A4 may be displayed in a state of overlapping with a part of the synthesized image SI. In this case, the dialog box A4 and / or the synthesized image SI may be moved on the screen.

[0356] Industrial Applicability

[0357] The present invention can be widely applied to a communication system for remotely maintaining equipment in a work site.

[0358] Explanation of symbols:

[0359] 100: communication system; 1: automatic warehouse system; 11: automatic warehouse; 13a: first external storage device; 13b: second external storage device; 15a: transportation management server; 15b: information management server; 17: operator terminal; 19: first gateway device; 2: maintenance base; 21a, 21b; maintenance terminal; 23: second gateway device; 3: maintenance server; 31: processor; 32: RAM; 33: ROM; 34: network interface; 35: storage device; 36: display; 37: input interface; 4: Relay server; 41: processor; 42: RAM; 43: ROM; 44: network interface; 45: storage device; 46: display; 47: input interface; A1: usage history display bar; A2: room selection bar; A3: user information change bar; A4: dialogue bar; AC: access information; AI: related information; B1: login button; B2: update button; B3: select button; B4: image change button; B5: update button; CA: camera; CA1~CA4: camera; CH: speech bubble; EL1: first device list Table; EL2: Second equipment list; FI: Operation information; FM: Field image; FS: File selection bar; GI: Synthetic image creation information; GL: Gateway list; I1: Login screen; I2: Initial screen; I3: Display screen; ICO1: First icon; ICO2: Second icon; ICO3: Third icon; ICO4: Fourth icon; ICO5: Fifth icon; ICO6: Sixth icon; ICO7: Seventh icon; ICO8: Eighth icon; IN: Identification information; L1: Utilization history list; LI: Login information; M1: Category change column; MI: Maintenance information; NW: Network; O: Object; PU: Pop-up window; R1: Device name recording part; R2: Address recording part; R3: Sub-ID recording part; R4: Sub-type recording part; R5: Status recording part; SE: Sensor; SH1: First switching hub; SH2: Second switching hub; SH3: Third switching hub; SI: Composite image; T1: User ID input column; T2: Password input column; T3: Memo column; T4: Information editing column; UI: User information.

Claims

1. A maintenance method, executed by a maintenance server for maintaining equipment in a work site using a maintenance terminal located at a maintenance base outside a work site, comprising: The step of creating a site image representing the work site; The step of obtaining access information for accessing the above-mentioned device; A step of generating a composite image by labeling an object representing the device in the on-site image with an icon indicating an association with the access information for accessing the device; The step of outputting the composite image to the maintenance terminal; a step of accepting a selection of the icon in the composite image using the maintenance terminal; and The step of establishing a VPN session between a gateway device connected to a device corresponding to the object marked with the selected icon and a gateway device connected to the maintenance terminal using the access information associated with the object marked with the selected icon, The steps of generating the composite image include: A step of determining the coordinate value of the configuration position of the above-mentioned object in the above-mentioned scene image; The step of associating the determined coordinate values ​​of the scene image with the access information; and The step of marking the above-mentioned icon at the determined coordinate position of the above-mentioned on-site image.

2. The maintenance method according to claim 1, wherein: There are multiple work sites mentioned above. The on-site image, the access information, and the synthesized image are generated for a plurality of work sites, respectively.

3. The maintenance method according to claim 1, wherein: The above maintenance methods also include: A step of receiving, from the device, abnormality information indicating an abnormality occurring in the device; and The step of associating the abnormality information with the object corresponding to the device in which the abnormality occurred to generate the synthetic image.

4. The maintenance method according to claim 2, wherein: The above maintenance methods also include: A step of receiving, from the device, abnormality information indicating an abnormality occurring in the device; and The step of associating the abnormality information with the object corresponding to the device in which the abnormality occurred to generate the synthetic image.

5. The maintenance method according to any one of claims 1 to 4, wherein: The above maintenance methods also include: If a device in the work site is changed after the synthetic image is generated, a new synthetic image is generated using new access information corresponding to the change in the device; and The step of outputting the new composite image to the maintenance terminal.

6. The maintenance method according to claim 5, wherein: The maintenance method further includes the step of notifying the maintenance terminal that the synthetic image has been updated when the new synthetic image is generated.

7. The maintenance method according to claim 6, wherein: The maintenance method further includes the step of displaying a dialogue bar together with the composite image, the dialogue bar providing a dialogue function between a maintenance worker using the maintenance terminal and a worker using a worker terminal provided at the work site, A notification that the composite image has been updated is displayed in the dialog box.

8. A maintenance server for maintaining equipment in the work site using a maintenance terminal located at a maintenance base outside the work site, The maintenance server comprises a control unit. The control unit is, Producing a site image representing the above-mentioned work site; Obtain access information for accessing the above-mentioned devices; An icon indicating association with the access information for accessing the device is annotated on the object representing the device in the on-site image to generate a composite image; Outputting the composite image to the maintenance terminal; accepting a selection of the icon in the composite image using the maintenance terminal; and Using the access information associated with the object labeled with the selected icon, a VPN session is established between a gateway device connected to the device corresponding to the object labeled with the selected icon and a gateway device connected to the maintenance terminal. The control unit determines the coordinate value of the configuration position of the object in the on-site image, associates the determined coordinate value of the on-site image with the access information, and marks the icon at the determined coordinate position of the on-site image, thereby generating the composite image.

9. The maintenance server according to claim 8, wherein: There are multiple work sites mentioned above. The control unit generates the on-site image, the access information, and the synthesized image for each of a plurality of work sites.

10. The maintenance server according to claim 8, wherein: The control unit receives, from the device, abnormality information indicating an abnormality occurring in the device, and generates the synthesized image by associating the abnormality information with the object corresponding to the device in which the abnormality occurred.

11. The maintenance server according to claim 9, wherein: The control unit receives, from the device, abnormality information indicating an abnormality occurring in the device, and generates the synthesized image by associating the abnormality information with the object corresponding to the device in which the abnormality occurred.

12. The maintenance server according to any one of claims 8 to 11, wherein: When a device in the work site is changed after the synthetic image is generated, the control unit generates a new synthetic image using new access information corresponding to the change in the device, and outputs the new synthetic image to the maintenance terminal.

13. The maintenance server according to claim 12, wherein: When the new composite image is generated, the control unit notifies the maintenance terminal that the composite image has been updated.

14. The maintenance server according to claim 13, wherein: The control unit displays a dialog box together with the composite image, and displays a notification that the composite image has been updated on the dialog box, wherein the dialog box provides a dialog function between a maintenance worker using the maintenance terminal and a worker using a worker terminal installed at the work site.

Images