State confirmation system

Abstract

This state confirmation system comprises a collection device and a display terminal. The collection device collects operation data including position data of traveling vehicles traveling in a physical distribution facility, and camera video data generated by images being captured by cameras provided in the traveling vehicles. The display terminal receives the operation data and the camera video data, and reproduces the camera video data and operation video data obtained by causing three-dimensional models of the traveling vehicles to travel on the basis of the operation data in a virtual space indicating the physical distribution facility. When a traveling vehicle experiences a fault, the display terminal performs a time-synchronized reproduction of the operation video data and the camera video data for the time period during which the fault occurred.

Description

Status check system

[0001] The present invention relates to a system for checking the status of a logistics facility (including an automated material handling system for transporting semiconductors).

[0002] Patent Literature 1 discloses a guided vehicle system installed in an automated warehouse or the like. The guided vehicle system includes a guided vehicle that transports packages and a management device that manages the guided vehicle. If an abnormality occurs in the guided vehicle, information about the abnormality is displayed on the display of the management device. In detail, an icon indicating the abnormality, an image captured at the time of the abnormality, and a two-dimensional route map showing the route of the guided vehicle at the time of the abnormality are displayed on the display of the management device.

[0003] Patent No. 6458411

[0004] The information displayed on the display of the management device in Patent Document 1 may not be enough to identify the cause of the abnormality in the transport vehicle or the circumstances under which the abnormality occurred, so there is room for improvement. This issue is not limited to automated warehouses that manage packages for delivery, but is a common issue in all logistics facilities, including automated transport systems that transport semiconductors.

[0005] The present invention has been made in consideration of the above circumstances, and its main object is to provide a status confirmation system that can identify in detail the cause of an abnormality in a traveling vehicle at a logistics facility or the circumstances under which the abnormality occurred. Means and effects for solving the problem

[0006] The problem to be solved by the present invention is as described above. Next, the means for solving this problem and the effects thereof will be explained.

[0007] According to a first aspect of the present invention, there is provided a status monitoring system having the following configuration. That is, the status monitoring system includes a collection device and a display terminal. The collection device collects operation data including position data of traveling vehicles traveling within a logistics facility and camera video data generated by cameras installed on the traveling vehicles. The display terminal receives the operation data and the camera video data, and plays back the operation video data in which a three-dimensional model of the traveling vehicle is traveling based on the operation data in a virtual space representing the logistics facility, and the camera video data. The display terminal plays back the operation video data and the camera video data in a time period in which an abnormality occurred in the traveling vehicle, in a synchronized manner.

[0008] By simultaneously viewing the time-synchronized operational footage and camera footage, the facility manager can accurately identify the cause of the vehicle's abnormality or the situation at the time the abnormality occurred. In particular, because the operational footage is 3D, the manager can easily intuitively and in detail grasp the situation of the vehicle when an abnormality occurs.

[0009] The status confirmation system preferably has the following configuration: the display terminal displays one or more of the traveling vehicles in which an abnormality has occurred and accepts selection; the display terminal plays back the operating video data so that the 3D model of the selected traveling vehicle is distinguished from the 3D models of the other traveling vehicles; and the display terminal plays back the camera video data of the selected traveling vehicle.

[0010] This allows administrators to view video footage of specific vehicles in the event of malfunctions in multiple vehicles.

[0011] The status confirmation system preferably has the following configuration: the collection device collects vibration data of the traveling vehicle in chronological order from a vibration sensor provided on the traveling vehicle, and the display terminal time-synchronizes and plays back the operating video data, the camera video data, and the vibration data of the traveling vehicle during a time period when an abnormality occurred.

[0012] This allows the administrator to more accurately identify the cause of the abnormality in the traveling vehicle.

[0013] In the above-described status checking system, it is preferable that the display terminal reproduces the operating video data so that three-dimensional models of elements arranged in the logistics facility other than the traveling vehicle are transparent or semi-transparent.

[0014] This allows the administrator to easily grasp the movements of the traveling vehicle or the like of interest.

[0015] In the above-described status checking system, it is preferable that the display terminal synchronizes the time of playback of the operating video data and the camera video data, with a starting point being a first predetermined time before the occurrence of the abnormality and an ending point being a second predetermined time after the occurrence of the abnormality.

[0016] This allows the administrator to easily check the footage before and after the abnormality.

[0017] According to a second aspect of the present invention, a status confirmation system with the following configuration is provided. That is, the status confirmation system comprises a collection device and a display terminal. The collection device collects operational data including location data of a vehicle traveling in a logistics facility, and sensor data acquired in time series by sensors installed on the vehicle. The display terminal receives the operational data and the sensor data, and plays back operational video data of a 3D model of the vehicle traveling in a virtual space representing the logistics facility based on the operational data, along with the sensor data. The display terminal plays back the operational video data and the sensor data in time synchronization for the period during which the abnormality of the vehicle occurred.

[0018] By simultaneously checking the time-synchronized operational video and sensor data, logistics facility managers can accurately identify the cause of a vehicle's abnormality or the situation at the time the abnormality occurred. In particular, because the operational video is 3D, managers can easily intuitively and in detail grasp the situation of the vehicle when an abnormality occurs.

[0019] A block diagram of a status confirmation system according to one embodiment of the present invention. A diagram showing the contents of a database related to operational data. A sequence diagram showing the exchange of various types of data. A diagram showing a display terminal on which operational video is displayed. A flowchart showing the process by which the collection server determines an abnormality in a moving vehicle and protects data related to the abnormality. A flowchart showing the process performed by the display terminal to verify the abnormality. A diagram showing an example of a screen displaying a list of abnormal moving vehicles. A diagram showing an example of a screen when operational video data, camera video data, and sensor data are played back in sync.

[0020] Next, embodiments of the present invention will be described with reference to the drawings. First, the configuration of the status confirmation system 1 will be described with reference to Figures 1 to 3.

[0021] Status confirmation system 1 is installed in a logistics facility. A logistics facility is a facility that handles goods and specifically includes automated material handling systems for transporting semiconductors. For example, in a logistics facility, receiving operations are carried out to transport goods and store them in storage locations, and outbound operations are carried out to retrieve goods stored in storage locations and transport them to designated locations.

[0022] Multiple mobile vehicles 10 are provided in the logistics facility. Mobile vehicles 10 are vehicles that autonomously travel within the logistics facility. Mobile vehicles 10 are, for example, transport vehicles that perform receiving and shipping operations. Specifically, transport vehicles include stacker cranes, AGVs, shuttle trolleys, and overhead transport vehicles. Furthermore, mobile vehicles are not limited to vehicles that transport goods. Mobile vehicles 10 may, for example, be vehicles that monitor the conditions within the logistics facility, or vehicles that clean the logistics facility.

[0023] As shown in Figure 1, the vehicle 10 comprises a drive unit 11 and a control unit 12. The drive unit 11 is an electric motor or engine that generates power to move the vehicle 10. The control unit 12 is a computer having a CPU, memory, storage, etc. The control unit 12 uses a program to control the drive unit 11 and a steering device (not shown) to make the vehicle 10 move autonomously.

[0024] The traveling vehicle 10 is provided with a module 20. The module 20 modularizes a camera 21, a sensor 22, and a communication device 23. The camera 21 captures images of the surroundings of the traveling vehicle 10. If the traveling vehicle 10 is a stacker crane, the camera 21 may be provided on a carriage that moves back and forth along rails installed on the ground or on a platform that moves up and down along a mast that extends up and down. Hereinafter, the image data captured and generated by the camera 21 will be referred to as "camera image data." The sensor 22 detects information about the traveling vehicle 10 or its surroundings and generates sensor data indicating the detection results. The sensor 22 detects, for example, temperature, vibration, and sound volume. The module 20 may be provided with multiple types of sensors 22, or multiple sensors of the same type. The communication device 23 is a wireless communication module capable of transmitting data via wireless communication. A UWB (Ultra-Wide Band) wireless chipset may be used as the wireless module of the communication device 23. In this case, even if the traveling vehicle 10 is an AGV that travels without a track on a two-dimensional plane, the position of the traveling vehicle 10 can be determined with high accuracy inside the building.

[0025] The data transmitted by the communication device 23 includes not only the camera image data and sensor data described above but also operation data. As shown in FIG. 2 , operation data is data indicating the operation status of the traveling vehicle 10, and includes at least position data of the traveling vehicle 10. The position data of the traveling vehicle 10 included in the operation data is, for example, information specifying one section when a facility is divided according to specific criteria. Alternatively, when the traveling vehicle 10 moves along rails or the like, the position data of the traveling vehicle 10 may be information specifying the position on the rail path. When the traveling vehicle 10 moves on a specific plane, the height position may be omitted. Because the traveling vehicle 10 travels autonomously under the control of the control device 12, the position of the traveling vehicle 10 can be specified based on the control content of the control device 12.

[0026] The operation data may include data other than the position data of the traveling vehicle 10. For example, if the traveling vehicle 10 is capable of changing its orientation (front direction), the operation data may include information on the orientation of the traveling vehicle 10. The operation data may include information indicating whether the traveling vehicle 10 is transporting cargo. If the traveling vehicle 10 has a tool (e.g., a transfer tool), the operation data may include information indicating the status of the tool. For example, if the traveling vehicle 10 is a stacker crane, the operation data may include information indicating the height position of a lifting platform serving as a transfer tool provided on the stacker crane.

[0027] The operation data, camera image data, and sensor data are all time-series data. That is, the operation data is data that indicates the operation status, such as the position of the traveling vehicle 10, at each time. The camera image data is data that indicates images at each time. The sensor data is data that indicates the detection values ​​of the sensor 22 at each time. In the following description, the operation data, camera image data, and sensor data may be collectively referred to as "collected data."

[0028] The status checking system 1 includes a collection server 2, a central server 3, and a display terminal 4.

[0029] The collection server 2 is installed within the logistics facility. The collection server 2 can communicate with the general server 3, the display terminal 4, and the traveling vehicles 10 via a network within the logistics facility. The collection server 2 is a server device equipped with a CPU, memory, storage, communication equipment, etc. The collection server 2 realizes various functions by the CPU executing programs.

[0030] The collection server 2 functions as a collection device. The collection device is a device that receives and stores collected data transmitted by the traveling vehicles 10, as shown in Fig. 3. The collection server 2 may receive the collected data directly from the traveling vehicles 10, or may receive the data via a relay device. The collection server 2 associates the collected data received from the traveling vehicles 10 with a unique device ID for identifying the traveling vehicles 10, and stores the data in a database.

[0031] The collection server 2 determines whether or not an abnormality has occurred in the vehicle 10 based on the collected data received from the vehicle 10. For example, it determines whether or not predetermined abnormality conditions are met. Abnormality conditions include, for example, the vehicle 10 remaining stationary for a predetermined period of time, the vehicle 10's position exceeding a predetermined range, the detection that the cargo's position is outside the acceptable range based on the analysis results of camera video data, or the sensor's detected value exceeding a threshold. The vehicle 10 may determine whether or not an abnormality has occurred and notify the collection server 2 of the determination result. As shown in Figure 3, if the collection server 2 determines that an abnormality has occurred, it notifies the overall server 3 of the detected abnormality.

[0032] The timing at which the collection server 2 receives the collected data from the vehicle 10 is arbitrary. The collection server 2 may, for example, receive the collected data from the vehicle 10 in real time. Alternatively, the vehicle 10 may store the collected data for a predetermined period of time (for example, several hours), and then the collection server 2 may receive the collected data for that predetermined period of time all at once.

[0033] Furthermore, as shown in Figure 1, the status confirmation system 1 is equipped with multiple collection servers 2. Each collection server 2 has the same function. However, each collection server 2 is responsible for acquiring data from different vehicles 10. For example, each collection server 2 stores a list of device IDs of vehicles 10 from which it should acquire data, and stores only the data of the device IDs listed in the list in its database.

[0034] The central server 3 may be located within the logistics facility, or it may be located in a monitoring facility geographically separated from the logistics facility. The central server 3 can communicate with the collection server 2 and the display terminal 4 via the network within the logistics facility. Alternatively, the central server 3 may be located on the cloud. In this case, the central server 3 can communicate with the collection server 2 and the display terminal 4 via the internet.

[0035] The general server 3 is a server device equipped with a CPU, memory, storage, communication equipment, etc. The CPU executes programs to enable the general server 3 to realize various functions, including a web server function, a model management function, and a state management function.

[0036] The web server function is a function that transmits data such as HTML data, CSS, and images required to display a specific web page to the display terminal 4 when the display terminal 4 accesses it using a browser. The browser on the display terminal 4 displays the web page based on the received data. Furthermore, when a button or link displayed on the web page is selected, the central server 3 transmits new data such as HTML data, CSS, and images corresponding to the selected button or link to the display terminal 4. Note that the web server function is not essential, and the display terminal 4 may store information regarding the screen layout, etc.

[0037] Next, the model management function will be described. In this embodiment, an image in which a three-dimensional model (three-dimensional CAD data) is arranged in a virtual space is generated. The three-dimensional model includes a three-dimensional model showing the environment of the logistics facility and a three-dimensional model showing the traveling vehicle 10. The three-dimensional model of the environment is data in which three-dimensional models of structures such as floors, shelves, ceilings, and stationary machines are arranged according to the actual positions of the logistics facility. The central server 3 stores the above-mentioned three-dimensional models and transmits a specified three-dimensional model in response to a request from the display terminal 4. Note that the model management function is not essential; for example, the display terminal 4 may store the three-dimensional models in advance.

[0038] The status management function is a function for managing the status of the traveling vehicle 10. For example, the status management function is a function for storing information that an abnormality has occurred in the traveling vehicle 10 in association with the device ID. Furthermore, the status management function is a function for notifying the display terminal 4 that an abnormality has occurred, and for notifying the mobile terminal of the administrator that an abnormality has occurred. Note that the status management function is not essential and can be omitted.

[0039] By using the above-described functions, the central server 3 transmits browser display data, three-dimensional models of the environment and the traveling vehicle, and abnormality information to the display terminal 4, as shown in FIG.

[0040] The status checking system 1 of this embodiment is provided with two types of server devices: a collection server 2 and a general server 3. Alternatively, the functions of the collection server 2 and the general server 3 may be integrated into a single server device. Alternatively, a separate server device may be provided for each of the multiple functions possessed by the general server 3.

[0041] Display terminal 4 is a terminal used by administrators to check the status of the logistics facility. Display terminal 4 can communicate with collection server 2 and management server 3 via the network within the logistics facility. Display terminal 4 can be, for example, a tablet device, smartphone, PC, or dedicated terminal. Display terminal 4 is equipped with a CPU, memory, storage, communication device, display, and input device. The CPU executes programs, enabling display terminal 4 to perform various functions. The display shows images and other information to check the status of the logistics facility. The input device is either a hardware key or a touch panel, and accepts operations from the administrator.

[0042] The display terminal 4 displays operational video on the screen by, for example, generating and playing operational video data. Operational video is video of a 3D model of the vehicle 10 moving based on operational data, with a 3D model of the logistics facility environment described above placed in a virtual space. Since the operational data includes position data, the display terminal 4 can generate operational video data by changing the display position of the 3D model in accordance with changes in the position data.

[0043] For example, if an administrator instructs the display of operational video for a designated area, the display terminal 4 requests and obtains collected data of the vehicles 10 located in that area from the collection server 2. Next, the display terminal 4 requests and obtains a 3D model of the environment and vehicles 10 located in that area from the central server 3. Then, based on the acquired collected data (especially operational data) and the 3D model, the display terminal 4 generates and plays operational video data, thereby displaying the operational video on the display as shown in Figure 4.

[0044] Alternatively, the central server 3 may generate the operational video data instead of the display terminal 4. In this case, the display terminal 4 plays back the operational video data received from the central server 3. Since the operational video is a three-dimensional image, the viewpoint can be changed. It is also possible to hide some of the three-dimensional models. By checking the three-dimensional operational video, the administrator can intuitively and comprehensively grasp the status of the logistics facility (especially the status of the moving vehicles 10).

[0045] Next, the function of checking for any abnormality occurring in the traveling vehicle 10 will be described in detail with reference to FIGS. 5 to 8. FIG.

[0046] First, let's explain how the collected data is handled. Since the collected data is time-series data, the total amount of collected data increases over time. On the other hand, the storage capacity of the collection server 2 is limited. Therefore, in this embodiment, the collection server 2 deletes collected data after a predetermined time has elapsed since acquisition (for example, one month, one week, or 72 hours). On the other hand, the collected data related to abnormalities in the vehicle 10 may be used later for cause verification and analysis. Therefore, the collection server 2 protects the collected data related to abnormalities in the vehicle 10. Specifically, the collection server 2 performs each process shown in the flowchart in Figure 5.

[0047] First, the collection server 2 determines whether or not an abnormality has occurred in the vehicle 10 (S101). The criteria for determining whether or not an abnormality has occurred are as described above. If the collection server 2 determines that an abnormality has occurred in the vehicle 10, it notifies the general server 3 of this fact (S102). The notification contents include, for example, the device ID of the vehicle 10, the time the abnormality occurred, and the details of the abnormality. The general server 3 may also notify the display terminal 4 or the administrator's mobile terminal of the abnormality notified by the collection server 2.

[0048] Next, the collection server 2 protects the collected data related to the abnormality occurrence. "Protection" means preventing the data from being deleted over time. Specifically, the collection server 2 extracts and protects the operation data, camera video data, and sensor data, starting from a first predetermined time before the abnormality occurrence and ending from a second predetermined time after the abnormality occurrence (S103).

[0049] By protecting the collected data, including the periods before and after the anomaly occurred, it becomes possible to recreate the situation at the time of the anomaly when verifying the cause of the anomaly later. Furthermore, since information prior to the time of the anomaly is important for verifying the cause of the anomaly, it is preferable that the first predetermined time is longer than the second predetermined time (the reverse is also acceptable).

[0050] Next, referring to Figures 6 to 8, we will explain the process for verifying the cause of the anomaly.

[0051] The flowchart in Fig. 6 shows the processing performed by the display terminal 4 when verifying the cause of an abnormality. First, the display terminal 4 displays a list of traveling vehicles 10 in which an abnormality has been detected, and accepts the selection of a traveling vehicle for which an abnormality is to be verified (S201). Hereinafter, this list will be referred to as the "abnormal traveling vehicle list." Fig. 7 shows an example of a screen displaying the abnormal traveling vehicle list.

[0052] Since abnormality information of traveling vehicles 10 is aggregated in the central server 3, the display terminal 4 acquires the abnormality information from the central server 3 and generates an abnormal traveling vehicle list based on the acquired abnormality information. Even if the abnormality has been resolved at the present time, it is important to verify abnormalities that occurred in the past. Therefore, the display terminal 4 lists traveling vehicles 10 that have experienced an abnormality during a predetermined period in the abnormal traveling vehicle list, regardless of the current abnormal state. However, the display terminal 4 may also list only traveling vehicles 10 that are currently abnormal in the abnormal traveling vehicle list.

[0053] Next, the display terminal 4 acquires protected operational data, camera video data, and sensor data for the selected vehicle 10 from the collection server 2 (S202). Based on the acquired operational data, the display terminal 4 generates operational video data in which the logistics facility environment and the unselected vehicles 10 are semi-transparent (S203). By making the logistics facility environment and the unselected vehicles 10 semi-transparent, the selected vehicle 10 is not hidden, and the selected vehicle 10 is made easier to see, thus making it easier to verify abnormalities. Alternatively, the environment and the unselected vehicles 10 may be made transparent (hidden).

[0054] Next, the display terminal 4 plays back the generated operational video data, camera video data, and sensor data in sync with the time (S204). The operational video data played back here is data that highlights the selected vehicle 10, as described above. The camera video data played back is data generated by the camera 21 installed on the selected vehicle 10. The sensor data played back is data generated by the sensor 22 installed on the selected vehicle 10. Therefore, these three sets of data are used to verify the status of the same vehicle 10. Since these three sets of data are played back in sync with the time, their interrelationships can be easily understood. For example, if vehicle 10 collides and an abnormality occurs, the operational video will show the collision of vehicle 10 from an objective viewpoint, the camera video will show the collision of vehicle 10 from a subjective viewpoint, and the sensor data will show, for example, a state in which vibrations increase sharply.

[0055] Figure 8 shows an example of a screen displaying synchronized playback of operational video data, camera video data, and sensor data. In the example in Figure 8, the display terminal 4 displays the operational video across the entire screen, and also overlays the camera video window 31 and the sensor data window 32 onto the operational video on the display. The camera video window 31 and the sensor data window 32 can be repositioned and resized. This allows administrators to view data in the event of an anomaly in their desired layout.

[0056] Furthermore, the display terminal 4 displays on its display a first seek bar 41 that is a seek bar for the operational video, a second seek bar 42 that is a seek bar for the camera video, and a third seek bar 43 that is a seek bar for the sensor data. Because the times of the three videos are synchronized, the time indicated by each seek bar is the same. When the administrator operates one of the seek bars, the other two seek bars move in the same way in response to that operation, and the times of the three videos are synchronized. In other words, when the administrator changes the target time for one piece of data on the display terminal 4, the display terminal 4 synchronizes the target times for the remaining pieces of data. Furthermore, an abnormality mark 44 indicating the time point at which an abnormality occurred is displayed on each of the three seek bars. This allows the administrator to grasp the time point at which an abnormality occurred at a glance.

[0057] The screen example in FIG. 8 is an example and can be changed as appropriate. For example, the operation video may not be displayed full screen, but may be displayed in a window. Also, instead of displaying three types of video on the display, only two types of video may be displayed on the display. For example, only the operation video and the camera video may be displayed on the display, or only the operation video and the sensor data may be displayed on the display. Furthermore, if multiple cameras 21 are installed on the traveling vehicle 10, the camera video from each camera 21 may be displayed in multiple camera video windows 31.

[0058] The display terminal 4 may also be capable of changing the target to be displayed semi-transparently in the operating video. For example, each time a selection is made in the operating video, the selected element may be displayed on the display in a semi-transparent state. Furthermore, instead of displaying unselected traveling vehicles 10 semi-transparently on the display, the selected traveling vehicles 10 and unselected traveling vehicles 10 may be displayed in different colors on the display.

[0059] As described above, the status confirmation system 1 of this embodiment comprises a collection server 2 and a display terminal 4. The collection server 2 collects operational data, including location data of vehicles 10 traveling within a logistics facility, and camera video data generated by cameras 21 installed on the vehicles 10. The display terminal 4 receives the operational data and camera video data and plays back operational video data and camera video data, which are generated by running a 3D model of the vehicles 10 in a virtual space representing the logistics facility based on the operational data. The display terminal 4 plays back the operational video data and camera video data for the time period in which the abnormality occurred in the vehicle 10, synchronizing the time. This is Feature 1.

[0060] By simultaneously reviewing synchronized operational video and camera footage, logistics facility managers can accurately identify the cause of any abnormalities in the moving vehicle 10 or the circumstances surrounding the abnormality. In particular, because the operational video is three-dimensional, managers can easily grasp the situation of the moving vehicle 10 at the time of the abnormality in an intuitive and detailed manner.

[0061] In the status confirmation system 1 of this embodiment, the display terminal 4 displays one or more vehicles 10 in which an abnormality has occurred and accepts the selection. The display terminal 4 plays back the operational video data so that the 3D model of the selected vehicle 10 is distinct from the 3D models of the other vehicles 10. The display terminal 4 plays back the camera video data of the selected vehicle 10. The above is Feature 2.

[0062] This allows the administrator to check the video of a desired traveling vehicle 10 when an abnormality occurs in multiple traveling vehicles 10.

[0063] In the status confirmation system 1 of this embodiment, the collection server 2 collects vibration data of the vehicle 10 in chronological order from a vibration sensor installed on the vehicle 10. The display terminal 4 plays back the operational video data, camera video data, and vibration data of the vehicle 10 where the abnormality occurred, synchronized with the time. The above is feature 3.

[0064] This allows the administrator to more accurately identify the cause of the abnormality in the traveling vehicle 10.

[0065] In the status monitoring system 1 of this embodiment, the display terminal 4 plays back the operation video data so that the three-dimensional models of the elements arranged in the logistics facility other than the traveling vehicle 10 are transparent or semi-transparent.

[0066] This allows the administrator to easily understand the movement of the traveling vehicle 10 of interest.

[0067] In the status monitoring system 1 of this embodiment, the display terminal 4 synchronizes the time of the operation video data and the camera video data and plays them back, starting from a first predetermined time before the occurrence of the abnormality and ending from a second predetermined time after the occurrence of the abnormality.

[0068] This allows the administrator to easily check the footage before and after the abnormality.

[0069] The status checking system 1 of this embodiment includes a collection server 2 and a display terminal 4. The collection server 2 collects operation data including position data of traveling vehicles 10 traveling within a logistics facility, and sensor data acquired in time series by sensors 22 installed on the traveling vehicles 10. The display terminal 4 receives the operation data and the sensor data, and plays back operation video data in which a three-dimensional model of the traveling vehicle 10 is traveling based on the operation data in a virtual space representing the logistics facility, and the sensor data. The display terminal 4 plays back the operation video data and the sensor data in a time period in which an abnormality occurred in the traveling vehicle 10, with the time synchronized. This is Feature 6.

[0070] By simultaneously checking the time-synchronized operation video and sensor data, the manager of the logistics facility can accurately identify the cause of the abnormality in the traveling vehicle 10 or the situation when the abnormality occurred. In particular, because the operation video is a three-dimensional video, the manager can easily intuitively and in detail grasp the situation of the traveling vehicle 10 when the abnormality occurred.

[0071] The above-mentioned features 1 to 6 can be combined, for example, as follows to realize a state confirmation system from the following perspectives: [Perspective 1] A state confirmation system having feature 1. [Perspective 6] A state confirmation system having feature 6. [Perspective 2] A state confirmation system having feature 2 in addition to perspective 1 or 6. [Perspective 3] A state confirmation system having feature 3 in addition to any one of perspectives 1, 2, or 6. [Perspective 4] A state confirmation system having feature 4 in addition to any one of perspectives 1, 2, 3, or 6. [Perspective 5] A state confirmation system having feature 5 in addition to any one of perspectives 1, 2, 3, 4, or 6. [Perspective 7] A state confirmation system having feature 1 or feature 6 (i.e., a system that plays back at least one of camera video data and sensor data in addition to operational video data, and synchronizes the time of the two (or three) data to be played back).

Claims

1. A status confirmation system comprising: a collection device that collects operational data including position data of vehicles traveling within a logistics facility and camera image data generated by cameras installed on the vehicles; and a display terminal that receives the operational data and the camera image data, and plays back operational image data in which a three-dimensional model of the vehicle is traveling in a virtual space representing the logistics facility based on the operational data, and the camera image data, wherein the display terminal plays back the operational image data and the camera image data in time synchronization during the time period in which an abnormality occurred in the vehicle.

2. A status confirmation system as described in claim 1, wherein the display terminal displays one or more of the traveling vehicles in which an abnormality has occurred and accepts selection, the display terminal plays back the operating video data so that the three-dimensional model of the selected traveling vehicle appears distinct from the three-dimensional models of the other traveling vehicles, and the display terminal plays back the camera video data of the selected traveling vehicle.

3. A status confirmation system as described in claim 1, wherein the collection device collects vibration data of the traveling vehicle in chronological order from a vibration sensor installed in the traveling vehicle, and the display terminal plays back the operating video data, the camera video data, and the vibration data in time synchronization during the time period when the abnormality occurred in the traveling vehicle.

4. A status confirmation system as described in claim 1, characterized in that the display terminal plays back the operating video data so that three-dimensional models of elements placed in the logistics facility other than the traveling vehicle become transparent or semi-transparent.

5. A status confirmation system as described in claim 1, characterized in that the display terminal plays back the operating video data and the camera video data in time synchronization, starting from a first predetermined time before the occurrence of the abnormality and ending from a second predetermined time after the occurrence of the abnormality.

6. A status confirmation system comprising: a collection device that collects operational data including position data of vehicles traveling within a logistics facility and sensor data acquired in time series by sensors installed on the vehicles; and a display terminal that receives the operational data and the sensor data and plays back operational video data in which a three-dimensional model of the vehicle is traveling in a virtual space representing the logistics facility based on the operational data, and the sensor data, wherein the display terminal plays back the operational video data and the sensor data in time synchronization during the time period in which an abnormality occurred in the vehicle.

Images