Information display device and program

Abstract

We provide an information display device and program that can assist in viewing items exhibited within a facility. [Solution] The information display device of the embodiment is an information display device used by a user moving within a facility where an item is displayed, and comprises: a first acquisition means for acquiring a photograph taken inside the facility; a second acquisition means for acquiring identification information that can identify the item, read from a wireless tag attached to the item; a position identification means for identifying the position of the device within the facility; and a display control means for displaying existence information indicating the presence of the item at a position on the photograph corresponding to the display position of the item, based on map information indicating the display position of the item within the facility and the position of the device, and for displaying the existence information in a way that distinguishes between items for which the identification information has been acquired and items for which it has not been acquired.

Description

[Technical Field]

[0001] Embodiments of the present invention relate to an information display device and a program. [Background Art]

[0002] Conventionally, in facilities such as museums and art galleries that exhibit articles, for the purpose of article management and the like, a wireless tag such as an RFID tag is attached to an article, and the wireless tag is read by a wireless tag communication device. For example, an information display device for providing explanations of exhibited articles to visitors is equipped with a wireless tag communication function, and based on information of the wireless tag read when a visitor moves within the facility with a terminal, technologies have been proposed for providing information related to articles to visitors and performing inventory of articles.

[0003] However, in the above-described conventional technology, information on articles that can be obtained by a terminal is limited to areas where wireless tag communication is available, so visitors cannot grasp the position and existence of other articles placed in the facility, resulting in poor convenience. Also, from the perspective of article management, since visitors cannot easily grasp the position and existence of each article exhibited in the facility, there is a possibility that some articles may not have their wireless tags read. [Summary of the Invention] [Problem to be Solved by the Invention]

[0004] The problem to be solved by the present invention is to provide an information display device and a program capable of supporting viewing of articles exhibited in a facility. [Means for Solving the Problem]

[0005] The information display device of the embodiment is an information display device used by a user moving within a facility where an item is displayed, and comprises: a first acquisition means for acquiring a photographed image taken within the facility; a second acquisition means for acquiring identification information that can identify the item, read from a wireless tag attached to the item; a position identification means for identifying the position of the device within the facility; and a display control means for displaying existence information indicating the presence of the item at a position on the photographed image corresponding to the display position of the item, based on map information indicating the display position of the item within the facility and the position of the device, and for displaying the existence information in a way that distinguishes between items for which identification information has been acquired and items for which it has not. [Brief explanation of the drawing]

[0006] [Figure 1] Figure 1 shows an example of the system configuration of an in-facility system according to the embodiment. [Figure 2] Figure 2 shows an example of the external configuration of a guide terminal according to this embodiment. [Figure 3] Figure 3 shows a first example of the hardware configuration of a guide terminal according to this embodiment. [Figure 4] Figure 4 shows a second example of the hardware configuration of the guide terminal according to the embodiment. [Figure 5] Figure 5 shows an example of the hardware configuration of a server device according to this embodiment. [Figure 6] Figure 6 shows an example of the data structure of the item information table according to the embodiment. [Figure 7] Figure 7 shows an example of the data structure of the inventory information table according to the embodiment. [Figure 8] Figure 8 shows an example of the functional configuration of the guide terminal and server device according to the embodiment. [Figure 9] Figure 9 shows an example of a screen displayed on the display unit of a guide terminal according to an embodiment. [Figure 10] Figure 10 is a flowchart showing an example of the processing performed by the guide terminal and server device in the embodiment. [Modes for carrying out the invention]

[0007] The following description of the information display device and program according to the embodiment will be given with reference to the drawings. In the following embodiment, the form used in a facility that displays items such as a museum or art gallery will be described, but the embodiment is not limited to this form. In this embodiment, "display" means setting up or permanently displaying multiple items in a predetermined location within the facility and making them available for viewing by visitors to the facility.

[0008] Figure 1 shows an example of the system configuration of an in-facility system according to an embodiment. The in-facility system includes a guide terminal 1 and a server device 6. The guide terminal 1 and the server device 6 are connected to each other via a network N so as to be able to communicate.

[0009] Network N can utilize the Internet, VPN (virtual private network), LAN (local area network), public communication network, mobile communication network, etc., individually or in appropriate combinations. In this embodiment, a LAN installed within a facility is used as an example of network N. The number of guide terminals 1 in the facility system is not limited to the illustrated example.

[0010] Guide terminal 1 is an example of an information display device. Guide terminal 1 is lent to visitors (hereinafter also referred to as users) who come to the facility, and users move around the facility with guide terminal 1. Visitors can obtain and check information about items displayed in the facility via guide terminal 1.

[0011] Figure 2 shows an example of the external configuration of the guide terminal 1 according to the embodiment. The guide terminal 1 shown in Figure 2 is a handheld information terminal that is operated by the user while being held in their hand. Note that the guide terminal 1 may also be mounted on a mobile device operated by the user.

[0012] Guide terminal 1 communicates with a wireless tag TG, such as an RFID tag. Guide terminal 1 reads the tag information recorded on the wireless tag TG by communicating with it.

[0013] The wireless tag TG is a wireless communication device that operates using radio waves (output signals) from the guide terminal 1. The wireless tag TG includes a processor, memory, communication circuit, and antenna. In response to a read command from the guide terminal 1, the wireless tag TG outputs a response signal containing tag information stored in its own memory. For example, a wireless tag TG is attached to each exhibit item. The wireless tag TG attached to an item stores tag information, including identification information (hereinafter also referred to as item ID) that can identify each item, in its internal memory.

[0014] In the configuration example shown in Figure 2, the guide terminal 1 is composed of a reader device 10 and an information terminal device 20. Here, the information terminal device 20 may be provided as the guide terminal 1. In this case, the guide terminal 1 as the information terminal device 20 is operated with the reader device 10, which is an external device, attached.

[0015] The reader device 10 comprises a housing on which an RFID tag interface having an antenna and a communication control circuit is installed. In the reader device 10, the antenna is positioned so that its directivity is strongest in the direction of arrow a shown in Figure 2. Here, the direction of arrow a is assumed to be the front (forward) direction of the guide terminal 1. Also, the front direction of the guide terminal 1 is assumed to be the direction of the antenna.

[0016] The reader device 10 has a gripping part 11 and a holding part 12. The gripping part 11 is the part that the operator grips. The holding part 12 is made up of a jig that holds the information terminal device 20. The holding part 12 holds the information terminal device 20 so that the display screen of the display unit 21 of the information terminal device 20 faces the operator who is gripping the gripping part 11. The guide terminal 1 is operated by the operator gripping the gripping part 11 with the information terminal device 20 set in the holding part 12.

[0017] The information terminal device 20 includes a display unit 21 and an input unit 22. The display unit 21 and the input unit 22 are, for example, a display device with a touch panel. When the information terminal device 20 is set in the holding unit 12 of the reader device 10, a user can visually recognize a screen displayed by the display unit 21. Further, when the information terminal device 20 is set in the holding unit 12 of the reader device 10, an operation instruction from the user is input to the input unit 22.

[0018] The guide terminal 1 is held by the user and moves through the facility together with the user. At this time, the guide terminal 1 reads tag information recorded in the wireless tag TG by receiving a response signal from the wireless tag TG existing within a reading area (reading range) of an antenna.

[0019] Further, the guide terminal 1 estimates the position of the own device using self-position estimation technology. For example, based on an image captured by a camera by the information terminal device 20 or the reader device 10, the guide terminal 1 estimates the self-position (the position of the own device) by a self-position estimation method used for AR (Augmented Reality) technology and the like. However, the method, configuration, and the like for estimating the position of the own device used in the guide terminal 1 are not limited to a specific one, as long as the self-position can be estimated even in an indoor environment or the like.

[0020] Returning to FIG. 1, the server device 6 manages information related to articles exhibited in the facility. Further, the server device 6 manages articles exhibited in the facility based on the tag information read by the guide terminal 1.

[0021] Hereinafter, the configurations of the guide terminal 1 and the server device 6 described above will be explained. First, a configuration example of the guide terminal 1 will be explained.

[0022] Figure 3 is a diagram showing a first configuration example of the hardware configuration of the guide terminal 1 according to the embodiment. In the configuration example shown in Figure 3, the guide terminal 1 has a processor 31, memory 32, wireless tag communication unit 33, sensor 34, communication interface (I / F) 35, camera 36, ​​display unit 21 and input unit 22. As shown in Figure 3, the guide terminal 1 can communicate with the wireless tag TG via the wireless tag communication unit 33.

[0023] The processor 31 controls each component. The processor 31 includes, for example, an arithmetic circuit such as a CPU (Central Processing Unit). The processor 31 performs control of each component and various data processing by executing a program. The processor 31 may also be equipped with internal memory. The processor 31 performs various processes by executing a program stored in memory 32 or the like.

[0024] Memory 32 includes various types of memory. For example, memory 32 includes ROM (Read Only Memory), RAM (Random Access Memory), NVM (Non-Volatile Memory), and other types of memory. ROM is a non-volatile memory that cannot be rewritten. ROM stores programs executed by the processor 31. RAM is a volatile memory that temporarily stores data. RAM operates as working memory or buffer memory.

[0025] The NVM of memory 32 is a rewritable, non-volatile memory. Memory 32 stores information such as control information, configuration information, and processing results in the NVM. Memory 32 also stores various programs in the NVM for the processor 31 to execute the processes described later. For example, memory 32 stores an operation support program in the NVM that outputs information to assist the operator in operating the chassis.

[0026] Furthermore, the NVM in memory 32 stores a spatial map 321 that associates the map of the facility with the location of each item displayed within the facility. Here, the map of the facility is a 3D map that represents the structure and equipment within the facility in three dimensions. For example, the 3D map may be created by 3D scanning the inside of the facility. In addition, the item ID of each item is associated with and stored at the display location of each item within the 3D map.

[0027] The wireless tag communication unit 33 is a communicator capable of communicating with the wireless tag TG. The wireless tag communication unit 33 is, for example, a communicator having a communication control circuit and an antenna. The wireless tag communication unit 33 may also be configured with a connection interface that connects to an external antenna.

[0028] In the wireless tag communication unit 33, the communication control circuit includes a control circuit for communicating with the wireless tag TG via the antenna. The communication control circuit causes the antenna to emit a transmission signal (radio wave) supplied from the processor 31 at a set output value. The antenna outputs the transmission signal supplied from the communication control circuit as a radio wave that the wireless tag TG can receive. The wireless tag communication unit 33 sets the range (reading area, reading range) in which the wireless tag TG can respond based on the direction of the antenna and the output value of the radio wave output from the antenna.

[0029] Furthermore, in the wireless tag communication unit 33, the communication control circuit not only outputs a transmission signal to the antenna, but also supplies the signal received by the antenna to the processor 31 as received data. The communication control circuit receives a response signal from the wireless tag TG, processes the response signal (received signal) received by the antenna, and supplies it to the processor 31. For example, the communication control circuit supplies the processor 31 with tag information contained in the received signal from the wireless tag TG.

[0030] The sensor (sensor group) 34 detects the movement of the guide terminal 1. For example, the sensor 34 may be an acceleration sensor, a gyroscope sensor, or a geomagnetic (compass) sensor. The sensor 34 may be configured to include multiple types of sensors. The sensor 34 may also include a position detection sensor that detects the self-position of the guide terminal 1 within the facility. The sensor 34 may also include a depth sensor capable of capturing the distance from the wireless tag communication unit 33 to surrounding objects and the shape of the objects as three-dimensional objects. The sensor 34 is an example of a position detection sensor.

[0031] The communication interface 35 is a wireless communication interface for communicating with external devices such as the server device 6. The communication interface 35 connects to the network N via a communication device such as a wireless LAN router, and communicates with the server device 6 connected to the network N.

[0032] Camera 36 captures images of the area around the guide terminal 1. The shooting direction of camera 36 changes according to the orientation of the guide terminal 1. For example, it is preferable to set the shooting direction of camera 36 to coincide with the orientation of the antenna of the wireless tag communication unit 33 (the front direction of the guide terminal 1). The images generated by camera 36 may be used not only for display on the display unit 21, but also for estimating the self-position of the guide terminal 1. In this case, camera 36 becomes an example of a position detection sensor. For example, guide terminal 1 estimates its own position based on the image captured by camera 36 using a self-position estimation method used in AR (augmented reality) technology, etc.

[0033] Furthermore, the estimation of the self-position is not limited to using the output from the position detection sensor mentioned above, but may also be estimated using other methods that utilize the output from other devices. For example, the self-position may be estimated using a positioning method that employs UWB (Ultra Wide Band).

[0034] The display unit 21 and input unit 22 of the information terminal device 20 are connected to the processor 31 via an internal interface. The display unit 21 is a device that displays information. For example, the display unit 21 displays various screens under the control of the processor 31. The display unit 21 is implemented by a display device having, for example, a display and / or an LED (Light Emitting Diode). The display unit 21 may also have sound devices such as a buzzer and a speaker.

[0035] The input unit 22 is a device for the user to input operation instructions, etc. The input unit 22 can be implemented, for example, by hard switches and / or buttons, a touch panel common to the display unit 21, etc. The input unit 22 supplies information indicating the content of the user's instructions to the processor 31.

[0036] The guide terminal 1 has a power supply that provides power for operation. In the case of a handheld guide terminal 1, for example, a rechargeable battery (secondary battery) is provided as the power supply. The battery supplies power to each part of the guide terminal 1 to operate it.

[0037] Figure 4 shows a second example of the hardware configuration of the guide terminal 1 according to the embodiment. In the example configuration shown in Figure 4, the information terminal device 20, which is the guide terminal 1, is connected to the reader device 10, which is an external device. In Figure 4, elements similar to those in Figure 3 are given the same reference numerals and their descriptions are omitted.

[0038] In addition to the wireless tag communication unit 33 described above, the reader device 10 also includes a processor 41, memory 42, and interface (I / F) 43.

[0039] The processor 41 controls each part of the reader device 10. The processor 41 includes, for example, an arithmetic circuit such as a CPU. The processor 41 performs control of each part and various data processing by executing a program. The processor 41 may also be equipped with internal memory. The processor 41 performs various processes by executing a program stored in memory 42 or the like.

[0040] Memory 42 includes various types of memory. For example, memory 42 includes ROM, RAM, NVM, and other types of memory. ROM is a non-volatile memory that cannot be rewritten. ROM stores programs executed by the processor 41. RAM is a volatile memory that temporarily stores data. RAM operates as working memory or buffer memory. NVM is a rewritable non-volatile memory. NVM stores information such as control information, configuration information, and processing results. The NVM in memory 42 stores various programs for the processor 41 to execute the processes described later.

[0041] Interface 43 only needs to be compatible with the interface 53 provided by the information terminal device 20. For example, interface 43 may be physically connected to an interface provided by the information terminal device 20, such as a USB (Universal Serial Bus) interface or a LAN interface. Alternatively, interface 43 may communicate wirelessly, such as through a Bluetooth® interface.

[0042] The reader device 10 has a power supply that provides power for operation. The reader device 10 is equipped with, for example, a rechargeable battery (secondary battery) as a power source. The battery supplies power to each part of the guide terminal 1 to operate it.

[0043] In the configuration example shown in Figure 4, the information terminal device 20 is set in the holding unit 12 of the reader device 10. The information terminal device 20 can be any device that includes a display unit 21, an input unit 22, etc., and communicates with the reader device 10. The information terminal device 20 is, for example, a portable information processing device such as a smartphone or a tablet PC. In this configuration example, the information terminal device 20 is an example of an information display device.

[0044] As shown in Figure 4, the information terminal device 20 includes a display unit 21, an input unit 22, a sensor 34, a communication interface 35, and a camera 36, ​​as well as a processor 51, memory 52, an interface (I / F) 53, and the like.

[0045] The processor 51 controls various parts of the information terminal device 20 and performs data processing, etc. The processor 51 is, for example, a CPU. The processor 51 performs various operations by executing programs stored in the memory 52.

[0046] Memory 52 includes various types of memory. For example, memory 52 includes ROM, RAM, NVM, etc. ROM is a non-volatile memory that cannot be rewritten. ROM stores programs executed by the processor 51. RAM is a volatile memory that temporarily stores data. RAM operates as working memory or buffer memory. NVM is a rewritable non-volatile memory. NVM stores information such as control information, configuration information, and processing results. The NVM in memory 42 stores various programs for the processor 41 to execute the processes described later. The NVM in memory 52 also stores the spatial map 321 described above.

[0047] Interface 53 is an interface (second communication interface) for communicating with interface 43. Interface 53 can be any interface that is compatible with interface 43. For example, interface 53 can be a LAN, USB, Bluetooth®, or other similar interface.

[0048] The information terminal device 20 has a power supply that provides power for its operation. The information terminal device 20 is equipped with, for example, a rechargeable battery (secondary battery) as its power source. The battery supplies power to each part of the information terminal device 20 to operate it.

[0049] Next, an example of the configuration of the server device 6 will be described. Figure 5 is a diagram showing an example of the hardware configuration of the server device 6 according to the embodiment. In the example configuration shown in Figure 5, the server device 6 has a processor 61, memory 62, and a communication interface (I / F) 63, etc.

[0050] The processor 61 controls each part of the server device 6. The processor 61 includes, for example, an arithmetic circuit such as a CPU. The processor 61 performs control of each part and various data processing by executing programs. The processor 61 may also be equipped with internal memory. The processor 61 performs various processes by executing programs stored in memory 62 or the like.

[0051] Memory 62 includes various types of memory. For example, memory 62 includes ROM, RAM, NVM, and other types of memory. ROM is a non-volatile memory that cannot be rewritten. ROM stores programs executed by the processor 61. RAM is a volatile memory that temporarily stores data. RAM operates as working memory or buffer memory.

[0052] The NVM of memory 62 is a rewritable, non-volatile memory. Memory 62 stores control information, configuration information, and processing results in the NVM. Memory 62 also stores various programs in the NVM for the processor 61 to execute the processes described later. For example, memory 62 stores a program in the NVM for performing processing in cooperation with the guide terminal 1.

[0053] Furthermore, the NVM in memory 62 stores the item information table 621 and the inventory information table 622. The item information table 621 is a data table or database that stores information about each item displayed within the facility.

[0054] Figure 6 shows an example of the data structure of the item information table 621 according to the embodiment. As shown in Figure 6, the item information table 621 stores item information for the item corresponding to the item ID of each item, associated with the item ID of that item. The item information includes, for example, the name of the item, a description, image data, 3D content, and orientation information.

[0055] Here, "name" refers to the name of the object the item represents or the title of the work the item is created for. The "informative text" includes text that explains the background of the item's discovery or creation, as well as information about the discoverer, creator, etc. The "image data" is two-dimensional image data and includes, for example, a photograph of the item, an illustration of the item, or an image used to describe the item.

[0056] 3D content includes three-dimensional image data (hereinafter also referred to as 3D images) that represent objects related to items in three dimensions. 3D images include, for example, three-dimensional images or data that represent an item corresponding to an item ID, a character modeled after the item, or a restored version of the item in three dimensions. 3D content may also include additional data to give movement or other effects to the 3D images.

[0057] Directional information is information that defines the orientation of a 3D image when it is placed in a facility space using AR technology or the like. For example, the orientation information is set based on the orientation of an item displayed in the facility, indicating the direction of the front of that item. The orientation information may be set using absolute directions such as east, west, north, and south, or it may be set using relative directions based on predetermined directions within the facility.

[0058] Furthermore, the inventory information table 622 is a data table or database that stores inventory information (hereinafter also referred to as inventory information) for each item displayed within the facility. Here, "inventory" means confirming the existence or non-existence of an item.

[0059] Figure 7 shows an example of the data structure of the inventory information table 622 according to the embodiment. As shown in Figure 7, the inventory information table 622 stores inventory information for items corresponding to each item ID, associated with the item ID of each item.

[0060] Inventory information includes, for example, date and time information indicating the date and time when the existence of an item was confirmed. This date and time information may be the most recent date and time when the existence of an item was confirmed, or it may be the dates and times for a predetermined number of times (for example, 5 times) when the existence of an item was confirmed.

[0061] Returning to Figure 5, the communication interface 63 is a wired or wireless communication interface for communicating with external devices such as the guide terminal 1. The communication interface 35 communicates with the guide terminal 1 connected to the network N by connecting to the network N.

[0062] Next, the functional configuration of the guide terminal 1 and the server device 6 will be described with reference to Figure 8. Figure 8 is a diagram showing an example of the functional configuration of the guide terminal 1 and the server device 6 according to this embodiment.

[0063] Guide terminal 1 includes, as its functional configuration, a camera image acquisition unit 101, a tag information acquisition unit 102, an item information acquisition unit 103, a position estimation unit 104, and a display control unit 105.

[0064] The functional configuration of guide terminal 1 is realized, for example, by the processor 31 of guide terminal 1 shown in Figure 3 executing a program stored in memory 32. Alternatively, the functional configuration of guide terminal 1 is realized, for example, by the processor 51 of information terminal device 20 shown in Figure 4 executing a program stored in memory 52. ​​Note that some or all of the functional configuration of guide terminal 1 may be realized by hardware configurations such as dedicated circuits.

[0065] The camera image acquisition unit 101 is an example of a first acquisition means. The camera image acquisition unit 101 acquires images captured by the camera 36 (hereinafter also referred to as captured images). The captured images are assumed to be real-time video captured by the camera 36, ​​but are not limited to this, and may be a series of still images taken at predetermined time intervals of relatively short periods of time.

[0066] The tag information acquisition unit 102 is an example of a second acquisition means. The tag information acquisition unit 102 acquires tag information read from the wireless tag TG. Specifically, the tag information acquisition unit 102 acquires tag information read from the wireless tag TG by the wireless tag communication unit 33. The tag information acquired by the tag information acquisition unit 102 is stored in, for example, the memory 32.

[0067] The item information acquisition unit 103 is an example of a third acquisition means. The item information acquisition unit 103 acquires item information corresponding to an item ID based on the item ID included in the tag information acquired by the tag information acquisition unit 102. Specifically, the item information acquisition unit 103 acquires item information corresponding to the specified item ID from the server device 6 by sending an item information request to the server device 6 specifying the item ID included in the tag information. The item information acquired by the item information acquisition unit 103 is stored in memory 32, for example, in association with the corresponding item ID.

[0068] Furthermore, the item information acquisition unit 103 may control the system so as not to send duplicate item information requests to the server device 6 for item IDs for which item information has already been acquired. In this case, the item information acquisition unit 103 refers to the tag information stored in the memory 32 and sends an item information request only if there is no tag information stored that overlaps with the tag information acquired by the tag information acquisition unit 102.

[0069] The position estimation unit 104 is an example of a position determination means. The position estimation unit 104 determines (estimates) its own position within the facility. Here, "self-position" is a concept that includes the position and orientation (3D orientation) of the guide terminal 1 within the facility. The method for estimating the self-position is not particularly limited, and multiple methods may be combined.

[0070] For example, the position estimation unit 104 may estimate its own position based on the spatial map 321 and the captured image acquired by the camera image acquisition unit 101, using a self-position estimation method used in AR technology, etc. In this case, the position estimation unit 104 may use the spatial map 321 and the sensing results of the sensor 34 together, or it may perform adjustments for positioning by using an AR marker, etc., when starting operation of the guide terminal 1.

[0071] Furthermore, the position estimation unit 104 may identify the location within the facility corresponding to the item ID read by the tag information acquisition unit 102 from the spatial map 321, and estimate its own position based on the identified location within the facility. The position estimation unit 104 may also use RSSI (Received Signal Strength Indicator) values ​​obtained when reading the tag information to estimate its own position.

[0072] The display control unit 105 is an example of a display control means. The display control unit 105 displays the captured image acquired by the camera image acquisition unit 101 on the display unit 21. In addition, the display control unit 105 supports viewing of items exhibited in the facility by overlaying various information on the captured image using AR technology, etc.

[0073] The display control unit 105 displays presence information indicating the presence of an item at the position on the captured image corresponding to the item's display location, based on the spatial map 321 and its own position estimated by the position estimation unit 104. The display control unit 105 also displays presence information for items whose tag information has been read and items whose tag information has not been read in a way that allows for identification.

[0074] Specifically, the display control unit 105 extracts the location and item ID of items displayed around its own location based on the spatial map 321 and its own location estimated by the location estimation unit 104. Here, it is preferable that the range from which items are extracted is larger than the reading range of the wireless tag TG. For example, items contained within a radius of 5m or 10m centered on the own location may be extracted.

[0075] Furthermore, the display control unit 105 identifies an item ID (hereinafter also referred to as a read item ID) from among the extracted item IDs that matches the item ID acquired by the tag information acquisition unit 102 (or the item ID from which the item information acquisition unit 103 has acquired item information). Then, based on the item information of the identified read item ID, the display control unit 105 displays presence information indicating the presence of the read item at the location in the captured image where the item corresponding to the read item ID (hereinafter also referred to as a read item) is displayed, or in the vicinity of that location (hereinafter collectively referred to as the display location).

[0076] For example, the display control unit 105 displays a 3D image as existence information at a position on the captured image corresponding to the display position of the scanned item, based on the 3D content included in the item information. Specifically, the display control unit 105 displays (positions) the 3D image of the scanned item facing a predetermined front direction, based on the 3D orientation of its own position estimated by the position estimation unit 104 and the orientation information included in the item information. The display control unit 105 also adjusts the size of the displayed 3D image according to the distance between its own position and the position of the scanned item. As a result, users of the guide terminal 1 can understand the position, distance, and orientation of the displayed item by looking at the displayed 3D image.

[0077] Furthermore, the display control unit 105 displays presence information indicating the existence of unread items at the display location of the item corresponding to the unread item ID (hereinafter also referred to as an unread item) in the captured image, for item IDs other than those already read (hereinafter also referred to as unread item IDs) among the extracted item IDs. For example, the display control unit 105 displays an unread image indicating the existence of an unread item at the location on the captured image corresponding to the display location of the unread item. Here, the unread image is displayed in a state that is identifiable as a three-dimensional image.

[0078] Furthermore, the display control unit 105 adjusts the size of the unread image displayed according to the distance between its own position and the position of the unread item, similar to how it adjusts the size of the unread image. This allows users of the guide terminal 1 to understand the location, distance, and orientation of the unread item by viewing the displayed unread image.

[0079] Figure 9 shows an example of a screen displayed on the display unit 21 of the guide terminal 1 according to an embodiment. As shown in Figure 9, the display control unit 105 displays the captured image acquired by the camera image acquisition unit 101, that is, the scenery inside the facility, on the display unit 21. In addition, if a read item exists around its own position, the display control unit 105 displays a 3D image at the position on the captured image corresponding to the display location of the read item, based on the item information of the read item. In Figure 9, it is shown that the read item is a dinosaur fossil Ga, and an example is shown in which a 3D image Gb of the dinosaur corresponding to the fossil Ga is displayed.

[0080] Furthermore, if an unread item exists in the vicinity of its own position, the display control unit 105 displays the unread image at the position on the captured image corresponding to the display position of the unread item. Figure 9 shows an example where a lizard fossil Gc is located further back from the user's perspective, and an egg-shaped unread image Gd is displayed as the unread image. The unread image may be any other image as long as it can be identified as a read item, but it should be pre-stored in memory 32 (memory 52), etc. The unread image may also be stored in association with the item ID of the spatial map 321. In this case, an unread image may be prepared for each item and each type of item.

[0081] In this way, the display control unit 105 displays items in the vicinity of the guide terminal 1 that have been read by the wireless tag TG and items that have not yet been read by the wireless tag TG, making them identifiable. This allows users of the guide terminal 1 to easily confirm which items have been read and which have not, and to easily understand where the read and unread items are displayed within the facility.

[0082] Furthermore, if an item is located in a position invisible to the user, such as behind a wall from the perspective of the guide terminal 1, the display control unit 105 may exclude that item from the display or include it in the display. In the latter case, the display control unit 105 may, for example, apply image processing to display a 3D image or an unreadable image to indicate that the item is located in a position invisible to the user.

[0083] As an example, the display control unit 105 may indicate that an item is located in a position invisible to the user by displaying the outline or silhouette of a 3D image or an unread image. As another example, the display control unit 105 may indicate that an item is located in a position invisible to the user by displaying a 3D image or an unread image in a transparent state. In Figure 9, it is assumed that an item is located behind the wall on the right side from the user's perspective, and an example is shown in which an egg-shaped unread image Gd is displayed transparently as the unread image of the item. This allows the user of the guide terminal 1 to easily understand that an item is displayed in a position invisible to them.

[0084] The information displayed by the display control unit 105 at the location of the read item is not limited to a 3D image. For example, the display control unit 105 may display information such as the name, explanatory text, and image data included in the item information. As one example, the display control unit 105 may display the name and explanatory text included in the item information along with the 3D image. As another example, when the display control unit 105 receives a touch operation or the like on the 3D image, it may display the name, explanatory text, image data, etc. of the read item corresponding to that 3D image. In this case, the display control unit 105 may switch from the screen of the captured image to display a screen showing the name, explanatory text, image data, etc.

[0085] Returning to Figure 8, the server device 6 includes a data management unit 601 and other components as part of its functional configuration. The functional configuration of the server device 6 is realized, for example, by the processor 61 of the server device 6 shown in Figure 5 executing a program stored in the memory 62. The functional configuration of the server device 6 may also be realized by hardware configurations such as dedicated circuits provided by the server device 6.

[0086] The data management unit 601 performs various data management tasks related to the operation of the information processing system. For example, the data management unit 601 provides item information in response to item information requests transmitted from the guide terminal 1. Specifically, when the data management unit 601 receives an item information request from the guide terminal 1, it searches the item information table 621 for the item ID specified in the item information request and reads the item information associated with that item ID. The data management unit 601 then provides the retrieved item information to the guide terminal 1 that sent the item information request.

[0087] Furthermore, when the data management unit 601 receives an item information request from the guide terminal 1, it searches the inventory information table 622 for the item ID specified in the item information request, and then searches for the item ID In association with this, the date and time when the item information request was received is stored as inventory information. Specifically, the data management unit 601 uses the inventory information table 622 to manage, for each item displayed in the facility, the most recent date and time when the existence of the item was confirmed, or the dates and times for the most recent predetermined number of times (e.g., 5 times) when the existence of the item was confirmed.

[0088] As described above, the server device 6 of this embodiment performs inventory checks (confirmation of the presence or absence of items displayed within the facility) via the guide terminal 1 used by the user. Furthermore, in the facility system of this embodiment, the configuration of the guide terminal 1 described above allows users to efficiently view the items displayed within the facility, so the server device 6 can efficiently collect the results of the presence or absence check for each item, i.e., the item ID. This makes it easy to determine, for example, the date and time when an item was last confirmed to be there if it goes missing from the facility.

[0089] Next, an example of the operation of the guide terminal 1 and the server device 6 will be described with reference to Figure 10. Figure 10 is a flowchart showing an example of the processing performed by the guide terminal 1 and the server device 6 in this embodiment.

[0090] As a prerequisite for this process, it is assumed that the user of guide terminal 1 is moving around the facility with guide terminal 1. Furthermore, it is assumed that the display unit 21 of guide terminal 1 displays the captured image acquired by the camera image acquisition unit 101, under the control of the display control unit 105. In addition, it is assumed that the position estimation unit 104 of guide terminal 1 is constantly estimating its own position or at predetermined time intervals.

[0091] First, the tag information acquisition unit 102 of the guide terminal 1 determines whether or not it has acquired tag information (step S11). If the tag information acquisition unit 102 determines that it cannot acquire tag information (step S11; No), the process proceeds to step S17.

[0092] When the tag information acquisition unit 102 of the guide terminal 1 determines that it has acquired tag information (step S11; Yes), it sends an item information request to the server device 6 specifying the item ID included in the tag information (step S12).

[0093] When the data management unit 601 of the server device 6 receives an item information request from the guide terminal 1, it reads the item information corresponding to the item ID specified in the item information request from the data management unit 601 (step S21). Next, the data management unit 601 transmits the read item information to the guide terminal 1 (step S22).

[0094] Furthermore, the data management unit 601 of the server device 6 searches the inventory information table 622 for the item ID specified in the item information request, and updates the inventory information table 622 by registering the date and time the item information request was received in association with the item ID (step S23).

[0095] When the tag information acquisition unit 102 of the guide terminal 1 acquires item information from the server device 6, it associates it with the item ID specified in the item information request and stores the item information corresponding to that item ID in memory 32 (memory 52) (step S13).

[0096] Next, the display control unit 105 of the guide terminal 1 extracts the item IDs of items located around its own position based on the spatial map 321 (step S14). Then, the display control unit 105 classifies the extracted item IDs into read items (read item IDs) and unread items (unread item IDs) based on the item IDs held in memory 32 (memory 52) (step S15).

[0097] Next, the display control unit 105 of the guide terminal 1 displays information indicating the presence of scanned and unscanned items at their respective display locations within the captured image (step S16). For example, the display control unit 105 displays a 3D image of a scanned item at its designated display location within the captured image. The display control unit 105 also displays an unscanned image at its designated display location within the captured image.

[0098] Next, the processor 31 (processor 51) of guide terminal 1 determines whether or not an operation to terminate the use of guide terminal 1 has been performed (step S17). If no termination operation is performed (step S17; No), the processor 31 (processor 51) returns to step S11.

[0099] On the other hand, if it is determined that a termination operation has been performed (step S17; Yes), the processor 31 (processor 51) of the guide terminal 1 stops the operation of each functional configuration and erases the item ID and item information held in the memory 32 (memory 52) (step S18), and terminates this process.

[0100] As described above, the guide terminal 1 of this embodiment acquires captured images taken inside the facility and reads tag information from wireless tags attached to items inside the facility. Furthermore, based on its own position inside the facility and the spatial map 321, the guide terminal 1 displays existence information indicating the presence of an item at a position on the captured image corresponding to the item's display location, and displays the existence information in a way that distinguishes between items that have been read and items that have not yet been read.

[0101] According to this embodiment, users of the guide terminal 1 can easily grasp the location of each item displayed in the facility by looking at the image displayed on the display unit 21. In addition, by looking at the screen displayed on the display unit 21, users can distinguish between items they have viewed (read items) and items they have not viewed (unread items), and can easily confirm the display location of those items. As a result, users of the guide terminal 1 can efficiently navigate the facility without overlooking any unviewed items. Therefore, the guide terminal 1 can assist in viewing items displayed in the facility.

[0102] Furthermore, the guide terminal 1 of this embodiment reads tag information from an item, acquires item information related to that item, and displays a 3D image, guidance text, etc., based on that item information as existence information at the display location of the item.

[0103] As a result, users of guide terminal 1 can access information such as 3D images and explanatory texts about exhibited items by approaching them, thereby enjoying a guide to the items. Therefore, guide terminal 1 can assist in viewing exhibited items within the facility.

[0104] Furthermore, the guide terminal 1 of this embodiment displays information about the existence of items displayed in an exhibition location that is invisible from the perspective of the device itself, making it possible to identify such items.

[0105] This allows users of guide terminal 1 to easily understand that items are displayed in locations they cannot see. Therefore, guide terminal 1 can assist in viewing items displayed within the facility.

[0106] The above-described embodiments can also be modified and implemented as appropriate by changing some of the configurations or functions of each of the above-described devices. Therefore, several modifications of the above-described embodiments will be described below as other embodiments. In the following, we will mainly describe the differences from the above-described embodiments, and will omit detailed explanations of points that are common with what has already been described. Furthermore, the modifications described below may be implemented individually or in combination as appropriate.

[0107] (Variation 1) In the above-described embodiment, the item information table 621 holds orientation information for the 3D content corresponding to each item, but the configuration is not limited to this. For example, the tag information of the wireless tag TG attached to each item may hold orientation information related to that item.

[0108] In this case, the display control unit 105 of the guide terminal 1 can achieve the same effects as in the embodiment described above by displaying a three-dimensional image based on the orientation information contained in the tag information.

[0109] (Modification 2) In the above-described embodiment, the guide terminal 1 is configured to acquire item information from the server device 6, but the configuration is not limited to this. For example, the guide terminal 1 may store an item information table similar to the item information table 621 in memory 32 (or memory 52), and use the item information stored in the item information table to display a 3D image or the like.

[0110] In this modified example, if the server device 6 manages inventory information for each item, the following process can be performed, for example. The tag information acquisition unit 102 of the guide terminal 1 notifies the server device 6 of the item ID included in the tag information when it acquires the tag information. Meanwhile, when the data management unit 601 of the server device 6 receives notification of the item ID from the guide terminal 1, it registers the date and time of receipt of the notification as inventory information for that item ID in the inventory information table 622.

[0111] This allows inventory information for each item to be managed using the guide terminal 1, similar to the embodiment described above.

[0112] In the above-described embodiment, the guide terminal 1 is configured to position the 3D image in a specific direction based on the orientation information included in the item information. However, the configuration is not limited to this, and the orientation in which the 3D image is positioned may be dynamically changed according to the positional relationship with the guide terminal 1.

[0113] For example, the display control unit 105 of the guide terminal 1 may be configured to position the front of the 3D image toward the guide terminal 1's own position. This allows users of the guide terminal 1 to view the 3D image of the scanned item from the same angle at any location within the facility, thereby improving convenience.

[0114] (Modification 4) In the above embodiment, the configuration is such that when the use of the guide terminal 1 ends, the information such as the item ID that has been read up to that point is cleared. However, the configuration is not limited to this, and for example, it may be configured to allow transfer to another information terminal.

[0115] In this case, for example, when the display control unit 105 of the guide terminal 1 receives an operation to end use, it encodes data such as item IDs and item information stored in memory 32 (memory 52) into a code symbol such as a two-dimensional code, and displays the encoded code symbol on the display unit 21. Furthermore, by reading the code symbol displayed on the guide terminal 1 with a mobile device such as a smartphone that the user possesses, information about the items read within the facility, in other words, information about the items viewed within the facility, can be transferred to the mobile device.

[0116] Furthermore, the item information transferred to the user's mobile device may be limited to certain types of information, such as two-dimensional image data. Alternatively, the system may provide other data corresponding to the item ID, either in place of or in conjunction with the item information. For example, data capable of displaying an image, such as a collection card representing the item corresponding to the item ID, may be provided via a code symbol.

[0117] In this case, by installing a dedicated application designed specifically for displaying collection cards on a mobile device, users can view and collect collection cards of items they have viewed within the facility. Furthermore, by designing the application to allow for competitive games using the collection cards, users can play games using the collection cards of items they have viewed within the facility.

[0118] By configuring the system to transfer data about items viewed by users within the facility to their information terminals, it is possible to increase users' interest in the items. Consequently, it is possible to increase the likelihood of users visiting the facility.

[0119] While embodiments (and modifications) of the present invention have been described above, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, combinations, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims of the invention and its equivalents.

[0120] Furthermore, the programs executed by each device in the above embodiments are provided pre-installed in the storage medium (ROM or storage unit) provided by the device, but are not limited to this. For example, the programs may be configured to be provided as installable or executable files recorded on a computer-readable recording medium such as a CD-ROM, flexible disk (FD), CD-R, or DVD (Digital Versatile Disk). Moreover, the storage medium is not limited to a medium independent of the computer or embedded system, but also includes storage media that have downloaded and stored or temporarily stored programs transmitted via a LAN, the Internet, etc.

[0121] Furthermore, the programs executed by each device of the above embodiment may be stored on a computer connected to a network such as the Internet and provided by allowing downloads via the network, or they may be configured to be provided or distributed via a network such as the Internet. [Explanation of Symbols]

[0122] 1 Guide terminal 6 Server devices 101 Camera image acquisition unit 102 Tag Information Acquisition Unit 103 Product Information Acquisition Department 104 Position estimation part 105 Display Control Unit 601 Data Management Department [Prior art documents] [Patent Documents]

[0123] [Patent Document 1] Japanese Patent Publication No. 2000-142925

Claims

1. An information display device used by a user moving within a facility where items are displayed, A first acquisition means for acquiring images taken inside the aforementioned facility, A second acquisition means for acquiring identification information that can identify the article, which is read from a wireless tag attached to the article, A location identification means for identifying the location of the device within the aforementioned facility, A display control means that, based on map information indicating the display location of the item within the facility and the location of the device, displays existence information indicating the presence of the item at the location on the captured image corresponding to the display location of the item, and displays the existence information in a way that distinguishes between items for which identification information has been acquired and items for which it has not been acquired. An information display device equipped with the following features.

2. The system further comprises a third acquisition means that acquires article information relating to an article identified by the identification information acquired by the second acquisition means, The display control means displays the item information of the item acquired by the third acquisition means as the existence information to be displayed at the display position of the item for which the identification information has been acquired. The information display device according to claim 1.

3. The aforementioned item information includes a three-dimensional image representing the object related to the item in three dimensions. The display control means displays the three-dimensional image relating to the article at the display position of the article for which the identification information has been acquired. The information display device according to claim 2.

4. The display control means displays the existence information of an article displayed in a display position that is invisible from the perspective of the device itself in an identifiable manner. The information display device according to claim 1.

5. The system further comprises a transmission means for transmitting the identification information acquired by the second acquisition means to an external device that manages the article, The information display device according to claim 1.

6. The computer of the information display device used by users moving around within a facility where items are displayed, A first acquisition means for acquiring images taken inside the aforementioned facility, A second acquisition means for acquiring identification information that can identify the article, which is read from a wireless tag attached to the article, A location identification means for identifying the location of the device within the aforementioned facility, A display control means that, based on map information indicating the display location of the item within the facility and the location of the device, displays existence information indicating the presence of the item at the location on the captured image corresponding to the display location of the item, and displays the existence information in a way that distinguishes between items for which identification information has been acquired and items for which it has not been acquired. A program to make it work.

Images