Image management system, image management method, reading device, and information processing device

Abstract

To provide a mechanism for facilitating access to a scene reflecting a specific object in a video.SOLUTION: A video management system for managing a video of a real space photographed by a photographing device is provided. The video management system includes first reading means for radiating an electromagnetic wave into a tag reading range, and utilizing the energy of the electromagnetic wave to read information returned from a radio frequency identification (RFID) tag, and data management means for associating a result of reading by the first reading means with the video to store the read result in a database so as to allow extraction of a first RFID tag storing first tag identification information and a part corresponding to the time when the first reading means reads the first tag identification information from the first RFID tag in the video photographed by the photographing device.SELECTED DRAWING: Figure 8

Description

[Technical Field]

[0001] The present disclosure relates to a video management system, a video management method, a reading device, and an information processing device. [Background technology]

[0002] In recent years, improvements in the performance of information and communication technology have made it possible to record video in a variety of situations. For example, at construction sites, in the past, in order to record the daily progress of construction work, a large number of photographs were taken and sorted by date or location, and then stored, but recently, video recording has become the norm. Prior art related to the former can be found in Patent Document 1, and prior art related to the latter can be found in Patent Document 2.

[0003] Patent Document 1 proposes displaying the location where a photograph was taken on a construction site in association with a construction site drawing in order to reduce management errors in photographs taken at the site. Patent Document 2 proposes that workers wear 360-degree cameras and take images while patrolling the site, and that the system automatically determines the work situation using image recognition. Patent Document 2 also discloses performing positioning in parallel with the shooting and storing the location data in association with the captured image. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2017-107420 [Patent Document 2] Japanese Patent Application Publication No. 2019-148946 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in environments where the situation of the managed object changes over time or where goods and people are frequently moving, it is not easy to find a scene that captures a specific object in the video. The difficulty of accessing a desired scene becomes more pronounced as the amount of recorded video increases. Methods that automatically identify specific objects using image recognition will not work effectively unless the appearance of the object is known and unchanging.

[0006] In view of the above, the present invention provides a mechanism for facilitating access to a scene in which a specific object appears in a video. [Means for solving the problem]

[0007] According to one aspect, there is provided an image management system for managing images of real space captured by an image capture device, the image management system including: a first reader that emits electromagnetic waves within a tag reading range and uses the energy of the electromagnetic waves to read information returned from an RFID tag, a first RFID tag that stores first tag identification information, and a data management means that stores the read result by the first reader in a database in association with the image so that a portion of the image captured by the image capture device corresponding to the time when the first reader read the first tag identification information from the first RFID tag can be extracted. Corresponding image management methods, readers, and information processing devices are also provided. [Effects of the Invention]

[0008] According to the present invention, it is possible to easily access a scene in which a specific object appears in a video. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a schematic diagram showing an example of the configuration of a video management system according to a first embodiment. [Figure 2] FIG. 2 is a block diagram showing an example of the configuration of a tag reader according to the first embodiment. [Figure 3]FIG. 2 is a block diagram showing an example of the configuration of a video server according to the first embodiment. [Figure 4] FIG. 2 is a block diagram showing an example of the configuration of a management server according to the first embodiment. [Figure 5] 3A to 3C are explanatory diagrams showing examples of the configurations of a floor table, a map table, and a position table according to the first embodiment. [Figure 6] FIG. 2 is an explanatory diagram showing an example of the configuration of a target table and a reader-camera table according to the first embodiment. [Figure 7] FIG. 3 is an explanatory diagram showing an example of the configuration of a movement amount table and a tag detection table according to the first embodiment. [Figure 8] FIG. 4 is an explanatory diagram for explaining association of a tag reading result with a video in the first embodiment. [Figure 9A] FIG. 10 is a first explanatory diagram for explaining a first embodiment of a scene selection UI. [Figure 9B] FIG. 2 is a second explanatory diagram for explaining the first embodiment of the scene selection UI. [Figure 10A] FIG. 10 is a first explanatory diagram for explaining a second embodiment of a scene selection UI. [Figure 10B] FIG. 20 is a second explanatory diagram for explaining a second embodiment of the scene selection UI. [Figure 11] FIG. 10 is a sequence diagram showing an example of the overall processing flow during shooting. [Figure 12] FIG. 10 is a sequence diagram showing an example of the overall processing flow during browsing. [Figure 13] 10 is a flowchart showing an example of the flow of a tag reading process executed by a tag reader. [Figure 14] 10 is a flowchart showing an example of the flow of a data reception process executed by the management server. [Figure 15] 10 is a flowchart showing an example of the flow of a display control process according to the first embodiment. [Figure 16] 10 is a flowchart showing an example of the flow of a display control process according to a second embodiment. [Figure 17] FIG. 10 is a schematic diagram showing an example of the configuration of a video management system according to a second embodiment. [Figure 18] FIG. 10 is a block diagram showing an example of the configuration of a management server according to the second embodiment. [Figure 19] 10A and 10B are explanatory diagrams showing examples of the configurations of a camera table, a reader table, and a tag detection table according to the second embodiment. [Figure 20] FIG. 11 is an explanatory diagram for explaining association of a tag reading result with a video in the second embodiment. [Figure 21] FIG. 11 is a schematic diagram showing an example of the configuration of a video management system according to a third embodiment. [Figure 22] FIG. 11 is a block diagram showing an example of the configuration of a management server according to the third embodiment. [Figure 23] FIG. 11 is an explanatory diagram showing an example of the configuration of a reader table, a camera table, and a tag detection table according to the third embodiment. [Figure 24] FIG. 11 is an explanatory diagram for explaining association of a tag reading result with a video in the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention claimed. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0011] 1. First Embodiment <1-1. System Overview> 1 is a schematic diagram showing an example of the configuration of a video management system 1 according to a first embodiment. The video management system 1 is a system that uses a database to manage videos of real space captured by a camera and enables users to view the videos.

[0012] 1 shows floors 10a and 10b of a building under construction as an example of a real space. Construction work is underway on floor 10a, and items 20a, 20b, 20c, and 20d used in the construction work are placed on floor 10a. Items 20a and 20b are equipment used for the work or its management. Items 20c and 20d are construction materials.

[0013] User 30a is a worker involved in construction work on floor 10a. User 30a periodically patrols floor 10a (for example, along the dashed arrow in the figure) and takes pictures with camera 50. The images taken by camera 50 are sent to a video server 150 (described later) and stored in a database. User 30b is a manager responsible for the progress of the construction work. User 30b uses a user terminal 70 to view the images stored by the video server 150 and check the progress of the construction work. Users 30a and 30b may be the same person or different people.

[0014] When a user 30b views video footage captured on the floor 10a, the user 30b may want to check a scene in which a specific object appears. For example, the user 30b may view the video footage to confirm where a specific piece of equipment is located, whether a specific material was used in the correct procedure, or whether construction on a specific section was completed as scheduled. However, the situation on the floor 10a changes over time, and objects and people frequently move. Therefore, it is not easy to find a scene in which a specific object appears among the many videos captured in the past. Therefore, in this embodiment, RFID (Radio Frequency IDentification) tags are incorporated into the video management system 1. An RFID tag is attached to each of one or more managed objects. In this specification, a managed object refers to an object that may appear in video footage managed by the system and is pre-selected in consideration of the purpose of management. For example, some of the above-mentioned objects may be selected as managed objects, and RFID tags may be attached to those objects. Not limited to movable objects, fixed objects such as ceilings, floors, or walls may also be selected as managed objects. Furthermore, a certain section in the real space may be selected as a management target, and an RFID tag may be attached to any position within that section.

[0015] In the example of FIG. 1, RFID tags 21a, 21b, and 21c are attached to objects 20a, 20b, and 20c, respectively. Furthermore, RFID tag 21e is attached to wall 22e, and RFID tag 21f is attached to wall 22f. User 30a wears a helmet equipped with camera 50 and carries tag reader 100 while patrolling floor 10a. During this time, camera 50 captures video, and tag reader 100 attempts to read information from each RFID tag. By storing the results of tag reader 100 reading information from each RFID tag in a database in association with the video captured by camera 50, it becomes possible to extract scenes corresponding to each reading time from the video.

[0016] 1, an RFID tag 11a is installed on floor 10a, and an RFID tag 11b is installed on floor 10b. The positions of these RFID tags 11a and 11b are known, and as will be explained in detail later, they can be used as reference positions when estimating the position of tag reader 100. In this specification, these RFID tags 11a and 11b are referred to as position tags. Furthermore, the RFID tags attached to each managed object are referred to as target tags.

[0017] In the following description, when it is not necessary to distinguish between the items 20a, 20b, ..., the alphabet at the end of the reference numeral will be omitted and these will be collectively referred to as the item 20. The same applies to the floors 10a, 10b, ... (floors 10), the position tags 11a, 11b, ... (position tags 11), the target tags 21a, 21b, ... (target tags 21), the users 30a, 30b, ... (users 30), and other components.

[0018] <1-2. Overall system configuration> The video management system 1 includes a camera 50, a user terminal 70, a tag reader 100, a video server 150, and a management server 200, in addition to the location tag 11 and target tag 21 described above. The user terminal 70, the tag reader 100, the video server 150, and the management server 200 are connected to each other via a network 5. The network 5 may be a wired network, a wireless network, or any combination thereof. Examples of the network 5 may include the Internet, an intranet, and a cloud network.

[0019] In this embodiment, each of the position tag 11 and the target tag 21 is a type of wireless device, specifically a passive RFID tag (passive tag). A passive tag is composed of a small IC (Integrated Circuit) chip with built-in memory and an antenna, and stores identification information and other information that identifies the tag in the memory. In this specification, the identification information is simply referred to as ID, and the identification information that identifies the tag is also referred to as tag ID. The IC chip of the passive tag operates using the energy of electromagnetic waves emitted from a tag reader, modulates the information stored in the memory into an information signal, and transmits (returns) the information signal from the antenna.

[0020] The camera 50 is a photographing device that captures and records video. The camera 50 may transmit the captured video to the video server 150 in real time. Alternatively, the camera 50 may store the video data in its internal memory. In the latter case, the video data stored in the memory may be uploaded from the camera 50 to the video server 150 later. In this embodiment, the camera 50 pre-stores camera identification information (also referred to as a camera ID) that uniquely identifies the camera 50. Video captured by the camera 50 can be identified by a combination of the camera ID and the capture time. The camera 50 may assign the camera ID and the capture time to video data representing each video and transmit the video data to the video server 150. Note that the camera 50 may be capable of capturing not only videos but also still images. In the example of FIG. 1, the camera 50 is attached to a helmet worn by the user 30a, but the camera 50 may also be carried by the user 30a.

[0021] The user terminal 70 is a terminal device used by a user 30 of the video management system 1. The user terminal 70 may be, for example, a general-purpose terminal such as a personal computer (PC), smartphone, or mobile phone, or may be a dedicated terminal specialized for the purpose of video management. The user terminal 70 typically includes a processor and memory, an input device for accepting user input, a communication interface for communicating with other devices, and a display device for displaying images and information. As an example, the user terminal 70 is used by the user 30b when viewing videos. Examples of user interactions that may take place via the user terminal 70 in connection with viewing videos will be further described below.

[0022] The tag reader 100 is a reading device that reads information from an RFID tag. The tag reader 100 is typically carried by a user and moves around the floor 10. The tag reader 100 periodically attempts to read a tag and transmits the read result to the management server 200. In this embodiment, the tag reader 100 is also capable of measuring the amount of relative movement in real space. The tag reader 100 may be capable of communicating directly with the management server 200, or may be capable of communicating indirectly with the management server 200 via some kind of relay device. An example of a specific configuration of the tag reader 100 will be further described later.

[0023] 1 depicts the camera 50 and the tag reader 100 as physically separate devices, an integrated device having the functions of both the camera 50 and the tag reader 100 may be provided. The camera 50 may have a tag reading function, or the tag reader 100 may have a photographing function. Also, instead of the user 30a wearing or carrying the camera 50 and the tag reader 100, the camera 50 and the tag reader 100 (or the integrated device) may be mounted on a mobile machine (e.g., a drone, a vehicle, or a robot) and move within real space. In this specification, the user 30 or such a mobile machine may also be referred to as a mobile subject.

[0024] The video server 150 is an information processing device having a database that stores and accumulates videos captured by the camera 50. The management server 200 is an information processing device having a database that stores the results of tag reading from the position tags 11 and target tags 21 by the tag reader 100. The video server 150 and the management server 200 may be implemented as an application server, a database server, or a cloud server using, for example, a high-performance general-purpose computer. An example of a specific configuration of the video server 150 and the management server 200 will be further described later.

[0025] The management function of the management server 200, which will be described in detail later, may be provided by a single device, or may be provided by multiple physically separate devices working together. The same applies to the video storage function of the video server 150. The video server 150 and the management server 200 may be realized as a single integrated server device. Also, part of the database, which will be described later, may be held by a device (e.g., a camera 50 or a tag reader 100) separate from the video server 150 and the management server 200.

[0026] <1-3. Example of tag reader configuration> 2 is a block diagram showing an example of the configuration of tag reader 100 according to the first embodiment. Referring to Fig. 2, tag reader 100 includes control unit 101, storage unit 102, communication unit 103, measurement unit 104, power source 105, and reading unit 106.

[0027] The control unit 101 includes a memory for storing a computer program and one or more processors (e.g., a CPU (Central Processing Unit)) for executing the computer program. The control unit 101 controls the overall functions of the tag reader 100 described in this specification. For example, the control unit 101 causes the reading unit 106 to read an RFID tag within a tag reading range and stores the read information and the read time in the memory unit 102 as read result data. In addition, while reading the RFID tag, the control unit 101 causes the measuring unit 104 to measure the amount of movement of the tag reader 100 and stores the amount of movement information indicating the measurement result and the measurement time in the memory unit 102 as measurement result data. The control unit 101 then transmits the read result data and the measurement result data stored in the memory unit 102, together with the reader identification information (also referred to as a reader ID) of the control unit 101, to the management server 200 via the communication unit 103.

[0028] The storage unit 102 may include any type of storage medium, such as a semiconductor memory such as a read only memory (ROM) or a random access memory (RAM), an optical disk, or a magnetic disk. In this embodiment, the storage unit 102 stores the reading result data, measurement result data, and the reader ID of the tag reader 100.

[0029] The communication unit 103 is a communication interface for the tag reader 100 to communicate with the management server 200. For example, the communication unit 103 may be a WLAN interface for communicating with a WLAN (Wireless Local Area Network) access point, or a cellular communication interface for communicating with a cellular base station. The communication unit 103 may also be a connection interface for connecting to a relay device (for example, a Bluetooth (registered trademark) interface or a USB (Universal Serial Bus) interface).

[0030] The measurement unit 104 is a measuring means that measures the relative movement amount of the tag reader 100 and outputs the measured movement amount to the control unit 101. The measurement unit 104 includes a triaxial acceleration sensor 104a, a gyro sensor 104b, and a geomagnetic sensor 104c. The triaxial acceleration sensor 104a measures the acceleration applied to the tag reader 100 in a device coordinate system specific to the tag reader 100 and outputs first sensor data. The gyro sensor 104a measures the angular velocity of the tag reader 100, i.e., the change in the attitude of the tag reader 100, and outputs second sensor data. The geomagnetic sensor 104c measures the orientation of the tag reader 100 in real space and outputs third sensor data. Based on the sensor data from these sensors, the measurement unit 104 can measure the relative movement amount of the tag reader 100 by accumulating the acceleration while converting the direction of the acceleration of the tag reader 100 into a direction in the coordinate system of the real space. The measurement of the amount of movement here may be performed according to any known self-position estimation method (also known as PDR (Pedestrian Dead Reckoning) method). The amount of relative movement output from measurement unit 104 to control unit 101 may be a two-dimensional vector within the plane of floor 10, or may be a three-dimensional vector including a component in the height direction. The starting point for measuring the amount of relative movement may be, for example, the position of tag reader 100 at the time tag reader 100 is started up.

[0031] 2 shows an example in which tag reader 100 includes measurement unit 104, measurement unit 104 may also be included in an external device that is capable of communicating with tag reader 100 and that is carried by the user together with tag reader 100. In that case, tag reader 100 receives movement amount information indicating the relative movement amount measured by measurement unit 104 from the external device.

[0032] Power supply 105 includes a battery and a DC-DC converter, and supplies power to control unit 101, memory unit 102, communication unit 103, measurement unit 104, and reading unit 106 of tag reader 100 to operate the electronic circuits. The battery may be a primary battery or a rechargeable secondary battery. Although not shown, tag reader 100 may have a connection terminal for connecting tag reader 100 to an external power source to charge power supply 105.

[0033] The reading unit 106 is a reading means capable of reading the identification information stored in each of the position tag 11 and the target tag 21. Referring to FIG. 2, the reading unit 106 includes an RF controller 110, a power amplifier 111, a filter 112, a first coupler 113, a second coupler 114, an antenna 115, a power detection unit 116, and a canceller 117. The RF controller 110 outputs a transmission signal (e.g., a signal modulated in the UHF band) from the TX terminal to the power amplifier 111 in accordance with the control of the control unit 101. The power amplifier 111 amplifies the transmission signal input from the RF controller 110 and outputs it to the filter 112. The filter 112 may be, for example, a low-pass filter, and is configured to remove unwanted components from the transmission signal after amplification by the power amplifier 111. frequencyThe first coupler 113 distributes the transmission signal that has passed through the filter 112 to the coupler 114 and the power detection unit 116. The second coupler 114 outputs the transmission signal input from the first coupler 113 to the antenna 115, and outputs the received signal input from the antenna 115 to the RF controller 110. The antenna 115 transmits the transmission signal input from the coupler 114 into the air as an electromagnetic wave. The antenna 115 also receives a signal returned from an RFID tag present within the reading range of the tag reader 100 in response to the transmission signal, and outputs the received signal to the coupler 114. The power detection unit 116 detects the power level of the signal input from the first coupler 113, and outputs a signal RF_DETECT indicating the detected power level to the control unit 101. The canceller 117 receives a signal CARRIER_CANCEL indicating the power level of the carrier wave from the control unit 101. Then, based on CARRIER_CANCEL, the canceller 117 cancels the carrier component of the signal received from the antenna 115 using the second coupler 114, thereby extracting the desired signal component of the received signal to be output to the RX terminal of the RF controller 110. The RF controller 110 demodulates the signal input from the RX terminal, acquires the tag ID and other information returned from the RFID tag, and outputs the acquired information to the control unit 101.

[0034] In this embodiment, the tag reading attempt by the reading unit 106 may be performed periodically (e.g., once per second) without requiring an explicit instruction from the user. Data transmission from the communication unit 103 to the management server 200 may also be performed periodically (e.g., once every few seconds) or each time a tag is read or a movement amount is measured, without requiring an explicit instruction from the user. In order to omit transmission of redundant data and reduce communication load, the control unit 101 may exclude from the data to be transmitted records that are identical to records already transmitted within the most recent predetermined period. Note that in other embodiments, one or both of the tag reading attempt by the reading unit 106 and the transmission of data to the management server 200 may be performed in response to user input via a user interface provided on the tag reader 100. When the communication unit 103 communicates indirectly with the management server 200 via a relay device, data transmission to the management server 200 may be performed only while the connection between the communication unit 103 and the relay device is active.

[0035] <1-4. Example of video server configuration> 3 is a block diagram showing an example of the configuration of the video server 150 according to the first embodiment. Referring to FIG. 3, the video server 150 includes a communication unit 160, a video management unit 170, a video database (DB) 180, and a playback control unit 190.

[0036] The communication unit 160 is a communication interface for the video server 150 to communicate with other devices. The communication unit 160 may be a wired communication interface or a wireless communication interface. In this embodiment, the communication unit 160 receives video data representing video captured by the camera 50 via the network 5. The communication unit 160 then outputs the received video data to the video management unit 170. Furthermore, when the communication unit 160 receives a playback request from the user terminal 70, it outputs the received playback request to the playback control unit 190. The communication unit 160 then sequentially transmits (e.g., streams) packets of the video to be played by the playback control unit 190 to the user terminal 70.

[0037] The video management unit 170 is a software module that provides a management function for managing video data stored in the video DB 180. The software module can be operated by one or more processors (not shown) of the video server 150 executing a computer program stored in a memory (not shown). The same applies to the playback control unit 190 described later.

[0038] For example, when video data is received via communication unit 160, video management unit 170 stores the video data in video DB 180. In the example of FIG. 3, data on multiple videos, including video data 181a and 181b, is accumulated in video DB 180. Video data 181a and 181b each include video files 182a and 182b containing the video itself, as well as video information such as a camera ID identifying the device that captured the video and the capture time. The capture time may include the capture start time and the capture end time, or may include the capture start time (or the capture end time) and duration. When a video information request specifying some search criteria is received, video management unit 170 may search the video DB to extract video information on one or more videos that match the specified search criteria and return the extracted video information to the requesting device.

[0039] The playback control unit 190 is a software module that plays back videos stored in the video DB 180 (or plays back still images extracted from videos). For example, the playback control unit 190 receives a playback request from the user terminal 70 via the communication unit 160. The playback request may include information that identifies the video to be played back (e.g., a camera ID and a shooting start time), as well as information that specifies the scene to be played back (e.g., the elapsed time from the shooting start time). In response to receiving such a playback request, the playback control unit 190 extracts the scene to be played back from the video data 181 of the specified video. Then, the playback control unit 190 sequentially transmits video packets of the extracted scene to the user terminal 70 via the communication unit 160.

[0040] In this specification, a scene means a portion of a video. A scene may be a partial video consisting of one or more frames, or may be a still image. When the playback request requests playback of a still image, the playback control unit 190 may transmit image data of the still image extracted from the video data 181 to the user terminal 70 via the communication unit 160 instead of a video packet.

[0041] <1-5. Management Server Configuration Example> <1-5-1. Basic configuration> 4 is a block diagram showing an example of the configuration of the management server 200 according to the first embodiment. Referring to FIG. 4, the management server 200 includes a communication unit 210, a read result DB 220, and a DB processing unit 290.

[0042] The communication unit 210 is a communication interface that allows the management server 200 to communicate with other devices. The communication unit 210 may be a wired communication interface or a wireless communication interface. The read result DB 220 is composed of a group of tables for managing data indicating the results of tag reading performed by the tag reader 100. In this embodiment, the read result DB 220 includes a floor table 230, a map table 235, a position tag table 240, a target tag table 250, a reader-camera table 260, a movement amount table 270, and a tag detection table 280. The DB processing unit 290 is a collection of multiple software modules that provide functions for controlling the storage of tag reading results by the read result DB 220 and the display of images based on the tag reading results. Each software module can be operated by one or more processors (not shown) of the management server 200 executing computer programs stored in memory (not shown). The memory here may include a non-transitory computer-readable storage medium. In this embodiment, the DB processing unit 290 includes a registration unit 291, a data management unit 292, and a user interface (UI) control unit 293.

[0043] <1-5-2. Data configuration example> 5(A) to 5(C) respectively show examples of the configuration of the floor table 230, map table 235, and position tag table 240 of the reading result DB 220. FIGS. 6(A) and 6(B) respectively show examples of the configuration of the target tag table 250 and reader-camera table 260 of the reading result DB 220. FIGS. 7(A) and 7(B) respectively show examples of the configuration of the movement amount table 270 and tag detection table 280 of the reading result DB 220.

[0044] The floor table 230 has two data items: floor ID 231 and name 232. Floor ID 231 is identification information that uniquely identifies each floor. Name 232 indicates the name of each floor. In the example of FIG. 5(A), the name of the floor identified by floor ID "F01" is "Floor A," and the name of the floor identified by floor ID "F02" is "Floor B." Note that "Floor A" and "Floor B" may actually be names such as "ABC Building Construction Site 1F" and "ABC Building Construction Site 2F," for example.

[0045] The map table 235 has three data items: a floor ID 236, a map image 237, and a scale 238. The floor ID 236 indicates the floor 10 to be associated with a map image in the map table 235 using the value of the floor ID 231 in the floor table 230. The map image 237 is a data item in which map information representing a map of the floor 10 identified by the floor ID 236 is stored. The map information here may typically be image information. The scale 238 indicates a ratio for converting a distance on the map of the map image 237 into a distance in real space (for example, how many meters one pixel on the image corresponds to in real space). Note that the map information stored in the map image 237 may be acquired from an external data source or uploaded by a user and updated as needed.

[0046] The position tag table 240 has four data items: tag ID 241, floor ID 242, tag position 243, and name 244. The tag ID 241 is identification information that uniquely identifies each position tag 11. The value of the tag ID 241 is the same as the tag ID value stored internally in the corresponding position tag 11. The floor ID 242 indicates the floor 10 on which each position tag 11 is installed using the value of the floor ID 231 in the floor table 230. In the example of FIG. 5(C), the position tag 11a identified by the tag ID "TG1A" is installed on the floor 10a identified by the floor ID "F01", and the position tag 11b identified by the tag ID "TG1B" is installed on the floor 10b identified by the floor ID "F02". The tag position 243 indicates the position coordinates that represent the position in real space where each position tag 11 is installed. The position coordinates here may be expressed in the coordinate system of the map information stored in the map image 237, in which case the scale 238 can be used to perform mutual conversion between the coordinate values ​​on the map and the corresponding coordinate values ​​in real space. The name 244 indicates the name of the location where each position tag 11 is installed. In the example of FIG. 5(C), the names 244 of the two position tags 11 identified by the tag IDs "TG1A" and "TG1B" are "Position A" and "Position B," respectively.

[0047] The target tag table 250 has three data items: tag ID 251, target ID 252, and name 253. The tag ID 251 is identification information that uniquely identifies the target tag 21 attached to each managed object. The value of the tag ID 251 is the same as the tag ID value stored internally in the corresponding target tag 21. The target ID 252 is identification information that uniquely identifies each managed object. The name 253 represents the name of each managed object. In the example of FIG. 6(A), the name of the managed object identified by the tag ID "TG2A" or the target ID "IT0A" is "equipment A." The name of the managed object identified by the tag ID "TG2B" or the target ID "IT0B" is "equipment B." The name of the managed object identified by the tag ID "TG2C" or the target ID "IT0C" is "material C." The name of the managed object identified by the tag ID "TG2E" or the target ID "IT0E" is "wall E."

[0048] The reader-camera table 260 is a table that maintains the association between the reader ID of a reading device that reads tags and the camera ID of a photographing device that photographs in parallel. The reader-camera table 260 has two data items: a reader ID 261 and a camera ID 262. In the example of FIG. 6(B), the first record in the reader-camera table 260 indicates that a tag reader identified by a reader ID "RD01" is associated with a camera identified by a camera ID "CM01." The second record indicates that a tag reader identified by a reader ID "RD02" is associated with a camera identified by a camera ID "CM02."

[0049] The movement amount table 270 is a table for storing records of measurement result data received from the tag reader 100 (hereinafter referred to as measurement result records). The movement amount table 270 has four data items: measurement time 271, reader ID 272, movement amount 273, and measurement position 274. The measurement time 271 indicates the time when the measurement was performed for the measurement result indicated by each measurement result record. The reader ID 272 is identification information that identifies the tag reader 100 that performed the measurement for the measurement result indicated by each measurement result record. In the example of FIG. 7(A), eight records in the movement amount table 270 indicate the results of movement amount measurements performed by the same tag reader 100 identified by the reader ID "RD01" at eight different times "T01" to "T08". The movement amount 273 indicates the relative movement amount of the tag reader 100 as a measurement result. The relative movement amount can be expressed, for example, in the form of a two-dimensional vector in a coordinate system of real space. The measurement position 274 is location information indicating where in the real space (floor 10) the device that performed the measurement for the measurement result indicated by each measurement result record was located. The measurement position 274 may also be a two-dimensional coordinate value. The measurement position 274 may be derived based on the amount of relative movement from the position where the tag ID was read from the position tag 11 by the tag reader 100 and the position coordinates of the position tag 11. The measurement position 274 of a measurement result record for measurement result data received before the detection of the position tag 11 may be blank or may be updated retroactively after the position tag 11 is detected.

[0050] The tag detection table 280 is a table for storing records of read result data received from the tag reader 100 (hereinafter referred to as read result records). The tag detection table 280 has four data items: tag ID 281, read time 282, reader ID 283, and detection position 284. The tag ID 281 indicates the tag ID read for each read result record. The read time 282 indicates the time when the tag ID indicated by the tag ID 281 was read. The reader ID 283 is identification information that identifies the device that performed tag reading for the read result indicated by each read result record. In the example of FIG. 7(B), the first record in the tag detection table 280 indicates that the tag reader 100 identified by the reader ID "RD01" read the tag ID "TG1A" at time "T02". Similarly, the second to fourth records indicate that the same tag reader 100 read tag IDs "TG2E," "TG2B," and "TG2C" at times "T04," "T06," and "T08," respectively. The detection position 284 is location information indicating the location in real space where the reader was located when the tag was read from the target tag 21 (i.e., when the target tag 21 was detected). In the example of FIG. 7(B), the second record in the tag detection table 280 indicates that wall E, identified by tag ID "TG2E," was located at position (U4, V4) at the time of tag reading. Similarly, the third and fourth records indicate that equipment B, identified by tag ID "TG2B," was located at position (U6, V6), and material C, identified by tag ID "TG2C," was located at position (U8, V8). The value of the detection position 284 can be transcribed from the measurement position 274 in the movement amount table 270 by the data management unit 292 (described later) based on the correlation between the measurement time 271 and the reading time 282 .

[0051] <1-5-3. Data registration> The contents of floor table 230, map table 235, location tag table 240, and object tag table 250 can be determined in advance by the user. Registration unit 291 receives input of the determined table contents and registers each content in the corresponding table.

[0052] The association between the reader ID and the camera ID in the reader-camera table 260 may also be predetermined and registered in advance by the user. Alternatively, in one variation, the association between the reader ID and the camera ID may be dynamically recognized when a tag reading attempt is made. For example, a third type of RFID tag (hereinafter referred to as a camera tag) different from the location tag 11 and the target tag 21 is attached to the camera 50. The tag reader 100 reads identification information (tag ID or camera ID) from the camera tag and transmits the reading result data together with the reader ID of the tag reader 100 to the management server 200. When the received reading result data includes a reading result for the camera tag, the registration unit 291 may register in the reader-camera table 260 the association between the reader ID of the tag reader 100 that transmitted the reading result data and the camera ID indicated by the reading result. In this variation, the reader-camera table 260 may include an additional data item indicating the validity period of each association (e.g., from the time the camera tag is detected to the end time of the reading attempt). According to this modification, it is possible to associate tag reading results with images captured and recorded daily while flexibly changing the association between devices. For example, even in cases where camera 50 is replaced with another camera depending on on-site needs, or where multiple users share camera 50, the user is not burdened with the work of registering data to change the association.

[0053] <1-5-4. Tag detection> Data management unit 292 adds one or more records of the measurement result data received from tag reader 100 via communication unit 210 to movement amount table 270. As described above, the measurement result data includes movement amount information indicating the relative movement amount of tag reader 100 measured by measurement unit 104 of tag reader 100. Data management unit 292 can derive the absolute position of tag reader 100 based on the relative movement amount indicated by this movement amount information and the known position coordinates (indicated by position tag table 240) of position tag 11 whose tag ID has been read by tag reader 100. For example, if the position coordinates of position tag 11 are (U0, V0), the relative movement amount at the time of detection of position tag 11 is (X0, Y0), and the latest relative movement amount is (X, Y), then the position coordinates (U, V) of the latest absolute position may be derived according to the following equation: (U,V)=(U0+(X-X0),V0+(Y-Y0)) Data management unit 292 adds the position coordinates indicating the absolute position of tag reader 100 derived in this way to the measurement position 274 column of movement amount table 270.

[0054] Furthermore, the data management unit 292 adds one or more records of the read result data received from the tag reader 100 to the tag detection table 280. As described above, the read result data includes the tag ID (and other information) of the RFID tag read by the reading unit 106 of the tag reader 100. The RFID tag here is either the position tag 11 or the target tag 21 (in the above-described variant, a camera tag may also be detected). For each read result record of the target tag 21, the data management unit 292 extracts from the movement amount table 270 the measurement result record having the measurement time closest to the read time indicated by the record (however, received from the same tag reader 100). Then, the data management unit 292 adds the value of the measurement position 274 indicated by the extracted measurement result record to the detection position 284 column of the corresponding read result record. As a result, the value of detection position 284 in tag detection table 280 indicates the estimated position in real space of each managed object to which target tag 21 detected by tag reader 100 is attached. The estimation error is at most approximately the same as the reading range of tag reader 100.

[0055] In this embodiment, adding the tag ID and the read time read from the target tag 21 by the tag reader 100 to the tag detection table 280 along with the reader ID means associating the read result with the video captured by the camera 50. This is because, if the reader ID of the read result record is converted to a camera ID in accordance with the association in the reader-camera table 260, it becomes possible to access the corresponding video scene in the video DB 180 based on the combination of the camera ID and the read time. In this way, the data management unit 292 stores the read result by the tag reader 100 in association with the video in the read result DB 220 so that the portion of the video captured by the camera 50 that corresponds to the time when the tag ID was read from the target tag 21 can be extracted.

[0056] FIG. 8 is an explanatory diagram for explaining the association of tag read results with the above-mentioned video. Here, as an example, three read result records in the tag detection table 280 indicate that tag IDs "TG2E," "TG2B," and "TG2C" were read at times "T04," "T06," and "T08," respectively. The reader IDs indicated by these read result records are associated with the camera ID "CM01" in the reader-camera table 260. The bottom section of FIG. 8 conceptually shows video 182a captured by the camera 50 identified by the camera ID "CM01" during a period beginning at start time "T00" and including times "T04," "T06," and "T08." The band extending to the right from video 182a represents the progression of the video along the time axis.

[0057] Assume now that the user wishes to view a scene showing device 20b ("device B"), one of the managed devices. The tag detection table 280 indicates that target tag 21b (identified by tag ID "TG2B") attached to device 20b was detected at time T06. Therefore, by displaying a scene corresponding to time T06 in video 182a (e.g., a scene starting a few seconds before time T06), the user can instantly view the scene showing device 20b without having to comprehensively examine the entire video 182a. The same applies to viewing scenes showing other managed devices.

[0058] In this embodiment, the camera 50 may be a 360-degree camera. A 360-degree camera synchronizes and captures images in parallel using multiple image sensors each with a different angle of view (e.g., two angles of view, forward and backward, or four angles of view, each 90 degrees apart). Multiple images captured at the same time may be combined into a single frame (e.g., a 360-degree horizontal angle of view) using a known stitching technique, or may be recorded separately. In the latter case, when displaying the video, the multiple images may be combined and displayed in a single frame, or may be displayed in parallel in separate windows. By using a 360-degree camera as the camera 50, the likelihood that each managed object tag 21 is captured in the video at the time of detection can be significantly increased, even if a user patrolling the real space does not intentionally aim the camera 50 in a specific direction. The tag reader 100 may also be a reading device capable of omnidirectional 360-degree reading. By configuring tag reader 100 as a reading device capable of omnidirectional reading (or a reading device with low directionality), the possibility of detecting each RFID tag can be increased without the user having to be aware of the orientation of tag reader 100.

[0059] 7(B) shows an example in which tag detection table 280 has reader ID 283 as a data item, tag detection table 280 may have a data item indicating the corresponding camera ID instead of (or in addition to) reader ID 283. In this case, when adding a reading result record to tag detection table 280, data management unit 292 obtains the value of the corresponding camera ID from reader-camera table 260 and includes it in the record.

[0060] <1-5-5. Display (playback) control> The UI control unit 293 accepts, for example, on the screen of a display device of the user terminal 70, a selection of a managed object that the user 30 wishes to view from among one or more managed objects under the management of the system. For convenience of explanation, the tag ID stored in the target tag 21 attached to the managed object selected here will be referred to as the first tag ID. The UI control unit 293 controls the user terminal 70 to display a scene corresponding to the read time of the first tag ID in a video associated with a read result including the first tag ID. Two examples of a UI for selecting a scene to be displayed will be described below with reference to FIGS. 9A to 10B.

[0061] (1) First Example 9A and 9B are explanatory diagrams for explaining a first example of a scene selection UI. FIG. 9A shows an example of a target selection screen 510. The target selection screen 510 includes a time period selection field 511, a floor selection field 512, a map display area 513, and a button 514. The time period selection field 511 is an object that allows the user to select the time period during which video of the managed object that the user wishes to view was captured. The floor selection field 512 is an object that allows the user to select the floor 10 on which the managed object that the user wishes to view was located. The map display area 513 is an area for presenting the user with a map of the floor 10 selected in the floor selection field 512. The UI control unit 293 displays the map of the selected floor 10A in the map display area 513 based on the map image 237 of the floor table 230. Furthermore, the UI control unit 293 displays icons (solid black circles in the figure) representing the detection locations of target tags 21 detected on the same floor 10 during the selected time period superimposed on the map image in the map display area 513. By operating (e.g., touching or clicking) one of the icons displayed in the map display area 513, the user can select the managed object corresponding to that icon as the managed object that the user wishes to view. In the example of FIG. 9A, device 20b named "Device B" is selected. In this state, when the user operates button 514, the screen transitions to video selection screen 520 in FIG. 9B.

[0062] FIG. 9B shows an example of a video selection screen 520. The video selection screen 520 includes a time period selection field 511, a floor selection field 512, a target selection field 523, a list presentation area 524, and a button 525. The target selection field 523 is an object that allows the user to select the management target object that the user wishes to view. In the example of FIG. 9B, the device 20b has already been selected as the desired management target object, but the user may reselect the management target object by operating the time period selection field 511, the floor selection field 512, or the target selection field 523. The list presentation area 524 is an area for presenting the user with a list of one or more candidate videos that may possibly show the management target object selected by the user. The UI control unit 293 identifies, in the tag detection table 280, a read result record that has a read time that belongs to the selected time period and indicates the tag ID of the target tag 21 of the selected management target object. Furthermore, the UI control unit 293 acquires from the video server 150 video information (e.g., the shooting time of the video) related to the video (i.e., the candidate video) associated with the identified reading result record. The UI control unit 293 then displays the acquired video information in list format in the list presentation area 524. In the example of FIG. 9B, the start time, end time, and target detection time (the read time of the tag ID for the selected managed object) of each candidate video are displayed in the list presentation area 524. The user can select one of the candidate videos presented in the list presentation area 524 and operate the button 525 to request the video server 150 to play back the scene corresponding to the detection time of the selected managed object.

[0063] The UI described using FIGS. 9A and 9B can typically be realized by the UI control unit 293 working in conjunction with the user terminal 70. For example, the target selection screen 510 and the video selection screen 520 may be web pages displayed by a web browser running on the user terminal 70. Furthermore, the target selection screen 510 and the video selection screen 520 may be application screens of applications downloaded to and executed on the user terminal 70. A processor of the user terminal 70 that executes a computer program for such a web browser or application may also be considered to be control means for controlling the UI. In the following description, the target selection screen 510 and the video selection screen 520 (as well as the video selection screen 530 and the location designation screen 540, which will be described later) may be collectively referred to as "viewing screens."

[0064] Regardless of the implementation form of the viewing screen, when the user selects one candidate video on the video selection screen 520 and operates the button 525, the user terminal 70 transmits a playback request including scene designation information along with the identification information of the selected video to the video server 150. As already explained, the identification information of the video may be, for example, a combination of the camera ID and the shooting start time (or shooting end time). However, without being limited to such an example, any identifier, file name, or URL that can be assigned to each candidate video by the video server 150 and provided as video information may be used as the identification information of the video. The scene designation information may be, for example, the elapsed time from the shooting start time. The UI control unit 293 can calculate the elapsed time as the scene designation information, for example, by subtracting the shooting start time from the object detection time.

[0065] (2) Second Example 10A and 10B are explanatory diagrams for explaining a second example of a scene selection UI. FIG. 10A shows an example of a video selection screen 530. The video selection screen 530 includes a floor selection field 531, a shooting date selection field 532, a list presentation area 533, and a button 534. The floor selection field 531 is an object that allows the user to select the floor 10 on which the managed object the user wishes to view was located. The shooting date selection field 532 is an object that allows the user to select the date on which the video of the managed object the user wishes to view was shot. In the example of FIG. 10A, the shooting date selection field 532 is a calendar-style object, and dates on which candidate videos exist are surrounded by circular marks. The list presentation area 533 is an area for presenting the user with a list of videos shot on a shooting date selected by the user. In the example at the bottom of FIG. 10A, June 14, 2021 is selected as the shooting date, and icons representing four videos shot at different times on this shooting date are displayed in list presentation area 533. The user can operate one of the icons displayed in list presentation area 533 to select the video corresponding to that icon as a candidate video. When the user then operates button 534, the screen transitions to location designation screen 540 in FIG. 10B.

[0066] FIG. 10B shows an example of a location designation screen 540. The location designation screen 540 includes a floor selection field 531, a shooting date selection field 532, a shooting time selection field 543, a location designation area 545, a first button 546, and a second button 547. In the example of FIG. 10B, an image captured on a specific floor 10 at a specific date and time has already been selected as a candidate image. However, the user may reselect the candidate image by operating the floor selection field 531, the shooting date selection field 532, or the shooting time selection field 543. The location designation area 545 is an area for presenting the user with a map of the floor 10 selected in the floor selection field 531. The UI control unit 293 displays the map of the selected floor 10A in the location designation area 545 based on the map image 237 of the floor table 230. The UI control unit 293 also displays the trajectory of tag reader 100's movement during the shooting period of the selected candidate video (the dashed line in the figure) superimposed on the map image in the point designation area 545 based on the sequence of position coordinates of measurement positions 274 in the movement amount table 270. The UI control unit 293 also displays an icon (a solid black circle in the figure) representing the detection position of target tag 21 detected during the shooting period of the selected candidate video superimposed on the map image. The user can designate any point in the point designation area 545 (e.g., a point near the management target that the user wishes to view) by, for example, touching or clicking. When the user operates the second button 547 with any point designated, a playback request is transmitted from the user terminal 70 to the video server 150. Here, the designated time is the time corresponding to the point on the trajectory of tag reader 100 that is the closest to the designated point. The UI control unit 293 can calculate the elapsed time from the start of the video for the scene to be played back, for example, by subtracting the shooting start time of the selected candidate video from the specified time. The playback request transmitted in response to the operation of the second button 547 may include identification information of the selected candidate video and scene designation information indicating this elapsed time. Note that when the user operates the first button 546, the screen may transition to the video selection screen 530 of FIG. 10A.

[0067] (2) Other Examples Although two examples of a UI for selecting a scene to be displayed have been described in detail in this section, other examples may also be envisioned. As one example, the UI control unit 293 may prompt the user to select a camera ID and a shooting date, present the user with a list of candidate videos identified by the selected camera ID and shooting date, and prompt the user to select one of the candidate videos in the list. As another example, on a scene playback screen displayed by a display device of the user terminal 70, after playback has started, a UI object may be provided for jumping to the immediately preceding or following tag detection time (tag ID read time).

[0068] <1-4. Processing flow> In this section, examples of several processing flows that can be executed in the video management system 1 according to this embodiment will be described using the sequence diagrams of Figures 11 and 12 and the flowcharts of Figures 13 to 16. In the following description, processing steps will be abbreviated as S (step).

[0069] <1-6-1. Overall processing flow when shooting> 11 is a sequence diagram showing an example of the overall processing flow during image capture according to this embodiment. The sequence shown in FIG. 11 mainly involves the camera 50, tag reader 100, video server 150, and management server 200.

[0070] Prior to the start of image capture and tag reading, in S10 the user sets up an association between the camera 50 and the tag reader 100, for example, via a UI provided by the management server 200. In S11, the registration unit 291 of the management server 200 registers the association entered by the user in the reader-camera table 260. Note that the setting and registration of the association here need only be performed once, as long as the combination of the camera 50 and the tag reader 100 remains the same.

[0071] When the time arrives to record the video, the user wears or carries camera 50 and tag reader 100 and begins patrolling floor 10. At the start of patrol, in S21, the user instructs tag reader 100 to begin tag reading. In S22, control unit 101 of tag reader 100 activates reading unit 106 and begins attempting to read the tag. Also, in S23, the user instructs camera 50 to begin filming. In S24, camera 50 begins filming video within floor 10.

[0072] While the user continues to patrol floor 10, in S31 measurement unit 104 of tag reader 100 repeatedly measures the relative movement amount of tag reader 100 (only steps related to one measurement are shown in FIG. 11). In S32, control unit 101 of tag reader 100 transmits the measurement result data to management server 200. In S33, data management unit 292 of management server 200 adds a measurement result record to movement amount table 270 of reading result DB 220 based on the measurement result data received from tag reader 100.

[0073] Furthermore, when a user approaches a position tag 11 or a target tag 21, in S36 the reading unit 106 of the tag reader 100 reads the tag ID from that tag. In response to reading the tag ID, in S37 the control unit 101 of the tag reader 100 transmits read result data to the management server 200. In S38, the data management unit 292 of the management server 200 adds a read result record to the tag detection table 280 of the read result DB 220 based on the read result data received from the tag reader 100. S36 to S38 can be repeated the number of times equal to the number of RFID tags detected within the floor 10.

[0074] When the user finishes patrolling floor 10, in S41 he instructs tag reader 100 to end tag reading. Also, in S42 the user instructs camera 50 to end image capture. In S43, camera 50 transmits the image data of the captured image (encoded as necessary and shaped into a predetermined file format) to image server 150. In S44, image management unit 170 of image server 150 stores the image data received from camera 50 (together with the camera ID and the image capture time) in image DB 180.

[0075] <1-6-2. Overall processing flow when browsing> 12 is a sequence diagram showing an example of the overall processing flow during browsing according to this embodiment. The sequence shown in FIG. 12 mainly involves the user terminal 70, the video server 150, and the management server 200.

[0076] First, in S51, the user operates the user terminal 70 to call up a viewing function in order to view a desired video scene. In response to this call, in S52 the user terminal 70 requests the management server 200 to display a viewing screen. In response to the request received from the user terminal 70, in S53 the UI control unit 293 of the management server 200 causes the display device of the user terminal 70 to display one of the viewing screens exemplified above.

[0077] Next, in S61, the user selects conditions related to the scene they wish to view on the displayed viewing screen. The conditions selected here may include, for example, at least one of floor, shooting time period, shooting date, management target, and location on the map. In S62, the user terminal 70 transmits information indicating the conditions selected by the user to the management server 200. In S63, the UI control unit 293 of the management server 200 searches the tag detection table 280 for tag reading results that match the selected conditions. Here, it is assumed that at least one tag reading result is identified as a search result. Next, in S64, the UI control unit 293 requests video information of the video associated with the tag reading result identified in S63 from the video server 150. In response to receiving the video information request, in S65, the video management unit 170 of the video server 150 obtains the requested video information from the video DB 180. Next, in S66, the video management unit 170 returns the video information obtained in S65 to the management server 200. In S67, the UI control unit 293 of the management server 200 presents to the user, on the viewing screen displayed on the user terminal 70, information relating to at least one candidate video that meets the conditions selected by the user.

[0078] Next, in S71, the user selects one of the presented candidate videos and instructs the start of scene playback. In response to the user's instruction, in S72 the user terminal 70 transmits a playback request including scene designation information to the video server 150. In response to receiving the playback request, in S73 the playback control unit 190 of the video server 150 extracts a scene to be played back from the video data 181 of the selected video stored in the video DB 180. Then, in S74, the playback control unit 190 plays or streams the extracted scene by sequentially transmitting video packets to the user terminal 70.

[0079] <1-6-3. Tag reading process (tag reader)> Fig. 13 is a flowchart showing an example of the flow of tag reading processing executed by tag reader 100. The tag reading processing in Fig. 13 corresponds to the processing in tag reader 100 from S22 to S37 in Fig. 11, and can be started when tag reader 100 is started or in response to some user operation on tag reader 100.

[0080] First, in S111, the reader 106 attempts to read the tag ID from a nearby RFID tag by emitting electromagnetic waves within the tag reading range. If the tag ID is received from the nearby RFID tag using the energy of the electromagnetic waves as a result of the tag reading attempt (S112-Yes), the process proceeds to S113. On the other hand, if the tag ID is not received (S112-No), the process proceeds to S115.

[0081] If a tag ID is received, in S113 the control unit 101 acquires the current time as the tag ID read time, for example by referring to an internal real-time clock. Next, in S114 the control unit 101 transmits read result data including the read tag ID, the read time, and the reader ID of the tag reader 100 to the management server 200 via the communication unit 103. Then the process proceeds to S115.

[0082] At S115, the measurement unit 104 measures the amount of relative movement of the tag reader 100 based on sensor data output from, for example, a three-axis acceleration sensor, a gyro sensor, and a geomagnetic sensor. Next, at S116, the control unit 101 acquires the current time as the measurement time. Then, at S117, the control unit 101 transmits measurement result data to the management server 200 via the communication unit 103, the measurement result data including the amount of relative movement measured by the measurement unit 104, the measurement time, and the reader ID of the tag reader 100.

[0083] Next, in S118, the control unit 101 determines whether or not to end the tag reading process. For example, if a user operation instructing to end reading is detected, the tag reading process is ended. If not, the above-mentioned steps S111 to S117 may be repeated again.

[0084] <1-6-4. Data reception process (management server)> Fig. 14 is a flowchart showing an example of the flow of data reception processing executed by the management server 200. The data reception processing in Fig. 14 corresponds to the processing in the management server 200 from S32 to S38 in Fig. 11, and can be repeatedly performed by the management server 200.

[0085] First, in S211, the data management unit 292 of the management server 200 receives measurement result data periodically transmitted from the tag reader 100. Next, in S212, the data management unit 292 derives the latest absolute position of the tag reader 100 based on the known position coordinates of the detected position tag 11 and the relative movement amount indicated by the received measurement result data. Note that S212 may be omitted until the position tag 11 is detected for the first time after the tag reader 100 is started. Next, in S213, the data management unit 292 adds a measurement result record to the movement amount table 270, which includes the measurement time, reader ID, and movement amount indicated by the received measurement result data, as well as position information indicating the absolute position derived in S212.

[0086] While periodically executing S211 to S213, the data management unit 292 waits in S214 to receive read result data from the tag reader 100. When read result data is received from the tag reader 100, the process proceeds to S215. In S215, the process branches depending on the type of RFID tag for which read result data is received. If read result data for the position tag 11 is received, the process proceeds to S216. On the other hand, if read result data for the target tag 21 is received, the process proceeds to S218.

[0087] In S216, the data management unit 292 adds a read result record including the tag ID, read time, and reader ID indicated by the read result data for the received position tag 11 to the tag detection table 280. Next, in S217, the data management unit 292 derives the absolute position for the measurement result record and read result record whose absolute position was unknown because they were received before the position tag 11 was detected, and updates the value of the position information (measurement position 274 or detection position 284). Then, the processing returns to S211.

[0088] In S218, the data management unit 292 identifies, in the movement amount table 270, the measurement result record having the measurement time closest to the read time of the received read result data, and determines the detection position of the target tag 21 by referring to the value of the measurement position 274 of the identified record. Next, in S219, the data management unit 292 adds, to the tag detection table 280, a read result record including the tag ID, read time, reader ID, and detection position indicated by the read result data for the received target tag 21. Here, as described in S11 of FIG. 11, prior to the data reception process of FIG. 14, the association between the camera 50 and the tag reader 100 was registered in the reader-camera table 260. Furthermore, the image captured by the camera 50 can be identified by the camera ID of the camera 50 and the capture time. Therefore, including the read time and reader ID in the read result record in S219 is equivalent to associating the read result by the tag reader 100 with the image via the camera ID and read time. Then, the process returns to S211.

[0089] Here, an example has been described in which measurement result data is transmitted and received between tag reader 100 and management server 200 each time the amount of movement is measured, and read result data is transmitted and received each time a tag is read. However, these data may be transmitted and received collectively after the image capture is completed. For example, when video is captured in a location where radio waves from a base station are difficult to reach, such as inside a building or underground, it is useful to temporarily store the data in tag reader 100. In this case, tag reader 100 may transmit the stored data to management server 200 afterwards (when communication becomes possible).

[0090] <1-6-5. Display control processing (management server / user terminal)> (1) First Example Fig. 15 is a flowchart showing an example of the flow of the display control process according to the first embodiment described above, which is executed by the management server 200. The display control process in Fig. 15 mainly corresponds to the processing by the management server 200 in the sequence shown in Fig. 12, although the user terminal 70 is involved in some steps.

[0091] First, in S231, the UI control unit 293 of the management server 200 causes the user terminal 70 to display a target selection screen 510, which is a type of viewing screen described with reference to FIG. 9A. Next, in S232, the UI control unit 293 accepts, via the target selection screen 510, a selection of the shooting time period of the video the user wishes to view. Also, in S233, the UI control unit 293 accepts, via the target selection screen 510, a selection of the floor where the video the user wishes to view was shot. Next, in S234, the UI control unit 293 identifies one or more managed objects that may appear in the video that meets the selected conditions by searching the tag detection table 280, and presents the results to the user on the target selection screen 510. For example, on the target selection screen 510, an icon representing the managed object may be superimposed on a map image of the selected floor, or a list of the managed objects may be displayed.

[0092] Next, in S235, the UI control unit 293 accepts the selection of the managed object that the user wishes to view via the target selection screen 510. Next, in S236, the UI control unit 293 acquires video information on the video associated with the selected managed object (i.e., candidate video) from the video server 150. Next, in S237, the UI control unit 293 displays the video selection screen 520 described with reference to FIG. 9B on the user terminal 70, and presents the user with a list of candidate videos together with the video information acquired from the video server 150. Next, in S238, the user terminal 70 accepts a playback start instruction from the user, including a selection of a video to be played, on the video selection screen 520. Then, in S239, the user terminal 70 triggers playback of the scene corresponding to the tag read time for the managed object selected in S235, by transmitting a playback request to the video server 150 in accordance with the accepted playback start instruction.

[0093] (2) Second Example Fig. 16 is a flowchart showing an example of the flow of the display control process according to the second embodiment described above, which is executed by the management server 200. The display control process in Fig. 16 mainly corresponds to the processing by the management server 200 in the sequence shown in Fig. 12, although the user terminal 70 is involved in some steps.

[0094] First, in S241, the UI control unit 293 of the management server 200 causes the user terminal 70 to display the video selection screen 530, which is a type of viewing screen described with reference to FIG. 10A. Next, in S242, the UI control unit 293 accepts, via the video selection screen 530, the selection of the floor where the video the user wishes to view was filmed. Also, in S243, the UI control unit 293 accepts, via the video selection screen 530, the selection of the filming date of the video the user wishes to view. Next, in S244, the UI control unit 293 acquires, from the movement amount table 270, the trajectory of movement of the tag reader 100 during the period when the video meeting the selected conditions was filmed. At this time, the UI control unit 293 can determine the filming period of the video meeting the selected conditions by inquiring about video information from the video server 150. Next, in S245, the UI control unit 293 displays the location designation screen 540 described using Figure 10B on the user terminal 70, and presents the user with the movement trajectory of the tag reader 100 along with the detection positions of the target tags 21 detected during the same period.

[0095] Next, in S246, the user terminal 70 accepts the designation of a location (within the selected floor) that the user wishes to view via the location designation screen 540. Next, in S247, the user terminal 70 accepts a playback start instruction from the user. Then, in S248, the user terminal 70 triggers playback of the scene corresponding to the location designated in S246 by transmitting a playback request to the video server 150.

[0096] <1-7. Summary of the First Embodiment> According to the first embodiment described in this section, while a real-space video is being captured by an imaging device, a tag reader reads tag IDs from target tags, which are passive RFID tags attached to one or more managed objects. Then, the tag reader's read results (which may include the tag ID and the tag ID read time) are stored in a database in association with the video, so that portions of the video captured by the imaging device corresponding to the read time of each tag ID can be extracted. Therefore, if a user wishes to view a scene showing a specific managed object, the scene showing the managed object can be accessed (e.g., a partial video or a still image can be played) based on the read time of the tag ID of the target tag attached to the managed object. Therefore, the user does not need to comprehensively examine the entire video. Furthermore, because the system according to this embodiment does not require identifying managed objects through image recognition, it functions effectively even in situations where the appearance of the managed object is not constant but may change over time.

[0097] Furthermore, each video is stored together with information identifying the camera device (e.g., a camera ID) and the time the video was captured, and the association between the read result by the tag reader and the video can be performed via the information identifying the camera device. Therefore, in order to associate the read result with the video, it is sufficient for the camera device to pre-store information identifying its own device and add that information to the video data of each video. It is also sufficient for the server or database that stores the video to store the video data of each video together with information identifying the camera device and the time the video was captured. With this configuration, the above-mentioned video management system can be built at low cost by utilizing camera devices, storage, or content servers already available on the market.

[0098] In the above-described embodiment, when a user selects a first managed object, a portion of a video associated with the first tag ID of a first RFID tag attached to the first managed object, corresponding to the time the first tag ID was read, can be played on the screen of the user terminal. Therefore, a user can immediately view scenes that may include the desired managed object simply by selecting the desired managed object via the UI. According to the first embodiment, a list of one or more candidate videos, each associated with one or more read results for the first tag ID, is presented to the user, and a corresponding scene in a video selected from the presented list can be played. In this case, the user can focus on one of the managed objects, which may be, for example, an item or a specific section, and sequentially view scenes of the managed object captured at different times, thereby efficiently checking how the managed object changes over time. According to the second embodiment, the tag reader is positioned during the capture period, and the trajectory of the tag reader's movement and the read positions where the tag IDs were read from each RFID tag are displayed on the screen of the user terminal. Then, a scene corresponding to a point specified by the user on the screen of the captured video can be played. In this case, the user can easily select and view a scene captured at a desired location while checking the positional relationship of the managed objects that existed at the site on the screen. If the tag reader positioning is performed using a self-location estimation method that does not rely on external radio waves such as those from a GPS satellite or base station, the above-mentioned mechanism can also be applied to the management of construction work carried out inside or underground of a building, for example.

[0099] 2. Second Embodiment In the first embodiment described above, an example was described in which a mobile subject such as user 30 carries or wears an image capture device and a reading device (or a single device that integrates them) and travels around a real space. In the second embodiment described in this section, the reading device is carried, worn, or mounted on the mobile subject, while the image capture device exists in a real space separately from the reading device. An RFID tag is attached to the image capture device as a target tag, and the reading device attempts to read the tag ID while the image capture device is capturing an image.

[0100] <2-1. System Overview> 17 is a schematic diagram showing an example of the configuration of a video management system 2 according to the second embodiment. Similar to the video management system 1, the video management system 2 is a system that uses a database to manage videos of real space captured by a camera and enables users to view the videos.

[0101] 17 shows a floor 10c that may be a construction site as an example of a real space. An item 20g is placed on the floor 10c, and an object tag 21g is attached to the item 20g. A position tag 11c is also installed on the floor 10c. In this embodiment, a camera 50 is also installed on the floor 10c, and a camera tag 51 is attached to the camera 50. Video captured by the camera 50 is transmitted to a video server 150 and stored in a database.

[0102] User 30c is a worker involved in construction work on floor 10c and is a mobile subject. User 30c carries tag reader 100c and periodically patrols floor 10c (for example, along dashed arrow 31c in the figure). FIG. 17 also shows vehicle 35d, another mobile subject. Vehicle 35d is equipped with tag reader 100d, and vehicle 35d moves within floor 10c (for example, along dashed arrow 36d in the figure).

[0103] While user 30c is patrolling floor 10c and while vehicle 35d is moving within floor 10c, camera 50 captures video and tag readers 100c and 100d attempt to read information from the RFID tags. By storing the results of tag readers 100c and 100d reading information from camera tags 51 in a database in association with the video captured by camera 50, it becomes possible to extract from the video a scene corresponding to the time when camera tag 51 was read.

[0104] <2-2. Overall system configuration> The video management system 2 includes a camera 50, a user terminal 70, a tag reader 100, a video server 150, and a management server 300. The user terminal 70, the tag reader 100, the video server 150, and the management server 300 are connected to each other via a network 5. The camera 50 is a photographing device that captures and records video. The camera tag 51 attached to the camera 50 is a passive tag similar to the location tag 11 and the target tag 21. The user terminal 70 is a terminal device used by a user 30 of the video management system 2. The tag reader 100 is a reading device that reads information from an RFID tag. The video server 150 is an information processing device having a database that stores and accumulates video captured by the camera 50. Examples of the configurations of these devices have already been explained in the previous section, so they will not be explained again here.

[0105] The management server 300 is an information processing device having a database that stores the results of tag reading by the tag reader 100 (e.g., tag readers 100c and 100d). The management server 300 may be implemented as an application server, a database server, or a cloud server using, for example, a high-performance general-purpose computer. As with the management server 200 according to the first embodiment, the management function of the management server 300 may also be provided by a single device, or may be provided by multiple physically separate devices working together. The video server 150 and the management server 300 may also be realized as a single integrated server device.

[0106] <2-3. Management Server Configuration Example> <2-3-1. Basic configuration> 18 is a block diagram showing an example of the configuration of a management server 300 according to the second embodiment. Referring to FIG. 18, the management server 300 includes a communication unit 210, a read result DB 320, and a DB processing unit 390.

[0107] The read result DB 320 is composed of a group of tables for managing data indicating the results of tag reading performed by one or more tag readers 100. In this embodiment, the read result DB 320 includes a floor table 230, a map table 235, a position tag table 240, a target tag table 250, a camera table 360, a reader table 365, a movement amount table 270, and a tag detection table 280. The DB processing unit 390 is a collection of multiple software modules that provide functions for controlling the storage of tag reading results by the read result DB 320 and the display of images based on the tag reading results. Each software module can be operated by one or more processors (not shown) of the management server 300 executing computer programs stored in memory (not shown). The memory here can include a non-transitory computer-readable storage medium. In this embodiment, the DB processing unit 390 includes a registration unit 391, a data management unit 392, and a UI control unit 393.

[0108] <2-3-2. Data configuration example> 19A to 19C show examples of the configurations of the camera table 360, the reader table 365, and the tag detection table 280 of the read result DB 320, respectively.

[0109] The camera table 360 ​​has three data items: tag ID 361, camera ID 362, and floor ID 363. The tag ID 361 is identification information that uniquely identifies the camera tag 51 attached to each image capture device. The camera ID 362 is identification information that uniquely identifies each image capture device. The floor ID 363 indicates the floor 10 on which each image capture device is installed using the value of the floor ID 231 in the floor table 230.

[0110] The reader table 365 has three data items: reader ID 366, moving entity ID 367, and name 368. The reader ID 366 is identification information that uniquely identifies each tag reader 100. The moving entity ID 367 is identification information that uniquely identifies a moving entity that moves with each tag reader 100. The name 368 represents the name of each moving entity. In the example of Figure 19 (B), the name of the moving entity identified by the reader ID "RD0C" or the moving entity ID "MV0C" is "User C". The name of the moving entity identified by the reader ID "RD0D" or the moving entity ID "MV0D" is "Vehicle D".

[0111] The configuration of the tag detection table 280 may be the same as that of the first embodiment. That is, the tag detection table 280 has four data items: tag ID 281, read time 282, reader ID 283, and detection position 284. In the example of FIG. 19(C), one of the illustrated records shows that the camera tag 51 identified by the tag ID "TG3A" was detected by the tag reader 100c identified by the reader ID "RD0C" at time "T22." Furthermore, the other record shows that the camera tag 51 identified by the tag ID "TG3A" was detected by the tag reader 100d identified by the reader ID "RD0D" at time "T28."

[0112] <2-3-3. Data registration> The contents of floor table 230, map table 235, position tag table 240, target tag table 250, and camera table 360 ​​can be determined in advance by the user. Registration unit 391 accepts input of the determined table contents and registers each content in the corresponding table.

[0113] The association between the reader ID and the mobile entity ID in the reader table 365 may also be predetermined by the user and pre-registered. Alternatively, in one variation, the association between the reader ID and the mobile entity ID may be dynamically recognized when tag reading is attempted. For example, the user 30 carries a user tag, which is a fourth type of RFID tag, and the tag reader 100 reads the user ID from the user tag and transmits the read result data to the management server 300. When the received read result data includes a read result for the user tag, the registration unit 391 may register in the reader table 365 the association between the reader ID of the tag reader 100 that transmitted the read result data and the user ID indicated by the read result. In this variation, the reader table 365 may include an additional data item indicating the validity period of each association.

[0114] <2-3-4. Tag detection> Similar to the data management unit 292 according to the first embodiment, the data management unit 392 adds one or more records of the measurement result data received from the tag reader 100 via the communication unit 210 to the movement amount table 270. The data management unit 392 can also derive the absolute position of the tag reader 100 based on the relative movement amount indicated by the measurement result data and the known position coordinates of the position tag 11 whose tag ID has been read by the tag reader 100. The data management unit 392 adds the position coordinates indicating the absolute position of the tag reader 100 derived in this way to the measurement position 274 column of the movement amount table 270.

[0115] Furthermore, the data management unit 392 adds one or more records of the read result data received from the tag reader 100 to the tag detection table 280. The RFID tag here may include the camera tag 51. The data management unit 392 may also add the value of the measurement position 274 indicated by the corresponding measurement result record in the movement amount table 270 to the detection position 284 column for the read result from the camera tag 51. Alternatively, for the camera tag 51 of a fixedly installed camera 50, the position information of the detection position 284 may be omitted.

[0116] In this embodiment, adding the tag ID and the read time read from the camera tag 51 by the tag reader 100 to the tag detection table 280 along with the reader ID means associating the read result with the video captured by the camera 50. This is because, if the tag ID of the read result record is converted to a camera ID in accordance with the definition in the camera table 360, it becomes possible to access the corresponding video scene in the video DB 180 based on the combination of the camera ID and the read time. In this way, the data management unit 392 stores the read result by the tag reader 100 in association with the video in the read result DB 320 so that the portion of the video captured by the camera 50 that corresponds to the read time of the tag ID from the camera tag 51 can be extracted.

[0117] FIG. 20 is an explanatory diagram for explaining the association of tag reading results with the above-mentioned video. Here, as an example, two reading result records in the tag detection table 280 indicate that the tag ID "TG3A" was read at times "T22" and "T28." The reader ID indicated by the first record is associated with the moving subject ID "MV0C" in the reader table 365, and the reader ID indicated by the second record is associated with the moving subject ID "MV0D" in the reader table 365. The bottom part of FIG. 20 conceptually shows video 182c captured by a camera 50 identified by the camera ID "CM01" during a period starting at start time "T20" and including times "T22" and "T28." The band extending to the right from video 182c represents the progression of the video along the time axis.

[0118] Here, assume that user 30b wishes to view a scene that shows user 30c ("user C"), one of the moving subjects. Tag detection table 280 indicates that camera tag 51 was detected at time T22 by tag reader 100c, which moves with user 30c. Therefore, by displaying the scene corresponding to time T22 in video 182c, user 30b can instantly view the scene that shows user 30c without having to comprehensively examine the entire video 182c. The same applies to the case where user 30b wishes to view a scene that shows vehicle 35d.

[0119] In this embodiment, the camera 50 may also be a 360-degree camera. Each tag reader 100 may also be a reading device capable of reading in all directions at 360 degrees.

[0120] <2-3-5. Display (playback) control> The UI control unit 393 accepts a selection of a moving subject that is of interest to the user 30, for example, on the screen of the user terminal 70, from among one or more moving subjects managed by the system. For convenience of explanation, the reader ID of the tag reader 100 that moves with the selected moving subject will be referred to as a first reader ID. The UI control unit 393 also accepts a selection of a camera 50 installed on the floor 10 that is of interest to the user 30. For convenience of explanation, the tag ID of the camera tag 51 of the selected camera 50 will be referred to as a first tag ID. The UI control unit 393 controls the user terminal 70 to display a scene corresponding to the reading time of the first tag ID in a video associated with a reading result including the first reader ID and the first tag ID. For example, the UI control unit 393 may present the user with a list of one or more candidate videos each associated with one or more reading results including the first reader ID and the first tag ID, and allow the user to select a desired video from the presented list. In this case, the user terminal 70 can trigger playback of a scene that may include the selected moving subject by sending a playback request to the video server 150 that includes scene designation information based on the time the first tag ID from the camera tag 51 was read.

[0121] <2-4. Summary of the second embodiment> According to the second embodiment described in this section, while a real-space video is being captured by a camera device, a reader traveling with each of one or more moving subjects reads a tag ID from a camera tag attached to the camera device. The results of tag reading by the reader are stored in a database in association with the video so that a portion of the video captured by the camera device corresponding to the time at which each tag ID was read can be extracted. Therefore, if a user wishes to view a scene in which a specific moving subject appears, the scene in which the moving subject appears can be accessed based on the time at which the reader traveling with the moving subject read the tag ID from the camera tag. Therefore, the user does not need to comprehensively examine the entire video. Furthermore, because the system according to this embodiment does not require identifying moving subjects through image recognition, it functions effectively even in situations where the appearance of the moving subject is unknown or not constant.

[0122] 3. Third Embodiment In the third embodiment described in this section, the image capturing device is carried, worn, or installed by a moving subject, while the reader is installed in real space. Therefore, the image capturing device is a physically separate device from the reader. An RFID tag is attached to the image capturing device as a target tag, and the reader attempts to read the tag ID while the image capturing device is capturing an image.

[0123] <3-1. System Overview> 21 is a schematic diagram showing an example of the configuration of a video management system 3 according to the third embodiment. Similar to the video management systems 1 and 2, the video management system 3 is a system that uses a database to manage videos of real spaces captured by cameras and enables users to view the videos.

[0124] 21 shows floor 10d as an example of a real space. An item 20h is placed on floor 10d, and a tag reader 100h is mounted on item 20h. Tag reader 100i is attached to wall 22i of floor 10d, and tag reader 100j is attached to floor 22j.

[0125] User 30d is a worker involved in construction work on floor 10d and is a mobile subject. User 30d wears a helmet equipped with a camera 50 and periodically patrols floor 10d (for example, along the dashed arrow in the figure). A camera tag 51 is attached to camera 50. Video captured by camera 50 is transmitted to video server 150 and stored in a database.

[0126] While user 30d is patrolling floor 10d, camera 50 captures video, and tag readers 100h, 100i, and 100j attempt to read information from the RFID tag. At this time, by storing the results of reading information from camera tag 51 by tag readers 100h, 100i, and 100j in a database in association with the video captured by camera 50, it becomes possible to extract scenes corresponding to each reading time from the video. Note that camera 50 may be carried, worn, or mounted by other types of mobile entities, rather than by users.

[0127] <3-2. Overall system configuration> The video management system 3 includes a camera 50, a user terminal 70, a tag reader 100, a video server 150, and a management server 400. The user terminal 70, the tag reader 100, the video server 150, and the management server 400 are connected to each other via a network 5. Examples of the configurations of the camera 50, the user terminal 70, the tag reader 100, and the video server 150 have already been described, so they will not be described again here.

[0128] The management server 400 is an information processing device having a database that stores the results of tag reading by the tag reader 100 (e.g., tag readers 100h, 100i, 100j). The management server 400 may be implemented as an application server, a database server, or a cloud server using, for example, a high-performance general-purpose computer. As with the above-described management servers 200 and 300, the management function of the management server 400 may also be provided by a single device, or may be provided by multiple physically separate devices working together. The video server 150 and the management server 400 may also be realized as a single integrated server device.

[0129] <3-3. Management Server Configuration Example> <3-3-1. Basic configuration> 22 is a block diagram showing an example of the configuration of a management server 400 according to the third embodiment. Referring to FIG. 22, the management server 400 includes a communication unit 210, a read result DB 420, and a DB processing unit 490.

[0130] The read result DB 420 is made up of a group of tables for managing data indicating the results of tag reading performed by one or more tag readers 100. In this embodiment, the read result DB 420 includes a floor table 230, a map table 235, a reader table 440, a camera table 450, and a tag detection table 480. The DB processing unit 490 is a collection of multiple software modules that provide functions for controlling the storage of tag reading results by the read result DB 420 and the display of images based on the tag reading results. Each software module can be operated by one or more processors (not shown) of the management server 400 executing computer programs stored in memory (not shown). The memory here can include a non-transitory computer-readable storage medium. In this embodiment, the DB processing unit 490 includes a registration unit 291, a data management unit 492, and a UI control unit 493.

[0131] <3-3-2. Data configuration example> 23A to 23C show examples of the configurations of the reader table 440, the camera table 450, and the tag detection table 480 of the read result DB 420, respectively.

[0132] The reader table 440 has three data items: a reader ID 441, a target ID 442, and a name 443. The reader ID 441 is identification information that uniquely identifies each tag reader 100. The target ID 442 is identification information that uniquely identifies the managed object to which each tag reader 100 is attached. The name 443 represents the name of each managed object. In the example of FIG. 23(A), the name of the managed object identified by the reader ID "RD0H" or the target ID "IT0H" is "Device H." The name of the managed object identified by the reader ID "RD0I" or the target ID "IT0I" is "Wall I." The name of the managed object identified by the reader ID "RD0J" or the target ID "IT0J" is "Floor J."

[0133] The camera table 450 has two data items: a tag ID 451 and a camera ID 452. The tag ID 451 is identification information that identifies the camera tag 51 attached to each image capturing device. The camera ID 452 is identification information that uniquely identifies each image capturing device.

[0134] The tag detection table 480 has three data items: tag ID 481, read time 482, and reader ID 483. That is, compared to the tag detection table 280 according to the first and second embodiments, the tag detection table 480 does not include a data item corresponding to the detection position 284. The first read result record in the tag detection table 480 illustrated in FIG. 23(C) indicates that the tag reader 100i identified by the reader ID "RD0I" detected the camera tag 51 identified by the tag ID "TG3A" at time "T33." The second read result record indicates that the tag reader 100h identified by the reader ID "RD0H" detected the same camera tag 51 at time "T35." The third read result record indicates that the tag reader 100j identified by the reader ID "RD0J" detected the same camera tag 51 at time "T37."

[0135] <3-3-3. Tag detection> The data management unit 492 adds one or more records of the read result data received from the tag reader 100 to the tag detection table 480. The RFID tag here may include the camera tag 51. In this embodiment, adding the tag ID and the read time read from the camera tag 51 by each tag reader 100 to the tag detection table 480 along with the reader ID means associating the read result with the video captured by the camera 50. This is because, if the tag ID of the read result record is converted to a camera ID according to the definition in the camera table 450, it becomes possible to access the corresponding video scene in the video DB 180 based on the combination of the camera ID and the read time. In this way, the data management unit 492 associates the read result by the tag reader 100 with the video and stores it in the read result DB 420 so that the portion of the video captured by the camera 50 corresponding to the read time of the tag ID from the camera tag 51 can be extracted.

[0136] FIG. 24 is an explanatory diagram for explaining the association of tag read results with the video described above. Here, as an example, three read result records in the tag detection table 480 indicate that tag ID "TG3A" was read at times "T33," "T35," and "T37." Furthermore, the reader IDs indicated by the first, second, and third records are associated with target IDs "IT0I," "IT0H," and "IT0J," respectively, in the reader table 440. The bottom section of FIG. 24 conceptually illustrates video 182d captured by a camera 50 identified by camera ID "CM01" during a period beginning at start time "T30" and including times "T33," "T35," and "T37." A band extending to the right from video 182d represents the progression of the video along the time axis.

[0137] Here, assume that user 30b wishes to view a scene showing floor 22j ("Floor J"), one of the managed objects. Tag detection table 480 indicates that camera tag 51 was detected at time T37 by tag reader 100j identified by reader ID "RD0J" attached to floor 22j. Therefore, by displaying the scene corresponding to time T37 in video 182d, user 30b can instantly view the scene showing floor 22j without having to comprehensively examine the entire video 182d. The same applies to the case where user 30b wishes to view a scene showing another managed object.

[0138] In this embodiment, the camera 50 may also be a 360-degree camera. Each tag reader 100 may also be a reading device capable of reading in all directions at 360 degrees.

[0139] <3-3-4. Display (playback) control> The UI control unit 493 accepts, for example, on the screen of the user terminal 70, a selection of a management target that the user 30 wishes to view from among one or more management targets under the management of the system. For convenience of explanation, the reader ID of the tag reader 100 attached to the selected management target will be referred to as a first reader ID. The UI control unit 493 also accepts a selection of a camera 50 that is of interest to the user 30. For convenience of explanation, the tag ID of the camera tag 51 of the selected camera 50 will be referred to as a first tag ID. The UI control unit 493 controls the user terminal 70 to display a scene corresponding to the reading time of the first tag ID in a video associated with a reading result including the first reader ID and the first tag ID. For example, the UI control unit 493 may present the user with a list of one or more candidate videos each associated with one or more reading results including the first reader ID and the first tag ID, and allow the user to select a desired video from the presented list. In this case, the user terminal 70 can trigger playback of a scene in which the selected managed object may appear by sending a playback request to the video server 150 that includes scene designation information based on the time the first tag ID from the camera tag 51 was read.

[0140] <3-4. Summary of the third embodiment> According to the third embodiment described in this section, while a real-space video is being captured by a camera device, a reader attached to each of one or more managed objects reads a tag ID from the camera tag attached to the camera device. Then, the results of tag reading by the reader are stored in a database in association with the video so that a portion of the video captured by the camera device corresponding to the time at which each tag ID was read can be extracted. Therefore, if a user wishes to view a scene showing a specific managed object, the scene showing the managed object can be accessed based on the time at which the reader attached to the managed object read the tag ID from the camera tag. Therefore, the user does not need to comprehensively examine the entire video. Furthermore, because the system according to this embodiment does not require identifying managed objects through image recognition, it functions effectively even in situations where the appearance of the managed object is not constant but may change over time.

[0141] The first, second, and third embodiments, and the first example, second example, and modified examples described in this specification may be combined with each other in any manner. Features and advantages described in relation to one embodiment may also be applied to other embodiments unless otherwise explicitly stated.

[0142] In this specification, an example in which the technology according to the present disclosure is applied to video recording at a construction site has been mainly described. However, the technology according to the present disclosure is not limited to construction sites and can be applied to various situations in which video recording may be performed. For example, the technology according to the present disclosure may be applied to situations such as logistics, inventory management, or livestock management in the livestock industry.

[0143] <4. Other embodiments> The above-described embodiment can also be realized in the form of a process in which a program for realizing one or more functions is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program, or by a circuit (e.g., ASIC) that realizes one or more functions.

[0144] The invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0145] 1, 2, 3: video management system, 10: floor, 11: location tag, 20: item, 21: target tag, 30: user, 50: camera (photographing device), 51: camera tag, 70: user terminal, 100: tag reader (reading device), 101: control unit (control means), 103: communication unit (communication means), 104: measurement unit (measurement means), 106: reading unit (reading means), 150: video server (information processing device), 180: video DB, 181, video data, 182: video, 200, 300, 400: management server (information processing device), 220, 320, 420: reading result DB, 292, 392, 492: data management unit (data management means), 293, 393, 493: UI control unit (control means)

Claims

1. An image management system for managing images of a real space captured by an image capturing device, a first reading means for emitting electromagnetic waves within a tag reading range and reading information returned from an RFID (Radio Frequency Identification) tag by utilizing the energy of the electromagnetic waves; a first RFID tag storing first tag identification information; a second RFID tag attached to the imaging device; a data management means for storing the read result by the first reading means in a database in association with the image captured by the imaging device so that a portion corresponding to the time when the first tag identification information was read from the first RFID tag by the first reading means can be extracted from the image; Including, the first reading means reads, from the second RFID tag, information for associating a read result of the first tag identification information from the first RFID tag with the image captured by the imaging device; Video management system.

2. the first reading means attempts to read tag identification information while the image capturing device is capturing the image; when the first tag identification information is read from the first RFID tag by the first reading means, the data management means stores the first tag identification information and the read time of the first tag identification information in the database in association with the video; The video management system according to claim 1 .

3. The video is stored together with information identifying the camera and the time the video was taken; the data management means associates the read result of the first tag identification information with the image via the information identifying the image capture device. The video management system according to claim 2 .

4. the first RFID tag is attached to a first management object; The video management system includes: a control means for, when the first management target is selected by a user from among one or more management targets, reproducing on a screen a portion of the video associated with the read result including the first tag identification information, the portion corresponding to the read time of the first tag identification information; The video management system according to claim 2 or 3, further comprising:

5. The video management system includes: a first reading device that is physically separate from the imaging device and includes the first reading means; The video management system according to any one of claims 1 to 4, comprising:

6. The video management system according to claim 5 , wherein the image capturing device and the first reading device are carried by, attached to, or mounted on a first moving subject.

7. The control means When the first management target is selected by the user, presenting to the user a list of one or more candidate videos each associated with one or more read results including the first tag identification information; a portion of the video selected by the user from the one or more candidate videos, the portion corresponding to the read time of the first tag identification information, is played on the screen; The video management system according to claim 4 .

8. An image management system for managing images of a real space captured by an imaging device, comprising: a first reading device carried, worn or mounted by a first mobile subject, the first reading device comprising a first reading means for emitting electromagnetic waves within a tag reading range and reading information returned from an RFID (Radio Frequency Identification) tag by utilizing the energy of the electromagnetic waves; a first RFID tag storing first tag identification information attached to the image capture device that is physically separate from the first reader; a data management means for storing the read result by the first reading means in a database in association with the image captured by the imaging device so that a portion corresponding to the time when the first tag identification information was read from the first RFID tag by the first reading means can be extracted from the image; Including, the first reading means attempts to read tag identification information while the image capturing device is capturing the image; when the first tag identification information is read from the first RFID tag by the first reader, the data management means stores, as the reading result, information for identifying the first reader, the first tag identification information, and the reading time of the first tag identification information in the database in association with the image; Video management system.

9. The video is identified by at least the time the video was taken; The video management system includes: a control means for, when the first moving subject is selected by a user from among one or more moving subjects, reproducing on a screen a portion of the video associated with the reading result including the information identifying the first reading device and the first tag identification information, the portion corresponding to the reading time of the first tag identification information; The video management system of claim 8 , further comprising:

10. An image management system for managing images of real space captured by a photographing device carried, worn, or mounted by a first mobile subject, comprising: a first reading means attached to the first management target, the first reading means emitting electromagnetic waves within a tag reading range and reading information returned from an RFID (Radio Frequency Identification) tag by utilizing the energy of the electromagnetic waves; a first RFID tag attached to the imaging device and storing first tag identification information; a data management means for storing the read result by the first reading means in a database in association with the image captured by the imaging device so that a portion corresponding to the time when the first tag identification information was read from the first RFID tag by the first reading means can be extracted from the image; Including, the first reading means attempts to read tag identification information while the image capturing device is capturing the image; when the first tag identification information is read from the first RFID tag by the first reading means, the data management means stores the first tag identification information and the read time of the first tag identification information in the database in association with the video; Video management system.

11. The video management system includes: a control means for, when the first management target is selected by a user from among one or more management targets, reproducing on a screen a portion of the video associated with the read result including the first tag identification information, the portion corresponding to the read time of the first tag identification information; The video management system of claim 10 further comprising:

12. An image management system for managing images of a real space captured by an imaging device, comprising: a first reading means for emitting electromagnetic waves within a tag reading range and reading information returned from an RFID (Radio Frequency Identification) tag by utilizing the energy of the electromagnetic waves; a measuring means that moves together with the first reading means and measures the amount of relative movement in the real space; a first RFID tag storing first tag identification information; a second RFID tag installed at a known position in the real space and storing second tag identification information; a data management means for storing in a database, in association with the image, position information based on the read result by the first reading means and the amount of movement measured by the measuring means, so that a portion of the image captured by the imaging device corresponding to the time when the first tag identification information was read from the first RFID tag by the first reading means can be extracted; and Including, the position information is derived based on a position where the second tag identification information is read from the second RFID tag by the first reading means and the amount of relative movement from that position measured by the measuring means; Video management system.

13. The video management system according to claim 12, wherein the measuring means measures the amount of relative movement based on sensor data output from a three-axis acceleration sensor, a gyro sensor, and a geomagnetic sensor.

14. The video management system includes: a control means for displaying on a screen a trajectory of movement of the first reading means and a reading position at which the first tag identification information is read from the first RFID tag by the first reading means, based on the position information; Furthermore, the control means plays back on the screen a portion of the video corresponding to a point designated by the user on the screen; The video management system according to claim 12 or 13.

15. The video management system according to any one of claims 1 to 14, wherein the image capturing device is a 360-degree camera.

16. While an image of the real space is being captured by the imaging device, a first reading device emits electromagnetic waves within a tag reading range, and attempts to read information returned from an RFID (Radio Frequency Identification) tag by utilizing the energy of the electromagnetic waves; storing a result of reading by the first reader in a database in association with the image so that, when the first tag identification information is read from a first RFID tag storing the first tag identification information by the first reader, a portion of the image corresponding to the time at which the first tag identification information is read can be extracted; Including, a second RFID tag is attached to the imaging device; information for associating a read result of the first tag identification information from the first RFID tag with the image captured by the imaging device is read from the second RFID tag by the first reading device; Video management methods.

17. a communication means for communicating with an information processing device that manages data related to video; a reading means for emitting electromagnetic waves within a tag reading range while an image of the real space is being captured by the imaging device, and for attempting to read information returned from an RFID (Radio Frequency Identification) tag by utilizing the energy of the electromagnetic waves; a control means for transmitting a read result of the first tag identification information to the information processing device via the communication means so that, when the first tag identification information is read by the reading means from a first RFID tag storing the first tag identification information, a portion of the image captured by the imaging device corresponding to the read time of the first tag identification information can be extracted, and for causing the information processing device to store the read result in a database in association with the image; Equipped with a second RFID tag is attached to the imaging device; the reading means reads, from the second RFID tag, information for associating a read result of the first tag identification information from the first RFID tag with the image captured by the imaging device; Reading device.

18. An information processing device for managing images of a real space captured by an imaging device, a communication means for communicating with a reader that reads information returned from an RFID (Radio Frequency Identification) tag by utilizing electromagnetic wave energy radiated within a tag reading range; a data management means for, when receiving a read result indicating that the reader has read the first tag identification information from a first RFID tag storing the first tag identification information via the communication means, storing the received read result in a database in association with the image captured by the photographing device so that a portion of the image captured by the photographing device corresponding to the time when the first tag identification information was read by the reader can be extracted; Equipped with a second RFID tag is attached to the imaging device; information for associating a result of reading the first tag identification information from the first RFID tag with the image captured by the imaging device is read from the second RFID tag by the reading device; Information processing device.

19. A reading device carried, worn, or mounted by a first mobile subject, which is physically separate from the photographing device to which the first RFID tag is attached, a communication means for communicating with an information processing device that manages data related to video; a reading means for emitting electromagnetic waves within a tag reading range while an image of the real space is being captured by the imaging device, and for attempting to read information returned from an RFID (Radio Frequency Identification) tag by utilizing the energy of the electromagnetic waves; a control means for transmitting information for identifying the reading device, the first tag identification information, and the read time of the first tag identification information to the information processing device via the communication means as a read result for the first tag identification information, so that when the first tag identification information is read by the reading means from the first RFID tag storing the first tag identification information, a portion of the image captured by the imaging device corresponding to the read time of the first tag identification information can be extracted, and causing the information processing device to store the read result in a database in association with the image; A reading device comprising:

20. An information processing device for managing images of real space captured by a photographing device carried, worn, or mounted by a first moving subject, comprising: a communication means for communicating with a reader that reads information returned from an RFID (Radio Frequency Identification) tag by utilizing electromagnetic wave energy radiated within a tag reading range; a data management means for, when receiving a read result indicating that the reader has read the first tag identification information from a first RFID tag attached to the photographing device and storing the first tag identification information via the communication means, storing the received read result in a database in association with the image so that a portion of the image photographed by the photographing device corresponding to the time when the reader read the first tag identification information can be extracted; Equipped with the reading device is attached to the first management target, and attempts to read tag identification information while the photographing device is photographing the image; when the first tag identification information is read from the first RFID tag by the reader, the data management means stores the first tag identification information and the time at which the first tag identification information was read in the database in association with the video; Information processing device.

21. An information processing device for managing images of a real space captured by a photographing device, comprising: a communication means for communicating with a reader that reads information returned from an RFID (Radio Frequency Identification) tag by utilizing electromagnetic wave energy radiated within a tag reading range; a data management means for, when receiving a read result indicating that the reader has read the first tag identification information from a first RFID tag storing the first tag identification information via the communication means, storing the received read result in a database in association with the image captured by the photographing device so that a portion of the image captured by the photographing device corresponding to the time when the first tag identification information was read by the reader can be extracted; Equipped with the reading device measures a relative movement amount in the real space; the data management means stores in the database the position information based on the amount of movement measured by the reading device, further associated with the image captured by the image capturing device; When second tag identification information is read by the reader from a second RFID tag that is installed at a known position in the real space and stores second tag identification information, the position information is derived based on a position at which the second tag identification information is read from the second RFID tag by the reader and the amount of relative movement measured by the reader from that position. Information processing device.

Images