Information processing device, information processing method, information processing system, base station, and terminal

Abstract

[Problem] To provide an information processing device, an information processing method, an information processing system, a base station, and a terminal that make it easier to acquire image information relating to a disappeared object. [Solution] An information processing device comprises: a positioning information processing unit that generates position coordinates in chronological order on the basis of radio wave ranging distances from a plurality of coordinates to a terminal attached to an article; and a first control unit that causes the display unit to display, on the basis of the time at which the distance is measured by radio waves, an image including, within the imaging range, a position within a space corresponding to at least one of the time-series position coordinates and a time at which radio wave ranging corresponding to the position within the space occurs.

Description

Technical Field

[0001] The present disclosure relates to an information processing device, a terminal, a base station, and an information processing system. Background Art

[0002] One of the main roles of surveillance cameras in stores is to monitor customers who illegally take away products. Therefore, surveillance cameras are installed in locations where the aisles between shelves can be seen. For example, there is a situation where a person squatting in the surveillance camera screen takes item 500 (DVD case size: 136 x 14 x 190 mm) taken from a shelf and puts it into a bag (see Fig.18 ).

[0003] Reference List

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application No. 2012-543148 Summary of the invention

[0006] Problems to be solved by the present invention

[0007] However, in the case where the store side notices the theft and searches for evidence from the camera image, the store side searches for items and bags that are only shown as small objects on the screen. In addition, there may be a case where another person or another object appears between the camera and the theft location, and the theft location is not visible from the camera. In that case, even if the camera image is searched, there is no captured scene showing that the object is put into the bag. In the event that a situation occurs where an object is put into the bag, "missing" is identified, but it is necessary to search for the "time" and "place" of its appearance from the images captured by all surveillance cameras.

[0008] Therefore, the present disclosure provides an information processing device, an information processing method, an information processing system, a base station, and a terminal that enable easier acquisition of image information about a lost object.

[0009] Solution to the problem

[0010] In order to solve the above problems, the present disclosure provides an information processing device, including:

[0011] a positioning information processing unit that generates position coordinates in time sequence based on distances from the plurality of sets of coordinates to a terminal attached to the article measured by radio distance measurement; and

[0012] A first control unit causes the display unit to display an image based on the time of radio distance measurement, the image having an imaging range including a position in space, the position corresponding to at least one set of time-series position coordinates, and the image corresponding to the time when the position in space has been measured by radio distance measurement.

[0013] The first control unit may cause the display unit to display an image when the position coordinates are within a predetermined area.

[0014] The radio distance measurement may be performed by a distance measurement method using a phase characteristic of a frequency of wireless communication between a plurality of base stations and a terminal.

[0015] When the position coordinates are within a predetermined area, the first control unit may issue a warning message.

[0016] The display unit may display an image captured at a point in time when the terminal starts moving.

[0017] may further include a storage unit, and

[0018] The positioning information processing unit may generate position coordinates associated with the time of the radio distance measurement.

[0019] A map image generating unit may be further included, the map image generating unit superimposing the temporal change of the position coordinates as trajectory information on the map image.

[0020] The display unit may also display, in parallel, a map image on which the trajectory information is superimposed and an image corresponding to a radio distance measurement time of the position coordinates included in the trajectory information.

[0021] A recognition processing unit that recognizes a region of the object from the image may be further included.

[0022] The recognition processing unit may superimpose a first mark indicating the area in the image.

[0023] The recognition processing unit may superimpose a second mark indicating a portion corresponding to the position coordinates in the image.

[0024] When the position coordinates are within the predetermined area, the first control unit may cause the recognition processing unit to start the recognition processing.

[0025] The method may further include a reading unit that reads first identification information about the item, and

[0026] The recognition processing unit may recognize the area through recognition processing corresponding to the first identification information.

[0027] The reading unit may be capable of reading second identification information about the terminal and storing the second identification information in the storage unit in association with the first identification information.

[0028] When the first identification information or the second identification information is read, the first control unit may stop generating the location coordinates of the object and stop the operation of the terminal.

[0029] The first control unit may select a base station that measures a distance to the terminal through wireless communication with the terminal based on the location coordinates, select a captured image from a plurality of sets of predetermined coordinates based on the location coordinates, and cause the display unit to display the image.

[0030] In order to solve the above problems, the present disclosure provides an information processing method, comprising:

[0031] a positioning information processing step of generating position coordinates in time sequence based on distances from the plurality of sets of coordinates to a terminal attached to the item measured by radio distance measurement; and

[0032] The first control step causes the display unit to display an image based on the time of radio distance measurement, the image having an imaging range including a position in space, the position corresponding to at least one set of time-series position coordinates, and the image corresponding to the time when the position in space has been measured by radio distance measurement.

[0033] In order to solve the above problems, the present disclosure provides an information processing system, including:

[0034] a plurality of base stations for measuring a distance to a terminal attached to an article by a distance measurement method using a phase characteristic of a frequency of wireless communication with the terminal;

[0035] an imaging unit that is included in the base station or the camera and captures an image of the terminal when the terminal attached to the article is placed within an imaging range; and

[0036] An information processing device, wherein

[0037] The information processing device includes:

[0038] a positioning information processing unit that generates position coordinates in time sequence based on distances from coordinates of a plurality of base stations to the terminal measured by radio distance measurement; and

[0039] In the image captured by the imaging unit,

[0040] A first control unit causes the display unit to display an image based on the time of radio distance measurement, the image having an imaging range including a position in space, the position corresponding to at least one set of time-series position coordinates, the image corresponding to the time when the position in space has been measured by radio distance measurement.

[0041] In order to solve the above problems, the present disclosure provides an imaging device, comprising:

[0042] a distance measurement unit for generating, in time sequence, position coordinates of a terminal attached to an item, and performing radio distance measurement to measure a distance to the terminal;

[0043] a wireless communication unit for wirelessly communicating with the terminal; and

[0044] An imaging unit includes an imaging optical system having a specified imaging range and an imaging element, and captures an image that can be associated with position coordinates in the captured image.

[0045] In order to solve the above problems, the present disclosure provides a terminal attached to an item, the terminal comprising:

[0046] a distance measurement unit for generating the position coordinates of the terminal in time sequence and performing radio distance measurement with a predetermined base station to measure the distance to the terminal; and

[0047] The wireless communication unit wirelessly communicates with a predetermined base station. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 : is a diagram illustrating an exemplary configuration of an information processing system 1 according to the first embodiment of the present technology.

[0049] Figure 2 is a floor plan of the interior of the store to be monitored.

[0050] Figure 3 is a block diagram showing an exemplary configuration of the base station 10 .

[0051] Figure 4 is a diagram for explaining the principle of measuring distance by a phase-based distance measurement method.

[0052] Figure 5 is a diagram showing an example of a graph for calculating the group delay τ.

[0053] Figure 6 2 is a block diagram showing an exemplary configuration of the terminal 20 .

[0054] Figure 7 is a block diagram showing an exemplary configuration of the information processing device 30 .

[0055] Figure 8 is a block diagram showing an exemplary configuration of the image processing unit 306 .

[0056] Fig. 9 is a plan view showing a situation where the terminal 20A has moved.

[0057] Fig.10 is a sequence diagram showing the operation of each component in the system in a case where the terminal 20A has moved.

[0058] Fig.11 20A is a plan view showing a situation where the terminal 20A has moved and performs monitoring end processing at the counter.

[0059] Fig.12 This is a sequence diagram for monitoring end processing.

[0060] Fig.13 It is a plan view showing a situation where a terminal enters an event occurrence area without receiving a monitoring end process.

[0061] Fig.14 is a timing chart showing a situation where an event occurs.

[0062] Fig.15 is a diagram illustrating an exemplary configuration of an information processing system according to a second embodiment of the present technology.

[0063] Fig.16 is a block diagram illustrating an example configuration of a camera.

[0064] Fig.17 is a floor plan of the interior of the store to be monitored.

[0065] Fig.18 It is a view of a scene in which a person squatting in the camera's frame puts an object the size of a DVD case into a bag. DETAILED DESCRIPTION

[0066] In the following description, embodiments of the information processing device, information processing method, information processing system, base station, and terminal will be described with reference to the accompanying drawings. Although the main components of the information processing device, information processing method, information processing system, base station, and terminal will be mainly described below, the information processing device, information processing method, information processing system, base station, and terminal may include components and functions not shown or described. The following description does not exclude components and functions not shown or described.

[0067] (First embodiment)

[0068] Figure 1 1 is a diagram showing an exemplary configuration of an information processing system 1 according to a first embodiment of the present technology. Figure 1 As shown, the information processing system 1 includes a plurality of base stations 10, a terminal 20, and an information processing device 30. The terminal 20 and the information processing device 30 may be plural. The base station 10 can capture an image of a monitoring range, and measure the distance to the terminal 20 using radio waves by communicating with the terminal 20. The terminal 20 is, for example, a tag formed, and is fixed to an object such as a DVD. The terminal 20 is able to measure the distance to the base station 10 by communicating with the base station 10. The information processing device 30 can control the base station 10 and the terminal 20, and monitor the position of the terminal 20. The specific structures of the base station 10, the terminal 20, and the information processing device 30 will be described later.

[0069] Figure 2is an example of a floor plan of the interior of a store to be monitored. Items are on shelves in the store. Figure 2 In the present invention, a plurality of base stations 10a, 10b, 10c, 10d, 10e and 10f, a plurality of terminals 20A and 20B and an information processing device 30 are arranged. The information processing system 1 is a system that monitors the positions of a plurality of terminals 20A and 20B fixed to, for example, an article, and is capable of issuing an alarm and presenting a related image when any of the terminals 20A and 20B enters an event occurrence area A100.

[0070] A plurality of base stations 10a, 10b, 10c, 10d, 10e, 10f are oriented, for example, with imaging units in a direction capable of monitoring aisles between shelves. An information processing device 30 is mounted on a counter. Terminals 20A and 20B are attached to items that need to be managed. When moving items, the attached terminals move with the items. Note that each of the base stations 10a, 10b, 10c, 10d, 10e, and 10f can send a presence information signal indicating its own presence at regular intervals to respond to the terminals 20A and 20B searching for the base station to be used. In addition, the orientation and imaging range A100 of the optical axis L100 of the imaging optical system of each of the base stations 10a, 10b, 10c, 10d, 10e, and 10f are defined in the world coordinate system. Similarly, the range of the radio distance measurement of the base stations 10a, 10b, 10c, 10d, 10e, and 10f is defined in the world coordinate system. Note that in the present embodiment, coordinates are described in the world coordinate system in some cases, but the present invention is not limited thereto. For example, the lower left corner of the store floor plan can be represented by a local coordinate system such as coordinates (x, y, z) = (0, 0, 0). With this configuration, even in the case where the plurality of terminals 20A and 20B do not have the position measurement unit 204, for example, processing can be performed using the local coordinate system of the information processing system 1.

[0071] Figure 3 is a block diagram showing an exemplary configuration of the base station 10. Figure 3 As shown, the base station 10 includes an imaging unit 100, a distance measuring unit 102, a communication unit 103, a storage unit 108, and a control unit 110. The base station 10 includes, for example, a central processing unit (CPU), and corresponding components are formed by executing various programs stored in the storage unit 108. In addition, the communication unit 103 includes a wireless communication unit 104 and a wired communication unit 106.

[0072] The imaging unit 100 includes an imaging optical system and an imaging element. The imaging unit 100 captures an image of the monitored area via the imaging optical system using the imaging element, and stores it in the storage unit 108. The imaging unit 100 captures an image of the environment in which the base station 10 exists. It should be noted that the base station 10 may include a plurality of imaging units 100. Alternatively, in the case where the monitored area can be sufficiently imaged by another base station in the information processing system 1, the base station 10 may not include the imaging unit 100. In addition, the imaging unit 100 associates the image of the monitored area with time and stores the images in the storage unit 108 in chronological order. These images can also be stored in the storage unit 302 of the information processing device 30 via the wired communication unit 106 described later.

[0073] As described above, the imaging optical system of the imaging unit 100 is fixed, and the imaging range A100 (see Figure 2 ) is preset. For example, the imaging range A100 is set in the world coordinate system. In addition, the optical axis L100 of the imaging optical system is specified (see Figure 2 ) and the positional relationship between the imaging element. Therefore, the position coordinates in the world coordinate system can be associated with the image captured by the imaging unit 100. This information is stored in the storage unit 108 of the information processing device 30 and the storage unit 302 described later. It is to be noted that in the case where the orientation of the optical axis L100 and the imaging range A100 of the imaging optical system are changed, the changed orientation of the optical axis L100 and the changed imaging range can be newly set in the world coordinate system.

[0074] The distance measurement unit 102 measures the distance to the terminal 20 by radio distance measurement through communication with the terminal 20. These distance information may also be stored in the storage unit 302 of the information processing device 30 via the wired communication unit 106 described later. In addition, the distance measurement unit 102 may also measure the distance to another base station 10 through communication with another base station 10.

[0075] In addition, for example, the distance measurement range of the distance measurement unit 102 is fixed, and the distance measurement range is preset. As described above, for example, the distance measurement range is set in the world coordinate system. This information is stored in the storage unit 108 of the information processing device 30 and the storage unit 302 described later. It should be noted that the details of the distance measurement unit 102 will be described later. Note that in the case where the distance measurement range is changed, the changed distance measurement range can be newly set in the world coordinate system.

[0076] As described above, the communication unit 103 can communicate with the terminal 20 and the information processing device 30, and includes the wireless communication unit 104 and the wired communication unit 106. The wireless communication unit 104 can wirelessly communicate with the terminal 20 and the information processing device 30 via the antenna 104a. The wireless communication unit 104 and the antenna 104a are connected by a wireless cable L104. Note that in the presence of another base station 10, communication with another base station 10 is also possible.

[0077] The wireless communication unit 104 can perform communication using, for example, Bluetooth (registered trademark) Low Energy (BLE). Through BLE, the time required for operations that require large power (such as connection establishment and data communication) can be minimized. As a result, power consumption can be reduced, and the size of the wireless communication unit 104 can be made smaller. In this way, the wireless communication unit 104 sends instructions, data, etc. to the outside of the base station 10 and receives instructions, data, etc. from the outside of the base station 10. Note that in the case where the base station 10 uses only the wired communication unit 106, the wireless communication unit 104 may not be included. Optionally, the wireless communication unit 104 may share the distance measurement radio wave to be used by the distance measurement unit 102, or may be formed independently of the distance measurement radio wave to be used by the distance measurement unit 102.

[0078] The wired communication unit 106 can perform wired communication with the information processing device 30, etc. The wired communication unit 106 transmits and receives instructions, data, etc. to and from the outside of the base station 10. Note that in the case where the base station 10 uses only the wireless communication unit 104, the wired communication unit 106 may not be included.

[0079] The storage unit 108 is formed of, for example, a hard disk drive (HDD), a solid state drive (SSD), or the like. In addition to the above-mentioned program and image data, the storage unit 108 can also store unique information. For example, the storage unit 108 stores the identification number of the base station 10, the position coordinates of the base station 10, wireless communication information, etc. as unique information. The wireless communication information is a radio parameter, etc., and includes, for example, information about the length of the wireless cable L104. In addition, as described above, the storage unit 108 can store, for example, the imaging range of the imaging unit 100 and the distance measurement range of the distance measurement unit 102 in the world coordinate system.

[0080] The control unit 110 controls the entire base station 10. In addition, the control unit 110 can also start driving the distance measurement unit 102 and the imaging unit 100 in response to a reception signal from the terminal 20 or the information processing device 30. That is, the control unit 110 can specify the distance measurement target of the distance measurement unit 102 in the terminal 20 and other base stations 10, and specify the timing of the distance measurement. The control unit 110 also performs control to send the image captured by the imaging unit 100 and the information about the distance measured by the distance measurement unit 102 to the outside of the base station 10 through the wireless communication unit 104 or the wired communication unit 106. In this way, the control unit 110 can process instructions given from the outside via the wireless communication unit 104 or the wired communication unit 106, and control the entire operation of the base station 10. Note that the image according to the present embodiment is referred to as a video image in some cases.

[0081] Here, reference Figure 4 and Figure 5 An example of processing performed by the distance measurement unit 102 is described. For example, the distance measurement unit 102 may measure the distance by a phase-based distance measurement method. Figure 4 is a diagram for explaining the principle of measuring distance by a phase-based distance measurement method. Figure 5 is a diagram showing an example of a graph for calculating a group delay τ. Figure 5 In the figure, the horizontal axis represents frequency and the vertical axis represents phase.

[0082] like Figure 4 As shown, by the phase-based distance measurement method, the wireless device (initiator) and the wireless device (reflector) transmit sine waves at the same frequency. Here, the wireless device (initiator) is described as the wireless communication unit 206 of the terminal 20 (refer to later). Figure 5 Description), and the wireless device (reflector) is described as the wireless communication unit 104 of the base station 10, but these devices are not limited thereto. For example, the wireless device (initiator) may be the wireless communication unit 104 of the base station 10, and the wireless device (reflector) may be the wireless communication unit 306 of the terminal 20.

[0083] By the phase-based distance measurement method, a terminal 20 (initiator) first transmits a sine wave with a frequency of f [Hz]. The radio wave emitted from the terminal 20 propagates toward the base station 10 (reflector) which is another wireless device at a distance D [m]. The base station 10 (reflector) receives the radio wave emitted from the terminal 20 (initiator) and receives the sine wave with a phase rotated according to the distance D [m] at that time. Assuming that the phase of the sine wave when transmitted from the terminal 20 (initiator) is 0 [rad], the sine wave arriving at the base station 10 (reflector) at a distance D [m] is received as a sine wave with a phase relative rotation of Φ = -2πfD / c [rad] relative to the phase emitted by the terminal 20 (initiator). Here, c represents the speed of light, and f represents the frequency of the sine wave.

[0084] When the transmission of the radio wave from the terminal 20 (initiator) to the base station 10 (reflector) ends, the base station 10 (reflector) next transmits the radio wave to the terminal 20 (initiator). At this time, the base station 10 (reflector) transmits a sine wave rotated by Φ [rad] in advance. Therefore, the phase of the sine wave to be received by the terminal 20 (initiator) is a phase rotated by an amount equivalent to the distance D [m] relative to the phase Φ [rad] transmitted by the base station 10 (reflector), which means that the terminal 20 (initiator) will receive a sine wave with a phase rotated by 2Φ [rad].

[0085] At this point in time, it is known that a sine wave having a phase of 0 [rad] has been first transmitted from the terminal 20 (initiator). Therefore, it is clear that when the radio wave reciprocates between the terminal 20 (initiator) and the base station 10 (reflector), the phase rotates by 2Φ [rad]. Therefore, on the terminal 20 (initiator) side, by equally dividing the observed amount of phase rotation of the sine wave from the base station 10 (reflector), the amount of unidirectional phase rotation Φ can be calculated.

[0086] The distance measurement unit 102 can map the distance between the terminal 20 (initiator) and the base station 10 (reflector) by reciprocating radio waves while changing the frequency f [Hz]. Figure 5 In an environment where only direct waves exist, the graph of the measurement result shows a straight line. In order to simplify the mathematical expression, the distance measurement unit 102 according to the present embodiment calculates the slope of the straight line as the group delay τ according to expression (1) using the phases Φ2 and Φ1 at the two frequencies f2 and f1.

[0087] [Math. 1]

[0088] τ=(Φ2-Φ1) / (2π(f2-f1)) expression (1)

[0089] [Math. 2]

[0090] D = -τc Expression (2)

[0091] Then, according to Expression (2) obtained by substituting Φ=-2πfD / c into Expression (1), the distance measurement unit 102 generates the distance D[m]. That is, the distance measurement unit 102 becomes able to calculate the distance between the terminal 20 (initiator) and the base station 10 (reflector) by obtaining the group delay τ from the phase measured while changing the frequency between the terminal 20 (initiator) and the base station 10 (reflector). It should be noted that, for convenience of explanation, the distance measurement unit 102 according to the present embodiment calculates the group delay τ as the slope of a straight line connecting two points on the graph. However, the group delay τ may not be calculated by this method.

[0092] In addition, the distance measurement unit 102 subtracts the length of the wireless cable L104 and the length of the wireless cable L206 of the terminal 20 from D in the expression (2). Therefore, the total length of the length of the wireless cable L104 and the length of the wireless cable L206 is prevented from becoming an error. The length of the wireless cable L206 can be acquired from the terminal 20 in advance through wireless communication.

[0093] In addition, when illuminated, the indirect wave is reflected by the walls, floors, etc. in the room. Therefore, its phase becomes greater than the phase of the direct wave. For example, in the case where multiple 2Φ [rad] are measured between the terminal 20 (initiator) and the base station 10 (reflector), the smallest 2Φ [rad] can be selected. Therefore, only the group delay τ of the direct wave can be generated. As described above, in the case of using the phase-based distance measurement method, even when an indirect wave occurs, the distance between the terminal 20 (initiator) and the base station 10 (reflector) can be measured with higher accuracy.

[0094] As can be seen from the above, by using the phase-based distance measurement method, even in a space where indirect waves are generated, such as in a store, the decrease in distance measurement accuracy can be reduced. It should be noted that the distance measurement unit 102 according to the present embodiment uses a phase-based distance measurement method, but is not limited thereto. For example, the distance can be measured by a distance measurement method using ultra-wideband (UWB) wireless communication.

[0095] Figure 6 2 is a block diagram showing an exemplary configuration of the terminal 20. Figure 6 As shown, the terminal 20 includes an acceleration detection unit 200, a distance measurement unit 202, a wireless communication unit 206, a storage unit 208, and a control unit 210. The terminal 20 includes, for example, a central processing unit (CPU), and forms corresponding components by executing various programs stored in the storage unit 208.

[0096] The acceleration detection unit 200 includes an acceleration sensor and outputs a movement signal to the control unit 210 when the terminal 20 moves. Note that the acceleration detection unit 200 according to the present embodiment includes the acceleration detection unit 200, but is not limited thereto. For example, the acceleration detection unit 200 may not be included in the configuration.

[0097] The distance measurement unit 202 has a configuration similar to that of the distance measurement unit 102 of the base station 10. That is, for example, the distance measurement unit 202 measures the distance to the base station 10 by communicating with the base station 10 using a phase-based distance measurement method. Note that in the case where a movement signal is output from the acceleration detection unit 200, for example, the distance measurement unit 202 can measure the distance to the base station 10 by communicating with the base station 10 at predetermined time intervals.

[0098] For example, the position measurement unit 204 measures the position of the terminal 20 using a global navigation satellite system (GNSS). Specifically, in an area where measurement using the global positioning satellite system can be performed, the position measurement unit 204 transmits a global positioning signal indicating the position to the information processing device 30 via the wireless communication unit 206 under the control of the control unit 210. Note that the terminal 20 according to the present embodiment includes the position measurement unit 204, but is not limited thereto. For example, the position measurement unit 204 may not be included in the configuration.

[0099] The wireless communication unit 206 can perform communication using, for example, Bluetooth (registered trademark) Low Energy (BLE). The wireless communication unit 206 transmits and receives instructions and the like to and from the outside of the terminal 20. Note that the wireless communication unit 206 may share the distance measurement radio wave to be used by the distance measurement unit 202, or may be formed independently of the distance measurement radio wave to be used by the distance measurement unit 202.

[0100] The storage unit 208 is formed of, for example, a hard disk drive (HDD), a solid state drive (SSD), etc. In addition to the above-mentioned various programs, for example, the storage unit 208 stores unique information as an identification number of the terminal 20, wireless communication information about the terminal 20, etc. The wireless communication information is radio parameters, etc., and includes, for example, information about the length of the wireless cable L206.

[0101] The control unit 210 controls the entire terminal 20. In addition, the control unit 210 can also start driving the distance measurement unit 202 and the position measurement unit 204 in response to a received signal from the terminal 20 or the information processing device 30. Specifically, the control unit 210 controls the position measurement unit 204 according to a control instruction received via the wireless communication unit 206 or a situation such as terminal movement detected by the acceleration detection unit 200, and performs control to measure its own position. In addition, the control unit 210 can perform control to send the measured distance and position information to the base station 10, etc. via the wireless communication unit 206.

[0102] Figure 7 is a block diagram showing an exemplary configuration of the information processing device 30. Figure 7 As shown, the information processing device 30 includes a reading unit 300, a storage unit 302, a positioning information processing unit 304, an image processing unit 306, a display and input / output unit 308, a wired communication unit 310, a wireless communication unit 312, and an overall control unit 314. The information processing device 30 is a computer device, for example, including a central processing unit (CPU), and the corresponding components are formed by executing various programs stored in the storage unit 302. Alternatively, the information processing device 30 can be formed using a portable device such as a smart phone. In addition, a plurality of information processing devices 30 can share the reading unit 300, the storage unit 302, the positioning information processing unit 304, the image processing unit 306, the display and input / output unit 308, the wired communication unit 310, the wireless communication unit 312, and the overall control unit 314 to perform cloud processing. Note that the overall control unit 314 according to the present embodiment corresponds to the first control unit.

[0103] In addition, in the case where the information processing system 1 includes a plurality of information processing devices 30, the information processing device 30 may include some components of the reading unit 300, the storage unit 302, the positioning information processing unit 304, the image processing unit 306, the display and input / output unit 308, the wired communication unit 310, the wireless communication unit 312, and the overall control unit 314. For example, a monitoring person patrolling a store carries a portable device such as a smart phone. In this case, the portable information processing device 30 may include the storage unit 302, the display and input / output unit 308, the wireless communication unit 312, and the overall control unit 314.

[0104] The reading unit 300 can read the identification number (e.g., ID) of the terminal 20 and the identification number of the item. The identification number of the item is associated with information such as the type of the item. The reading unit 300 is, for example, a barcode reader, and the identification number of the terminal 20 and the identification number of the item are attached to the terminal 20 and the item as barcodes. The reading unit 300 associates the read identification number of the terminal 20 with the read identification number of the item, and stores these identification numbers in the storage unit 302. In addition, in the case of paying a fee for an item to which the terminal 20 is fixed, the reading unit 300 reads the identification number of the item, associates the identification number with information indicating that the item has been paid, and stores the identification number and the information in the storage unit 302. In addition, when the information indicating that the payment has been made is associated with the identification number of the item, the identification number of the terminal 20 can be erased from the storage unit 302. For example, when the identification number of the terminal 20 is erased, the terminal 20 can be associated with a new item. Note that, in the case of paying for an item to which the terminal 20 is fixed, the store clerk removes and collects the terminal 20 from the item.

[0105] The storage unit 302 is formed of, for example, a hard disk drive (HDD), a solid state drive (SSD), etc. The storage unit 302 can store various information transmitted from the base station 10 and the terminal 20 .

[0106] The storage unit 302 associates the identification numbers (e.g., IDs) of the plurality of base stations 10 with the coordinates in the world coordinate system and stores the identification numbers associated with the coordinates. In addition, the storage unit 302 stores map information related to the area to be monitored. The map information is associated with the world coordinates. In addition, the identification number of the terminal 20 is associated with the identification number of the item and is stored as described above.

[0107] In addition, as described above, the imaging direction and imaging range of each base station 10 are set in advance. As a result, the storage unit 302 can also store the imaging range associated with the world coordinates and the information about the imaging optical systems of the plurality of base stations 10 and the example measurement range. It should be noted that the storage unit 302 according to the present embodiment is referred to as an information accumulation unit in some cases.

[0108] The positioning information processing unit 304 generates the coordinates of the terminal 20 in the world coordinate system according to the principle of triangulation, for example, using the identification number of the terminal 20 and the distance information sent from the plurality of base stations 10. The positioning information processing unit 304 associates the coordinates of the terminal 20 in the world coordinate system with time, for example, and stores the coordinates in the storage unit 302 in chronological order.

[0109] The image processing unit 306 can perform various information processing under the control of the overall control unit 314 . Figure 8 is a block diagram showing an exemplary configuration of the image processing unit 306. Figure 8 As shown, the image processing unit 306 includes a map image generating unit 306a and a recognition processing unit 306b.

[0110] The map image generation unit 306a is capable of generating a trajectory image in which the positions of the terminal stored in chronological order are superimposed and displayed as a trajectory on the map information stored in the storage unit 302. Under the control of the overall control unit 314, the map image generation unit 306a generates a trajectory image in the case where, for example, the target enters the event occurrence area A100. Note that in the case of paying a charge for an item to which the terminal 20 is attached, it may be set so that when the terminal 20 enters the event occurrence area A100, a series of processes related to the alarm process are not performed.

[0111] The recognition processing unit 306b includes a plurality of recognition processing units corresponding to the identification number of the article. For example, the recognition processing unit 306b includes a plurality of recognition processing units that have learned the shapes of the corresponding categories to which the identification number of the article belongs. As a result, when information about the identification number of the article is provided, the overall control unit 314 uses the recognition processing corresponding to the identification number of the article to identify the area corresponding to the identification number of the article from the image captured by the plurality of base stations 10. For example, in the case where the identification number of the article corresponds to a quadrilateral, the recognition processing unit 306b identifies the quadrilateral area and superimposes a mark on the image. For example, under the control of the overall control unit 314, in the case where the target enters the event occurrence area A100, the recognition processing unit 306b starts the recognition processing of the captured image stored in the storage unit 302.

[0112] In addition, the recognition processing unit 306b can recognize the object to which the terminal 20 is attached from the captured image by referring to the coordinates of the terminal 20 in the world coordinate system generated by the positioning information processing unit 304. Thus, the recognition processing unit 306b can perform processing for recognizing an object mainly from the coordinate information, thereby improving recognition accuracy. In addition, the recognition processing unit 306b can mark the coordinates of the terminal 20 in the world coordinate system in the captured image.

[0113] Return to reference Figure 7 , the display and input / output unit 308 includes a display unit and an input device as shown in the figure. The display unit is, for example, a monitor, and the input device is formed by a keyboard, a mouse, etc. The display unit of the display and input / output unit 308 can display a trajectory image, etc., in which the position of the terminal 20 is superimposed and displayed as a trajectory. In addition, the display unit can display a captured image in which an area of ​​the item to which the terminal 20 is attached, which is recognized by the recognition processing unit 306b, is marked.

[0114] In this case, the coordinates of the terminal 20 fixed to the article in the world coordinate system are measured. Thus, the image processing unit 306 can read an image including a position in space corresponding to the position coordinates of the terminal 20 in the world coordinate system from the storage unit 302 under the control of the overall control unit 314, arrange the trajectory of the article and the image obtained by imaging the article in chronological order, and display the trajectory and the image on the display unit. In addition, the recognition processing unit 306b can also perform recognition processing on the image containing the coordinate position of the terminal 20 fixed to the article in the world coordinate system in the imaging range, and mark the coordinates in the captured image. In this case, the coordinates of the terminal 20 fixed to the article in the world coordinate system are measured, and therefore, the image processing unit 306 can mark the coordinates of the terminal 20 in the world coordinate system in the captured image.

[0115] As a result, when the terminal 20 enters the event occurrence area A100, the clerk can arrange the trajectory of the object and the image of the marked object, and confirm the trajectory and image in chronological order. As described above, the clerk can easily observe the movement state of the object based on the movement trajectory and the captured image without performing any operation to retrieve the image.

[0116] The wired communication unit 310 can perform wired communication with the base station 10 or the like. The wired communication unit 310 can perform transmission of control instructions to the base station 10 or the like, data reception, etc. The wireless communication unit 312 can perform wireless communication via the antenna 312a, for example, using Bluetooth (registered trademark) Low Energy (BLE). The wireless communication unit 312 can perform transmission of control instructions to the base station 10 or the like, data reception, etc. Although the exemplary configuration according to the present embodiment has been described above, an example of control of the information processing system 1 will be described below.

[0117] (Movement of Terminal 20A)

[0118] Fig. 9 20A is a plan view showing a situation where the terminal 20A has moved. Fig.10 is a sequence diagram showing the operation of each component in the system in a case where the terminal 20A has moved.

[0119] like Fig. 9 and Fig.10 As shown in , when the terminal 20A is taken out from the shelf, the acceleration sensor of the acceleration detection unit 200 in the terminal 20A detects physical movement and outputs a movement signal to the control unit 210 (step S100). The control unit 210 detects the movement signal, instructs the wireless communication unit 206 to search for the base station 10, and searches for an available one among the neighboring base stations 10a to 10f (step S102).

[0120] exist Fig. 9 , the presence information signals of the base stations 10b, 10c, 10e and 10f can be received. On the other hand, in the case where the presence information signals from the base stations 10a, 10d do not reach the terminal due to the long distance from the terminal, etc. (steps S106a, S106d), the control unit 210 of the terminal 20A obtains the base stations 10b, 10c, 10e, 10f that can perform wireless communication as the search result (step S108). Subsequently, after the search for the base stations 10b, 10c, 10e and 10f is completed, the terminal 20A measures the distances to the corresponding base stations 10b, 10c, 10e and 10f, from which the presence information signals have been successfully received (step S110).

[0121] Next, between the distance measurement unit 202 of the terminal 20A and the distance measurement units 102 of the base stations 10b, 10c, 10e, and 10f, sinusoidal communication required to calculate the group delay τ from the phase measured while changing the frequency of the radio wave is repeated (steps S112b, S112c, S112e, and S112f). Subsequently, the distance measurement unit 202 of the terminal 20A calculates the group delay τ from the information obtained through the communication with the base stations 10b, 10c, 10e, and 10f, calculates the distances from the base stations 10b, 10c, 10e, and 10f, and when the calculation process has been completed, outputs an end signal to the control unit 210 (step S114).

[0122] Furthermore, when a mobile signal is sensed, the control unit 210 causes the distance measurement unit 202 to perform position measurement (step S116 ). If the global positioning satellite system is available, the distance measurement unit 202 transmits the position measurement result to the control unit 210 (step S118 ).

[0123] At the same time, the distance measurement unit 102 of the base stations 10b, 10c, 10e, and 10f stores information such as the own position coordinates and antenna parameters in the storage unit (information accumulation unit) 302 of the information processing device via the wired communication unit 106 and the wired communication unit 310 of the information processing device, and transmits the information to the positioning information processing unit 304 (steps S120 to S126). Subsequently, via the wireless communication unit 206, the terminal 20A stores various position measurement results in the storage unit (information accumulation unit) 302 of the information processing device via the wireless communication unit 104 and the wired communication unit 106 of each of the base stations 10b, 10c, 10e, and 10f and the wired communication unit 310 of the information processing device, and transmits the position measurement results to the positioning information processing unit 304 (steps S128 to S136).

[0124] Next, the positioning information processing unit 304 calculates the world coordinates of the terminal 20A according to the principle of triangulation using various information (step S138). The positioning information processing unit 304 causes the storage unit (information accumulation unit) 302 to store the world coordinates of the terminal 20A associated with the measurement time, and causes the display and output unit 308 to present the world coordinates via the overall control unit 314 (steps S140 to S144). Then, the overall control unit 314 uses the information on the measurement ranges of the base stations 10b, 10c, 10e, and 10f stored in the storage unit (information accumulation unit) 302 to select the base station 10 mainly used for the position measurement of the terminal 20A from the base stations 10b, 10c, 10e, and 10f (step S146).

[0125] The selection is notified to the base station selected for position measurement of the terminal 20A and the terminal 20A via the wired communication unit 310 of the information processing device 30, the wired communication unit 106 and the wireless communication unit 104 of the base station 10, and the wireless communication unit 206 of the terminal 20A (steps S148 to S154). The terminal 20A performs the next and subsequent distance measurement operations with the selected base station 10 based on the base station information provided from the overall control unit 314 of the information processing device 30.

[0126] (Monitoring of terminal 20A ends)

[0127] Fig.11 20A is a plan view showing a situation where the terminal 20A has moved and monitoring is finished at the counter. Fig.12 This is a sequence diagram for monitoring end processing.

[0128] In the case where the terminal 20A is brought into the counter and there is no problem in taking it away, the user (store clerk) of the counter uses the reading unit 300 of the information processing device 30 to read the barcode of the item and the terminal 20A, and notifies the overall control unit 314 of the monitoring end signal related to the terminal 20A (step S200). The overall control unit 314 causes the display and input unit 308 to display information about the item and the terminal 20A (step S202). At this time, the user (store clerk) can remove the terminal 20A from the item.

[0129] Next, the overall control unit 314 transmits a stop instruction signal to the control unit 210 of the terminal 20A via the wireless communication unit 312 and the wireless communication unit 206 of the terminal 20A (steps S204 to S208). Subsequently, the control unit 210 of the terminal 20A transmits a stop confirmation signal to the overall control unit 314 via the wireless communication unit 206 of the terminal 20A and the wireless communication unit 312 of the information processing device 30 (steps S210 to S214).

[0130] Next, the terminal 20A sends a stop instruction confirmation signal to the overall control unit 314 of the information processing device 30, and then stops its own operation (step S216). Upon receiving the stop time confirmation signal from the terminal 20A, the overall control unit 314 of the information processing device 30 ends the monitoring process of the terminal 20A being performed by the overall control unit 314.

[0131] (If the terminal enters the area where the event occurred, no monitoring end processing will be performed)

[0132] Fig.13 2 is a plan view showing a situation where the terminal 20B has entered the event occurrence area without performing monitoring end processing at the counter. Fig.13 In the invention, a plurality of information processing devices 30 and 30a to 30c are provided. The information processing devices 30a to 30c are portable smart phones and are carried by security personnel, for example. Fig.14 is a sequence diagram showing the situation of entering the event occurrence area.

[0133] The overall control unit 314 of the information processing device 30 calculates the position information about the terminal 20B from the positioning information processing unit 304 (step S300), and notifies the overall control unit 314 of the position information (step S302). Next, the overall control unit 314 determines that the terminal 20B is in the event occurrence area A100 based on the calculated position information (step S304).

[0134] When it has been determined that the terminal 20B is in the event occurrence area A100, the overall control unit 314 presents the abnormality occurrence of the terminal 20B to the user as warning information via the display and input / output unit 308. In addition, the overall control unit 314 presents the abnormality occurring in the terminal 20B to the security personnel as alarm information via the display and input / output units 308 of the other information processing devices 30a to 30c (step S306).

[0135] At the same time, the overall control unit 314 also causes the image processing unit 306 to start image processing related to the terminal 20B (step S308). In the case of receiving the image processing instruction related to the terminal 20B, the image processing unit 306 obtains information about the movement trajectory and movement time of the terminal 20B from the storage unit (information accumulation unit) 302 (steps S310, S312). For example, the image processing unit 306 obtains an image showing the movement start position at the movement start time of the terminal 20B from the storage unit 302.

[0136] exist Fig.13In the case shown, the image processing unit 306 acquires images captured by the base stations 10b and 10e at the start time of movement based on the information about the imaging range of the base stations 10a to 10f, wherein the movement start position of the terminal 20B is within the imaging range. In addition, the image processing unit 306 acquires images that can be captured by the base stations 10a and 10d that can be imaged from the opposite side at the start time of movement, based on the movement trajectory and movement time of the terminal 20B. The image at the start of movement is usually captured before the item is put into a handbag, etc. Therefore, the object to which the terminal 20B is attached and the person moving the object are likely to be recognized from the image captured at the start of movement.

[0137] The recognition processing unit 306b of the image processing unit 306 uses a recognition process compatible with the shape of the object on which the terminal 20B is installed to perform a process of identifying the object to which the terminal 20B is attached in the acquired image (step S314). In the case where the recognition is successfully performed, the first mark is superimposed on the object in the image. In the case where the recognition is not successfully performed, a second mark different from the first mark is superimposed on a position corresponding to the movement start position in the image. The first mark and the second mark may have different colors, shapes, etc. In this way, the positioning information processing unit 304 of the information processing device 30 wirelessly measures the position of the terminal 20B. Therefore, even when a person or the like is an obstacle and the object to which the terminal 20B is attached is not directly shown in the image captured by the imaging unit 100 of the base station 10, the position of the object to which the terminal 20B is attached can be identified from the image.

[0138] The image processing unit 306 sends the image that has undergone image processing to the overall control unit 314 (step S316). Then, the overall control unit 314 presents the image processed by the image processing unit 306 as information about the terminal 20B to the user via the display and input / output unit 308. In addition, the same image as that presented by the information processing device 30 is presented via the display and input / output units 308 of the other information processing devices 30a to 30c.

[0139] As described above, according to the present embodiment, the position of the terminal 20 attached to the item to be monitored is measured in time series and stored in the storage unit 302 of the information processing device 30, and a captured image in which time series position information is associated with time information is stored in the storage unit 302. As a result, in the case where the item to be monitored is in the event occurrence area A100, an alarm can be issued and an image showing the movement trajectory of the terminal 20 can be presented to the user in time series via the display and input / output unit 308. Thus, it is possible to more easily observe the image moving from the removal of the item to be monitored to the event occurrence area A100.

[0140] In addition, the recognition processing unit 306b of the image processing unit 306 performs a process of identifying an object to which the terminal 20B is attached. In the case where the recognition is successfully performed, the first mark is superimposed on the object in the image. In the case where the recognition is not successfully performed, a second mark different from the first mark is superimposed on a position corresponding to the moving start position in the image. As a result, when the terminal 20 enters the event occurrence area A100, the user (store clerk) and the like can confirm the trajectory of the object to which the terminal 20B is attached, and can arrange the images showing the marked objects in chronological order. In this case, the time information is also linked between the two images, so the moving state of the object to which the terminal 20B is attached can be easily identified. In addition, even in the case where the object to which the terminal 20B is attached is placed in a handbag, etc., the trajectory of the object to which the terminal 20B is attached, and the images showing the marked objects and arranged in chronological order can be checked. Thus, it is also easy to identify a handbag, etc., of an object to which the terminal 20B is attached.

[0141] (Second embodiment)

[0142] The information processing system 1 according to the second embodiment is different from the information processing system 1 according to the first embodiment in further including a camera 40. In the following description, the difference from the information processing system 1 according to the first embodiment will be described.

[0143] Fig.15 : is a diagram showing an exemplary configuration of an information processing system 1 according to a second embodiment of the present technology. Fig.15 As shown, the information processing system 1 further includes a camera 40. A plurality of cameras 40 may be included.

[0144] Fig.16 1 is a block diagram showing an exemplary configuration of the camera 40. The camera 40 has a configuration similar to that of the base station 10 except for the distance measuring unit 102. That is, the camera 40 includes an imaging unit 100, a wireless communication unit 104, a wired communication unit 106, a storage unit 108, and a control unit 110. The base station 10 includes, for example, a central processing unit (CPU), and the corresponding components are formed by executing various programs stored in the storage unit 108. The storage unit 108 of the camera 40 stores the identification number of the camera 40, the placement coordinates of the camera 20, information related to the imaging optical system, wireless communication information, etc. as unique information.

[0145] Fig.17 is an example of a floor plan of the interior of a store to be monitored. Items are on shelves in the store. Fig.17In the invention, a plurality of base stations 10a, 10c, 10d and 10f, a plurality of terminals 20A and 20B, an information processing device 30, and a plurality of cameras 40a and 40b are arranged. For example, in a case where the entire area within a store can be covered by the radio wave irradiation range of the base stations 10a, 10c, 10d, 10f, cameras 40a and 40b can be arranged instead of the base station 10 to cover the imaging range. The plurality of base stations 10a, 10c, 10d, 10f and the cameras 40a and 40b are configured to be able to capture images of the entire area within the store. It should be noted that Fig.18 is a view of a scene in which a person squatting in the screen of the camera puts the item 500 into a bag.

[0146] As described above, in the information processing system 1 of the present embodiment, in a case where the entire area within the store can be covered by the radio wave irradiation range of the base stations 10a, 10c, 10d, 10f, the cameras 40a, 40b for covering the imaging range are provided instead of the base station 10. Therefore, the radio wave overlapping area can be reduced while maintaining the imaging range in the store.

[0147] It should be noted that the present technology can have the following configurations.

[0148] (1) An information processing device comprising:

[0149] a positioning information processing unit that generates position coordinates in time sequence based on distances from the plurality of sets of coordinates to a terminal attached to the article measured by radio distance measurement; and

[0150] A first control unit causes the display unit to display an image based on the time of radio distance measurement, the image having an imaging range including a position in space, the position corresponding to at least one set of time-series position coordinates, and the image corresponding to the time when the position in space has been measured by radio distance measurement.

[0151] (2) The information processing device according to (1), wherein the first control unit causes the display unit to display the image when the position coordinates are located within a predetermined area.

[0152] (3) The information processing device according to (2), wherein the radio distance measurement is performed by using a phase characteristic of a frequency of wireless communication between the plurality of base stations and the terminal.

[0153] (4) The information processing device according to (2), wherein the first control unit issues warning information when the position coordinates are located within a predetermined area.

[0154] (5) The information processing device according to (1), wherein the display unit displays an image captured at a time point when the terminal starts moving.

[0155] (6) The information processing device according to (1), further comprising:

[0156] Storage unit,

[0157] The positioning information processing unit generates position coordinates associated with the time of the radio distance measurement and stores the position coordinates in the storage unit.

[0158] (7) The information processing device according to (6), further comprising:

[0159] The map image generating unit superimposes the temporal change of the position coordinates as trajectory information on the map image.

[0160] (8) The information processing device according to (7), wherein the display unit displays the map image on which the trajectory information is superimposed, and an image corresponding to a time of radio distance measurement of position coordinates included in the trajectory information side by side.

[0161] (9) The information processing device according to (6), further including a recognition processing unit that recognizes a region of the object from the image.

[0162] (10) The information processing device according to (9), wherein the recognition processing unit superimposes a first mark indicating the area in the image.

[0163] (11) The information processing device according to (10), wherein the recognition processing unit superimposes a second mark indicating a portion corresponding to the position coordinates in the image.

[0164] (12) The information processing device according to (11), wherein the first control unit causes the recognition processing unit to start recognition processing when the position coordinates are located within a predetermined area.

[0165] (13) The information processing device according to (12), further comprising:

[0166] A reading unit reads first identification information about the item,

[0167] The recognition processing unit recognizes the area through recognition processing corresponding to the first identification information.

[0168] (14) The information processing device according to (13), wherein the reading unit is capable of reading second identification information about the terminal and storing the second identification information associated with the first identification information in the storage unit.

[0169] (15) The information processing device according to (14), wherein when the first identification information or the second identification information is read, the first control unit stops generating the position coordinates of the item and stops the operation of the terminal.

[0170] (16) The information processing device according to (1), wherein the first control unit selects a base station that measures a distance to the terminal through wireless communication with the terminal based on the position coordinates, selects images captured from a plurality of sets of predetermined coordinates based on the position coordinates, and causes the display unit to display the images.

[0171] (17) An information processing method comprising:

[0172] a positioning information processing step of generating position coordinates in time sequence based on distances from the plurality of sets of coordinates to a terminal attached to the item measured by radio distance measurement; and

[0173] The first control step causes the display unit to display an image according to the time of the radio distance measurement, the image having an imaging range including a position in space, the position corresponding to at least one set of time-series position coordinates, and the image corresponding to the time when the position in space has been measured by the radio distance measurement.

[0174] (18) An information processing system comprising:

[0175] a plurality of base stations for measuring a distance to a terminal attached to an article by a distance measurement method using a phase characteristic of a frequency of wireless communication with the terminal;

[0176] an imaging unit that is included in the base station or the camera and captures an image of a terminal attached to an item when the terminal is placed within an imaging range; and

[0177] information processing device,

[0178] The information processing device includes:

[0179] a positioning information processing unit that generates position coordinates in time sequence based on distances from coordinates of a plurality of base stations to the terminal measured by radio distance measurement, and

[0180] In the image captured by the imaging unit,

[0181] A first control unit causes the display unit to display an image based on the time of radio distance measurement, the image having an imaging range including a position in space, the position corresponding to at least one set of time-series position coordinates, and the image corresponding to the time when the position in space has been measured by radio distance measurement.

[0182] (19) An imaging device comprising:

[0183] a distance measurement unit for generating, in time sequence, position coordinates of a terminal attached to an item, and performing radio distance measurement to measure a distance to the terminal;

[0184] a wireless communication unit for wirelessly communicating with the terminal; and

[0185] An imaging unit, including an imaging optical system having a specified imaging range and an imaging element, captures an image in which position coordinates can be associated with the image.

[0186] (20) A terminal attached to an article, the terminal comprising:

[0187] a distance measurement unit, configured to generate the position coordinates of the terminal in time sequence, and perform radio distance measurement with a predetermined base station to measure the distance to the terminal; and

[0188] The wireless communication unit wirelessly communicates with a predetermined base station.

[0189] The various aspects of the present disclosure are not limited to the above-mentioned embodiments, but include various modifications that can be thought of by those skilled in the art, and the effects of the present disclosure are not limited to the above-mentioned contents. That is, various additions, modifications, and partial deletions may be made without departing from the concept and spirit of the present disclosure obtained from the contents defined in the claims and their equivalents.

[0190] Reference Symbols List

[0191] 1 Information Processing System

[0192] 10, 10a to 10f Base Station

[0193] 20, 20A, 20B terminals

[0194] 30, 30a to 30c Information processing device

[0195] 40, 40a, 40b Camera

[0196] 100 Imaging Units

[0197] 102 Distance measurement unit

[0198] 103 Communication unit

[0199] 104 Wireless Communication Unit

[0200] 202 Distance measurement unit

[0201] 302 Storage Unit

[0202] 306a Map image generation unit

[0203] 306b Identification Processing Unit

[0204] 308 Display output unit

[0205] 500 items.

Claims

1. An information processing device, comprising: a positioning information processing unit that generates position coordinates in time sequence based on distances from the plurality of sets of coordinates to a terminal attached to the article measured by radio distance measurement; as well as A first control unit causes a display unit to display an image based on the time of the radio distance measurement, the image having an imaging range including a position in space, the position corresponding to at least one set of time-series position coordinates, the image corresponding to the time when the position in space has been measured by radio distance measurement.

2. The information processing device according to claim 1, wherein: The first control unit causes the display unit to display the image when the position coordinates are located within a predetermined area.

3. The information processing device according to claim 2, wherein: The radio distance measurement is performed by a distance measurement method using a phase characteristic of frequencies of wireless communications between a plurality of base stations and the terminal.

4. The information processing device according to claim 2, wherein: When the position coordinates are within the predetermined area, the first control unit issues a warning message.

5. The information processing device according to claim 1, wherein: The display unit displays an image captured at a point in time when the terminal starts moving.

6. The information processing device according to claim 1, further comprising: Storage unit, The positioning information processing unit generates the position coordinates associated with the time of the radio distance measurement, and stores the position coordinates in the storage unit.

7. The information processing device according to claim 6, further comprising: The map image generating unit superimposes the temporal change of the position coordinates as trajectory information on the map image.

8. The information processing device according to claim 7, wherein: The display unit displays the map image on which the trajectory information is superimposed and an image corresponding to the time of the radio distance measurement of the position coordinates contained in the trajectory information side by side.

9. The information processing device according to claim 6, further comprising: The recognition processing unit recognizes the area of ​​the object from the image.

10. The information processing device according to claim 9, wherein: The recognition processing unit superimposes a first mark indicating the area in the image.

11. The information processing device according to claim 10, wherein: The recognition processing unit superimposes a second mark indicating a portion corresponding to the position coordinates in the image.

12. The information processing device according to claim 11, wherein: When the position coordinates are located within a predetermined area, the first control unit causes the recognition processing unit to start recognition processing.

13. The information processing device according to claim 12, further comprising: a reading unit, reading first identification information about the item, The recognition processing unit recognizes the area through recognition processing corresponding to the first identification information.

14. The information processing device according to claim 13, wherein: The reading unit is capable of reading second identification information about the terminal and storing the second identification information in the storage unit in association with the first identification information.

15. The information processing device according to claim 14, wherein: When the first identification information or the second identification information is read, the first control unit stops generating the location coordinates of the object and stops the operation of the terminal.

16. The information processing device according to claim 1, wherein: The first control unit selects a base station that measures a distance to the terminal through wireless communication with the terminal based on the position coordinates, selects images captured from a plurality of sets of predetermined coordinates based on the position coordinates, and causes the display unit to display the images.

17. An information processing method, comprising: a positioning information processing step of generating position coordinates in time sequence based on distances from the plurality of sets of coordinates to a terminal attached to the item measured by radio distance measurement; as well as The first control step causes the display unit to display an image based on the time series position coordinates and the time of the radio distance measurement, wherein the image has an imaging range including a position in space, the position corresponding to at least one set of the time series position coordinates, and the image corresponds to the time when the position in the space has been measured by the radio distance measurement.

18. An information processing system comprising: a plurality of base stations for measuring a distance to a terminal attached to an article by a distance measurement method using a phase characteristic of a frequency of wireless communication with the terminal; an imaging unit included in the base station or the camera and capturing an image of the terminal when the terminal attached to the item is placed within an imaging range; as well as information processing device, Wherein, the information processing device includes: a positioning information processing unit that generates position coordinates in time sequence based on distances from coordinates of a plurality of base stations to the terminal measured by radio distance measurement, and In the image captured by the imaging unit, A first control unit causes a display unit to display an image based on the time series position coordinates and the time of the radio distance measurement, wherein the image has an imaging range including a position in space, the position corresponding to at least one set of time series position coordinates, and the image corresponds to the time when the position in the space has been measured by radio distance measurement.

19. An imaging device comprising: a distance measurement unit for generating, in time sequence, position coordinates of a terminal attached to an item, and performing radio distance measurement to measure a distance to the terminal; A wireless communication unit, communicating wirelessly with the terminal; as well as The imaging unit includes an imaging optical system having a specified imaging range and an imaging element, and captures an image that can be associated with the position coordinates in a captured image.

20. A terminal attached to an article, the terminal comprising: a distance measurement unit, configured to generate the position coordinates of the terminal in time sequence and perform radio distance measurement with a predetermined base station to measure the distance to the terminal; as well as The wireless communication unit is configured to communicate wirelessly with the predetermined base station.

Images