Equipment information provision system and equipment information provision method

Abstract

According to the present disclosure, wireless tags (2) are attached to items (1) that are used in a facility (200). Wireless tag readers (10) and cameras (20) are installed at entrances and exits (202) of the facility. Servers (12, 22) store an image of each item (1) in association with identification information, and stores captured image data from the cameras (20). The server (12) acquires sets of inventory information about the items (1) at an opening time and a closing time, on the basis of a reception history of the wireless tag readers (10), and, by comparing these sets of inventory information, detects the identification information of items (1) that have been carried out of the facility (200) during business hours. Using the detected identification information, the server (12) estimates times at which items (1) were carried out, from the reception history of the wireless tag readers (10). By analyzing the captured image data at the estimated times at which items were carried out, the server (22) detects captured images including images of the items (1) and provides item information including the detected captured images.

Description

[Technical field]

[0001] The present disclosure relates to an equipment information providing system and an equipment information providing method for providing information about equipment used in a facility. [Background technology]

[0002] Japanese Patent Laid-Open Publication No. 2009-217364 (Patent Document 1) discloses a system for monitoring the theft of shopping baskets used in a store. In the system described in Patent Document 1, an IC tag capable of transmitting a radio signal including an ID for identifying the shopping basket is attached to the shopping basket. In the store, multiple sensors capable of receiving the radio signal transmitted from the IC tag are installed.

[0003] In the above system, the monitoring device determines that no shopping baskets have been stolen if the IDs corresponding to all shopping baskets have been acquired, and if it determines that the ID of any shopping basket has not been acquired, the monitoring device issues an alarm to the effect that the shopping basket has been stolen from a speaker provided near the sensor that acquired the ID of the shopping basket determined to have been stolen. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2009-217364 A Summary of the Invention [Problem to be solved by the invention]

[0005] According to the system described in Patent Document 1, an employee can immediately know that a shopping basket has been stolen and the location where the theft occurred. In addition, the theft can be notified to shoppers in the vicinity of the location where the shopping basket was stolen, which increases the number of witnesses.

[0006] However, some shoppers may temporarily take their shopping baskets out of the store to carry purchased items to a parking lot or bicycle parking area. If the management device determines that the shopping basket has been stolen and issues an alarm in the event of such a temporary removal, there is a concern that this may undermine the trust of the shopper. Therefore, a mechanism is needed to accurately detect theft of the shopping basket and prevent it from happening again.

[0007] The present disclosure has been made to solve such problems, and the purpose of the present disclosure is to provide an equipment information providing system and an equipment information providing method that can accurately detect when equipment used within a facility has been removed, and provide information about the person who removed the equipment. [Means for solving the problem]

[0008] An equipment information providing system according to an embodiment of the present disclosure provides equipment information regarding equipment used in a facility. The equipment information providing system includes a plurality of wireless tags, at least one wireless tag reader, a camera, and at least one server. The wireless tags are attached to a plurality of pieces of equipment used in the facility, respectively, and store identification information of the corresponding equipment. The at least one wireless tag reader is installed in the facility and receives signals including the identification information from the plurality of wireless tags. The camera is installed at an entrance / exit of the facility and captures an image of a person exiting through the entrance / exit. The at least one server is communicatively connected to the at least one wireless tag reader and the camera. The at least one wireless tag reader includes at least a first wireless tag reader installed at the entrance / exit. The at least one server includes at least one processor, a memory that stores a program executed by the at least one processor, and a storage device. The storage device stores an image of each of the plurality of pieces of equipment in association with the corresponding identification information, and stores image data captured by the camera. The at least one processor acquires inventory information of the equipment at a first time and inventory information of the equipment at a second time when a predetermined time has elapsed since the first time, based on a signal reception history of the at least one wireless tag reader. The at least one processor detects identification information of the equipment taken out of the facility during the predetermined time by comparing the inventory information at the first time with the inventory information at the second time. The at least one processor estimates a take-out time, which is a time when the equipment was taken out of the facility, from the reception history of the first wireless tag using the detected identification information. The at least one processor acquires an image of the equipment taken out of the facility from the storage device using the detected identification information. The at least one processor detects a captured image including an image of the equipment by analyzing captured image data at the estimated take-out time using the acquired image of the equipment, and provides equipment information including the detected captured image.

[0009] An equipment information providing method according to one aspect of the present disclosure is an equipment information providing method for providing equipment information regarding equipment used in a facility. A plurality of wireless tags are attached to a plurality of equipment used in the facility, respectively. Each of the plurality of tags is configured to store identification information of the corresponding equipment. At least one wireless tag reader that receives signals including the identification information from the plurality of wireless tags and a camera that captures an image of a person exiting through an entrance / exit of the facility are installed in the facility. The at least one wireless tag reader includes at least a first wireless tag reader installed at the entrance / exit. The equipment information providing method includes the steps of: linking images of each of a plurality of pieces of equipment with corresponding identification information and storing them in a storage device; storing image data captured by the camera in the storage device; acquiring, by a computer, inventory information of the equipment at a first time and inventory information of the equipment at a second time a predetermined time after the first time based on a reception history of signals from at least one wireless tag reader; detecting, by a computer, identification information of equipment taken out of the facility during a predetermined time by comparing the inventory information at the first time with the inventory information at the second time; using the detected identification information, estimating, by a computer, a removal time, which is the time when the equipment was taken out of the facility, from the reception history of signals from at least one wireless tag reader at the predetermined time; acquiring, by a computer, images of the equipment taken out of the facility from the storage device using the detected identification information; detecting, by a computer, a captured image including an image of the equipment by analyzing the captured image data at the estimated removal time using the acquired equipment image; and providing, by a computer, equipment information including the detected captured image. Effect of the Invention

[0010] According to the present disclosure, it is possible to accurately detect when equipment used within a facility has been removed, and provide information on the person who removed the equipment. [Brief description of the drawings]

[0011] [Figure 1] 1 is an overall configuration diagram of an equipment information providing system according to an embodiment. [Diagram 2] FIG. [Diagram 3] 2 is a diagram illustrating a hardware configuration of the equipment information providing system illustrated in FIG. 1. [Figure 4] FIG. 2 is a diagram showing an example of information managed in a server of the equipment management system. [Diagram 5] FIG. 4 is a diagram showing an example of car image data. [Figure 6] FIG. 13 is a diagram illustrating an example of a store entrance / exit table. [Figure 7] FIG. 13 is a diagram illustrating an example of a camera system ID table. [Figure 8] 11 is a diagram showing an example of a cart state DB, pre-opening inventory data, and post-closing inventory data. FIG. [Figure 9] FIG. 11 is a diagram illustrating an example of lost cage data. [Figure 10] 11A and 11B are diagrams illustrating an example of camera inquiry data and an image request signal. [Figure 11] 11A and 11B are diagrams illustrating a generation process of an image response signal in a server of the camera system. [Figure 12] FIG. 13 is a diagram showing an example of an analysis result image. [Figure 13] FIG. 13 is a diagram showing an example of display of information regarding a lost basket. [Figure 14] 11 is a flowchart illustrating an example of a processing procedure executed by the equipment management system. [Figure 15] 11 is a flowchart illustrating an example of a processing procedure executed by the equipment management system. [Figure 16] 10 is a flowchart illustrating an example of a processing procedure executed by the camera system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In the drawings, the same or corresponding parts are designated by the same reference characters and the description thereof will not be repeated.

[0013] <Application example of equipment information provision system> The equipment information providing system according to this embodiment is a system for providing information about equipment used in a facility to a manager of the facility. The information about the equipment includes information for identifying equipment that has been taken out of the facility due to theft or the like, and information for identifying the person who has taken the equipment out of the facility. In the following, a commercial facility will be described as an example of a facility to which the equipment information providing system according to this embodiment is applied.

[0014] In commercial facilities, shopping baskets are used by shoppers to carry the products they purchase. After purchasing the products, the shoppers are required to return the shopping baskets to basket storage areas installed in the facilities. However, some shoppers do not return the shopping baskets and end up stealing them.

[0015] As a measure to prevent such shopping basket theft, the above-mentioned Patent Document 1 describes a system that immediately issues an alarm when a shopping basket is stolen. However, there are cases where shoppers temporarily take their shopping baskets out of the facility to carry purchased items to a parking lot or bicycle parking area. Treating such temporary removal of the shopping basket as a shopping basket theft may undermine the trust of shoppers.

[0016] The equipment information providing system according to this embodiment is configured to accurately detect the theft of a shopping basket and to obtain an image of the person who has stolen the shopping basket.

[0017] <Overall configuration of equipment information provision system> FIG. 1 is an overall configuration diagram of an equipment information providing system according to an embodiment of the present disclosure.

[0018] As shown in Fig. 1, an equipment information providing system according to this embodiment is realized by cooperation between an equipment management system 100 and a camera system 120. The equipment management system 100 is a system for managing equipment used in a commercial facility 200. The equipment includes a shopping basket 1. The camera system 120 is a system for monitoring the inside of the commercial facility 200.

[0019] Fig. 2 is an external view of the shopping basket 1. As shown in Fig. 2, the shopping basket 1 has a basket body that contains products and a handle attached to the basket body. The shopping basket 1 is equipped with an RFID (Radio Frequency Identification) tag 2. In the example of Fig. 2, the RFID tag 2 is attached to the basket body.

[0020] The RFID tag 2 is configured to perform wireless communication with an RFID reader (described later) using a short-distance wireless communication method. The RFID tag 2 corresponds to one embodiment of a "wireless tag." Note that the wireless tag is not limited to an RFID tag, and may be, for example, an NFC (Near Field Communication) tag.

[0021] The RFID tag 2 includes an antenna, a processing circuit, and a memory. The memory stores identification information (hereinafter, also referred to as "RFID tag number") assigned to each shopping basket. This RFID tag number is read from the memory based on an instruction from the processing circuit.

[0022] Generally, wireless tags are classified into passive tags and active tags depending on whether they have a power source. Passive tags are wireless tags that do not have a power source and operate using radio waves from a wireless tag reader as their energy source. Active tags are wireless tags with a built-in power source. In this embodiment, the RFID tag 2 is a passive tag that operates using radio waves received from an RFID reader. This allows the equipment management system 100 to be constructed at low cost.

[0023] Returning to Fig. 1, the equipment management system 100 includes a plurality of RFID readers 10 and a server 12. The plurality of RFID readers 10 are installed in a commercial facility 200. The RFID readers 10 are installed in a location where shopping baskets 1 are collected, such as a basket storage area installed in the commercial facility 200. The RFID readers 10 may be portable readers.

[0024] At least some of the multiple RFID readers 10 are installed at entrances 202 through which shoppers enter and exit the commercial facility 200. In the example of Fig. 1, the commercial facility 200 has three entrances 202, and three RFID readers 10 are installed corresponding to the three entrances 202, respectively. The multiple RFID readers 10 are configured to be able to communicate with the server 12 and each other. The communication between the RFID readers 10 and the server 12 may be wired or wireless.

[0025] The RFID reader 10 operates by receiving power from a power source (not shown) and is configured to wirelessly communicate with the RFID tag 2 mounted on the shopping basket 1 using a short-range wireless communication method. As described above, the RFID tag 2 is a passive tag and is therefore operated by radio waves from the RFID reader 10. The RFID reader 10 corresponds to one embodiment of a "wireless tag reader."

[0026] The RFID reader 10 includes an antenna and a processing circuit. The RFID reader 10 transmits a request signal to the surroundings via the antenna at regular intervals in accordance with a short-range wireless communication standard. The RFID tag 2 mounted on the shopping basket 1 receives the request signal emitted from the RFID reader 10 and operates accordingly. The RFID tag 2 generates a response signal including the RFID tag number stored in the memory, modulates it, and transmits it to the RFID reader 10.

[0027] When the RFID reader 10 receives a response signal transmitted from the RFID tag 2 in response to the request signal, the RFID reader 10 stores the reception time of the response signal in association with the RFID tag number included in the response signal.

[0028] The server 12 has a function for managing the fixtures used in the commercial facility 200. As an example, the server 12 is installed in an operating company that operates the commercial facility 200. The server 12 can be used by a manager entrusted with the management of the commercial facility by the operating company.

[0029] The RFID reader 10 transmits the response signal reception history together with its own identification information (hereinafter also referred to as "RFID reader ID") at a predetermined period (for example, every few minutes) to the server 12. At this time, by deleting a part of the reception history already stored in the RFID reader 10, it is possible to reduce the memory capacity used for storing the reception history.

[0030] In addition, the RFID reader 10 is not limited to being configured to periodically transmit the reception history to the server 12 in accordance with the above-mentioned specified cycle, but may be configured to transmit the reception history for the most recent specified period (e.g., 3 minutes) in response to a transmission request from the server 12.

[0031] When the server 12 receives the reception history from each RFID reader 10, the server 12 registers in a storage device the reception history of each RFID reader 10. When multiple RFID readers 10 are installed in the commercial facility 200, the reception history of each RFID reader 10 is registered in the storage device.

[0032] Camera system 120 includes multiple cameras 20 and a server 22. The multiple cameras 20 are installed in a commercial facility 200. Specifically, the multiple cameras 20 are installed at entrances 202 of the commercial facility 200. The number of cameras 20 installed matches the number of entrances 202. In the example of FIG. 1, three cameras 20 are installed corresponding to the three entrances ENT, respectively.

[0033] The camera 20 is disposed so as to include the corresponding entrance / exit 202 in its field of view. The camera 20 is configured to capture an image of a person leaving the commercial facility 200. The camera 20 corresponds to one embodiment of a "camera."

[0034] Server 22 has a function for monitoring the inside of commercial facility 200. Server 22 is installed, for example, in an operating company that operates commercial facility 200, and can be used by a manager entrusted with managing the commercial facility by the operating company.

[0035] The server 22 is configured to be able to communicate with the camera 20 and the server 12. The communication between the camera 20 and the server 22 may be wired or wireless. The communication between the server 12 and the server 22 is performed via a communication network NW (typically, the Internet).

[0036] The camera 20 transmits captured image data consisting of moving images to the server 22. The server 22 stores the captured image data from each camera 20 in a storage device (database) by linking the captured image data with the identification information (hereinafter also referred to as "camera ID") of the camera 20 that transmitted the data. Note that the server 22 is configured to read out the captured image stored in the database and transmit it to the server 12 in response to an image request signal from the server 12, as described later.

[0037] <Hardware configuration of equipment information provision system> FIG. 3 is a diagram showing a hardware configuration of the equipment information providing system shown in FIG.

[0038] 3, in the equipment management system 100, the server 12 includes a CPU (Central Processing Unit) 111, a ROM (Read Only Memory) 112, a RAM (Random Access Memory) 113, a storage device 114, an input unit 115, a display unit 116, and a communication IF (Interface) 117. These components exchange various types of data via a communication bus.

[0039] The CPU 111 loads a program stored in the ROM 112 into the RAM 113 and executes it. In the program stored in the ROM 112, processes to be executed by the server 12 are written.

[0040] The storage device 114 is a storage for storing various information, and stores information about the commercial facility 200 and information about the equipment (including the shopping basket 1) used in the commercial facility 200. The storage device 114 is, for example, a hard disk drive (HDD: Hard Disk Drive) or a solid state drive (SSD: Solid State Drive).

[0041] The input unit 115 is made up of a touch panel, a mouse, a keyboard, etc., and receives user operations. The display unit 116 is made up of a display, various indicators, etc., and outputs the processing results from the CPU 111, etc.

[0042] The communication IF 117 is an interface for transmitting signals and data to and from the server 22 of the RFID reader 10 and the camera system 120 .

[0043] FIG. 3 shows an example of a configuration in which the CPU 111 executes a program to provide necessary processing, but some or all of the provided processing may be implemented using a dedicated hardware circuit (for example, an ASIC (Application Specific Integrated Circuit) or an FPGA (Field-Programmable Gate Array)).

[0044] In the camera system 120, the server 22 includes a web server 24, an application server 26, and a database server .

[0045] The WEB server 24 includes a CPU 241, a ROM 242, a RAM 243, a storage device 244, and a communication IF 245. These components exchange various data through a communication bus.

[0046] The application server 26 includes a CPU 261, a ROM 262, a RAM 263, a storage device 264, and a communication IF 265. These components exchange various types of data through a communication bus.

[0047] In each of the Web server 24 and the application server 26, the CPU loads a program stored in the ROM into the RAM and executes it. The program stored in the ROM describes the processes to be executed by each server.

[0048] The storage device is a storage device that stores various information. The storage device 244 of the WEB server 24 stores information including an analysis result image 272 generated by the image analysis process in the application server 26. The storage device 264 of the application server 26 stores information including image analysis data 282 and extraction target image data 283 used in the image analysis process.

[0049] The database server 28 stores a database 280. The database 280 includes image data 281 captured by the cameras 20 installed in the commercial facility 200.

[0050] <Functional configuration of equipment management system> Next, the functional configuration of the equipment management system 100 will be described with reference to FIGS.

[0051] Fig. 4 is a diagram showing an example of information managed in the server 12 of the equipment management system 100. Referring to Fig. 4, the storage device 114 of the server 12 stores a facility information DB 130 in which information related to the commercial facility 200 is stored, and a basket status DB 134 in which information related to the status of the shopping basket 1 within the commercial facility 200 is stored.

[0052] The facility information DB 130 includes basket image data 131, a store entrance / exit table 132, and a camera system ID table 133. FIG.

[0053] Referring to FIG. 5, the basket image data 131 includes image files of all shopping baskets 1 managed by the equipment management system 100. An RFID tag 2 that stores identification information (RFID tag number) of the shopping basket 1 is attached to each shopping basket 1. The image file of each shopping basket 1 is registered in the basket image data 131 in association with the RFID tag number. FIG. 5 illustrates an example of basket image data 131 when there is one type of shopping basket 1 to be managed. In this case, the RFID tag number of each shopping basket 1 is linked to an image file (file name: kago.png) of one type of shopping basket 1.

[0054] Fig. 6 is a diagram showing an example of the store entrance / exit table 132. Referring to Fig. 6, the store entrance / exit table 132 includes data of a "store name" which is the name of the commercial facility 200, an "entrance / exit name" which is the name of the entrance / exit 202 of the commercial facility 200, an "entrance / exit ID" which is identification information of the entrance / exit 202, a "camera ID" which is identification information of the camera 20 installed in the commercial facility 200, and an "RFID reader ID" which is identification information of the RFID reader 10 installed in the commercial facility 200.

[0055] 6, the store entrance table 132 includes data on the entrance names and entrance IDs of three entrances 202 installed in the commercial facility 200 (see FIG. 1). The store entrance table 132 further includes the camera IDs of the three cameras 20 installed at the three entrances 202, respectively, and the RFID reader IDs of the three RFID readers 10.

[0056] Fig. 7 is a diagram showing an example of camera system ID table 133. Referring to Fig. 7, camera system ID table 133 includes data of "camera system ID" which is identification information of camera system 120, and "camera ID" of camera 20 included in each camera system 120. Camera system ID table 133 further includes data of "destination address" which is an address assigned for accessing server 22 which manages each camera 20.

[0057] The car image data 131, the store entrance / exit table 132, and the camera system ID table 133 are stored in the facility information DB 130 by the manager of the commercial facility 200.

[0058] 4 , the basket status DB 134 includes data on identification information (RFID tag number) of the shopping basket 1 present in the commercial facility 200. The server 12 generates pre-opening inventory data 136 and post-closing inventory data 138 using the data related to the shopping basket 1 stored in the basket status DB 134, and stores the generated inventory data 136, 138 in the storage device 114.

[0059] Fig. 8 is a diagram showing an example of the car state DB 134, the pre-opening inventory data 136, and the post-closing inventory data 138. Referring to Fig. 8, the car state DB 134 registers a reception history of a response signal in the RFID reader 10 installed in the commercial facility 200. The reception history includes the time when the response signal was received from the RFID tag 2, the identification information (RFID tag number) of the RFID tag 2 that sent the response signal, and the identification information (RFID reader ID) of the RFID reader 10.

[0060] When a plurality of RFID readers 10 are installed, the car status DB 134 stores the reception history of each RFID reader 10. In the car status DB 134, the reception history of each RFID reader 10 is stored in order of reception time for each business day of the commercial facility 200.

[0061] The server 12 extracts the reception history of the RFID reader 10 immediately before the opening time of the commercial facility 200 on a certain business day from the basket status DB 134, and stores the extracted reception history in the storage device 114 as pre-opening inventory data 136. The pre-opening inventory data 136 corresponds to inventory information of the shopping basket 1 at the opening time of the commercial facility 200 on a certain business day. The opening time corresponds to an example of "first time", and the pre-opening inventory data 136 corresponds to an example of "inventory information at first time". In this embodiment, the opening time of the commercial facility 200 is 10:00 AM, the closing time is 10:00 PM, and the hours from 10:00 AM to 10:00 PM are the business hours.

[0062] The server 12 extracts the reception history for a predetermined length of time immediately before the opening time (10:00 a.m.) from the car status DB 134. In the example of Fig. 8, the reception history for 10 minutes from 9:50 a.m. to 10:00 p.m. is extracted and stored in the storage device 114 as pre-opening inventory data 136.

[0063] The server 12 further extracts the reception history of the RFID reader 10 immediately after the closing time of the business day from the basket status DB 134, and stores the extracted reception history in the storage device 114 as the after-closing inventory data 138. The after-closing inventory data 138 corresponds to inventory information of the shopping basket 1 at the closing time of the commercial facility 200 on a certain business day. The closing time corresponds to an example of the "second time", and the after-closing inventory data 138 corresponds to an example of the "inventory information at the second time". Specifically, the server 12 extracts the reception history for a predetermined length of time immediately after the closing time (10:00 PM) from the basket status DB 134. In the example of FIG. 8, the reception history for 10 minutes from 10:00 PM to 10:10 PM is extracted and stored in the storage device 114 as the after-closing inventory data 138.

[0064] Returning to Fig. 4, the server 12 generates lost basket data 140 based on the pre-opening inventory data 136 and the post-closing inventory data 138 stored in the storage device 114. The lost basket data 140 is data indicating shopping baskets 1 (hereinafter also referred to as "lost baskets") that have been taken out of the commercial facility 200 during the time from opening time to closing time, i.e., during the business hours of the commercial facility 200.

[0065] Fig. 9 is a diagram showing an example of the lost basket data 140. Referring to Fig. 9, the server 12 compares the before-opening inventory data 136 and the after-closing inventory data 138 stored in the storage device 114. The server 12 detects reception histories included in the before-opening inventory data 136 that are not included in the after-closing inventory data 138.

[0066] As described above, the pre-opening inventory data 136 is inventory information of the shopping cart 1 at opening time, and the after-closing inventory data 138 is inventory information of the shopping cart 1 at closing time. A reception history that is included in the pre-opening inventory data 136 but not included in the after-closing inventory data 138 corresponds to a reception history of a response signal from a shopping cart 1 that was present within the commercial facility 200 at opening time but was not present within the commercial facility 200 at closing time. The server 12 determines that this shopping cart 1 is a shopping cart 1 (lost basket) that was taken out of the commercial facility 200 during business hours, and treats the reception history from this shopping cart 1 as lost basket data 140.

[0067] In the example of FIG. 9, the pre-opening inventory data 136 includes reception histories from shopping baskets 1 with RFID tag numbers "tag1", "tag2", "tag3", "tag4", and "tag5". Therefore, it is determined that the commercial facility 200 holds these five shopping baskets 1 at the opening time. The post-closing inventory data 138 includes reception histories from shopping baskets 1 with RFID tag numbers "tag1", "tag2", "tag3", and "tag5". Therefore, it is determined that the commercial facility 200 holds these four shopping baskets 1 at the closing time.

[0068] When the before-opening inventory data 136 and the after-closing inventory data 138 are compared, the after-closing inventory data 138 does not include a reception history from the shopping basket 1 with the RFID tag number "tag4." Therefore, the reception history from the shopping basket 1 having the RFID tag number "tag4" is extracted as the lost basket data 140.

[0069] Next, the server 12 refers to the reception history during business hours of the commercial facility 200, which is stored in the basket status DB 134. The server 12 searches the reception history during business hours for a reception history that includes the RFID tag number included in the lost basket data 140. The server 12 then detects the reception history with the latest reception time (i.e., the latest) from the search results of the reception history that includes the RFID tag number of the lost basket. The server 12 rewrites the lost basket data 140 using the detected reception history. At this time, the server 12 rewrites the "reception time" and "RFID reader ID" included in the lost basket data 140 to the "reception time" and "RFID reader ID" included in the detected latest reception history, respectively.

[0070] In the example of FIG. 9, the server 12 searches for reception history including the RFID tag number "tag4" from the reception history of the RFID reader 10 during business hours. The server 12 then detects the reception history with the latest reception time (latest) from among at least one reception history included in the search result. If the reception history with the reception time "2022 / 01 / 27 17:45" is the latest reception history, the server 12 rewrites the lost basket data 140 using the reception time "2022 / 01 / 27 17:45" and the RFID reader ID "R1" included in this reception history.

[0071] The reception history with the latest reception time among at least one reception history included in the search result corresponds to the last reception history from shopping cart 1 (lost cart) during business hours. This last reception history indicates that reception of a response signal from shopping cart 1 was discontinued after that reception history.

[0072] Therefore, the server 12 estimates that the reception time of this last reception history is the take-out time when shopping basket 1 (lost basket) was taken out of the commercial facility 200. The server 12 rewrites the "reception time" included in the lost basket data 140 to the estimated take-out time. Furthermore, the server 12 rewrites the "RFID reader ID" included in the lost basket data 140 to the RFID reader ID of the RFID reader 10 that received the last response signal from shopping basket 1. As a result, the lost basket data 140 becomes data that includes the take-out time of shopping basket 1 (lost basket) and the RFID reader ID of the RFID reader 10 that received the response signal from shopping basket 1 when shopping basket 1 (lost basket) was taken out through the entrance / exit 202.

[0073] Returning to Fig. 4, the server 12 generates camera inquiry data 142 based on the lost basket data 140. The server 12 generates an image request signal 144 using the generated camera inquiry data 142. Fig. 10 is a diagram showing an example of the camera inquiry data 142 and the image request signal 144.

[0074] 10, the camera inquiry data 142 includes the “reception time”, “RFID reader ID”, and “RFID tag number” included in the lost basket data 140. In FIG. 10, the camera inquiry data 142 includes the reception time “2022 / 01 / 27 17:45”, the RFID reader ID “R1”, and the RFID tag number “tag4”.

[0075] The camera inquiry data 142 further includes a "camera ID." This "camera ID" is generated based on the store entrance table 132 (see FIG. 6). The server 12 detects, in the store entrance table 132, a "camera ID" linked to the "RFID reader ID" included in the lost basket data 140, and includes the detected "camera ID" in the camera inquiry data 142.

[0076] 6, the RFID reader ID "R1" is associated with the store name "SH1", the entrance name "ENT1", the entrance ID "E1", and the camera ID "C1". This indicates that an RFID reader 10 having an RFID reader ID "R1" and a camera 20 having a camera ID "C1" are installed at the entrance 202 of the entrance name "ENT1" in the commercial facility 200 of the store name "SH1". Therefore, the server 12 includes the camera ID "C1" in the camera inquiry data 142.

[0077] Next, the server 12 generates an image request signal 144 using the camera inquiry data 142. The image request signal 144 is a signal for requesting the camera system 120 to transmit an image captured by the camera 20. As shown in Fig. 10, the image request signal 144 includes an "equipment management system ID" which is identification information for the equipment management system 100, an "RFID tag number", a "car image", a "camera ID", a "received time", and a "time span". Of these data, the "RFID tag number", "camera ID", and "received time" use data included in the camera inquiry data 142.

[0078] "Basket image" indicates an image file of shopping basket 1 (lost basket) having "RFID tag number." "Basket image" is generated based on basket image data 131 (see FIG. 5). "Time span" indicates the length of the captured image cut out from the captured image data of camera 20. In the example of FIG. 10, the image request signal 144 contains the image file "kago.png" of shopping basket 1 having RFID tag number "tag4" and a time span of "10 seconds."

[0079] According to the image request signal 144 shown in Figure 10, in the camera system 120, from the image data captured by the camera 20 having the camera ID "C1", an image captured for a time range (10 seconds) including the reception time "2022 / 01 / 27 17:45" of the response signal from the shopping cart 1 having the RFID tag number "tag4" is cut out.

[0080] The server 12 transmits the generated image request signal 144 to the server 22 of the camera system 120 via the communication network NW. Upon receiving the image request signal 144 from the server 12, the server 22 generates an image response signal 300 in response thereto.

[0081] <Functional configuration of the camera system> Next, the functional configuration of the camera system 120 will be described with reference to FIG.

[0082] FIG. 11 is a diagram for explaining the process of generating an image response signal 300 in the server 22 of the camera system 120. As shown in FIG.

[0083] 11, in the server 22, the application server 26 acquires image analysis data 282 from the database server 28 based on the "camera ID" included in the image request signal 144. Specifically, the application server 26 reads out the captured image data 281 of the camera 20 with the camera ID "C1" stored in the database server 28. The application server 26 cuts out a captured image (moving image) having the reception time and time width included in the image request signal 144 from the read captured image data 281. This captured image has a time width of 10 seconds centered around the reception time "2022 / 01 / 27 17:45". The application server 26 acquires the captured image cut out from the captured image data 281 as image analysis data 282.

[0084] Furthermore, the application server 26 acquires the basket image included in the image request signal 144 as the extraction target image data 283. In the example of Fig. 11, the application server 26 acquires, as the extraction target image data 283, an image file "kago.png" showing an image of the shopping basket 1 having the RFID tag number "tag4".

[0085] Next, the application server 26 executes an image analysis process 270 using the image analysis data 282 and the extraction target image data 283. Specifically, the application server 26 executes a process for detecting the basket image indicated by the extraction target image data 283 from the image analysis data 282. In this image analysis process 270, the application server 26 generates an analysis result image 272 by superimposing a mark for identifying the basket image contained in the photographed image on the photographed image. FIG. 12 is a diagram showing an example of the analysis result image 272.

[0086] As shown in FIG. 12, the analysis result image 272 includes an image captured by a camera 20 having a camera ID "C1" and a mark for identifying the extraction target image data 283 (basket image) contained in the captured image.

[0087] The captured image is an image of an entrance 202 having an entrance name "ENT1" and an entrance ID "E1" in a commercial facility 200 with a store name "SH1" captured by a camera 20. The captured image includes an image of a person M leaving through the entrance 202.

[0088] The basket image detected from this captured image is an image that matches the image (image file "kago.png") of shopping basket 1 (lost basket) having RFID tag number "tag4" acquired as extraction target image data 283. The mark is displayed, for example, superimposed on the captured image so as to surround the detected image of shopping basket 1.

[0089] Furthermore, the identification information (RFID tag number "tag4") of shopping basket 1 (lost basket) included in the captured image is superimposed and displayed on the analysis result image 272.

[0090] This analysis result image 272 captures a person who took the shopping basket 1 (lost basket) out of the commercial facility 200 around the time (2022 / 01 / 27 17:45) when the shopping basket 1 (lost basket) was taken out of the commercial facility 200. Therefore, it is possible to identify the person who stole the shopping basket 1 from the analysis result image 272.

[0091] The application server 26 transmits the created analysis result image 272 to the WEB server 24. The WEB server 24 stores the analysis result image 272 in the storage device 244.

[0092] Furthermore, application server 26 generates an image response signal 300 for server 12 of equipment management system 100. Image response signal 300 is obtained by adding to image request signal 144 (see FIG. 10) received from server 12 a "camera system ID" that is identification information for camera system 120, a URL (Uniform Resource Locator) describing the location where analysis result image 272 is saved (i.e., WEB server 24), and information on a key used to access the saved location.

[0093] The application server 26 transmits the generated image response signal 300 to the server 12 of the equipment management system 100 via the communication network NW.

[0094] <Provision of equipment information> In the equipment management system 100, when the server 12 receives an image response signal 300 from the server 22 of the camera system 120, the server 12 generates information 302 about the lost basket using the image response signal 300. Then, the server 12 provides the information 302 about the lost basket in response to an instruction from the manager of the commercial facility 200. For example, the server 12 causes the display unit 116 to display the information 302 about the lost basket. Fig. 13 is a diagram showing an example of how the information 302 about the lost basket is displayed.

[0095] The information 302 about the lost basket includes image search result data 304 and information 306 about the time the lost basket was taken out. The server 12 generates the image search result data 304 by adding the URL and key information of the Web server 24, which is included in the image response signal 300 from the server 22, to the camera inquiry data 142 (see FIG. 10). The server 12 further adds to the image search result data 304 information indicating the store name "SH1" of the commercial facility 200 and the entrance / exit name "ENT1" of the entrance / exit 202 from which the shopping basket 1 was taken out.

[0096] The information 306 relating to the time of removal includes the identification information of the lost basket (RFID tag number), the image file name of the lost basket, the time when the lost basket was removed from the commercial facility 200, and information regarding the time span during which the camera system 120 performed the image analysis.

[0097] With the image search result data 304 displayed on the display unit 116, the input unit 115 can accept a user operation for displaying the analysis result image 272. For example, when the administrator moves the mouse pointer 308 to the URL of the WEB server 24 and clicks, the server 12 uses the URL and key information to access the WEB server 24 and obtains the analysis result image 272 from the WEB server 24. The server 12 displays the obtained analysis result image 272 on the display unit 116.

[0098] By checking the information displayed on the display unit 116, the manager of the commercial facility 200 can obtain the identification information (RFID tag number) and basket image of the shopping basket 1 taken out of the commercial facility 200, information about the commercial facility 200 and the entrance / exit 202 from which the shopping basket 1 was taken, information about the estimated time the shopping basket 1 was taken out, and an image of the person who took the shopping basket 1 out.

[0099] <Equipment information provision process flow> Next, the flow of processing executed by the equipment information providing system will be described.

[0100] 14 and 15 are flowcharts showing an example of a procedure of a process executed by the equipment management system 100. FIG.

[0101] 14, each of the multiple RFID readers 10 installed in the commercial facility 200 transmits a request signal at regular time intervals (step S01). At least some of the multiple RFID readers 10 are installed at the entrance / exit 202 of the commercial facility 200 (see FIG. 1).

[0102] The RFID tag 2 mounted on the shopping basket 1 used in the commercial facility 200 is activated upon receiving a request signal emitted from the RFID reader 10. The RFID tag 2 generates a response signal including an RFID tag number, which is identification information of the shopping basket 1, and transmits it to the RFID reader 10.

[0103] Each RFID reader 10 judges whether or not a response signal has been received from the RFID tag 2 mounted in the shopping basket 1 (step S02). When a response signal has been received from the RFID tag 2 (YES in S02), the RFID reader 10 stores a response signal reception history in a basket status DB 134 provided in the storage device 114 of the server 12 (step S03). As shown in Fig. 8, the basket status DB 134 stores the reception time of the response signal in each RFID reader 10, the RFID reader ID of the RFID reader 10 that received the response signal, and the RFID tag number of the RFID tag 2 mounted in the shopping basket 1 in association with each other.

[0104] The server 12 judges whether the opening time of the commercial facility 200 has arrived (step S04). When it is judged that the opening time of the commercial facility 200 has arrived (YES judgment in S04), the server 12 generates pre-opening inventory data 136 using the reception history stored in the car state DB 134, and stores it in the storage device 114 (step S05). In S05, the server 12 extracts the reception history of each RFID reader 10 for a predetermined length of time immediately before the opening time of the commercial facility 200 from the car state DB 134, and stores it in the storage device 114 as the pre-opening inventory data 136.

[0105] Next, the server 12 judges whether the closing time of the commercial facility 200 has arrived (step S06). If it is judged that the closing time of the commercial facility 200 has arrived (YES in S06), the server 12 uses the reception history stored in the cart status DB 134 to generate after-closing inventory data 138 and stores it in the storage device 114 (step S07). In S07, the server 12 extracts the reception history of each RFID reader 10 for a predetermined length of time immediately after the closing time of the commercial facility 200 from the cart status DB 134, and stores it in the storage device 114 as after-closing inventory data 138.

[0106] If the store opening time has not yet arrived (NO in S04) or if the store closing time has not yet arrived (NO in S06), the process returns to S01. The wireless communication between the RFID reader 10 and the RFID tag 2 shown in S01 to S03 continues, and the reception history of the RFID reader 10 is stored in the car status DB 134.

[0107] When the after-closing inventory data 138 is stored in the storage device 114, the server 12 compares the before-opening inventory data 136 and the after-closing inventory data 138 stored in the storage device 114 (step S08). The server 12 then generates lost basket data 140 (see FIG. 9) based on the comparison result in S08 (step S09). In S09, the server 12 extracts, as the lost basket data 140, the reception history that is included in the before-opening inventory data 136 but not included in the after-closing inventory data 138. The server 12 rewrites the "reception time" and "RFID reader ID" included in this lost basket data 140 to the "reception time" and "RFID reader ID" included in the reception history with the latest reception time among the reception histories from lost baskets during business hours.

[0108] Next, the server 12 generates camera inquiry data 142 (see FIG. 10) based on the lost basket data 140 generated in S09 (step S10). In S10, the server 12 detects the "camera ID" of the camera 20 that captured the image of the lost basket by using the lost basket data 140 and the store entrance / exit table 132 (see FIG. 6), and includes the detected "camera ID" in the camera inquiry data 142.

[0109] Next, the server 12 generates an image request signal 144 (see FIG. 10) based on the camera inquiry data 142 (step S11), and transmits the generated image request signal to the server 22 of the camera system 120 (step S12).

[0110] 15, after transmitting the image request signal 144, the server 12 determines whether or not it has received the image response signal 300 (see FIG. 11) from the server 22 of the camera system 120 (step S13).

[0111] When the image response signal 300 is received from the server 22 (YES in S13), the server 12 generates image search result data 304 (see FIG. 13) using the information included in the image response signal 300 and the camera inquiry data 142 (step S14). The server 12 displays information 302 about the lost basket, including the generated image search result data 304, on the display unit 116 (step S15).

[0112] The server 12 determines whether the input unit 115 has received a user operation to click on the URL of the WEB server 24 included in the image search result data 304 (step S16). If a user operation has been received (YES in S16), the server 12 uses the URL and key information to access the WEB server 24 and acquires the analysis result image 272 from the WEB server 24 (step S17). The server 12 displays the acquired analysis result image 272 on the display unit 116 (step S18).

[0113] FIG. 16 is a flowchart showing an example of a procedure of processing executed by the camera system 120.

[0114] Referring to FIG. 16, server 22 stores image data captured by camera 20 installed at entrance / exit 202 of commercial facility 200 in database 280 in database server 28 (step S21).

[0115] The server 22 (application server 26) determines whether or not it has received an image request signal 144 from the server 12 of the equipment management system 100 (step S22). If it has received the image request signal 144 (YES in S22), the server 22 (application server 26) uses the information on the "camera ID," "reception time," and "time span" included in the image request signal 144 to extract image analysis data 282 from the captured image data of the camera 20 stored in the database 280 and stores the extracted data in the storage device 264 (step S23). In S23, the server 22 reads out the captured image data 281 of the camera 20 having the "camera ID" included in the image request signal 144 from the database 280, and cuts out a captured image (moving image) having the "reception time" and "time span" included in the image request signal 144 from the read captured image data 281 (see FIG. 11).

[0116] The server 22 (application server 26) further stores the "basket image" included in the image request signal 144 in the storage device 264 as extraction target image data 283 (step S24).

[0117] Next, the server 22 (application server 26) executes image analysis processing using the image analysis data 282 and the extraction target image data 283 (step S25). In S25, the server 22 executes processing to detect the car image indicated by the extraction target image data 283 from the image analysis data 282. The server 22 generates an analysis result image 272 (see FIG. 12) by superimposing a mark for identifying the car image contained in the photographed image on the photographed image (step S26).

[0118] The server 22 (application server 26) transmits the analysis result image 272 generated in S26 to the WEB server 24. The WEB server 24 stores the analysis result image 272 in the storage device 244 (step S27).

[0119] Next, the server 22 (application server 26) generates an image response signal 300 (see FIG. 11) for the server 12 of the equipment management system 100 (step S28). The server 22 transmits the generated image response signal 300 to the server 12 of the equipment management system 100 via the communication network NW (step S29).

[0120] As described above, in the equipment information provision system according to the present embodiment, an RFID reader 10 and a camera 20 are installed at the entrance / exit 202 of the commercial facility 200, and an RFID tag 2 is attached to a shopping basket 1 used within the commercial facility 200. In this configuration, inventory information of the shopping basket 1 at the opening time of the commercial facility 200 (pre-opening inventory data) and inventory information of the shopping basket 1 at the closing time of the commercial facility 200 (post-closing inventory data) are obtained based on the reception history from the RFID tag 2. Then, based on a comparison of these two pieces of inventory information, information regarding the shopping basket 1 (lost basket) taken out of the commercial facility 200 during business hours is obtained.

[0121] Furthermore, the time when the lost basket was taken out and the entrance / exit from which the lost basket was taken out are estimated from the reception history of the RFID reader 10 during business hours, and based on this estimated information, an image of the entrance / exit around the time of taking the basket out is acquired from the image data captured by the camera 20. Then, an image analysis process is performed to detect an image of the lost basket from the acquired captured image, thereby acquiring an image of the person who took the shopping basket 1 out of the commercial facility 200.

[0122] According to this embodiment, when a shopper temporarily takes a shopping basket out of a commercial facility, the lost basket data 140 acquired based on the pre-opening inventory data and the post-closing inventory data does not include information about the shopping basket. Therefore, it is possible to prevent the temporary removal of a shopping basket from being mistakenly determined as theft of the shopping basket. This makes it possible to accurately detect the theft of a shopping basket.

[0123] Furthermore, according to this embodiment, an image of the person who has taken the shopping basket is acquired, so that the manager can identify the person who has taken the shopping basket based on the acquired image.

[0124] The equipment information system according to this embodiment is intended to improve the accuracy of theft detection by detecting theft of a shopping basket after the fact, and is not intended to catch a shopping basket theft in the act. Therefore, it cannot directly prevent shopping basket theft. However, by informing shoppers that the equipment information system according to this embodiment can identify the person who has stolen the shopping basket, it is expected that the system will have a deterrent effect on shopping basket theft.

[0125] In the above-described embodiment, a shopping basket used in a commercial facility is given as an example of equipment used within a facility, however, the facilities and equipment to which the equipment provision system according to this embodiment can be applied are not limited to this.

[0126] The embodiments disclosed herein should be considered as illustrative and not restrictive in all respects. The present disclosure is defined by the claims rather than the above description, and is intended to include all modifications within the meaning and scope of the claims. [Explanation of symbols]

[0127] 1 shopping basket, 2 RFID tag, 10 RFID reader, 12, 22 server, 20 camera, 24 WEB server, 26 application server, 28 database server, 100 equipment management system, 111, 241, 261 CPU, 112, 242, 262 ROM, 113, 243, 263 RAM, 114, 244, 264 storage device, 115 input unit, 116 display unit, 117, 245, 265 communication IF, 120 camera system, 130 facility information DB, 131 basket image data, 132 store entrance / exit table, 133 camera system ID table, 134 basket status DB, 136 pre-opening inventory data, 138 post-closing inventory data, 140 lost basket data, 142 camera inquiry data, 144 image request signal, 200 commercial facility, 202 Entrance / exit, 270 image analysis processing, 272 analysis result image, 280 database, 281 captured image data, 282 image analysis data, 283 extraction target image data, 300 image response signal, 302 image search result data, NW communication network.

Claims

1. An equipment information providing system for providing equipment information regarding equipment used in a facility to a manager of the facility, a plurality of wireless tags attached to a plurality of pieces of equipment used in the facility, each of the wireless tags storing identification information of the corresponding pieces of equipment; at least one wireless tag reader that is installed within the facility and receives signals including the identification information from the wireless tags; A camera installed at an entrance of the facility for photographing people exiting through the entrance; a first server communicatively connected to the at least one wireless tag reader; a second server communicatively connected to the camera and the first server; the at least one wireless tag reader includes at least a first wireless tag reader installed at the entrance; the first server includes a first storage device for storing an image of each of the plurality of pieces of equipment in association with the corresponding identification information; the second server includes a second storage device for storing image data captured by the camera; The first server comprises: acquiring inventory information of the equipment at a first time and inventory information of the equipment at a second time that is a predetermined time after the first time, based on a reception history of the signal of the at least one wireless tag reader; detecting the identification information of the equipment taken out of the facility during the predetermined time by comparing the inventory information at the first time with the inventory information at the second time; using the detected identification information, and estimating a take-out time, which is a time when the equipment was taken out of the facility, from a signal reception history of the at least one wireless tag reader during the predetermined time; acquiring, from the first storage device, an image of the equipment removed from the facility using the detected identification information; The second server comprises: acquiring the captured image data at the estimated take-out time from the second storage device; Analyzing the photographed image data at the estimated take-out time using the acquired image of the furniture to detect a photographed image including an image of the furniture; The first server comprises: An equipment information providing system that provides the equipment information including the detected photographed image to a manager of the facility.

2. The first server comprises: acquiring the identification information included in the reception history of the at least one wireless tag reader at the first time as the inventory information at the first time; acquiring the identification information included in the reception history of the at least one wireless tag reader at the second time as the inventory information at the second time; The equipment information providing system of claim 1, wherein identification information that is included in the inventory information at the first time but not included in the inventory information at the second time is detected as the identification information of the equipment taken out of the facility.

3. An equipment information providing system as described in claim 1 or 2, wherein the captured images detected by the second server include images of the equipment taken out of the facility and images of the person who took the equipment out of the facility.

4. The equipment information providing system according to claim 1 , wherein the wireless tag is a passive tag that operates by radio waves from the wireless tag reader.

5. The facility is a commercial facility, The equipment information providing system according to claim 1 , wherein the equipment includes a shopping basket.

6. A plurality of wireless tags attached to a plurality of pieces of equipment used in a facility, each of which stores identification information of the corresponding piece of equipment; at least one wireless tag reader that is installed within the facility and receives signals including the identification information from the wireless tags; A camera installed at an entrance of the facility for photographing people exiting through the entrance; a first server communicatively connected to the at least one wireless tag reader; a second server communicatively connected to the camera and the first server; the at least one wireless tag reader includes at least a first wireless tag reader installed at the entrance; The first server includes a first storage device for storing an image of each of the plurality of pieces of equipment in association with the corresponding identification information; and a second server providing equipment information on equipment used in the facility to a manager of the facility by a system having a second storage device for storing image data captured by the camera, a step in which the first server acquires inventory information of the equipment at a first time and inventory information of the equipment at a second time that is a predetermined time after the first time, based on a reception history of the signal of the at least one wireless tag reader; The first server detects the identification information of the equipment taken out of the facility during the predetermined time by comparing the inventory information at the first time with the inventory information at the second time; a step in which the first server estimates a take-out time, which is a time when the equipment was taken out of the facility, from a signal reception history of the at least one wireless tag reader during the predetermined time period using the detected identification information; The second server acquires, from the second storage device, an image of the equipment taken out of the facility using the detected identification information; The second server analyzes the photographed image data at the estimated take-out time using the acquired image of the furniture to detect a photographed image including an image of the furniture; The equipment information providing method comprises a step of the first server providing the equipment information including the detected photographed image to a manager of the facility.

7. The step of acquiring the inventory information at the first time and the inventory information at the second time includes: acquiring the identification information included in the reception history of the at least one wireless tag reader at the first time as the inventory information at the first time; acquiring the identification information included in the reception history of the at least one wireless tag reader at the second time as the inventory information at the second time, The equipment information providing method of claim 6, wherein the step of detecting the identification information of the equipment taken out of the facility includes a step of detecting the identification information that is included in the inventory information at the first time but is not included in the inventory information at the second time as the identification information of the equipment taken out of the facility.

8. The equipment information providing method according to claim 6 , wherein the step of detecting the captured image includes a step of detecting an image of the equipment from the captured image data.

Images