Store leaving management system, store leaving management method and program

Abstract

To provide a store leaving management system capable of supporting a countermeasure against an abnormality with a commodity carried by a leaving customer when the registration and the settlement are performed for the commodity through a portable terminal operated by the customer.SOLUTION: A store leaving management system according to the present disclosure includes: code acquisition means that, when the registration and the settlement are performed for a commodity through a portable terminal operated by a customer, acquires a settlement completion code from the portable terminal; paid commodity information acquisition means that acquires the number of paid commodities related to the settlement completion codes; estimation means that estimates the number of commodities carried by a customer from an image obtained by capturing luggage of the leaving customer; abnormality detection means that detects an abnormality on the basis of the acquired number of the paid commodities and the estimated number of the carried commodities; and abnormality output means that outputs detection of the abnormality.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a store exit management system and the like. [Background technology]

[0002] Some stores allow customers to register products and settle their bills by operating a terminal. At the gate or counter of such stores, customers are sometimes required to scan a terminal before leaving the store.

[0003] Patent Document 1 discloses a system in which a customer registers a product using a portable terminal. In Patent Document 1, when the terminal is returned to the terminal management device, a payment process for the registered product is performed. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-099441 Summary of the Invention [Problem to be solved by the invention]

[0005] When a customer registers and pays for products, there may be a discrepancy between the amount of products registered and paid for and the amount of products taken out. For example, this discrepancy may occur if a customer takes out products without paying for them, or if a customer leaves paid-for products inside the store. When such a discrepancy occurs, it is preferable for the customer or store clerk to take action.

[0006] The present disclosure aims to provide a store exit management system, etc., that can assist in dealing with any abnormalities that occur regarding products taken out by customers as they leave the store when products are registered and settled via a portable terminal operated by the customer. [Means for solving the problem]

[0007] The store exit management system according to the present disclosure comprises a code acquisition means for acquiring a payment code from a portable terminal operated by a customer when the product is registered and settled via the portable terminal; a payment product information acquisition means for acquiring the amount of paid products associated with the payment code; an estimation means for estimating the amount of products to be taken out by the customer from an image of the customer's luggage as they leave the store; an abnormality detection means for detecting an abnormality based on the acquired amount of paid products and the estimated amount of products to be taken out; and an abnormality output means for outputting that the abnormality has been detected.

[0008] The store exit management method disclosed herein, when a product is registered and settled via a portable terminal operated by a customer, obtains a settled code from the portable terminal, obtains the amount of settled products associated with the settled code, estimates the amount of products to be taken out by the customer from an image of the customer's luggage as they leave the store, detects an abnormality based on the obtained amount of settled products and the estimated amount of products to be taken out, and outputs a message that the abnormality has been detected.

[0009] A program according to the present disclosure causes a computer to execute a process, when a product is registered and settled via a portable terminal operated by a customer, to acquire a settled code from the portable terminal, acquire the amount of settled products associated with the settled code, estimate the amount of products to be taken out by the customer from an image of the customer's luggage as the customer leaves the store, detect an abnormality based on the acquired amount of settled products and the estimated amount of products to be taken out, and output a notification that the abnormality has been detected. The program may be stored in a computer-readable non-transitory recording medium. [Effects of the Invention]

[0010] According to the present disclosure, when product registration and payment are performed via a portable terminal operated by a customer, it is possible to assist in dealing with abnormalities regarding products taken out by customers as they leave the store. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a block diagram showing an example of the configuration of a store system according to a first embodiment. [Figure 2] FIG. 2 is a schematic diagram showing an example of installation of a reader and a camera. [Figure 3] FIG. 10 is a schematic diagram showing another example of installation of a reader and a camera. [Figure 4] FIG. 2 is a block diagram showing an example of the configuration of a portable terminal 20. [Figure 5] FIG. 10 is a diagram showing an example of a display screen for a payment completion code. [Figure 6] 4 is a flowchart showing an example of operation according to the first embodiment. [Figure 7] FIG. 10 is a block diagram showing an example of the configuration of a store system according to a second embodiment. [Figure 8] 10 is a flowchart showing an operation example according to the second embodiment. [Figure 9] FIG. 10 is a block diagram showing an example of the configuration of a store system according to a third embodiment. [Figure 10] 10 is a flowchart showing an operation example according to the third embodiment. [Figure 11] FIG. 2 is a block diagram illustrating an example of a hardware configuration of a computer. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, with reference to the drawings, embodiments of a store exit management system, a store exit management method, a program, and a recording medium for non-temporarily recording the program according to the present disclosure will be described. The present embodiments do not limit the disclosed technology.

[0013] [First embodiment] The store exit management system 100 according to the first embodiment is a system that manages the amount of merchandise that a customer takes away when leaving a store and the amount of merchandise that the customer has paid for. The store exit management system 100 is used as a store system 1. FIG. 1 is a block diagram showing an example configuration of the store system 1 according to the first embodiment. The store exit management system 100 includes a code acquisition unit 101, a payment merchandise information acquisition unit 102, an estimation unit 103, an abnormality detection unit 104, and an abnormality output unit 105.

[0014] In addition to the store exit management system 100, the store system 1 includes a POS (Point of Sales) server 31, a product database 32, and a sales database 33. The store exit management system 100, the POS server 31, the product database 32, and the sales database 33 are connected to each other so that they can communicate with each other via wire or wirelessly, as appropriate, depending on the specific configuration of each system. The POS server 31 is also connected to a portable terminal 20 so that they can communicate with each other via wire or wirelessly. The portable terminal 20 is a terminal operated by a customer.

[0015] The POS server 31 manages, for example, various information related to products sold in a store. The POS server 31 may be an application server. In addition to the portable terminal 20, the POS server 31 may also manage product registration and payment using terminals other than the portable terminal 20, such as a POS terminal operated by a store clerk. In other words, both an existing POS terminal and the portable terminal 20 may be used in a store to register products and pay for them.

[0016] The POS server 31 also transmits a payment completion code to the portable terminal 20. The transmitted payment completion code is stored in the portable terminal 20. There are no particular limitations on the payment completion code, as long as it indicates that the customer has completed payment. The payment completion code is also associated with information related to the payment. The information related to the payment includes, for example, the number of items paid for, identification information of the items paid for, or the total amount. The payment completion code is also called a store exit code, as it is used when the customer leaves the store.

[0017] For example, a settlement ID (Identifier) ​​may be used as the settlement code. The settlement ID uniquely identifies each settlement. Alternatively, a transaction ID or a customer ID may be used as the settlement code. The transaction ID uniquely identifies each transaction, with the entire process from the start of the product registration process to the completion of settlement being considered as one transaction. Also, the customer ID uniquely identifies the customer. The transaction ID and customer ID can be used as the settlement code, for example, by being linked to information indicating that the settlement has been completed.

[0018] The product database 32 stores product identification information and product information in association with each other for each product. For example, if the product identification information is a product code, the product information is information such as the product name and product price. Note that the product information is not particularly limited.

[0019] The sales database 33 stores, for example, information about sales for each store. The sales database 33 also manages, for example, information about each settlement for each store. The sales database 33 may store information about each settlement in association with a settlement code.

[0020] The store system 1 also includes a reader 41 and a camera 42. Multiple readers 41 and multiple cameras 42 may be provided. The reader 41 and the camera 42 are installed in the store. However, the camera 42 may be realized by a camera provided in the portable terminal 20. The store exit management system 100 is connected to the reader 41 and the camera 42 so that they can communicate with each other via wired or wireless communication.

[0021] The reader 41 reads the payment completion code from the portable terminal 20. The method by which the reader 41 reads the payment completion code is not particularly limited. For example, the reader 41 may read a barcode or two-dimensional code that encodes the payment completion code and is displayed on the display of the portable terminal 20. Alternatively, the reader 41 may receive the payment completion code from the portable terminal 20 via wireless communication such as NFC (Near Field Communication), Wi-Fi (registered trademark), or Bluetooth (registered trademark). The reader 41 transmits the read payment completion code to the code acquisition unit 101 of the store exit management system 100.

[0022] The camera 42 takes an image of the customer's luggage and transmits the captured image to the store exit management system 100.

[0023] The package is at least one object carried by the customer. The camera 42 may photograph the package placed in front of the camera 42. Alternatively, the camera 42 may photograph the package being carried by the customer by photographing the customer holding the package.

[0024] The camera 42 takes an image of, for example, the baggage of a customer leaving the store. In this case, the camera 42 may be installed near the exit of the store.

[0025] 2 is a schematic diagram showing an example of the installation of the reader 41 and the camera 42. The reader 41 is installed, for example, on a counter. The camera 42 is installed, for example, from above the counter so that it can capture an image of a package placed on the counter. For example, before leaving the store, customer C1 places package C2 on the counter and has the reader 41 read the payment completion code displayed on the portable terminal 20. After having the reader 41 read the payment completion code, customer C1 leaves the store.

[0026] FIG. 3 is a schematic diagram showing another example of installation of the reader 41 and the camera 42. The reader 41 is installed, for example, at the exit gate 44. The exit gate 44 is a gate through which customers passing to leave the store. When a payment completion code is presented, the exit gate 44 becomes ready for the customer to pass through. The camera 42 is installed, for example, so that it can photograph customers passing through the exit gate 44. For example, customer C1, carrying luggage C2, has the reader 41 read the payment completion code displayed on the portable terminal 20. After the payment completion code is read, customer C1 passes through the exit gate 44.

[0027] The installation example of the reader 41 and the camera 42 is not limited to the above example. The reader 41 and the camera 42 may be installed in different locations within the store. In other words, the reading of the payment completion code and the capture of the image may be performed in different locations.

[0028] In the first embodiment, a case will be described in which the store system 1 further includes a detected anomaly database 34 and an alarm device 43. The store system 1 does not necessarily have to include the detected anomaly database 34 and the alarm device 43. The store exit management system 100 is communicably connected to the detected anomaly database 34 and the alarm device 43.

[0029] The detected anomaly database 34 stores anomalies detected in the store exit management system 100. The notification device 43 may refer to the detected anomaly database 34 and notify the detected anomaly in response to a request from the store manager.

[0030] The notification device 43 notifies an abnormality detected in the store exit management system 100. The notification device 43 is, for example, a display, a speaker, or a lamp. Multiple notification devices 43 may be installed in the store. The notification device 43 may notify a customer leaving the store of the abnormality. The notification device 43 may also notify a store clerk of the abnormality. In this case, the notification device 43 may be a terminal operated by the store clerk.

[0031] The alarm device 43 may be installed at the counter where the reader 41 is installed. The alarm device 43 may also be installed at an exit gate 44 as shown in FIG. 3. By installing the alarm device 43 near the reader 41, the alarm device 43 can notify customers leaving the store of the abnormality. The alarm device 43 can also notify store staff near the customer leaving the store and other customers of the abnormality.

[0032] The store exit management system 100, POS server 31, and each database related to the store system 1 may be installed, for example, in the back yard of the store, but the installation location is not particularly limited. Some of the devices constituting the store system 1 may be realized by a cloud server.

[0033] Each system, including the store exit management system 100, POS server 31, and each database, related to the store system 1 may be realized by a separate information processing device. Alternatively, the functions of each system may be realized by a single information processing device. For example, each database related to the store system 1 may be included in the information processing device that realizes the POS server 31. Furthermore, each system may be realized by a combination of multiple information processing devices.

[0034] The portable terminal 20 is, for example, a mobile terminal such as a smartphone or a tablet. The type of portable terminal 20 is not particularly limited. The portable terminal 20 may be owned by a customer or rented out by a store. The portable terminal 20 may be attached to a shopping basket or shopping cart, or may be provided integrally with the shopping basket or shopping cart. When the portable terminal 20 is attached to a shopping cart, the shopping cart may be self-propelled by being provided with, for example, moving parts such as tires and a driving device such as a motor.

[0035] 4 is a block diagram showing an example configuration of the portable terminal 20. The portable terminal 20 includes a processing unit 21 and a storage unit 22. An application program may be installed in the storage unit 22 to realize the following functions of the processing unit 21. The processing unit 21 may not have some of the functions described below.

[0036] The configurations of the store system 1 and the portable terminal 20 will be described in more detail below.

[0037] <Enter identification information> The processing unit 21 of the portable terminal 20 accepts the input of the identification information of the product. The POS server 31 may display a screen for accepting the input of the identification information.

[0038] The processing unit 21 accepts input of product identification information, for example, via a camera (not shown). The camera is not particularly limited, but may be, for example, a camera provided in the portable terminal 20. The processing unit 21 may accept input of the identification information by reading a one-dimensional code or two-dimensional code of the product photographed by an imaging device. Alternatively, the processing unit 21 may accept input of the identification information by identifying the product from an image of the product photographed by the imaging device using image recognition technology.

[0039] Furthermore, the processing unit 21 may acquire the identification information by reading a tag to which an RFID (Radio Frequency Identifier) ​​is attached via a reading device. When an RFID is used, the reading device is an RFID reader.

[0040] Furthermore, the processing unit 21 may accept product identification information by accepting a product selection from the product search results. For example, the method of product search is not particularly limited. Product search may be performed using text or images. For example, the processing unit 21 may accept input of product identification information by accepting a product selection from a displayed product image.

[0041] Furthermore, the processing unit 21 may receive input of identification information by receiving input of a number such as a product code via an input device (not shown). Examples of the input device include a touch panel display and a keyboard that can input numbers and letters.

[0042] The processing unit 21 of the portable terminal 20 transmits the product identification information received as described above to the POS server 31. For the product whose identification information has been transmitted and registered, the processing unit 21 may display product information in accordance with the control of the POS server 31.

[0043] <Product registration> The POS server 31 registers the product based on the identification information received from the portable terminal 20. The POS server 31 references the product database 32 to acquire product information. The POS server 31 then registers the product purchased by the customer using the acquired product information. As part of the product registration process, the POS server 31 generates, for example, a registered product list for each customer.

[0044] <Settlement processing> Furthermore, the processing unit 21 of the portable terminal 20 executes processing related to the settlement of the registered products. Below, we will explain the cases where the settlement is performed via a payment server and where the settlement is performed using a settlement device installed in the store. However, the settlement method is not limited to these.

[0045] When settlement is performed via a settlement server, the processing unit 21 of the portable terminal 20 may communicate with the settlement server (not shown) and instruct the settlement server to perform settlement as part of the settlement process. The processing unit 21, for example, sends a settlement instruction including payment information and settlement information to the settlement server. The payment information is information that identifies the customer's credit card, debit card, electronic money, points, etc. The settlement information is information that identifies the registered product or the purchase amount of the registered product. The settlement server executes cashless payment using the payment information and settlement information. For example, the payment server notifies the portable terminal 20 and the POS server 31 that settlement has been completed.

[0046] The processing unit 21 of the portable terminal 20 may instruct the payment server to settle the account via the POS server 31 or the store exit management system 100. In addition, some or all of the functions of the payment server may be realized by the store exit management system 100 or the POS server 31.

[0047] Alternatively, when settlement is performed using a settlement device installed in the store, the processing unit 21 of the portable terminal 20 may output information identifying settlement information to the settlement device as a process related to settlement. The information identifying the settlement information is, for example, stored in association with the settlement information by the POS server 31. The settlement device acquires the settlement information from, for example, the POS server 31 based on the output information identifying the settlement information. Based on the acquired settlement information, the settlement device executes settlement processing by any method, such as cash payment or cashless payment. For example, the settlement device notifies the portable terminal 20 and the POS server 31 that settlement is complete.

[0048] The method of outputting the information specifying the settlement information to the settlement device is not particularly limited. For example, the processing unit 21 may display a barcode or two-dimensional code that encodes the information specifying the settlement information on the display of the portable terminal 20. The settlement device reads the information specifying the settlement information displayed on the display. Alternatively, the processing unit 21 may transmit the information specifying the settlement information to the settlement device via wireless communication such as NFC (Near Field Communication), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0049] The processing unit 21 may output the settlement information to the settlement device as part of the settlement process. In this case, the settlement device acquires the settlement information from the portable terminal 20.

[0050] The POS server 31 stores information about the settlement in the sales database 33. After the settlement is completed, the POS server 31 transmits a settlement completion code to the portable terminal 20.

[0051] <Anomaly detection> When product registration and payment are performed via the portable terminal 20, the code acquisition unit 101 of the store exit management system 100 acquires a payment completion code from the portable terminal 20.

[0052] The code acquisition unit 101 of the store exit management system 100 acquires the payment completion code from the portable terminal 20 via the reader 41. In this case, the processing unit 21 of the portable terminal 20 first outputs the payment completion code.

[0053] For example, the processing unit 21 transmits a payment completion code to the reader 41 by wireless communication.

[0054] Alternatively, the processing unit 21 displays a barcode or two-dimensional code that encodes the payment completion code on the display of the portable terminal 20. FIG. 5 is a diagram showing an example of a display screen of the payment completion code displayed on the portable terminal 20. The display screen in FIG. 5 displays a two-dimensional code that encodes the payment completion code. Before leaving the store, the customer has the reader 41 read the barcode displayed on the portable terminal 20.

[0055] Then, the code acquisition unit 101 acquires the read settlement code from the reader 41.

[0056] The payment product information acquisition unit 102 acquires the amount of the paid product associated with the payment code. For example, the payment product information acquisition unit 102 refers to the sales database 33 and acquires the amount of the paid product associated with the payment code.

[0057] The amount of paid-for products may be the number of paid-for products. Alternatively, the amount of paid-for products may be the volume of the paid-for products. For example, the paid-for product information acquisition unit 102 may estimate the volume of the products from the number of paid-for products. Alternatively, the paid-for product information acquisition unit 102 may refer to the volume of the products stored in the product database 32 and acquire the volume of the paid-for products.

[0058] The estimation unit 103 estimates the amount of merchandise to be taken out by a customer from an image of the customer's baggage as he or she leaves the store. The amount of merchandise to be taken out may be the number of merchandise to be taken out. Alternatively, the amount of merchandise to be taken out may be the volume of the merchandise to be taken out.

[0059] The following describes a method for estimating the amount of goods being taken out.

[0060] The estimation unit 103 acquires, for example, an image of a customer's luggage as they leave the store from the camera 42. The estimation unit 103 recognizes the luggage from the acquired image. The estimation unit 103 may recognize merchandise as luggage. The estimation unit 103 may recognize merchandise in a container from an image of the inside of the container. The container for the merchandise is, for example, a bag, a paper bag, a plastic bag, a cardboard box, or a shopping basket. The estimation unit 103 may estimate the amount of merchandise that has been recognized as the amount of merchandise to be taken out.

[0061] When products are placed in a container, it may be difficult to recognize all of the products. For example, if an image of a customer's luggage is taken from the outside of the container, the products inside the container may not be visible. Therefore, the estimation unit 103 may recognize the container as luggage. The estimation unit 103 may then recognize the volume of the container using existing technology related to image-based size measurement. Furthermore, when a customer places products in a container with a known volume, the estimation unit 103 may recognize the volume of the container by recognizing the type of container. The estimation unit 103 may estimate the volume of the container thus recognized as the amount of products to be taken out.

[0062] For example, a plastic bag may have a larger volume the more products it contains, so the volume of the plastic bag can be estimated as the volume of the products contained in the bag.

[0063] Furthermore, there are cases where a customer does not fill a container with products to the brim, so the estimation unit 103 may estimate the amount of products in the container as the amount of products to be taken out.

[0064] The estimation unit 103 may estimate the amount of goods in a container using the proportion of the container's volume that the goods occupy. The estimation unit 103 may recognize the volume of the container using existing technology related to image-based size measurement. Furthermore, when a customer puts goods into a container with a known volume, the estimation unit 103 may recognize the type of container to recognize the volume of the container. Then, for example, the estimation unit 103 recognizes the position of the topmost product in the container from the image. Furthermore, the estimation unit 103 recognizes the bottom and top of the container from the image. Then, the estimation unit 103 calculates the proportion of the container's volume that the goods occupy by dividing the height from the bottom of the container to the position of the topmost product by the height of the container. The estimation unit 103 may estimate the volume obtained by multiplying the container's volume by the proportion of the goods occupying it as the amount of goods in the container.

[0065] The estimation unit 103 may also estimate the amount of products in a container using the amount of products in the top layer of the container and the number of product layers in the container. For example, the estimation unit 103 recognizes the product in the top layer of the container from an image taken from above of the inside of the container. The estimation unit 103 also recognizes the position of the product in the top layer of the container and the position of the bottom of the container from the image. Furthermore, the estimation unit 103 estimates the number of product layers in the container from the height from the bottom of the container to the product in the top layer. The estimation unit 103 may then estimate the value obtained by multiplying the amount of product in the surface layer by the estimated number of product layers as the total amount of products in the container.

[0066] For example, a customer may place paid items in a shopping cart and take the items out of a store. If the volume of the shopping cart is known, the estimation unit 103 can estimate the volume of the items in the cart based on the position of the topmost item in the shopping cart.

[0067] The estimation unit 103 may estimate the number of products to be taken out from the volume of the products to be taken out. For example, the estimation unit 103 may estimate the number of products to be taken out by dividing the result of estimating the volume of the products to be taken out by the average volume of the products.

[0068] Furthermore, the estimation unit 103 may estimate the amount of merchandise to be taken out from an image of the customer's baggage as he or she leaves the store and an image of the customer's baggage before he or she picks up the merchandise to be taken out.

[0069] The camera 42, for example, captures an image of a customer's luggage before the customer picks up the product to be taken out. The camera 42 capturing the image before the customer picks up the product may be installed, for example, near the entrance of the store. The estimation unit 103 acquires the image from the camera 42.

[0070] For example, the estimation unit 103 recognizes the customer's luggage for each image and estimates the volume of the recognized luggage. The estimation unit 103 may estimate the volume obtained by subtracting the volume of luggage recognized for the customer before picking up the products from the volume of luggage recognized for the customer leaving the store as the amount of products to be taken out.

[0071] As an example, a customer may put paid-for items into a bag that they brought with them. The bag may contain items that the customer brings into the store from outside. For example, the camera 42 may take an image of the bag near the entrance before the customer puts items into it and send it to the estimation unit 103. The camera 42 may also take an image of the bag near the exit after the customer puts items into it and send it to the estimation unit 103. The estimation unit 103 may then calculate the volume of the bag from each image and estimate the amount of items being taken out.

[0072] For example, a customer may register which bag the customer will put the items in before carrying them in the store. A customer may also register the bag by having the camera 42 take a picture of the bag they have brought with them.

[0073] The estimation unit 103 may also perform object recognition on each piece of luggage included in the image. Then, for example, the estimation unit 103 may recognize, as the product to be taken out, objects recognized in the image of the luggage of a customer leaving the store, excluding objects recognized in the image of the luggage before the customer picked up the product in the store.

[0074] The above describes a method for estimating the amount of goods to be taken out. The above-described methods may be used in combination. Furthermore, the method for estimating the amount of goods to be taken out is not limited to the above example.

[0075] The recognition of the customer's luggage may be performed by the camera 42. The camera 42 may transmit the recognition result to the store exit management system 100. The estimation unit 103 acquires the recognition result of the customer's luggage from the camera 42.

[0076] The anomaly detection unit 104 detects an anomaly based on the amount of paid-for products acquired by the paid-for product information acquisition unit 102 and the amount of products being taken out estimated by the estimation unit 103. An anomaly occurs when there is a difference between the amount of paid-for products and the amount of products being taken out. For example, if the amount of products being taken out is greater than the amount of paid-for products, there is a possibility that the customer has fraudulently taken out unpaid products. Furthermore, if the amount of products being taken out is less than the amount of paid-for products, there is a possibility that the customer has left paid-for products behind in the store. Therefore, the anomaly detection unit 104 detects an anomaly based on the difference between the amount of paid-for products and the amount of products being taken out.

[0077] The anomaly detection unit 104 may detect an anomaly when the amount of goods to be taken out estimated by the estimation unit 103 is greater than the amount of paid goods acquired by the paid goods information acquisition unit 102. In other words, the anomaly detection unit 104 may detect an anomaly when it is estimated that there are goods to be taken out that have not been paid for.

[0078] Furthermore, the anomaly detection unit 104 may detect an anomaly when the amount of merchandise to be taken out estimated by the estimation unit 103 is less than the amount of merchandise that has been paid for and acquired by the payment merchandise information acquisition unit 102. That is, the anomaly detection unit 104 may detect an anomaly when, for example, it is estimated that some merchandise has been left behind in the store despite the payment having been made.

[0079] When the anomaly detection unit 104 detects an anomaly, a threshold may be set for the difference between the amount of merchandise paid for and the amount of merchandise taken out. For example, the anomaly detection unit 104 may detect an anomaly when the difference between the amount of merchandise taken out and the amount of merchandise paid for exceeds the threshold. Setting a threshold can reduce the number of anomalies detected.

[0080] The threshold may be set depending on the accuracy of the estimation of the amount of goods to be taken out. If the difference between the amount of goods to be taken out and the amount of goods paid for is small, the anomaly detection unit 104 may erroneously detect an anomaly. Therefore, for example, if the estimation accuracy of the estimation unit 103 is low, the threshold may be set high. If the difference does not exceed the threshold, the anomaly detection unit 104 may determine that there is an error in the estimation.

[0081] The threshold value may also be set according to the products that the customer has paid for. Depending on the product, it may be difficult to recognize the volume from an image, which may reduce the estimation accuracy by the estimation unit 103. Therefore, if the products that the customer has paid for include a product whose volume is difficult to recognize from an image, the threshold value may be set high.

[0082] The threshold may also be set according to the amount of merchandise paid for by a customer. For example, the threshold may be set low when the payment amount is high. This allows the anomaly detection unit 104 to detect an anomaly for customers who pay a high amount, even if the difference in the amount of merchandise is small. Therefore, if a customer who made a high-priced purchase may have left a high-priced item in the store after paying for it, the anomaly detection unit 104 can appropriately detect an anomaly. Alternatively, the threshold may be set low when the payment amount is low. This allows for appropriate detection of an anomaly for customers who may have lowered their payment amount by paying for an amount of merchandise that is less than the amount they actually took out, even if the difference in the amount of merchandise is small.

[0083] The threshold may also be set according to the degree of congestion. The degree of congestion may be the real-time congestion level or the predicted congestion level for each time period. For example, when the store is crowded, the threshold may be set high. This makes it difficult to detect abnormalities, thereby preventing further congestion due to the confirmation of abnormalities. Alternatively, the threshold may be set low when the store is crowded. This makes it easy to detect abnormalities, thereby preventing products that have not yet been paid for from being taken out due to the congestion.

[0084] The threshold may also be set depending on the customer. For example, the threshold may be set lower for a customer for whom anomalies have been detected in the past. This allows for anomalies to be detected for customers who are more likely to commit fraud.

[0085] The abnormality detection unit 104 may notify the abnormality output unit 105 of the difference between the amount of merchandise already paid for and the estimated amount of merchandise to be taken out as the degree of abnormality.

[0086] <Abnormality Notification> The abnormality output unit 105 outputs that an abnormality has been detected. For example, the abnormality output unit 105 may output that an abnormality has been detected to the alarm device 43, causing the alarm device 43 to notify the abnormality. Alternatively, the abnormality output unit 105 may output that an abnormality has been detected to the detected abnormality database 34, causing the detected abnormality to be stored in the database.

[0087] By storing the detected anomaly in the detected anomaly database 34, the store manager can check for the anomaly even after the customer has left the store. The manager can then carefully observe the customer the next time they visit the store.

[0088] The abnormality output unit 105 may cause the alarm device 43 to output an alarm sound or an alarm display.

[0089] Alternatively, the abnormality output unit 105 may detect an abnormality and cause the alarm device 43 to notify the number of paid items by voice or display. In other words, if no abnormality is detected, the abnormality output unit 105 does not notify the number of paid items. Therefore, the store clerk or customer can determine whether an abnormality has been detected based on whether or not the number of paid items is notified. The abnormality output unit 105 may obtain the number of paid items from, for example, the abnormality detection unit 104, the paid item information acquisition unit 102, or the sales database 33. Note that other information related to the paid items may be used instead of the number of paid items.

[0090] The abnormality output unit 105 may change the output notification device 43 based on the difference between the amount of goods being taken out and the amount of goods that has been paid for. For example, if the amount of goods being taken out is greater than the amount of goods that has been paid for, the abnormality output unit 105 may cause the notification device 43 for the store clerk to notify that an abnormality has been detected. This allows, for example, the store clerk to check the customer's belongings to see if any goods have been taken out fraudulently. Also, if the amount of goods being taken out is less than the amount of goods that has been paid for, the abnormality output unit 105 may cause the notification device 43 for the customer to notify that an abnormality has been detected. This allows, for example, the customer to check to see if any goods have been left behind in the store.

[0091] The abnormality output unit 105 may output that an abnormality has been detected, depending on the degree of the detected abnormality.

[0092] As an example, the abnormality output unit 105 may change the level of notification in the notification device 43 depending on the difference between the amount of merchandise that has been paid for and the estimated amount of merchandise to be taken out. For example, when the difference is large, the abnormality output unit 105 causes the notification device 43 to issue a higher level of notification than when the difference is small. A higher level of notification is more likely to attract people's attention.

[0093] If the difference is small, it is possible that an abnormality has been falsely detected, so a low-level alert may be issued. If the difference is large, it is highly likely that an abnormality has occurred, so a high-level alert may be issued. Furthermore, if the difference is large, it is possible that more items have been taken out without being paid for, or that more paid-for items have been left behind in the store. Therefore, a high-level alert may be issued, which conveys a greater degree of urgency.

[0094] When the notification device 43 is a speaker, the abnormality output unit 105 may, for example, cause the speaker to output a short notification sound when the difference is small. When the difference is large, the abnormality output unit 105 may, for example, cause the speaker to output an alarm sound. When the notification device 43 is a display, the abnormality output unit 105 may, for example, cause the abnormality to be displayed in yellow when the difference is small and in red when the difference is large.

[0095] Next, we will explain the operation of the store exit management system 100. Figure 6 is a flowchart showing an example of the operation of the store exit management system 100 according to the first embodiment. For example, the store exit management system 100 may start the process of Figure 6 in response to the reader 41 reading the settled code.

[0096] The code acquisition unit 101 of the store exit management system 100 acquires a payment completion code from the portable terminal 20 via the reader 41 (step S1). The code acquisition unit 101 notifies the payment completion code acquired to the payment product information acquisition unit 102.

[0097] The paid product information acquisition unit 102 acquires the paid product quantity associated with the payment completion code (step S2). The paid product information acquisition unit 102 notifies the abnormality detection unit 104 of the acquired paid product quantity.

[0098] The estimation unit 103 estimates the amount of merchandise to be taken away by the customer from an image of the customer's baggage as he or she leaves the store (step S3). The estimation unit 103 notifies the anomaly detection unit 104 of the estimated amount of merchandise to be taken away by the customer.

[0099] The abnormality detection unit 104 detects an abnormality based on the amount of merchandise paid for and the estimated amount of merchandise to be taken out (step S4). The abnormality detection unit 104 notifies the abnormality output unit 105 of the detected abnormality. If no abnormality is detected, the store exit management system 100 ends operation.

[0100] The abnormality output unit 105 outputs that an abnormality has been detected (step S5). For example, when the abnormality is output, the store exit management system 100 ends its operation. Note that the store exit management system 100 repeatedly executes the above-described series of operations, for example, for each customer as appropriate.

[0101] According to the first embodiment, when product registration and payment are performed via the portable terminal 20, the code acquisition unit 101 acquires a payment completion code from the portable terminal 20. Then, the payment product information acquisition unit 102 acquires the amount of paid products associated with the payment completion code. The estimation unit 103 estimates the amount of products to be taken away by the customer from an image of the customer's luggage as they leave the store. Furthermore, the anomaly detection unit 104 detects an abnormality based on the amount of paid products and the estimated amount of products to be taken away. Then, the anomaly output unit 105 outputs a signal that an abnormality has been detected. Therefore, according to the first embodiment, when product registration and payment are performed via a portable terminal operated by the customer, it is possible to support the handling of abnormalities related to the products taken away by the customer as they leave the store.

[0102] [Second embodiment] Next, as a second embodiment, another example of a store system 1 equipped with a store exit management system 100 will be described. The configuration of the second embodiment is similar to part of the configuration of the first embodiment. Therefore, a description of the similar configuration will be omitted. Figure 7 is a block diagram showing an example of the configuration of the store system 1 according to the second embodiment.

[0103] The store exit management system 100 according to the second embodiment differs from the store exit management system 100 according to the first embodiment in that it includes a gate control unit 106.

[0104] In the second embodiment, the store system 1 further includes an exit gate 44. The store exit management system 100 is connected to the exit gate 44 so that it can communicate with the store.

[0105] An example of the installation of the exit gate 44 is as shown in Fig. 3. However, the installation example of the exit gate 44 is not limited to this example.

[0106] The gate control unit 106 controls whether the exit gate 44 is in a passable state or a passable state. For example, the gate control unit 106 may control the opening and closing of the exit gate 44. The gate control unit 106 may open the exit gate 44 to allow customers to pass through, or close the exit gate 44 to prevent customers from passing through.

[0107] When an abnormality is detected, the gate control unit 106 may make the exit gate 44 passable. For example, the abnormality output unit 105 outputs a signal that an abnormality has been detected to the gate control unit 106. When the gate control unit 106 receives an output from the abnormality output unit 105 indicating that an abnormality has been detected, it makes the exit gate 44 passable.

[0108] Next, the operation of the store exit management system 100 according to the second embodiment will be described. Fig. 8 is a flowchart showing an example of the operation of the store exit management system 100 according to the second embodiment. For example, the store exit management system 100 may start the process of Fig. 8 in response to the reader 41 installed at the store exit gate 44 reading the settled code.

[0109] The code acquisition unit 101 of the store exit management system 100 acquires the payment completion code from the portable terminal 20 via the reader 41 (step S11). The code acquisition unit 101 notifies the payment completion code acquired to the payment product information acquisition unit 102.

[0110] The paid product information acquisition unit 102 acquires the paid product quantity associated with the paid product code (step S12). The paid product information acquisition unit 102 notifies the abnormality detection unit 104 of the acquired paid product quantity.

[0111] The estimation unit 103 estimates the amount of merchandise to be taken away by the customer from an image of the customer's baggage as he or she leaves the store (step S13). The estimation unit 103 notifies the anomaly detection unit 104 of the estimated amount of merchandise to be taken away by the customer.

[0112] The abnormality detection unit 104 detects an abnormality based on the amount of merchandise that has been paid for and the estimated amount of merchandise to be taken out. If the abnormality detection unit 104 detects an abnormality (step S14; Yes), it notifies the abnormality output unit 105 of the detected abnormality.

[0113] The abnormality output unit 105 outputs the detection of an abnormality to the gate control unit 106 (step S15). In response to the output indicating the detection of an abnormality, the gate control unit 106 makes the exit gate 44 passable (step S16).

[0114] If the abnormality detection unit 104 does not detect an abnormality (step S14; No), it outputs a signal indicating that there is no abnormality to the gate control unit 106. In response to the signal indicating that there is no abnormality, the gate control unit 106 sets the exit gate 44 to a state in which it is possible to pass through (step S17).

[0115] For example, the store exit management system 100 ends its operation due to gate control by the gate control unit 106.

[0116] According to the second embodiment, if an abnormality is detected, the gate control unit 106 makes the exit gate 44 non-passable. Therefore, according to the second embodiment, if there is an abnormality with the product being taken out, control can be performed to prevent the customer from leaving the store.

[0117] [Third embodiment] Next, as a third embodiment, a further example of a store system 1 equipped with a store exit management system 100 will be described. The configuration of the third embodiment is similar to part of the configuration of the first embodiment. Therefore, a description of the similar configuration will be omitted. Figure 9 is a block diagram showing an example of the configuration of the store system 1 according to the third embodiment.

[0118] The store exit management system 100 according to the third embodiment differs from the store exit management system 100 according to the first embodiment in that it includes a payment product information output unit 107. The store system 1 according to the third embodiment also includes an alarm device 43.

[0119] In response to obtaining the payment completion code, the payment product information output unit 107 outputs the number of payment completion products to the notification device 43. In the third embodiment, the payment product information output unit 107 causes the notification device 43 to notify the number of payment completion products regardless of whether an abnormality is detected or not.

[0120] The paid product information output unit 107 acquires the number of paid products, for example, from the paid product information acquisition unit 102. Then, the paid product information output unit 107 outputs the acquired number of paid products to the notification device 43.

[0121] In the third embodiment, the notification device 43 notifies the customer of the number of items for which payment has been made. The notification device 43 is, for example, a display or a speaker. As in the first embodiment, the notification device 43 also notifies the customer of any abnormalities detected in the store exit management system 100.

[0122] The notification device 43 may notify the customer of the number of items that have been paid for. This allows a customer leaving the store to know that the number of items that have been paid for is known. This is expected to prevent customers from leaving the store without paying for some of the items.

[0123] The notification device 43 may also notify the store clerk of the number of items that have been paid for. This allows the store clerk to check the amount of items taken out by the customer as they leave the store to see if there is a difference between the number of items that have been paid for and the amount of items that have been taken out. In other words, this allows for a double check: the store exit management system 100 detects an abnormality, and the store clerk confirms the abnormality.

[0124] Instead of the number of paid items, other information regarding the paid items may be used and reported.

[0125] 10 is a flowchart showing an example of the operation of the store exit management system 100 according to the third embodiment. For example, the store exit management system 100 may start the process of FIG. 10 in response to the reader 41 reading the payment completion code.

[0126] The flowchart in Fig. 10 includes the basic operations of the store exit management system 100 in the flowchart in Fig. 6 according to the first embodiment. Therefore, only the parts of the flowchart in Fig. 10 that differ from the flowchart in Fig. 6 will be described.

[0127] In step S2, for example, the paid product information acquisition unit 102 notifies the paid product information output unit 107 of the number of paid products out of the amount of paid products.

[0128] After step S2, the paid product information output unit 107 outputs the number of paid products to the notification device 43 (step S22). The notification device 43 receives the output and notifies the number of paid products.

[0129] The operation after step S22 is as described with reference to Fig. 6. If no abnormality is detected (step S4; No), the store exit management system 100 ends the operation.

[0130] The timing of execution of step S22 is not particularly limited as long as it is performed after step S2. For example, step S22 may be performed after an abnormality is detected in step S4. In this case, the payment product information output unit 107 may obtain the number of payment-completed products from the abnormality detection unit 104.

[0131] Furthermore, there is no particular limitation on the timing at which the notification device 43 notifies the customer of the number of paid-for products. For example, the notification device 43 may notify the customer of the number of paid-for products at the same time as notifying the customer that an abnormality has been detected.

[0132] According to the third embodiment, the paid-for product information output unit 107 outputs the number of paid-for products to the notification device 43 in response to obtaining the payment completion code. The notification device 43 then notifies the number of paid-for products. Therefore, according to the third embodiment, for example, abnormalities regarding products taken out by customers leaving the store can be prevented. Furthermore, according to the third embodiment, for example, it is possible to assist store staff in checking whether there is a difference between the number of paid-for products and the amount of products taken out by customers leaving the store.

[0133] [Variations] The above embodiments may be modified. The store exit management system 100 is applicable to both a single customer and a group of multiple customers. When there is a group of customers, each customer belonging to the group is associated with a single payment code. The estimation unit 103 can then estimate the total amount of merchandise taken out by the group from images of each customer's belongings. According to this modification, even when a representative of the group settles the bill and multiple customers cooperate to take out merchandise, it is possible to detect abnormalities in the merchandise being taken out.

[0134] [Hardware configuration] In each of the above-described embodiments, each component of the store exit management system 100 represents a functional block. Some or all of the components of the store exit management system 100 may be realized by any combination of a computer 500 and a program.

[0135] Fig. 11 is a block diagram showing an example of the hardware configuration of a computer 500. Referring to Fig. 11, the computer 500 includes, for example, a processor 501, a read only memory (ROM) 502, a random access memory (RAM) 503, a program 504, a storage device 505, a drive device 507, a communication interface 508, an input device 509, an input / output interface 511, and a bus 512.

[0136] The program 504 includes instructions for realizing each function of the store exit management system 100. The program 504 is stored in advance in the ROM 502, RAM 503, and storage device 505. The processor 501 executes the instructions included in the program 504 to realize each function of the store exit management system 100. The RAM 503 may also store data processed in each function of the store exit management system 100. There is no particular limitation on the number of ROM (Read Only Memory) 502, RAM (Random Access Memory) 503, and storage device 505, and they may be one or more.

[0137] The processor 501 controls the entire computer 500. The processor 501 may be, for example, a CPU (Central Processing Unit). The number of processors 501 is not particularly limited, and there may be one or more processors 501.

[0138] The drive device 507 reads and writes data from and to the recording medium 506. The communication interface 508 provides an interface with a communication network. The input device 509 is, for example, a mouse or keyboard, and receives information input from a store manager or the like. The output device 510 is, for example, a display, and outputs (displays) information to a store manager or the like. The input / output interface 511 provides an interface with peripheral devices. The bus 512 connects these hardware components. The program 504 may be supplied to the CPU 501 via a communication network, or may be stored in advance on the recording medium 506, read by the drive device 507, and supplied to the CPU 501.

[0139] It should be noted that the hardware configuration shown in FIG. 11 is an example, and other components may be added, or some components may not be included.

[0140] There are various variations in the method of realizing the store exit management system 100. For example, the store exit management system 100 may be realized by any combination of different computers and programs for each component. Furthermore, multiple components included in the store exit management system 100 may be realized by any combination of a single computer and program.

[0141] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, the configurations in the respective embodiments can be combined with each other without departing from the scope of the present disclosure.

[0142] Some or all of the above embodiments may be described as, but are not limited to, the following supplementary notes.

[0143] [Appendix 1] a code acquisition means for acquiring a payment completion code from a portable terminal operated by a customer when the product is registered and payment is made via the portable terminal; A payment product information acquisition means for acquiring the amount of the payment product associated with the payment completion code; an estimation means for estimating the amount of merchandise to be taken away by the customer from an image of the customer's baggage as the customer leaves the store; an anomaly detection means for detecting an anomaly based on the acquired amount of the paid-for goods and the estimated amount of the goods to be taken out; an abnormality output means for outputting the detection of the abnormality; A store exit management system equipped with the above.

[0144] [Appendix 2] The abnormality detection means detects the abnormality based on a difference between the estimated amount of merchandise to be taken out and the amount of merchandise that has already been paid for. The store exit management system described in Appendix 1.

[0145] [Appendix 3] The abnormality detection means detects the abnormality when a difference between the estimated amount of merchandise to be taken out and the amount of merchandise already paid for exceeds a threshold value. The store exit management system described in Appendix 2.

[0146] [Appendix 4] The abnormality detection means detects the abnormality when the estimated amount of merchandise to be taken out is greater than the amount of merchandise that has already been paid for. 4. A store exit management system according to any one of appendices 1 to 3.

[0147] [Appendix 5] The abnormality detection means detects the abnormality when the estimated amount of merchandise to be taken out is less than the amount of merchandise that has already been paid for. 5. A store exit management system according to any one of appendices 1 to 4.

[0148] [Appendix 6] The amount of the purchased product is the volume of the purchased product, The amount of the product to be taken out is the volume of the product to be taken out. 6. A store exit management system according to any one of appendices 1 to 5.

[0149] [Appendix 7] The amount of the checked-out items is the number of the checked-out items; The quantity of goods to be removed is the number of goods to be removed. 7. A store exit management system according to any one of appendices 1 to 6.

[0150] [Appendix 8] The estimation means estimates the amount of the goods to be taken out from the image of the baggage of the customer leaving the store and an image of the baggage of the customer before picking up the goods to be taken out. 8. A store exit management system according to any one of appendices 1 to 7.

[0151] [Appendix 9] Further provided is a gate control means for controlling an exit gate through which the customer exiting the store passes, The gate control means makes the exit gate unpassable when the abnormality is detected. 9. A store exit management system according to any one of appendices 1 to 8.

[0152] [Appendix 10] a payment product information output means for outputting the number of the paid products to a notification device in response to acquisition of the payment completion code; 10. The store exit management system according to any one of appendices 1 to 9, further comprising:

[0153] [Appendix 11] When product registration and payment are performed via a portable terminal operated by a customer, a payment completion code is obtained from the portable terminal; Obtaining a quantity of the purchased product associated with the purchased code; Estimating the amount of merchandise taken away by the customer from an image of the customer's baggage as the customer leaves the store; Detecting an abnormality based on the acquired amount of the paid-for goods and the estimated amount of the goods to be taken out; Output that the abnormality has been detected Exit management methods.

[0154] [Appendix 12] When product registration and payment are performed via a portable terminal operated by a customer, a payment completion code is obtained from the portable terminal; Obtaining a quantity of the purchased product associated with the purchased code; Estimating the amount of merchandise taken away by the customer from an image of the customer's baggage as the customer leaves the store; Detecting an abnormality based on the acquired amount of the paid-for goods and the estimated amount of the goods to be taken out; Output that the abnormality has been detected A recording medium that non-temporarily records a program that causes a computer to execute a process. [Explanation of symbols]

[0155] 1. Store system 100 Exit Management System 101 Code Acquisition Department 102 Settlement product information acquisition department 103 Estimation part 104 Abnormality detection unit 105 Abnormal output section 106 Gate control section 107 Checkout product information output section 20 Portable terminal 21 Processing section 22 Memory section 31 POS Server 32 Product Database 33 Sales Database 34 Detected Anomaly Database 41 Leader 42 Camera 43 Alarm device 44 Exit Gate C1 Customer C2 Luggage

Claims

1. a code acquisition means for acquiring a code from a portable terminal operated by a customer when the product is registered and settled via the portable terminal; a payment product information acquisition means for acquiring information about a payment product associated with the code; an estimation means for estimating information about products in the shopping cart from an image of the shopping cart of the customer who is leaving the store taken from above; an anomaly detection means for detecting an anomaly based on information about the paid-for items and information about the items in the shopping cart; A store exit management system equipped with the above.

2. the payment product information acquisition means acquires at least one of the number of products and the volume of the products as information about the payment product; The estimation means estimates at least one of the number of products and the volume of the products as information about the products in the shopping cart. The store exit management system according to claim 1 .

3. The abnormality detection means detects the abnormality when a difference between the quantity of products indicated by the information about the products in the shopping cart and the quantity of products indicated by the information about the paid-for products exceeds a threshold. The store exit management system according to claim 1 or 2.

4. The abnormality detection means detects the abnormality when the quantity of products indicated by the information about the products in the shopping cart is greater than the quantity of products indicated by the information about the purchased products. The store exit management system according to any one of claims 1 to 3.

5. The abnormality detection means detects the abnormality when the amount of products indicated by the information about the products in the shopping cart is less than the amount of the purchased products. The store exit management system according to any one of claims 1 to 4.

6. The estimation means estimates information about the products in the shopping cart using a ratio of the product volume to the volume of the shopping cart. The store exit management system according to any one of claims 1 to 5.

7. The estimation means estimates information about the products in the shopping cart based on the position of the product in the top layer of the shopping cart. The store exit management system according to any one of claims 1 to 6.

8. Further provided is a gate control means for controlling an exit gate through which the customer exiting the store passes, The gate control means makes the exit gate unpassable when the abnormality is detected. The store exit management system according to any one of claims 1 to 7.

9. When product registration and payment are performed via a portable terminal operated by a customer, a code is obtained from the portable terminal; Obtaining information about a purchased item associated with the code; Inferring information about the products in the shopping cart from an image of the shopping cart of the customer leaving the store; Detecting an abnormality based on the information about the paid items and the information about the items in the shopping cart Exit management methods.

10. When product registration and payment are performed via a portable terminal operated by a customer, a code is obtained from the portable terminal; Obtaining information about a purchased item associated with the code; Inferring information about the products in the shopping cart from an image of the shopping cart of the customer leaving the store; Detecting an abnormality based on the information about the paid items and the information about the items in the shopping cart A program that causes a computer to perform a process.

Images