System, article registration monitoring method, and program

Abstract

To provide a system, etc., that can reliably prevent omissions of article registration by customers.SOLUTION: A system includes: prescribed article detection unit 112 that detects the presence of prescribed articles other than those placed in a shopping basket in a store; an action detection unit 113 that detects prescribed article registration behavior using images of article registration work by a person registering articles; and an output unit 114 that outputs an alert when the presence of the prescribed articles is detected by the prescribed article detection unit 112 and the prescribed article registration behavior for the prescribed articles is not detected by the action detection unit 113.SELECTED DRAWING: Figure 8

Description

[Technical field]

[0001] The present invention relates to a system, a method, and a program for monitoring product registration. [Background technology]

[0002] 2. Description of the Related Art Conventionally, techniques have been proposed for self-checkout registers that automatically perform checkout processes, allowing the process to be carried out smoothly even when a shopping basket contains an item that is too large to fit inside the basket. For example, the accounting machine described in Patent Document 1 includes a large item detection unit that detects that the item is a large item that does not fit into a container, and an alert output unit that outputs an alert when the large item is detected. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-105936 Summary of the Invention [Problem to be solved by the invention]

[0004] The system described in Patent Document 1 is intended to prevent overlooking registration of large items that are too large to fit in a shopping cart, and does not take into consideration preventing overlooking of other items, such as items that are not large but are placed on the bottom shelf of a cart or items that are held by a customer. The present invention aims to provide a system etc. that can prevent customers from failing to register products with a high degree of accuracy. [Means for solving the problem]

[0005] The present invention, which was completed with the above-mentioned objective in mind, is a system comprising a first detection unit that detects the presence of a specified product other than products placed in containers pre-placed in a store, a second detection unit that detects a specified product registration action using images captured of a product registrant performing a product registration operation, and an output unit that outputs an alert when the first detection unit detects the presence of the specified product and the second detection unit does not detect the specified product registration action for the specified product. Here, the first detection unit may detect the presence of the predetermined product by using an image captured of the product registrant. Alternatively, the first detection unit may detect the presence of the specified product when the product registrant inputs information indicating that the specified product exists. The predetermined product may also be a product placed on the lower level of a cart that carries the container. Alternatively, the specified product may be a product held by the product registrant. The second detection unit may detect the predetermined product registration behavior based on a skeleton recognition process of the product registrant. The output unit may output the alert when the product registrant declares that the product registration work has been completed. The system may further include a third detection unit which detects that a receipt has been discharged, and a fourth detection unit which detects that the product registrant has left the accounting device which performs accounting processing, and the output unit may output the alert when the fourth detection unit detects that the product registrant has left but the third detection unit has not detected that the receipt has been discharged. Also, from another perspective, the present invention is a system comprising a first detection unit that detects the presence of multiple containers placed in advance in a store, a second detection unit that detects a specified product registration action using an image captured of a product registrant performing a product registration operation, and an output unit that outputs an alert when the first detection unit detects the presence of multiple containers and the second detection unit does not detect the specified product registration action for the multiple containers. Also, from another perspective, the present invention is a product registration monitoring method which detects the presence of a specified product other than products placed in containers placed in advance in a store, detects a specified product registration action using images captured of a product registrant performing a product registration operation, and outputs an alert when the presence of the specified product is detected but the specified product registration action for the specified product is not detected. Also, from another perspective, the present invention is a product registration monitoring method which detects the presence of multiple containers placed in a store in advance, detects a specified product registration behavior using images captured of a product registrant performing the product registration work, and outputs an alert if the presence of multiple containers is detected and the specified product registration behavior for the multiple containers is not detected. From another perspective, the present invention is a program that enables a computer to implement a first detection function of detecting the presence of a specified product other than products placed in containers placed in advance in a store, a second detection function of detecting a specified product registration action using an image captured of a product registration work performed by a product registrant, and an output function of outputting an alert when the presence of the specified product is detected by the first detection function and the specified product registration action for the specified product is not detected by the second detection function. From another perspective, the present invention is a program that enables a computer to implement a first detection function of detecting the presence of multiple containers placed in advance in a store, a second detection function of detecting a specified product registration behavior using images captured of a product registrant performing a product registration operation, and an output function of outputting an alert when the first detection function detects the presence of multiple containers and the second detection function does not detect the specified product registration behavior for the multiple containers. Effect of the Invention

[0006] According to the present invention, it is possible to provide a system etc. that can prevent customers from failing to register products with a high degree of accuracy. [Brief description of the drawings]

[0007] [Figure 1] 1 is a diagram illustrating an example of the schematic configuration of a transaction system according to a first embodiment. [Diagram 2] 1 is a diagram illustrating an example of a schematic configuration of a transaction device according to a first embodiment. [Diagram 3] 2 is a block diagram showing an example of the functional configuration of the accounting device according to the first embodiment. FIG. [Figure 4] FIG. 13 is a diagram showing an example of a transaction start screen displayed on the display operation unit. [Diagram 5] FIG. 13 is a diagram showing an example of a selection screen displayed on a display operation unit. [Figure 6] FIG. 11 is a diagram showing an example of a request screen displayed on a display operation unit. [Figure 7] FIG. 13 is a diagram showing an example of a product list screen displayed on a display operation unit. [Figure 8] FIG. 2 is a diagram illustrating an example of a schematic configuration of a server according to the first embodiment. [Figure 9] 10 is a flowchart illustrating an example of a monitoring process performed by a control unit. [Figure 10] 10 is an example of a flowchart illustrating a monitoring process according to a second embodiment, which is performed by a control unit. [Figure 11] FIG. 13 is a diagram illustrating an example of a schematic configuration of a system according to a third embodiment. [Figure 12] FIG. 1 is a schematic diagram showing a customer pushing a cart while waiting to check out; [Figure 13] 10 is a flowchart illustrating an example of a monitoring process performed by a control unit. [Figure 14] FIG. 13 is a diagram illustrating an example of a schematic configuration of a system according to a fourth embodiment. [Figure 15] FIG. 13 is a diagram illustrating an example of a schematic configuration of a system according to a fifth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. First Embodiment FIG. 1 is a diagram showing an example of the schematic configuration of a transaction system 1 (hereinafter sometimes simply referred to as "system 1") according to a first embodiment. FIG. 2 is a diagram showing an example of a schematic configuration of the accounting device 10 according to the first embodiment. FIG. 3 is a block diagram showing an example of the functional configuration of the accounting device 10 according to the first embodiment. The system 1 comprises a self-checkout accounting device 10 where customers themselves register products and make payments, and a server 100 that can communicate with the accounting device 10 via a network 8. Fig. 1 shows an example in which four accounting devices 10 are placed in an accounting area A1 located next to the product display area within a sales floor of a store such as a supermarket, and the server 100 is placed in a back office.

[0009] The system 1 also includes a camera 150 that is connected to the server 100 via the network 8 and captures a predetermined area A0 in the store, including the checkout area A1. The camera 150 is installed, for example, on the ceiling, and has a wide-angle lens so that it can capture a wide area at once. The predetermined area A0 also includes a waiting area A2 where customers who enter the checkout area A1 wait. The network 8 is not particularly limited as long as it is a communication network used for data communication between devices, and can be, for example, a LAN (Local Area Network). The communication line used for data communication can be either wired or wireless, or a combination of both. The wireless LAN can be, for example, WiFi (registered trademark) or Bluetooth (registered trademark).

[0010] The system 1 is a system that can suppress failure to register a predetermined product, which is a product other than the product placed in the shopping basket 180. Here, the shopping basket 180 is a container that is placed in advance in a store and is used to hold products to be purchased. Examples of the predetermined product include products placed in the lower level 192 of a cart 190 that holds the shopping basket 180, and products held by a customer. Examples of the products placed in the lower level 192 of the cart 190 include rice, a dozen drinks, toilet paper, and the like. Examples of the product held by a customer include a rolled up mat, a straw mat, or the like. In other words, examples of the predetermined product include products that are difficult to put in the shopping basket 180. The accounting device 10 and the server 100 will be described in detail below.

[0011] (Accounting device 10) The accounting device 10 includes a POS register 20 that registers products, a change machine 30 that deposits and dispenses currency, a light 60 for calling a store clerk, an unregistered product counter 71 on which unregistered products are placed, and a registered product counter 72 on which registered products are placed. The POS register 20 and the change machine 30 will be described in detail later. The light 60 has an LED (light-emitting diode) and lights up or flashes when the LED emits light.

[0012] (POS register 20) The POS register 20 includes a display operation unit 21, a scanner 22, a card reader 23, a printing unit 24, a communication unit 25, and a hand scanner 26. The display operation unit 21 can be exemplified by a liquid crystal touch panel display. The scanner 22 obtains information such as the product name and price by optically reading the image of the barcode attached to the product.

[0013] The card reader 23 reads information from a credit card, a debit card, a prepaid card, and the like. The printing unit 24 discharges a receipt from a receipt discharge port 24a on which transaction details such as the registered product name, price, amount of inserted coins, amount of change, date and time, etc. are printed. The communication unit 25 is a communication interface for communicating with the change machine 30 and the server 100.

[0014] Hand scanner 26 is disposed on the top of device housing 11. A holder 12 that holds hand scanner 26 with the tip of hand scanner 26 hooked thereon is provided on the top of device housing 11. Hand scanner 26 is held and operated by a customer, and obtains information such as the name and price of a product by optically reading an image of a barcode attached to the product. The scanner 22 and the hand scanner 26 may be any type that can read information for identifying a product, and may be, for example, any type that can read information other than a barcode.

[0015] The POS register 20 also has a CPU (Central Processing Unit) (not shown), a ROM (Read Only Memory) (not shown) which is a memory area for storing programs such as a BIOS (Basic Input Output System), and a RAM (Random Access Memory) (not shown) which is an execution area for the programs, and is equipped with a POS control unit 27 which controls the entire POS register 20. The POS register 20 also includes a storage unit 28 that stores various programs such as an OS (Operating System) and applications, input data for the various programs, output data from the various programs, etc. The storage unit 28 can be exemplified by a storage device such as an HDD (Hard Disk Drive) or a semiconductor memory.

[0016] The POS control unit 27 includes a purchase product registration unit 271 , a purchase amount calculation unit 272 , a payment processing unit 273 , and a display control unit 274 . When the scanner 22 or the hand scanner 26 reads a barcode attached to a product and obtains information such as the product's name and price, the purchased product registration unit 271 registers the product using the obtained information. In addition, when the purchased product registration unit 271 obtains information that a product displayed on the display operation unit 21 has been selected, the purchased product registration unit 271 registers the name, price, etc. of the selected product.

[0017] The purchased product registration unit 271 uses the measurement results of a weight sensor (not shown) provided on the unregistered product stand 71 and a weight sensor (not shown) provided on the registered product stand 72 to determine whether or not a registered product has been placed on the registered product stand 72, and when the registered product has been placed on the registered product stand 72, the next product can be registered. That is, since a decrease in the measurement result of the weight sensor provided on the unregistered product stand 71 that occurs before the product is registered corresponds to the weight of the product, when the measurement result of the weight sensor provided on the registered product stand 72 increases by the weight of the product, it determines that "the registered product has been placed on the registered product stand 72," and enables the registration of the next product. In addition, the purchased product registration unit 271 terminates the registration of the product when it determines that the measurement result of the weight sensor installed on the unregistered product stand 71 corresponds to the weight of the shopping basket 180 and the last registered product has been placed on the registered product stand 72.

[0018] When the purchased product registration unit 271 finishes registering products, the purchase amount calculation unit 272 calculates the purchase amount by adding up the prices of all registered products. The payment processing unit 273 completes the transaction by settling the purchase amount calculated by the purchase amount calculation unit 272, and issues a receipt. In other words, the payment processing unit 273 causes the printing unit 24 to discharge from the receipt discharge port 24a a receipt on which transaction details such as the registered product name, price, amount of inserted coins, amount of change, date and time, etc. are printed. The purchase amount may be settled by payment card or cash. If payment by cash is selected, the payment processing unit 273 transmits the purchase amount to the change machine 30, and completes the settlement when it receives a settlement completion notice from the change machine 30.

[0019] Next, the display control unit 274 will be described. FIG. 4 is a diagram showing an example of a transaction start screen 210 displayed on the display operation unit 21. As shown in FIG. During standby, the display control unit 274 causes the display operation unit 21 to display the checkout start screen 210. The display control unit 274 causes a declaration button 211 for the customer to declare the start of checkout, on the checkout start screen 210. In the example shown in Fig. 4, the declaration button 211 displays the words "Checkout Start".

[0020] FIG. 5 is a diagram showing an example of the selection screen 220 displayed on the display operation unit 21. As shown in FIG. When the declaration button 211 is pressed, the display control unit 274 causes the display operation unit 21 to display a selection screen 220 for selecting whether or not there is a product placed on the lower level 192 of the cart 190 or a product held by the customer. For example, in order to have the customer select whether or not there is a product placed on the lower level 192 of the cart 190 or a product held by the customer, the display control unit 274 causes the display operation unit 21 to display text information 221 prompting the customer to select "Are there any products placed on the lower level of the cart or any products held by the customer?", as well as a Yes button 222 displaying the text information "Yes" and a No button 223 displaying the text information "No" for selecting an answer.

[0021] FIG. 6 is a diagram showing an example of a request screen 230 displayed on the display operation unit 21. As shown in FIG. When the Yes button 222 or the No button 223 displayed on the selection screen 220 is pressed, the display control unit 274 causes the display operation unit 21 to display a request screen 230 for prompting the user to register the product to be purchased. For example, the display control unit 274 causes the request screen 230 to display text information 231 saying "Please scan the barcode of the product" as shown in FIG.

[0022] FIG. 7 is a diagram showing an example of a product list screen 240 displayed on the display operation unit 21. As shown in FIG. The purchased product registration unit 271 registers the product when the scanner 22 or the hand scanner 26 reads a barcode attached to the product, and the display control unit 274 causes the display operation unit 21 to display a list 241 of the products registered by the purchased product registration unit 271 on the product list screen 240. In addition, the display control unit 274 causes the purchase price calculated by the purchase price calculation unit 272 to be displayed on the product list screen 240. The display control unit 274 also displays a checkout button 242 on the product list screen 240, which allows the customer to declare the start of checkout after completing product registration. The display control unit 274 also displays a cancel button 243 on the product list screen 240, which allows the customer to declare that he or she will cancel product registration. In the example shown in Fig. 7, the checkout button 242 and the cancel button 243 display the words "checkout" and "cancel", respectively. In addition, when payment by cash is selected, the display control unit 274 causes the display operation unit 21 to display the deposit amount, the amount of change, etc.

[0023] (Change machine 30) 3, the change machine 30 includes a communication unit 31, a display operation unit 32, and a memory unit 33. The communication unit 31 is a communication interface for communicating with the POS register 20 and the server 100. The display operation unit 32 is a liquid crystal touch panel display or the like. The memory unit 33 can be exemplified as a storage device such as a HDD or semiconductor memory.

[0024] The change machine 30 also includes a change control unit 34 having a CPU (not shown), a ROM (not shown), and a RAM (not shown) and controlling the entire change machine 30, a banknote processing unit 40, and a coin processing unit 50. The change machine 30 also includes a banknote insertion slot 41 (see FIG. 2) used for inserting banknotes, a banknote ejection outlet 42 (see FIG. 2) used for ejecting banknotes, a coin insertion slot 51 (see FIG. 2) used for inserting coins, and a coin ejection outlet 52 (see FIG. 2) used for ejecting coins.

[0025] The change control unit 34 has a deposit processing unit 341 which performs a process of depositing money, and a withdrawal processing unit 342 which performs a process of withdrawing money. When the deposit processing unit 341 receives the purchase amount from the POS register 20, it permits the insertion of currency into the banknote handling unit 40 and the coin handling unit 50. Thereafter, the deposit processing unit 341 obtains the number of currency of each denomination inserted from the banknote handling unit 40 and the coin handling unit 50, and calculates the total amount of the inserted currency. When the total amount of the inserted currency exceeds the purchase amount, the deposit processing unit 341 calculates the difference between the total amount and the purchase amount as change, and instructs the withdrawal processing unit 342 to dispense the change. On the other hand, when the total amount of the inserted currency matches the purchase amount or the withdrawal processing unit 342 has normally dispensed the change, the deposit processing unit 341 transmits a settlement completion notice to the POS register 20.

[0026] When the withdrawal processing unit 342 receives a change dispensing instruction from the deposit processing unit 341, it determines the denomination and number of coins to be dispensed according to the amount of change, and performs the dispensing process by outputting a dispensing instruction to the banknote processing unit 40 and the coin processing unit 50. Then, when the withdrawal of the change has been completed normally, the withdrawal processing unit 342 notifies the deposit processing unit 341.

[0027] When a customer registers an item in a shopping basket 180, the customer places the item to be purchased in the shopping basket 180 on an unregistered item stand 71, takes out the item from the shopping basket 180, holds the barcode on the item over a scanner 22 to register it, and then repeats the operation of moving the item to a registered item stand 72. On the other hand, when a customer registers an item placed on the lower level 192 of the cart 190, the customer may, for example, read the barcode on the item with the hand scanner 26. Also, when a customer registers an item that he or she is holding in his or her hand, the customer may, for example, read the barcode on the item with the hand scanner 26.

[0028] (Server 100) FIG. 8 is a diagram illustrating an example of a schematic configuration of the server 100 according to the first embodiment. The server 100 includes a CPU, a ROM, and a RAM, and is equipped with a control unit 110 that controls the entire server 100. The server 100 also includes a storage unit 131 that stores various programs such as an OS and applications, input data for the various programs, output data from the various programs, etc. The storage unit 131 can be exemplified by a storage device such as a HDD or a semiconductor memory.

[0029] The server 100 includes a display unit 132 used to display an operation reception screen and images, an operation unit 133 that receives input operations from the user, and a communication unit 134 that is used for communication with external devices. The display unit 132 can be, for example, a liquid crystal display or an organic EL display. Examples of the operation unit 133 include a keyboard, a mouse, and a touch panel. When the operation unit 133 is a touch panel, the touch panel functions as the operation unit 133 and the display unit 132. The communication unit 134 can be exemplified as a communication interface. The server 100 can be exemplified as a computer that performs the function of transmitting and receiving information to and from the client accounting device 10. The server 100 may be a notebook PC, a desktop PC, a tablet PC, a tablet terminal, a personal digital assistant (PDA), or a multi-function mobile phone (a so-called "smartphone").

[0030] (Control unit 110) The control unit 110 includes an acquisition unit 111 that acquires images captured by the camera 150 and information output from the accounting device 10, a predetermined product detection unit 112 that detects the presence of a predetermined product, and a behavior detection unit 113 that detects that a product registrant has performed a predetermined product registration behavior. The control unit 110 also includes an output unit 114 that outputs an alert when the presence of a predetermined product is detected by the predetermined product detection unit 112 and the behavior detection unit 113 does not detect a product registration behavior for the predetermined product, since it is considered that the predetermined product has not been registered. The control unit 110 also includes a completion detection unit 115 that detects that product registration by a customer has been completed. The control unit 110 also includes a discharge detection unit 116 that detects that a receipt has been discharged from the receipt discharge port 24a of the POS register 20. The control unit 110 also includes a departure detection unit 117 that detects that a customer has left the accounting device 10.

[0031] Acquiring unit 111 acquires still images captured by camera 150 of predetermined area A0 and still images captured of the face of a customer within predetermined area A0, and when a customer is detected within predetermined area A0, it captures the customer's movement as a movement of a human-like skeleton including multiple joints and stores the three-dimensional coordinates of each joint in a three-dimensional space set in predetermined area A0 in a time series as human body information in storage unit 131. By storing the human body information capable of reproducing the customer's skeleton and customer's movement as text-format data recording the three-dimensional coordinates of each joint in a time series, the amount of data to be stored can be reduced compared to storing video images of the customer's movement.

[0032] An example of the predetermined product detection unit 112 detecting the presence of a predetermined product based on an image captured by the camera 150 of a customer waiting in line in the waiting area A2 to enter the checkout area A1 or a customer who has entered the checkout area A1 (hereinafter, these customers may be referred to as "customers waiting to checkout"). For example, the predetermined product detection unit 112 detects the presence of a predetermined product when it determines that an item is placed on the lower level 192 of the cart 190 based on an image captured by the camera 150 of a cart 190 being pushed by a customer waiting to checkout. Furthermore, the predetermined product detection unit 112 detects the presence of a predetermined product based on an image captured by the camera 150 of the customer waiting to checkout when the customer is holding a product in his or her hand.

[0033] In addition, the predetermined product detection unit 112 detects the presence of a predetermined product when the customer declares the presence of the predetermined product via the POS register 20. For example, the predetermined product detection unit 112 detects the presence of a predetermined product when the Yes button 222 on the selection screen 220 is pressed.

[0034] Behavior detection unit 113 detects that a customer who is a product registrant has performed a predetermined product registration behavior. An example of the predetermined product registration behavior is a customer using hand scanner 26 to read a barcode attached to a product placed on lower level 192 of cart 190. Another example of the predetermined product registration behavior is a customer using hand scanner 26 to read a barcode attached to a product that is being held in the hand.

[0035] The behavior detection unit 113 detects a predetermined product registration behavior based on a skeleton recognition process of the product registrant using the data stored in the storage unit 131. For example, when the behavior detection unit 113 wants to detect the above-mentioned predetermined product registration behavior, the behavior detection unit 113 can search for human body information in text format based on the three-dimensional coordinates of each joint.

[0036] Note that the acquisition unit 111 can set a trigger setting in advance for determining whether the customer's movement or posture satisfies a predetermined condition so that it can detect a movement or posture such as when the hand scanner 26 reads a barcode attached to a product placed on the lower level 192 of the cart 190, and can store a predetermined period of video in which the movement or posture that satisfies the trigger setting is stored. Then, if the behavior detection unit 113 specifies the trigger setting when searching for data that the user wants to view from among the stored data, it can search for and check the stored video that satisfies the trigger setting.

[0037] The output unit 114 outputs an alert when the presence of a predetermined product is detected by the predetermined product detection unit 112 and a product registration action for the predetermined product is not detected by the behavior detection unit 113. Examples of the alert include displaying predetermined text information on the display operation unit 21 of the POS register 20 to the effect that the product has not been registered, displaying predetermined text information on the display unit of the terminal looked by the store clerk standing in the accounting area A1, and turning on or blinking the light 60. Therefore, when the presence of a predetermined product is detected by the predetermined product detection unit 112 and a product registration action for the predetermined product is not detected by the behavior detection unit 113, the output unit 114 outputs a command to the accounting device 10 or the terminal to display predetermined text information on the display operation unit 21 of the POS register 20 or on the display unit of the terminal looked by the store clerk standing in the accounting area A1. Alternatively, when the presence of a specified product is detected by the specified product detection unit 112 and no product registration behavior for the specified product is detected by the behavior detection unit 113, the output unit 114 outputs a command to the accounting device 10 to turn on or flash the light 60.

[0038] The completion detection unit 115 detects that the registration of the product is completed when the checkout button 242 on the product list screen 240 is pressed. The discharge detection unit 116 detects that a receipt has been discharged when an image of the receipt discharged from the receipt discharge slot 24a is included in the image captured by the camera 150.

[0039] The exit detection unit 117 detects that a customer has left using images captured by the camera 150. For example, the exit detection unit 117 detects that a customer has left in front of the checkout device 10 when the customer moves from within a predetermined range centered on the checkout device 10 to outside the predetermined range. Alternatively, the exit detection unit 117 detects that a customer has left the checkout area A1 when the customer moves from within the checkout area A1 to outside the checkout area A1.

[0040] [Monitoring process] The monitoring process performed by control unit 110 will be described below with reference to a flowchart. 9 is a flowchart showing an example of a monitoring process performed by the control unit 110. The control unit 110 repeatedly executes the monitoring process at preset fixed time intervals (for example, 1 millisecond). The control unit 110 judges whether or not the predetermined product is present (S901). The process of S901 is a process for judging whether or not the predetermined product detection unit 112 has detected the predetermined product as described above. If the predetermined product is not present (NO in S901), the control unit 110 ends the monitoring process.

[0041] On the other hand, if the predetermined product is present (YES in S901), the control unit 110 judges whether or not a predetermined product registration behavior has been detected (S902). The process of S902 is a process for judging whether or not the behavior detection unit 113 has detected the above-mentioned predetermined product registration behavior. If the predetermined product registration behavior has been detected (YES in S902), the control unit 110 ends the monitoring process.

[0042] On the other hand, if the predetermined product registration behavior has not been detected (NO in S902), the control unit 110 judges whether the end of the registration behavior has been detected (S903). The process of S903 is, for example, a process in which the end detection unit 115 judges whether the checkout button 242 on the product list screen 240 has been pressed. If the end of the registration behavior has not been detected (NO in S903), the control unit 110 performs the processes from S902 onwards.

[0043] On the other hand, when the end of the registration behavior is detected (YES in S903), the control unit 110 outputs an alert (S904). The process of S904 is a process performed by the output unit 114. As a result, an alert is output when the behavior detection unit 113 does not detect a product registration behavior for the specific product even though the specific product is present and the registration behavior has ended.

[0044] As described above, the system 1 includes a predetermined product detection unit 112 (an example of a first detection unit) that detects the presence of a predetermined product other than a product placed in a shopping basket 180 (an example of a container) placed in advance in a store, and a behavior detection unit 113 (an example of a second detection unit) that detects a predetermined product registration behavior using an image captured of a product registration operation by a product registrant. The system 1 also includes an output unit 114 that outputs an alert when the predetermined product detection unit 112 detects the presence of a predetermined product and the behavior detection unit 113 does not detect a predetermined product registration behavior for the predetermined product.

[0045] According to the system 1 configured as above, when the presence of a predetermined product is detected by the predetermined product detection unit 112 and the behavior detection unit 113 does not detect a predetermined product registration behavior for the predetermined product, it is highly likely that the customer has missed registering the predetermined product, so by outputting an alert, it is possible to highly reliably prevent the customer from missing the registration of the product. For example, if the alert is to display predetermined text information on the display operation unit 21 of the POS register 20 indicating that the product has not been registered, the customer himself / herself will notice the missed registration of the product by looking at the text information displayed on the display operation unit 21 and will register the product. Also, if the alert is to display predetermined text information on the display operation unit 21 of the POS register 20 indicating that the product has not been registered, for example, a store clerk standing in the checkout area A1 can notice the missed registration of the product by the customer by looking at the text information displayed on the display operation unit 21 and encourage the customer to register the product. Also, if the alert is to display predetermined text information on the display unit of the terminal that the store clerk standing in the checkout area A1 is looking at, indicating that the product has not been registered, the store clerk can notice the missed registration of the product by the customer by looking at the text information displayed on the display unit of the terminal and encourage the customer to register the product. Also, if the alert is to turn on or blink the light 60, for example, a store clerk standing in the checkout area A1 can notice that a customer has forgotten to register an item and urge the customer to register the item.

[0046] The predetermined product detection unit 112 then detects the presence of the predetermined product using an image of the product registrant. For example, based on an image of a customer waiting to pay taken by the camera 150, the predetermined product detection unit 112 detects the presence of the predetermined product when it is determined that the customer is holding a product in their hand or that an item is placed on the lower level 192 of the cart 190. Alternatively, the predetermined product detection unit 112 detects the presence of the predetermined product when the product registrant inputs that the predetermined product is present (for example, when the Present button 222 on the selection screen 220 is pressed). This makes it possible to detect the presence of the predetermined product with a high degree of accuracy.

[0047] Furthermore, as described above, the behavior detection unit 113 detects a predetermined product registration behavior based on the skeleton recognition process of the product registrant. This allows the behavior detection unit 113 to detect the predetermined product registration behavior with a high degree of accuracy. Furthermore, the output unit 114 outputs an alert when the product registrant declares that the product registration work is completed. For example, the output unit 114 outputs an alert when the completion detection unit 115 presses the checkout button 242 on the product list screen 240. This allows the customer who is the product registrant to notice the missed registration more quickly. Alternatively, the output unit 114 may output an alert when the exit detection unit 117 detects that a customer has left the accounting device 10 or the accounting area A1. This makes it possible to prevent a customer from leaving the store even if they have not yet finished registering a specific product.

[0048] 1 shows one camera 150, there may be multiple cameras 150 capturing images of the specified area A0. Also, the accounting device 10 may have a camera that captures images of a specified area centered on the accounting device 10.

[0049] The product registration monitoring method performed by the system 1 detects the presence of a predetermined product other than the products placed in the shopping basket 180 placed in advance in the store, detects the predetermined product registration behavior using an image taken of the product registration work by the product registrant, and outputs an alert when the presence of the predetermined product is detected and the predetermined product registration behavior for the predetermined product is not detected. This product registration monitoring method can highly reliably prevent customers from failing to register products.

[0050] Moreover, the above-described processes performed by the server 100 and the accounting device 10 can be realized by the cooperation of software and hardware resources. In this case, the CPU of the control unit (e.g., the control unit 110) of each device (e.g., the server 100) executes a program that realizes each function of the device, and realizes each function. For example, a non-transient computer-readable recording medium on which a program is recorded is provided to each device (e.g., the server 100), and the CPU of the device (e.g., the server 100) reads the program stored in the recording medium. Alternatively, the CPU of the device (e.g., the server 100) downloads the program via the network 8. In these cases, the program read from the recording medium or the downloaded program itself realizes the functions of the above-described embodiments, and the program itself and the recording medium on which it is recorded constitute the present invention.

[0051] The program that realizes the functions of the server 100 enables a computer to realize a first detection function that detects the presence of a specified product other than products placed in a shopping basket 180 that has been placed in advance in the store, a second detection function that detects a specified product registration behavior using images captured of the product registration work being performed by a product registrant, and an output function that outputs an alert when the first detection function detects the presence of the specified product and the second detection function does not detect the specified product registration behavior for the specified product. Examples of recording media for supplying such programs include flexible disks, CD-ROMs, DVD-ROMs, hard disks, optical disks, magneto-optical disks, CD-Rs, magnetic tapes, non-volatile memory cards, and ROMs.

[0052] <Second embodiment> The system according to the second embodiment is different from the system 1 according to the first embodiment in the monitoring process performed by the control unit 110 of the server 100. In the first and second embodiments, components having the same functions are denoted by the same reference numerals, and detailed description thereof will be omitted. Fig. 10 is an example of a flowchart showing a monitoring process according to the second embodiment performed by the control unit 110. In the flowchart shown in Fig. 10, the same processes as those in the flowchart shown in Fig. 9 are denoted by the same process numbers, and detailed descriptions thereof will be omitted.

[0053] The control unit 110 judges whether or not the predetermined product is present (S901). If the predetermined product is not present (NO in S901), the control unit 110 ends the monitoring process. On the other hand, if the predetermined product is present (YES in S901), the control unit 110 judges whether or not a predetermined product registration action has been detected (S902).

[0054] If a predetermined product registration action has not been detected (NO in S902), the control unit 110 judges whether or not the checkout button 242 on the product list screen 240 has been pressed (S1003). The process of S1003 is a process in which the end detection unit 115 judges whether or not the checkout button 242 on the product list screen 240 has been pressed. If the checkout button 242 has not been pressed (NO in S1003), the control unit 110 performs the processes from S902 onwards.

[0055] On the other hand, when the checkout button 242 is pressed (YES in S1003), the control unit 110 outputs an alert (S1004). The process of S1004 is a process of displaying predetermined text information on the display operation unit 21 of the POS register 20 to the effect that the product has not been registered.

[0056] On the other hand, if a predetermined product registration action is detected (YES in S902), the control unit 110 determines whether or not a receipt has been discharged (S1005). The process of S1005 is a process in which the discharge detection unit 116 determines whether or not a receipt has been detected as having been discharged from the receipt discharge outlet 24a. If a receipt has been discharged (YES in S1005), the control unit 110 ends the monitoring process.

[0057] On the other hand, if a receipt has not been ejected (NO in S1005), the control unit 110 determines whether the customer has left in front of the checkout device 10 or from the checkout area A1 (S1006). The process of S1006 is a process in which the departure detection unit 117 determines whether or not the customer has detected that the customer has left in front of the checkout device 10 or from the checkout area A1. If the customer has not left in front of the checkout device 10 or from the checkout area A1 (NO in S1006), the control unit 110 performs the processes from S1005 onwards.

[0058] On the other hand, if the customer leaves the accounting device 10 or accounting area A1 (YES in S1006), the control unit 110 outputs an alert (S1007). The process of S1007 is performed by the output unit 114, and examples of the process include displaying predetermined text information to the effect that the product has not been registered on the display operation unit 21 of the POS register 20, displaying predetermined text information to the effect that the product has not been registered on the display of a terminal looked by a store clerk standing in the accounting area A1, and turning on or blinking the light 60.

[0059] As described above, the system according to the second embodiment includes a discharge detection unit 116 (an example of a third detection unit) that detects when a receipt has been discharged, and a departure detection unit 117 (an example of a fourth detection unit) that detects when a product registrant has left the accounting device 10 that performs accounting. When the departure detection unit 117 detects that a product registrant has left, but the discharge detection unit 116 has not detected that a receipt has been discharged, the output unit 114 outputs an alert. This makes it possible to prevent a customer from leaving the store even though a receipt has not been discharged, in other words, even though the settlement process has not been completed.

[0060] In order to prevent a customer from leaving the store before the settlement process is complete, the following determination may be made instead of determining whether or not a receipt has been ejected (S1005). For example, in S1005, the control unit 110 may determine whether or not the checkout button 242 on the product list screen 240 has been pressed. If the checkout button 242 has been pressed, the control unit 110 may end the monitoring process, and if the checkout button 242 has not been pressed, the control unit 110 may determine whether or not the customer has left (S1006). This makes it possible to prevent the customer from leaving the store even if the checkout process has not been completed. Alternatively, the system according to the second embodiment may be equipped with a reading device at the exit of the checkout area A1 that reads the image of the barcode on the receipt, and in S1005 the control unit 110 may determine whether the reading device has read the image of the barcode on the receipt (in other words, whether the customer has held the receipt over the reading device). The control unit 110 may then end the monitoring process if the reading device has read the barcode, and on the other hand, determine whether the customer has left if the reading device has not read the image of the barcode (S1006). This makes it possible to prevent a customer from leaving the store without holding the receipt over the exit of the checkout area A1.

[0061] In the above-described system 1 according to the first embodiment and the system according to the second embodiment, instead of the output unit 114 outputting an alert, the control unit 110 may store the history in the storage unit 131. For example, when the predetermined product detection unit 112 detects the presence of the predetermined product based on the image captured by the camera 150, the control unit 110 recognizes the face of the customer who is waiting to be checked out and who is carrying the predetermined product, and recognizes which accounting device 10 the customer has registered the product with. When the predetermined product detection unit 112 detects the presence of the predetermined product and the behavior detection unit 113 does not detect the product registration behavior for the predetermined product, the control unit 110 stores in the storage unit 131 an image of the face of the customer who is waiting to be checked out and who is carrying the predetermined product, in association with the number of the accounting device 10, together with information that the predetermined product has not been registered. This allows, for example, a store clerk to know the face of the customer who has failed to register the product by checking the history stored in the storage unit 131. Then, the store clerk can call out to the customer who has failed to register and visits the store at a later date, thereby preventing the customer who has failed to register from failing to register again.

[0062] <Third embodiment> FIG. 11 is a diagram showing an example of a schematic configuration of a system 3 according to the third embodiment. FIG. 12 is an example of a schematic diagram showing a customer pushing a cart 190 while waiting to check out. The system 3 according to the third embodiment is different from the system 1 according to the first embodiment and the system according to the second embodiment in that the condition under which the output unit 314 corresponding to the output unit 114 outputs an alert is different. In the first and second embodiments, components having the same functions are denoted by the same reference numerals, and detailed description thereof will be omitted.

[0063] The system 3 includes a server 300 equivalent to the server 100 , and the server 300 includes a control unit 310 equivalent to the control unit 110 . The control unit 310 has an acquisition unit 111, a plurality of detection units 312 that detect the presence of a plurality of shopping baskets 180, and a behavior detection unit 313 that detects that a product registrant has performed a predetermined product registration behavior. The control unit 310 also has an output unit 314 that outputs an alert when the presence of a plurality of shopping baskets 180 is detected by the plurality of detection units 312 and the predetermined product registration behavior for the plurality of shopping baskets 180 is not detected by the behavior detection unit 313, since it is considered that the products placed in the plurality of shopping baskets 180 have not been registered. The control unit 310 also has an end detection unit 115, a discharge detection unit 116, and a removal detection unit 117.

[0064] One example of the plurality detection unit 312 is to detect that there are a plurality of shopping baskets 180 based on an image of a customer waiting to pay or a cart 190 being pushed by a customer waiting to pay, captured by the camera 150. For example, the plurality detection unit 312 detects that there are a plurality of shopping baskets 180 when it is determined that shopping baskets 180 are placed on the upper level 191 and the lower level 192 of a cart 190 being pushed by a customer waiting to pay, captured by the camera 150.

[0065] The behavior detection unit 313 detects that a customer who is a product registrant has performed a predetermined product registration behavior. An example of the predetermined product registration behavior is when the customer places both of the shopping baskets 180 placed on the upper level 191 and the lower level 192 of the cart 190 on the unregistered product stand 71. The timing of placing the multiple shopping baskets 180 on the unregistered product stand 71 does not have to be simultaneous. For example, after the registration of the products placed in the shopping basket 180 placed on the upper level 191 is completed, the shopping basket 180 placed on the lower level 192 may be placed on the unregistered product stand 71. The behavior detection unit 313 uses the data stored in the storage unit 131 to detect a predetermined product registration behavior based on a skeleton recognition process of the product registrant.

[0066] Note that the acquiring unit 111 can set a trigger setting in advance for determining whether the customer's movement or posture satisfies a predetermined condition so as to detect the movement or posture of, for example, the shopping basket 180 placed on the upper level 191 or lower level 192 of the cart 190 being placed on the unregistered product stand 71, and can store a predetermined period of video in which the movement or posture that satisfies the trigger setting is stored. Then, if the behavior detecting unit 313 specifies the trigger setting when searching for the data that the user wants to view from among the stored data, it can search for and confirm the stored video that satisfies the trigger setting.

[0067] The output unit 314 outputs an alert when the plurality detection unit 312 detects that there are a plurality of shopping baskets 180 and the behavior detection unit 313 does not detect a predetermined product registration behavior for the plurality of shopping baskets 180.

[0068] [Monitoring process] The monitoring process performed by control unit 310 will be described below with reference to a flowchart. Fig. 13 is a flowchart showing an example of a monitoring process performed by control unit 310. In the flowchart shown in Fig. 13, the same processes as those in the flowchart shown in Fig. 9 are given the same process numbers, and detailed descriptions thereof will be omitted. Control unit 310 repeatedly executes the monitoring process at preset fixed time intervals (for example, 1 millisecond).

[0069] The control unit 310 determines whether or not there are multiple shopping baskets 180 (S1301). The process of S1301 is a process for determining whether or not the multiple detection unit 312 has detected, as described above, that there are multiple shopping baskets 180. If there are not multiple shopping baskets 180 (NO in S1301), the control unit 310 ends the monitoring process.

[0070] On the other hand, if there are multiple shopping baskets 180 (YES in S1301), the control unit 310 judges whether or not a predetermined product registration behavior has been detected (S1302). The process of S1302 is a process for judging whether or not the behavior detection unit 313 has detected the above-mentioned predetermined product registration behavior. If the predetermined product registration behavior has been detected (YES in S1302), the control unit 310 ends the monitoring process.

[0071] On the other hand, if the predetermined product registration behavior has not been detected (NO in S1302), the control unit 310 judges whether the end of the registration behavior has been detected (S903). If the end of the registration behavior has not been detected (NO in S903), the control unit 310 performs the processes from S1302 onwards. On the other hand, if the end of the registration behavior has been detected (YES in S903), the control unit 310 outputs an alert (S904). The process of S904 is a process performed by the output unit 314. As a result, if there are multiple shopping carts 180 and the registration behavior has been completed, but the behavior detection unit 313 has not detected the predetermined product registration behavior for multiple shopping carts 180, an alert is output.

[0072] As described above, the system 3 includes the plurality detection unit 312 (an example of a first detection unit) that detects the presence of a plurality of shopping baskets 180 placed in advance in the store, and the behavior detection unit 313 (an example of a second detection unit) that detects a predetermined product registration behavior using an image capturing the state of the product registration work by the product registrant. The system 3 also includes an output unit 314 that outputs an alert when the plurality detection unit 312 detects the presence of a plurality of shopping baskets 180 and the behavior detection unit 313 does not detect the predetermined product registration behavior for the plurality of shopping baskets 180.

[0073] According to the system 3 configured as described above, if the multiple detection unit 312 detects that there are multiple shopping baskets 180 and the behavior detection unit 313 does not detect a specified product registration behavior for the multiple shopping baskets 180, it is highly likely that the customer has failed to register a specified product, so by outputting an alert, it is possible to highly reliably prevent customers from failing to register products.

[0074] Note that the control unit 310 may output an alert when the exit detection unit 117 detects that a product registrant has left but the discharge detection unit 116 does not detect that a receipt has been discharged, as in the monitoring process according to the second embodiment described using the flowchart in Fig. 10. This makes it possible to prevent a customer from leaving the store even though a receipt has not been discharged, in other words, even though the settlement process has not been completed.

[0075] The product registration monitoring method performed by the system 3 detects that there are multiple shopping baskets 180 placed in advance in the store, detects a predetermined product registration behavior using images captured of the product registration work by the product registrant, and outputs an alert when it is detected that there are multiple shopping baskets 180 and the predetermined product registration behavior for the multiple shopping baskets 180 is not detected. This product registration monitoring method can highly reliably prevent customers from failing to register products.

[0076] In addition, the program that realizes the functions of the server 300 causes the computer to realize a first detection function that detects the presence of multiple shopping baskets 180 already placed in the store, a second detection function that detects a specified product registration behavior using images captured of the product registration work being performed by a product registrant, and an output function that outputs an alert when the first detection function detects the presence of multiple shopping baskets 180 and the second detection function does not detect the specified product registration behavior for the multiple shopping baskets 180.

[0077] <Fourth embodiment> FIG. 14 is a diagram showing an example of a schematic configuration of a system 4 according to the fourth embodiment. System 4 according to the fourth embodiment differs from system 1 according to the first embodiment and the system according to the second embodiment in that system 4 confirms that a product has been registered by recognizing that an LED, which is lit when an image of a barcode attached to a product is read by scanner 22 and hand scanner 26, has been turned on based on an image captured by camera 150. Components having the same functions in the first and second embodiments and the fourth embodiment are denoted by the same reference numerals, and detailed description thereof will be omitted.

[0078] In the system 4 according to the fourth embodiment, a control unit 410 of a server 400 corresponding to the server 100 has an illumination detection unit 418 that detects that the LED of the scanner 22 or the hand scanner 26 is illuminated. The lighting detector 418 detects, based on an image captured by the camera 150, that the LED of the scanner 22 or the hand scanner 26 has been turned on.

[0079] The output unit 114 according to the fourth embodiment outputs an alert when the predetermined product detection unit 112 detects the presence of a predetermined product and the behavior detection unit 113 does not detect a product registration behavior for the predetermined product, and can also output an alert in the following cases. That is, the output unit 114 according to the fourth embodiment outputs an alert when the predetermined product detection unit 112 detects the presence of a predetermined product and the behavior detection unit 113 detects a product registration behavior for the predetermined product, and the lighting detection unit 418 does not detect that the LED of the scanner 22 or the hand scanner 26 has been lit in association with the product registration behavior for the predetermined product.

[0080] According to system 4 configured as described above, even if predetermined product detection unit 112 detects the presence of a predetermined product and, for example, behavior detection unit 113 detects that a customer has performed an action of reading a barcode attached to the predetermined product with hand scanner 26, if it does not detect that the LED of hand scanner 26 has been lit, it is highly likely that the customer has performed a deceptive act of reading the barcode attached to the predetermined product with hand scanner 26. According to system 4, when it is highly likely that the customer has performed a deceptive act, an alert is output, thereby making it possible to accurately prevent customers from failing to register products.

[0081] <Fifth embodiment> FIG. 15 is a diagram showing an example of a schematic configuration of a system 5 according to the fifth embodiment. System 5 according to the fifth embodiment differs from system 1 according to the first embodiment and the system according to the second embodiment in that, as a result of scanner 22 and hand scanner 26 reading an image of a barcode attached to a product, the system confirms that the product has been registered by recognizing, based on an image captured by camera 150, that a new product has been added to list 241 on product list screen 240 displayed on display operation unit 21 of POS register 20 of accounting device 10. Components having the same functions in the first and second embodiments and the fifth embodiment are denoted by the same reference numerals, and detailed explanations thereof will be omitted.

[0082] The system 5 according to the fifth embodiment has a control unit 510 of a server 500 corresponding to the server 100, and a registration detection unit 519 that detects that a new product has been added to the list 241 of the product list screen 240 when the scanner 22 or the hand scanner 26 reads an image of a barcode attached to a product. The registration detection unit 519 detects, based on an image captured by the camera 150 of the display operation unit 21 of the accounting device 10, that a new product has been added to the list 241 of the product list screen 240 displayed on the display operation unit 21 of the accounting device 10 when the scanner 22 or hand scanner 26 reads an image of a barcode attached to a product.

[0083] The output unit 114 according to the fifth embodiment outputs an alert when the presence of a predetermined product is detected by the predetermined product detection unit 112 and a product registration action for the predetermined product is not detected by the behavior detection unit 113, and can also output an alert in the following cases: That is, the output unit 114 according to the fifth embodiment outputs an alert when the presence of a predetermined product is detected by the predetermined product detection unit 112 and a product registration action for the predetermined product is detected by the behavior detection unit 113, and a new product is not added to the list 241 of the product list screen 240 due to the scanner 22 or hand scanner 26 reading an image of a barcode attached to the product in association with the product registration action for the predetermined product.

[0084] As described above, the system 5 includes a registration detection unit 519 (an example of a first detection unit) that detects a predetermined operation of the POS register 20 (an example of a product registration terminal) (for example, the addition of a new product to the list 241 of the product list screen 240 displayed on the display operation unit 21) based on an image captured of the POS register 20 (an example of a product registration terminal). The system 5 also includes a behavior detection unit 113 (an example of a second detection unit) that detects a predetermined product registration behavior (for example, the act of reading a barcode with the hand scanner 26) using an image captured (an image captured by the camera 150) of the product registration work performed by the product registrant. The system 5 also includes an output unit 114 that outputs an alert when the registration detection unit 519 does not detect a predetermined operation in response to the predetermined product registration behavior detected by the behavior detection unit 113.

[0085] According to the system 5 configured as above, even if the predetermined product detection unit 112 detects the presence of a predetermined product and the behavior detection unit 113 detects a predetermined product registration behavior for the predetermined product, if the output unit 114 does not detect a predetermined operation of the POS register 20, it outputs an alert. Here, even if the predetermined product detection unit 112 detects the presence of a predetermined product and, for example, the behavior detection unit 113 detects that a customer has performed an action of reading a barcode attached to the predetermined product with the hand scanner 26, if the behavior detection unit 113 does not detect the predetermined operation of the POS register 20, it is highly likely that the customer has performed a deceptive act of reading a barcode attached to the predetermined product with the hand scanner 26. According to the system 5, it is possible to accurately prevent a customer from failing to register a product by outputting an alert when it is highly likely that the customer has performed a deceptive act.

[0086] It should be noted that the registration detection unit 519 does not necessarily detect that a new product has been added to the list 241 of the product list screen 240 displayed on the display operation unit 21, for example, when the hand scanner 26 reads an image of a barcode attached to a product, based on an image captured by the camera 150. For example, the system 5 may include a dedicated camera for capturing an image of the display operation unit 21 of the POS register 20, separate from the camera 150, and the registration detection unit 519 may detect that a new product has been added to the list 241 of the product list screen 240 based on an image captured by the dedicated camera.

[0087] Instead of confirming that a product has been registered by recognizing that a new product has been added to the list 241 of the product list screen 240 displayed on the display operation unit 21 of the POS register 20, it is also possible to confirm that a product has been registered by having the voice recognition unit recognize that a sound (e.g., a registration sound such as a "beep") is generated by the POS register 20 when the scanner 22 and the hand scanner 26 read the image of the barcode attached to the product. Even in the case of recognizing a sound, it is possible to highly reliably prevent a customer from committing a counterfeit act. [Explanation of symbols]

[0088] 1,3,4,5...system, 10...accounting device, 20...POS register, 100,300,400,500...server, 110,310,410,510...control unit, 112...predetermined product detection unit, 113,313...behavior detection unit, 114,314...output unit, 116...ejection detection unit, 117...exit detection unit, 150...camera, 180...shopping basket, 190...cart, 312...multiple detection unit

Claims

1. a first detection unit that detects the presence of a predetermined commodity other than the commodity contained in the container placed in advance in the store; a second detection unit that detects a predetermined product registration behavior by using an image captured of a product registration operation by a product registrant; an output unit that outputs an alert when the first detection unit detects the presence of the predetermined product and the second detection unit does not detect the predetermined product registration behavior for the predetermined product; A system comprising:

2. The first detection unit detects the presence of the predetermined product by using an image of the product registrant. The system of claim 1 .

3. the first detection unit detects the presence of the predetermined product when the product registrant inputs that the predetermined product exists; The system of claim 1 .

4. The predetermined product is a product placed on a lower shelf of a cart that carries the container. The system of claim 1 .

5. The predetermined product is a product held by the product registrant. The system of claim 1 .

6. The second detection unit detects the predetermined product registration behavior based on a skeleton recognition process of the product registrant. The system of claim 1 .

7. the output unit outputs the alert when the product registrant declares that the product registration work has been completed. The system of claim 1 .

8. a third detection unit that detects that a receipt has been discharged; a fourth detection unit that detects when the product registrant leaves a checkout device that performs checkout processing; Further equipped with the output unit outputs the alert when the fourth detection unit detects that the merchandise registrant has left and the third detection unit has not detected that the receipt has been discharged. The system of claim 1 .

9. A first detection unit that detects the presence of a plurality of containers placed in advance in the store; a second detection unit that detects a predetermined product registration behavior by using an image captured of a product registration operation by a product registrant; an output unit that outputs an alert when the first detection unit detects that there are a plurality of the containers and the second detection unit does not detect the predetermined product registration behavior for the plurality of the containers; A system comprising:

10. Detecting the presence of a predetermined product other than the products contained in the containers placed in advance in the store; A predetermined product registration action is detected using an image captured of a product registration operation by a product registrant; outputting an alert when the presence of the predetermined product is detected and the predetermined product registration action for the predetermined product is not detected; How to monitor product registration.

11. Detecting that there are multiple containers placed in advance in the store, A predetermined product registration action is detected using an image captured of a product registration operation by a product registrant; outputting an alert when it is detected that there are a plurality of the containers and the predetermined product registration behavior for the plurality of the containers is not detected; How to monitor product registration.

12. On the computer, A first detection function for detecting the presence of a predetermined commodity other than the commodity contained in the container placed in advance in the store; A second detection function for detecting a predetermined product registration behavior using an image captured of a product registration operation by a product registrant; an output function that outputs an alert when the first detection function detects the presence of the predetermined product and the second detection function does not detect the predetermined product registration behavior for the predetermined product; A program to achieve this.

13. On the computer, A first detection function for detecting the presence of a plurality of pre-placed containers in the store; A second detection function for detecting a predetermined product registration behavior using an image captured of a product registration operation by a product registrant; an output function that outputs an alert when the first detection function detects that there are a plurality of the containers and the second detection function does not detect the predetermined product registration behavior for the plurality of the containers; A program to achieve this.

Images