Monitoring system

Abstract

To provide a monitoring system capable of efficiently reducing user's fraudulent payment for a commodity.SOLUTION: A monitoring system includes: a self-checkout machine with which a user registers or pays for a commodity; an imaging device which captures the image of the user who operates the self-checkout machine; an in-store management device which is used in a store in which the self-checkout machine is disposed; a storage device which stores a determination result including identification information of a specific user who is determined to have omitted registration of commodities in the past, on the basis of the image captured by the imaging device and the operation history of the self-checkout machine; and a control device which determines whether a user who visits the store is the specific user, on the basis of the determination result, and notifies the in-store management device of the fact that the user is the specific user, when the user is the specific user.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to a monitoring system for monitoring fraudulent accounting in self-checkout.

Background Art

[0002] In recent years, in stores such as convenience stores or supermarkets, the introduction of self-checkout systems in which users register the products they purchase and perform accounting themselves has been progressing. In self-checkout systems, fraudulent accounting may occur, such as omission of product registration by inexperienced users or intentional non-registration of products by users. Therefore, for example, in Patent Document 1, it is proposed to collate the total weight of all products measured by a weighing scale with the total weight of products registered by a user, and display an error screen when the collation result is inappropriate, thereby suppressing fraudulent accounting by the user.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, when checking for registration omissions using the weight of products as described in Patent Document 1, if a user does not place on the weighing scale a product that they intentionally do not register, or if they place on the weighing scale a product of the same weight that is different from the registered product, there is a risk of erroneously judging that the collation result is appropriate. Also, although it is possible to suppress fraudulent accounting by having store employees monitor the accounting of users, in this case, the number of store personnel cannot be reduced, and the advantages of the self-checkout system cannot be obtained.

[0005] The present invention is for solving the above problems, and an object thereof is to provide a monitoring system capable of efficiently reducing fraudulent accounting of products by users.

Means for Solving the Problems

[0006] The monitoring system according to the present invention includes a self-checkout device for a user to register and account for products, a photographing device for photographing an image of the user operating the self-checkout device, a store management device used in a store where the self-checkout device is arranged, a storage device for storing a determination result including identification information of a specific user determined to have had a registration omission of a product in the past based on the image photographed by the photographing device and the operation history of the self-checkout device, and a control device for determining whether a user visiting the store is the specific user based on the determination result and notifying the store management device if the user is the specific user.

Effects of the Invention

[0007] According to the monitoring system of the present invention, based on the determination result of a specific user determined to have had a registration omission of a product in the past, it is determined whether a user visiting the store is the specific user, and by notifying the store management device, fraudulent accounting by the user can be efficiently reduced.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Embodiments for Carrying Out the Invention

[0009] Hereinafter, with reference to the drawings, the monitoring system 100 according to the embodiment of the present invention will be described. In each figure, the same or corresponding parts are denoted by the same reference numerals, and the description thereof will be omitted or simplified as appropriate. In addition, regarding the configuration shown in each figure, the shape, size, arrangement, etc. can be appropriately changed within the scope of the present invention.

[0010] Embodiment 1. FIG. 1 is a schematic configuration diagram of the monitoring system 100 according to Embodiment 1. The monitoring system 100 of the present embodiment is a system used for monitoring fraudulent accounting of users in a store such as a convenience store or a supermarket. As shown in FIG. 1, the monitoring system 100 includes a plurality of self-checkout devices 1, a plurality of imaging devices 2, an in-store management device 3, a management server 4, and a fraud determination device 5. The plurality of self-checkout devices 1, the plurality of imaging devices 2, the in-store management device 3, the management server 4, and the fraud determination device 5 are connected so as to be able to communicate wired or wirelessly via a network 200 such as a WAN or the Internet.

[0011] A plurality of self-checkout devices 1, a plurality of imaging devices 2, and a store management device 3 are provided in the store. In FIG. 1, one store management device 3 is shown, but a plurality of store management devices 3 may be used in the store. Also, in FIG. 1, a plurality of self-checkout devices 1, a plurality of imaging devices 2, and a store management device 3 provided in one store are shown, but the monitoring system 100 may include a plurality of self-checkout devices 1, a plurality of imaging devices 2, and a plurality of store management devices 3 provided in a plurality of stores. The management server 4 and the fraud determination device 5 are provided in a system headquarters that oversees a plurality of stores.

[0012] The self-checkout device 1 is a POS terminal installed in the checkout area of the store and used by customers who visit the store to register and check out their own products. The imaging device 2 is a device equipped with a camera function and a communication function for capturing an image of the user operating the self-checkout device 1. The imaging device 2 is installed above or on the side of the self-checkout device 1, or on the wall of the checkout area. The imaging device 2 and the self-checkout device 1 may be provided in a one-to-one correspondence, or one imaging device 2 may be provided for a plurality of self-checkout devices 1.

[0013] The store management device 3 is an information terminal such as a tablet, smartphone, or PC operated by store employees. The store management device 3 displays information regarding fraudulent checkouts transmitted from the management server 4.

[0014] The management server 4 stores and manages operation history information including the operation history of the user's self-checkout device 1 used for monitoring fraudulent checkouts, and image information including the image of the user captured by the imaging device 2. In addition, the management server 4 of the present embodiment stores and manages the determination result and fraud information regarding the user's fraudulent checkout transmitted from the fraud determination device 5, and monitors the store visitors based on the stored information. The fraud determination device 5 is a computer such as a PC and is used to determine the presence or absence of the user's fraudulent checkout based on the operation history information and image information stored in the management server 4 by the determination operator.

[0015] Figure 2 is a control block diagram of the monitoring system 100 according to Embodiment 1. As shown in FIG. 2, the self-checkout device 1 includes a control device 11, a display device 12, an operation device 13, a reading device 14, a printing device 15, a settlement device 16, and a communication device 17.

[0016] The control device 11 controls each part of the self-checkout device 1. The control device 11 is composed of a computer including a memory for storing data and programs necessary for control and a processor such as a CPU for executing programs, a dedicated processing circuit such as an ASIC or an FPGA, or both. The control device 11 registers and settles the products read by the reading device 14 as purchased products. In addition, the control device 11 performs settlement processing for the registered products. In the self-checkout device 1 of the present embodiment, settlement can be made in cash, prepaid card, credit card, electronic money, or barcode. When the user selects settlement by cash, prepaid card, credit card, electronic money, or barcode, the control device 11 performs settlement processing for the input to the settlement device 16.

[0017] The display device 12 is, for example, a liquid crystal display, and displays information on products to be registered, information on product settlement, and the like. The operation device 13 is, for example, a touch panel integrally formed with the display device 12. The operation device 13 is operated by the user, and inputs such as payment instructions and selection of a settlement method are input. The reading device 14 is a device that irradiates a product with a laser scanning line and reads a barcode attached to the product. The reading device 14 is a fixed scanner, a handy scanner, or both. The information on the product read by the reading device 14 is transmitted to the control device 11.

[0018] When the settlement by the control device 11 is completed, the printing device 15 prints and outputs a receipt on which information such as the name of the product purchased by the user, the selling price of the product, the deposited amount, and the change amount is described. The settlement device 16 includes a card reader, a barcode reader, a change ticket machine, and a change machine, accepts payments by cash, credit card, prepaid card, electronic money, or barcode, and returns change as necessary.

[0019] The communication device 17 is a network interface that performs two-way communication with the management server 4. The communication device 17 transmits operation history information including the login information of the user using the self-checkout device 1 and the operation history of the user to the management server 4 according to the instructions of the control device 11. Note that the communication device 17 may perform two-way communication with the imaging device 2 or the in-store management device 3 via the network 200.

[0020] The imaging device 2 includes a control device 21, a camera 22, and a communication device 23. The control device 21 controls each part of the imaging device 2. The control device 21 is composed of a computer including a memory for storing data and programs necessary for control and a processor such as a CPU for executing programs, a dedicated processing circuit such as an ASIC or FPGA, or both. The control device 21 controls the camera 22 to acquire an image of the user operating the self-checkout device 1.

[0021] The camera 22 captures an image of the user operating the self-checkout device 1. The image captured by the camera 22 is transmitted to the control device 21. The image captured by the camera 22 may be either a still image or a moving image.

[0022] The communication device 23 is a network interface that performs two-way communication with the management server 4. The communication device 23, in accordance with an instruction from the control device 21, transmits image information including an image captured by the camera 22, the date and time when the image was captured, the identification information of the store where the imaging device 2 is installed, and the identification information of the self-checkout device 1 that is the imaging target, to the management server 4. Note that the communication device 23 may perform two-way communication with the self-checkout device 1 or the in-store management device 3.

[0023] The in-store management device 3 includes a control device 31, a display device 32, an operation device 33, and a communication device 34. The control device 31 controls each part of the in-store management device 3. The control device 31 is composed of a computer including a memory that stores data and programs necessary for control, a processor such as a CPU that executes programs, a dedicated processing circuit such as an ASIC or an FPGA, or both. The control device 31 receives visit information of users who have committed fraudulent accounting in the past from the management server 4, and causes the display device 32 to display the received visit information.

[0024] The display device 32 is, for example, a liquid crystal display, and displays visit information received from the management server 4 in accordance with an instruction from the control device 21. The operation device 33 is, for example, a touch panel integrally formed with the display device 32. An employee operates the operation device 33 to make selections for display and the like.

[0025] The communication device 34 is a network interface that performs two-way communication with the management server 4. The communication device 34 receives visit information from the management server 4. Note that the communication device 34 may perform two-way communication with the self-checkout device 1 or the imaging device 2.

[0026] The management server 4 includes a control device 41, a storage device 42, and a communication device 43. The control device 41 controls each part of the management server 4. The control device 41 is composed of a computer including a memory for storing data and programs necessary for control and a processor such as a CPU for executing programs, a dedicated processing circuit such as an ASIC or an FPGA, or both. The control device 41 stores the information received from the self-checkout device 1 and the fraud determination device 5 in the storage device 42. Further, based on the information stored in the storage device 42, the control device 41 determines whether the user who has come to the store is a user who has had a registration omission of goods in the past, and notifies the in-store management device 3 of the determination result as store visit information.

[0027] The storage device 42 is composed of, for example, a non-volatile semiconductor memory such as a ROM or a flash memory, a volatile semiconductor memory such as a RAM, an HDD, or an SSD, and stores information used for the management of fraudulent accounting in the monitoring system 100. Specifically, the storage device 42 has an image information database 421, an operation history information database 422, and a fraud information database 423. In FIG. 2, "database" is abbreviated as "DB".

[0028] The image information database 421 stores the image information transmitted from the imaging device 2. The image information includes an image of the user captured by the imaging device 2, the date and time when the image was captured, the identification information of the store where the imaging device 2 is installed, and the identification information of the self-checkout device 1 that is the imaging target.

[0029] The operation history information database 422 stores the operation history information received from the self-checkout device 1. The operation history information database 422 stores the multiple operation histories in one accounting as one piece of operation history information. The operation history includes the identification information of the self-checkout device 1 where the operation was performed, the identification information of the store where the self-checkout device 1 is installed, the user's login information, the department (category) of the registered product, the product code, the product name, the number of product points, and the registration information such as the price, the department (category) of the product whose registration was canceled, the product code, the product name, the number of product points, and the cancellation information such as the price, and the date and time when each operation was performed. The user's login information is identification information such as a membership number or a prepaid card number.

[0030] The fraud information database 423 stores the determination result received from the fraud determination device 5 and the fraud information. The determination result is information including the identification information of the user for whom a product registration omission was confirmed in the past and the type of registration omission. The types of registration omission are "fraudulent" when the user deliberately did not register a product and "negligent" when a product registration omission occurred due to the user's unfamiliarity. The fraud information includes the image of the user at the time of the product registration omission and the operation history information at that time. The fraud information is stored as evidence that the user committed fraudulent accounting.

[0031] The communication device 43 is a network interface that performs two-way communication with the self-checkout device 1, the imaging device 2, the in-store management device 3, and the fraud determination device 5. The communication device 43 receives operation history information from the self-checkout device 1, receives image information from the imaging device 2, and receives the determination result and the fraud information from the fraud determination device 5. Also, the communication device 43 transmits the store visit information to the in-store management device 3.

[0032] The fraud determination device 5 includes a control device 51, a display device 52, an operation device 53, and a communication device 54. The control device 51 controls each part of the fraud determination device 5. The control device 51 is composed of a computer including a memory that stores data and programs necessary for control, a processor such as a CPU that executes programs, a dedicated processing circuit such as an ASIC or an FPGA, or both. The control device 51 requests image information and operation history information from the management server 4 in response to an input from a determination operator who determines the presence or absence of fraud accounting by the user, and causes the display device 52 to display the image information and operation history information received from the management server 4. Further, the control device 51 transmits the determination result and fraud information input by the determination operator to the management server 4. In the following description, a user who has had a registration omission of a product in the past is referred to as a "specific user". A specific user is a user who has been determined as "fraudulent" or "negligent" in the determination result of the fraud determination device 5.

[0033] The display device 52 is, for example, a liquid crystal display. The display device 52 displays the image information and operation history information received from the management server 4. The operation device 53 is a mouse, a keyboard, or a touch panel integrally formed with the display device 52, etc. The operation device 53 is used when the determination operator inputs the determination result and fraud information.

[0034] The communication device 54 is a network interface that performs two-way communication with the management server 4. The communication device 54 receives image information and operation history information from the management server 4, and transmits the determination result and fraud information to the management server 4.

[0035] FIG. 3 is a diagram for explaining the operation flow of the monitoring system 100 according to Embodiment 1. First, in the store, the self-checkout device 1 is operated by the user, and login and product registration are performed. Then, the operation history information associated with the operation of the self-checkout device 1 is transmitted to the management server 4 (S1). Further, the imaging device 2 captures an image of the user operating the self-checkout device 1, and the image information including the captured image is transmitted to the management server 4 (S2).

[0036] The management server 4 stores the information received from the self-checkout device 1 and the imaging device 2 in the storage device 42 (S3). Specifically, the management server 4 stores the operation history information received from the self-checkout device 1 in the operation history information database 422, and stores the image information received from the imaging device 2 in the image information database 421. By repeating the processes of steps S1 to S3, the operation history information and the image information from a plurality of self-checkout devices 1 and a plurality of imaging devices 2 are accumulated in the operation history information database 422 and the image information database 421 of the management server 4.

[0037] Then, the fraud determination process by the fraud determination device 5 is performed (S4). The fraud determination process is performed when an investigation request for a user suspected of fraudulent accounting is made from a store where an abnormal accounting such as cancellation of registration has occurred, or a store with a high loss rate. FIG. 4 is a flowchart showing the flow of the fraud determination process by the fraud determination device 5 according to the first embodiment. In this process, first, the operation history information of the user to be determined is extracted from the management server 4 (S41). Here, the identification information of the user to be determined is input to the fraud determination device 5 by the determination operator, and the identification information input from the fraud determination device 5 to the management server 4 is transmitted. The identification information of the user to be determined is communicated to the determination operator at the time of the investigation request. Then, among the operation history information stored in the operation history information database 422 in the management server 4, the operation history information corresponding to the received identification information is extracted and transmitted to the fraud determination device 5.

[0038] Subsequently, the date and time corresponding to the operation history information and the image information of the self-checkout device 1 are extracted from the management server 4 (S42). Here, the date and time corresponding to the extracted operation history information (the date and time when the operation history included in the operation history information was performed) and the identification information of the self-checkout device 1 are input into the fraud determination device 5 by the determination operator, and the date and time and the identification information of the self-checkout device 1 input from the fraud determination device 5 are transmitted to the management server 4. Then, among the image information stored in the image information database 421 in the management server 4, the image information corresponding to the input date and time and the identification information of the self-checkout device 1 is extracted and transmitted to the fraud determination device 5. The extracted operation history information and image information are displayed on the display device 52 (S43).

[0039] The determination operator compares the operation history information and the image information displayed on the display device 52 to check whether there is any omission in the registration of goods. For example, when the goods have been moved to the shelf after registration in the image but there is no registration information in the operation history information, it is determined that there is an omission in the registration of goods. And when an omission in registration occurs, the image and the operation history information at the time of the omission in registration are created as fraud information (S44). The fraud information is, for example, a spreadsheet with the image and the operation history information pasted on it.

[0040] Based on the fraud information, the determination operator determines which of "no fraud", "fraud exists", or "negligence exists" the user to be determined belongs to, and inputs it as the determination result when it belongs to "fraud exists" or "negligence exists" (S45). As an example, when no fraud information is created, that is, when no omission in registration is found as a result of checking the image of the user to be determined, the determination operator determines "no fraud". On the other hand, when fraud information of the user to be determined is created even once, that is, when there is one or more omissions in the registration of goods, the determination operator determines fraud exists or negligence exists according to the number of omissions in registration. For example, when there is only one omission in the registration of goods out of three checkouts, it is determined as "negligence exists" as it is not intentional, and when there are two or more omissions in the registration of goods out of three checkouts, it is determined as "fraud exists" as it is intentional.

[0041] As another example, the determination operator may determine that there is fraud when the number of product registration omissions in one accounting is 1 / 3 or more of the number of purchases. Alternatively, the determination operator may determine whether there is fraud based on the operation history of the user to be determined. Specifically, the determination operator may determine that there is fraud when a product registration omission occurs after the user has input the quantity of the product or performed an operation on the self-checkout device 1 such as a cancellation immediately before.

[0042] Returning to FIG. 3, the fraud information and the determination result input by the determination operator are transmitted to the management server 4 (S5). The management server 4 stores the fraud information and the determination result received from the fraud determination device 5 in the fraud information database 423 (S6). By repeating the processes of steps S4 to S6, information regarding specific users with past product registration omissions is accumulated in the fraud information database 423.

[0043] Then, when the self-checkout device 1 is newly operated by the user, the operation history information is transmitted to the management server 4 (S7). The management server 4 determines whether the user operating the self-checkout device 1 is a specific user based on the login information included in the operation history information and the determination result stored in the fraud information database 423 (S8). Specifically, the control device 41 of the management server 4 determines that the user operating the self-checkout device 1 is a specific user when the identification information of the user included in the login information matches any of the identification information of the specific users included in the determination result. If the user logging in to the self-checkout device 1 is not a specific user (S8: NO), the process proceeds to step S3 and the operation history information is stored.

[0044] On the other hand, when the user who has logged in to the self-checkout device 1 is a specific user (S8: YES), store arrival information notifying the store management device 3 from the management server 4 that the specific user has arrived at the store is transmitted (S9). The store arrival information includes that the specific user has arrived at the store, the identification information of the specific user, the type of the specific user, and the identification information of the self-checkout device 1 that the specific user is operating. Then, the store arrival information is displayed on the display device 52 of the store management device 3 (S10).

[0045] The store employees can know from the display of the store management device 3 that a specific user who had a past omission in product registration has arrived at the store. Also, when the type of the specific user is "with negligence", support for product registration by the user can be provided to prevent the occurrence of product registration omissions. Further, when the type of the specific user is "with fraud", fraud in accounting can be prevented by taking measures such as monitoring the accounting operation of the specific user or performing an accounting check on all products after the specific user's accounting. Note that the processing of steps S1 to S6 and the processing of steps S7 to S10 are executed in parallel.

[0046] As described above, in the monitoring system 100 of the present embodiment, using the fraud determination device 5, it is determined whether there is an omission in product registration by the user based on the past operation history and images of the user, and the determination result is stored in the management server 4. Then, based on the stored determination result and the identification information of the arriving user, it is determined whether the arriving user is a specific user who had a past omission in product registration, and by notifying the store management device 3, fraud in accounting can be efficiently reduced.

[0047] Embodiment 2. The monitoring system 100 of Embodiment 2 will be described. The monitoring system 100 of Embodiment 2 is different from Embodiment 1 in the operation of the management server 4. The configuration and other controls of the monitoring system 100 of Embodiment 2 are the same as those of Embodiment 1.

[0048] FIG. 5 is a diagram for explaining the operation flow of the monitoring system 100 according to the second embodiment. In the present embodiment, the same processes of steps S1 to S6 as in the first embodiment are performed. Further, in the present embodiment, in the management server 4, it is possible to estimate the future store visits of a specific user after the next time and transmit the estimation result to the in-store management device 3.

[0049] Specifically, when the management server 4 receives the determination result and the unauthorized information from the unauthorized determination device 5 and stores them in the unauthorized information database 423 (S6), it extracts the past store visit history of the specific user (S21). Specifically, the control device 41 of the management server 4 uses the identification information of the specific user stored in the unauthorized information database 423 to extract the operation history information corresponding to the specific user from the operation history information stored in the operation history information database 422. Then, the past store visit history of the specific user is extracted from the date and time included in the operation history information.

[0050] Then, the management server 4 estimates the next store visit date of the specific user using the extracted past store visit history (S22). FIG. 6 is an example of the past store visit history of a specific user extracted by the management server 4 according to the second embodiment. In the example of FIG. 6, it can be seen that the specific user visits the store frequently on Saturdays. Therefore, the control device 41 of the management server 4 estimates that the next store visit date of the specific user is Saturday. Note that the method for estimating the next store visit date is not limited to the above, and it may be estimated according to the interval between store visits and the like.

[0051] The management server 4 transmits estimation information including the identification information of a specific user and the estimated next visit date to the in-store management device 3 (S23). The in-store management device 3 displays the visit information received from the management server 4 on the display device 52 (S24). Thereby, the store employees can know the visit date of the specific user estimated. And measures for preventing fraudulent accounting can be taken, such as requesting the dispatch of anti-theft guards on the estimated visit date. Or, the estimated warning time can be estimated using the estimated visit time zone, and measures for preventing fraudulent accounting can be taken at the estimated warning time. Note that also in this embodiment, the processes of steps S7 to S10 of Embodiment 1 are executed in parallel.

[0052] As described above, in the monitoring system 100 of this embodiment, in addition to the effects of Embodiment 1, by estimating the visit date of a specific user, measures for preventing fraudulent accounting in advance can be taken, and fraudulent accounting can be reduced more efficiently.

[0053] Embodiment 3. The monitoring system 100 of Embodiment 3 will be described. The monitoring system 100 of Embodiment 3 is different from Embodiment 1 in the operation after the visit information is displayed on the in-store management device 3. The configuration and other controls of the monitoring system 100 of Embodiment 3 are the same as those of Embodiment 1.

[0054] FIG. 7 is a diagram for explaining the operation flow of the monitoring system 100 according to Embodiment 3. In this embodiment, the same processes of steps S1 to S10 as in Embodiment 1 are executed. And in this embodiment, on the in-store management device 3, the operation history information of the visiting specific user can be displayed in real time.

[0055] Specifically, the identification information of the self-checkout device 1 for which operation history information is to be displayed is input via the operation device 53 of the store management device 3 (S31), and the operation history information is requested from the self-checkout device 1 corresponding to the input identification information (S32). Here, the identification information of the self-checkout device 1 included in the displayed store visit information is input to the store management device 3 by a store employee.

[0056] Then, the operation history information is transmitted from the self-checkout device 1 corresponding to the input identification information to the store management device 3 (S33). The store management device 3 displays the operation history information received from the self-checkout device 1 on the display device 52 in real time (S34).

[0057] The input of the identification information of the self-checkout device 1 is performed via the operation history screen 520 displayed on the display device 52, and the operation history information of the self-checkout device 1 is displayed on the operation history screen 520. FIG. 8 is an example of the operation history screen 520 displayed on the store management device 3 in the third embodiment. As shown in FIG. 8, the operation history screen 520 includes an input unit 521 for inputting the identification information of the self-checkout device 1, a registered product display unit 522 for displaying information on the registered products among the operation history information, and a canceled product display unit 523 for displaying information on the canceled products among the operation history information.

[0058] When a product is registered in the self-checkout device 1, the information on the registered product is displayed on the registered product display unit 522. Similarly, when the registration of a product is canceled in the self-checkout device 1, the information on the canceled product is displayed on the canceled product display unit 523. The store employee can confirm the presence or absence of unregistered products by monitoring the display contents of the registered product display unit 522 and the canceled product display unit 523 and the accounting operations of a specific user.

[0059] As described above, in the monitoring system 100 of the present embodiment, by displaying the operation history information on the in-store management device 3 in real time, the store employees can confirm in real time whether there is any omission in product registration. As a result, fraudulent accounting can be reduced more efficiently. In the above description, the operation history screen 520 is displayed on the in-store management device 3. However, the operation history screen 520 may be displayed on a device other than the in-store management device 3, such as a terminal device held by a shoplifter prevention guard. Further, the in-store management device 3 may be communicably connected to the imaging device 2, and an image of a specific user captured by the imaging device 2 may be displayed in real time on the in-store management device 3 or a terminal device together with the operation history screen 520. Thereby, it is possible to confirm in real time the occurrence of registration omission even from a remote location where the specific user cannot be directly visually recognized.

[0060] The above is the description of the embodiment. However, the present invention is not limited to the above embodiment, and various modifications or combinations can be made without departing from the gist of the present invention. For example, in the above embodiment, the self-checkout device 1 and the imaging device 2 are described as separate devices. However, the imaging device 2 may be incorporated in the self-checkout device 1. Further, in the above embodiment, the self-checkout device 1 is installed in the accounting area of the store. However, the self-checkout device 1 may be a terminal such as a tablet attached to a shopping cart or a smartphone. In this case, the imaging device 2 may be a camera incorporated in the tablet or the smartphone.

[0061] Also, each function of each device may be implemented by another device. For example, the fraud determination process of the fraud determination device 5 may be implemented by the management server 4. In this case, the fraud determination device 5 can be omitted. Further, the determination of the visit of a specific user in the management server 4 may be implemented by the in-store management device 3. Furthermore, part or all of each database stored in the storage device 42 of the management server 4 may be stored in a device other than the management server 4.

[0062] In the above-described embodiment, the determination as to whether a user is a specific user is made based on the identification information input by the user at the time of login. However, the present invention is not limited to this. For example, an image of the face of the user photographed by the photographing device 2 may be stored as the identification information of the user, and when the user visits the store, face authentication of the user may be performed to determine whether the user is a specific user. In this case, even when the user has not registered the identification information, it is possible to determine whether the user is a specific user and to determine the visit to the store.

[0063] Furthermore, in the above-described embodiment, the process performed by the determination operator may be automatically performed by the fraud determination device 5. For example, when the control device 51 of the fraud determination device 5 extracts the operation history information of the user to be determined from the management server 4, the control device 51 may automatically extract the corresponding image information based on the date and time information included in the operation history information and the identification information of the self-checkout device 1, and display it on the display device 52. In addition, the control device 51 of the fraud determination device 5 may perform image recognition on the extracted image to identify the movement operation of the product, and automatically determine whether there is a registration omission of the product by comparing it with the operation history information. Then, the control device 51 of the fraud determination device 5 may automatically create fraud information and the determination result based on the determination result, and transmit them to the management server 4.

[0064] In the above-described embodiment, the fraud determination process is performed when a request for investigation of a user suspected of fraud accounting is made from a store where an abnormal accounting such as cancellation of registration has occurred, or a store with a high loss rate. However, the present invention is not limited to this. For example, the fraud determination process may be performed for all users in an arbitrary time period of a store where an abnormal accounting has occurred or a store with a high loss rate. In particular, when the fraud determination process is automatically performed as described above, it is possible to quickly perform the fraud determination for a plurality of users.

Explanation of Signs

[0065] 1 Self-checkout device, 2 Photographing device, 3 In-store management device, 4 Management server, 5 Fraud determination device, 11 Control device, 12 Display device, 13 Operating device, 14 Reading device, 15 Printing device, 16 Settlement device, 17 Communication device, 21 Control device, 22 Camera, 23 Communication device, 31 Control device, 32 Display device, 33 Operating device, 34 Communication device, 41 Control device, 42 Storage device, 43 Communication device, 51 Control device, 52 Display device, 53 Operating device, 54 Communication device, 100 Monitoring system, 200 Network, 421 Image information database, 422 Operation history information database, 423 Fraud information database, 520 Operation history screen, 521 Input section, 522 Registered product display section, 523 Cancelled product display section.

Claims

1. A self-checkout device for a user to register and account for goods, A photographing device for photographing an image of the user operating the self-checkout device, An in-store management device used in a store where the self-checkout device is arranged, A storage device that stores a determination result including identification information of a specific user determined to have had a registration omission of goods in the past based on the image photographed by the photographing device and the operation history of the self-checkout device, A monitoring system comprising: a control device that determines whether a user visiting the store is the specific user based on the determination result, and notifies the in-store management device when the user is the specific user.

2. The monitoring system according to claim 1, wherein the control device determines whether a user visiting the store is the specific user based on the determination result and the identification information of the user input to the self-checkout device.

3. The monitoring system according to claim 1, wherein the control device determines whether a user visiting the store is the specific user based on the image photographed by the photographing device.

4. The determination result includes information regarding the type of the past registration omission of the specific user, The monitoring system according to any one of claims 1 to 3, wherein the type includes intentional fraud and non-intentional negligence.

5. The monitoring system according to any one of claims 1 to 3, wherein the control device estimates a future visit date of the specific user based on the identification information and the operation history of the specific user, and notifies the in-store management device.

6. The monitoring system according to any one of claims 1 to 3, wherein the in-store management device includes a display device that displays the operation history of the self-checkout device in real time.

Images