Information processing device, control method, and program

Abstract

To provide, as one object, a technology for accurately checking registration of a commodity as a settlement target.SOLUTION: A registration checking device (2000) determines, for a target transaction, whether a relation between the number of times that a scan operation is performed and the number of commodities registered as settlement targets satisfies a predetermined condition. The scan operation is detected by analyzing a moving image (12). The registration checking device (2000) performs predetermined output processing when the above-described relation does not satisfy the predetermined condition. Here, the predetermined condition is a condition satisfied when the number of registered commodities is more than or equal to the detected number of times of the scan operation.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a technique for registering a product as an object to be settled.

Background Art

[0002] When purchasing a product at a store, the product to be purchased is read by a scanner (such as a barcode reader), and the product is registered as an object to be settled. The device used for registering the product is called a product registration device or the like.

[0003] Regarding the scanning of products by such a product registration device, techniques for detecting unauthorized operations have been developed. Patent Document 1 discloses a technique for detecting fraud by comparing the number of scanned products with the number of products passing detected by an optical sensor provided near the scanner. More specifically, in an operation where a user causes a product to be scanned by a scanner (hereinafter referred to as a scanning operation), two sensors of a reflection-type optical detection structure including a light emitter and a light receiver are provided in a separated state at a location where the user's hand passes. Then, when an object is detected by the two sensors in a predetermined order, the number of products passing is counted.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In the invention of Patent Document 1, since a sensor composed of a light emitter and a light receiver is used to detect the passage of a product, it is considered that detection omissions and false detections are likely to occur. For example, when a user moves their body, if the user's body happens to pass in front of the two sensors, that movement may be falsely detected as a scanning operation.

[0006] The present invention has been made in view of the above problems, and one of its objects is to provide a technique for accurately confirming that a product has been registered as a settlement target.

Means for Solving the Problems

[0007] The first registration confirmation device of the present invention includes a scan operation detection unit that detects a scan operation by analyzing a video including the user's hand, and for a target transaction, determines whether the relationship between the number of times the scan operation has been performed and the number of products registered as settlement targets satisfies a predetermined condition, and an output unit that performs a predetermined output process when the relationship does not satisfy the predetermined condition. The predetermined condition is a condition that is satisfied when the number of registered products is greater than or equal to the number of detected scan operations.

[0008] The second registration confirmation device of the present invention includes a scan operation detection unit that detects a scan operation by analyzing a video including the user's hand, a determination unit that determines whether a product has been registered as a settlement target according to the scan operation, and an output unit that performs a predetermined output process when the product has not been registered as a settlement target according to the scan operation. In the video, when the hand enters the reading area of the scanner and then exits the reading area, a scan operation is detected.

[0009] The first control method of the present invention is executed by a computer. The control method includes a scan operation detection step of detecting a scan operation by analyzing a video including the user's hand, a determination step of determining whether the relationship between the number of times the scan operation has been performed and the number of products registered as settlement targets for a target transaction satisfies a predetermined condition, and an output step of performing a predetermined output process when the relationship does not satisfy the predetermined condition. The predetermined condition is a condition that is satisfied when the number of registered products is greater than or equal to the number of detected scan operations.

[0010] The second control method of the present invention is executed by a computer. The control method includes a scan operation detection step of detecting a scan operation by analyzing a video including a user's hand, a determination step of determining whether a product is registered as a settlement target according to the scan operation, and an output step of performing a predetermined output process when the product is not registered as a settlement target according to the scan operation. In the video, when a hand enters the reading area of the scanner and then exits the reading area, a scan operation is detected.

Effects of the Invention

[0011] According to the present invention, a technique for accurately confirming that a product is registered as a settlement target is provided.

Brief Description of the Drawings

[0012]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Embodiments for Carrying Out the Invention

[0013] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, the same components are denoted by the same reference numerals, and the description will be omitted as appropriate. Also, unless otherwise specified, in each block diagram, each block represents a configuration of a functional unit, not a configuration of a hardware unit. In the following description, unless otherwise specified, various predetermined values (such as threshold values) are stored in advance in a storage device accessible from the functional components that utilize the values.

[0014] [Embodiment 1] <Overview> FIG. 1 is a diagram for explaining the overview of the registration confirmation device 2000 of the present embodiment. Note that FIG. 1 is an illustration for facilitating the understanding of the registration confirmation device 2000, and the functions of the registration confirmation device 2000 are not limited to those shown in FIG. 1.

[0015] The registration confirmation device 2000 is used to confirm whether a product has been registered as an item to be settled (as an object for payment of the price) when a product is scanned by the scanner 22 of the product registration device 20 in the store where the product is purchased. The product registration device 20 is a device used to register a product as an item to be settled. The scanner 22 is a reader that reads a predetermined symbol such as a barcode attached to the product, a camera that images the product, or the like. Hereinafter, a product registered as an item to be settled is also referred to as a registered product.

[0016] Here, in the store where the registration confirmation device 2000 is operated, the device used for registering products (i.e., the product registration device 20) and the device used for settlement (payment of the price) of the registered products may be provided integrally or separately. The device used for settlement of the registered products is called a settlement device or the like. Note that the user who operates the product registration device 20 to register products may be a store clerk or a customer.

[0017] Here, when a product is normally scanned by each scan operation, one or more products are registered each time the scan operation is performed. Therefore, when a product is normally scanned in each of all the scan operations performed for one transaction, a predetermined relationship holds between the number of registered products registered for the transaction and the number of scan operations performed for the transaction. Specifically, the number of registered products is equal to or greater than the number of scan operations. Note that the number of registered products is more than the number of scan operations when a plurality of products are scanned in one scan operation. Also, "one transaction" refers to a transaction for one or more products that are settled together at once (a transaction for one or more products purchased by one customer at once).

[0018] In view of the relationship between the number of scan operations and the number of registered products described above, the registration confirmation device 2000 checks whether a product is normally registered in all scan operations in a certain transaction by performing the following processing. Here, the transaction that is the target of the check as to whether a product is normally registered is called the target transaction. The registration confirmation device 2000 detects a scan operation by analyzing the video 12 obtained from the camera 10 that images the user's actions. The registration confirmation device 2000 determines whether the relationship between the number of detected scan operations and the number of registered products satisfies a predetermined condition. The predetermined condition is a condition that holds when the number of registered products is equal to or greater than the number of scan operations.

[0019] When the above-described predetermined conditions are not satisfied (that is, when there is a scanning operation in which a product is not registered), the registration confirmation device 2000 performs a predetermined output process. For example, this output process is the output of a message that causes the user to recognize the existence of a product that has not been registered.

[0020] <An example of the operation and effect> According to the registration confirmation device 2000 of the present embodiment, it is determined whether or not the relationship between the number of scanning operations and the number of registered products satisfies a predetermined condition. Then, when the predetermined condition is not satisfied, a predetermined output process is performed. Here, the registration confirmation device 2000 detects the scanning operation by analyzing the video 12 in which the scanning operation is recorded. Therefore, compared with the case where a sensor that senses the passage of an object is used to detect the scanning operation, the scanning operation can be detected with high accuracy. As a result, the number of scanning operations can be accurately counted. Therefore, it is possible to accurately confirm whether or not a product has been registered as a settlement target by the scanning operation.

[0021] Hereinafter, the present embodiment will be described in more detail.

[0022] <An example of the functional configuration> FIG. 2 is a diagram illustrating the functional configuration of the registration confirmation device 2000 according to Embodiment 1. The registration confirmation device 2000 includes a scanning operation detection unit 2020, a first determination unit 2040, and a first output unit 2060. The scanning operation detection unit 2020 detects the scanning operation by analyzing the video 12. The first determination unit 2040 determines whether or not the number of detected scanning operations and the number of registered products satisfy a predetermined condition. The first output unit 2060 performs a predetermined output process when the above-described predetermined condition is not satisfied.

[0023] <An example of the hardware configuration of the registration confirmation device 2000> Each functional component of the registration confirmation device 2000 may be implemented by hardware (e.g., a hard-wired electronic circuit, etc.) that realizes each functional component, or may be implemented by a combination of hardware and software (e.g., a combination of an electronic circuit and a program that controls it, etc.). Hereinafter, the case where each functional component of the registration confirmation device 2000 is implemented by a combination of hardware and software will be further described.

[0024] FIG. 3 is a diagram illustrating a computer 1000 for realizing the registration confirmation device 2000. The computer 1000 is an arbitrary computer. For example, the computer 1000 is a stationary computer such as a PC (Personal Computer) or a server machine. In addition, for example, the computer 1000 may be a portable computer such as a smartphone or a tablet terminal. When the registration confirmation device 2000 and the product registration device 20 are provided integrally, the computer 1000 can be any computer (such as a POS (Point of Sales) terminal) capable of realizing the product registration device 20.

[0025] The computer 1000 may be a dedicated computer designed to realize the registration confirmation device 2000, or may be a general-purpose computer. In the latter case, for example, by installing a predetermined application on the computer 1000, the functions of the registration confirmation device 2000 are realized in the computer 1000. The above application is composed of programs for realizing each functional component of the registration confirmation device 2000.

[0026] The computer 1000 has a bus 1020, a processor 1040, a memory 1060, a storage device 1080, an input / output interface 1100, and a network interface 1120. The bus 1020 is a data transmission path for the processor 1040, the memory 1060, the storage device 1080, the input / output interface 1100, and the network interface 1120 to transmit and receive data to and from each other. However, the method of connecting the processor 1040, etc. to each other is not limited to bus connection.

[0027] The processor 1040 is various processors such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and an FPGA (Field-Programmable Gate Array). The memory 1060 is a main memory device realized using, for example, a RAM (Random Access Memory). The storage device 1080 is an auxiliary storage device realized using a hard disk, an SSD (Solid State Drive), a memory card, or a ROM (Read Only Memory).

[0028] The input / output interface 1100 is an interface for connecting the computer 1000 and an input / output device. For example, an input device such as a keyboard and an output device such as a display device are connected to the input / output interface 1100.

[0029] The network interface 1120 is an interface for connecting the computer 1000 to a communication network. This communication network is, for example, a LAN (Local Area Network) or a WAN (Wide Area Network). In FIG. 3, a camera 10 and a product registration device 20 are connected to the network interface 1120.

[0030] The method of communicably connecting the camera 10 and the product registration device 20 to the computer 1000 is not limited to the method of connecting via a network. Also, the camera 10 and the product registration device 20 do not necessarily have to be communicably connected to the computer 1000. In this case, for example, information is exchanged between these devices via a storage device that can be commonly accessed from these devices. Also, the registration confirmation device 2000 and the product registration device 20 may be provided integrally. In this case, the computer 1000 functions not only as the registration confirmation device 2000 but also as the product registration device 20.

[0031] The storage device 1080 stores program modules (program modules that implement the aforementioned applications) that implement each functional component of the registration confirmation device 2000. The processor 1040 reads out and executes these program modules in the memory 1060, thereby implementing the functions corresponding to the respective program modules.

[0032] <Flow of processing> FIG. 4 is a flowchart illustrating the flow of processing executed by the registration confirmation device 2000 according to Embodiment 1. The scan operation detection unit 2020 analyzes the video 12 for the target transaction and detects the scan operation from the video 12, thereby identifying the number of scan operations performed for the target transaction (S102). The first determination unit 2040 determines whether the number of scan operations and the number of registered products for the target transaction satisfy a predetermined condition (S104). If the predetermined condition is not satisfied (S104: NO), the first output unit 2060 performs a predetermined output process (S106). If the predetermined condition is satisfied (S104: YES), the processing in FIG. 4 ends.

[0033] Note that the first output unit 2060 may perform some output process even when the predetermined condition is satisfied. However, different output processes are performed depending on whether the predetermined condition is satisfied or not. For example, when the predetermined condition is not satisfied, a message prompting the user to re-register the product is output, and when the predetermined condition is satisfied, a message indicating that the product registration has been completed normally is output. Further details of the output process will be described later.

[0034] <Flow of operations on the product registration device 20> To facilitate understanding of the usage mode of the registration confirmation device 2000, an example of the flow of user operations performed by the user on the product registration device 20 in the target transaction will be described. The flow of user operations described here is the same in Embodiment 2 described later.

[0035] FIG. 5 is a diagram illustrating the flow of user operations on the product registration device 20. The user performs an operation indicating the start of product registration on the product registration device 20 (S202). In response to this operation, the product registration device 20 shifts to a state where it accepts product registration. For example, this operation is an operation of pressing a button displayed as "Start scanning the product".

[0036] The user performs a scanning operation on each product to be purchased and causes each product to be read by the scanner 22 (S204, S206: NO). When the scanning operation has been completed for all products (S206: YES), the user performs an operation indicating the end of product registration (S208). In response to this operation, the state of the product registration device 20 transitions to a state where it does not accept product registration. Also, a notification indicating that product registration has been completed is sent from the product registration device 20 to the registration confirmation device 2000. However, when the product registration device 20 and the registration confirmation device 2000 are provided integrally, this notification is unnecessary.

[0037] In response to the completion of product registration, the registration confirmation device 2000 compares the number of scanning operations with the number of registered products (S104). It is preferable that the detection of the scanning operation and the counting of its number are performed in parallel while product registration is being performed.

[0038] Note that the imaging by the camera 10 may be performed at all times, or may be performed only during the period from the time when product registration is started (registration start time) to the time when product registration is completed (registration end time). In FIG. 5, the registration start time and the registration end time are the times of S202 and S208, respectively. When imaging is performed only during the period from the registration start time to the registration end time, for example, the camera 10 receives from the product registration device 20 a notification indicating that an operation indicating the start of product registration has been performed, and a notification indicating that an operation indicating the end of product registration has been performed. The camera 10 starts imaging in response to receiving the former notification and ends imaging in response to receiving the latter notification.

[0039] <Acquisition of Video 12> The scan operation detection unit 2020 analyzes the video 12 corresponding to the target transaction and detects the scan operation from the video 12. To that end, the scan operation detection unit 2020 acquires the video 12. Here, the scan operation detection unit 2020 may acquire all of the images (video frames) constituting the video 12 together, or may acquire the video frames in multiple portions. In the latter case, it is preferable that the scan operation detection unit 2020 performs in parallel the process of repeatedly acquiring video frames and the process of analyzing the already acquired video frames to detect the scan operation.

[0040] There are various methods for acquiring the video frames constituting the video 12 in multiple portions. For example, each time a new video frame is generated by the camera 10, the scan operation detection unit 2020 acquires the new video frame. Additionally, for example, the scan operation detection unit 2020 may periodically (for example, at a frequency of once per second) acquire the unacquired video frames together. For example, when a 30fps (frames / second) camera is used as the camera 10 and video frames are acquired at a frequency of once per second, 30 video frames are acquired at a time.

[0041] The scan operation detection unit 2020 may acquire the video 12 from the camera 10, or may acquire the video 12 from a source other than the camera 10. In the latter case, for example, the scan operation detection unit 2020 accesses the storage device in which the video 12 is stored by the camera 10 to acquire the video 12.

[0042] The scan operation detection unit 2020 uses, as video 12, the video generated by the camera 10 from the registration start time to the registration end time. For example, the scan operation detection unit 2020 acquires information indicating the registration start time and the registration end time from the merchandise registration device 20. Then, the scan operation detection unit 2020 acquires, as video 12, the portion of the video generated by the camera 10 between the registration start time and the registration end time. Additionally, for example, as described above, imaging by the camera 10 may be performed only from the registration start time to the registration end time. In this case, the scan operation detection unit 2020 acquires, as video 12, the entire video generated by the camera 10 for the target transaction.

[0043] <Detection of scan operation: S102> The scan operation detection unit 2020 detects a scan operation from video 12 (S102). For example, the scan operation detection unit 2020 has a detection model that detects a scan operation from the video in response to the input of a video in which the scan operation is recorded. Here, the "video in which the scan operation is recorded" means that the video is generated by imaging the scan operation with a camera.

[0044] For example, the detection model sequentially receives the time-series video frames constituting the video, and when a scan operation is detected from one or more received video frames, outputs an indication that the scan operation has been detected. After that, the detection model initializes its internal state and then further receives subsequent video frames. By doing so, each of the one or more scan operations included in video 12 is sequentially detected. Note that it is preferable for the detection model to output information indicating the start point and the end point of the scan operation (for example, the frame numbers of the video frames at the start point and the end point of the scan operation) when a scan operation is detected from the video.

[0045] The detection model is pre-trained using videos in which the scanning operation is recorded. This training is performed using, at least, videos in which the scanning operation is recorded as positive example teacher data. Also, negative example data (i.e., videos in which the scanning operation is not recorded) may be further used for the training.

[0046] Various videos can be used as teacher data. For example, as positive example teacher data (i.e., videos in which the scanning operation is recorded), videos including a series of actions where 1) a hand is positioned outside the reading area of the scanner 22 (i.e., the area where a product can be scanned), 2) the hand enters the reading area, and 3) the hand exits the reading area are used.

[0047] However, only operations performed with an object held in the hand may be treated as the scanning operation. That is, videos including a series of actions where 1) a hand holding an object is positioned outside the reading area of the scanner 22, 2) the hand enters the reading area while holding the object, and 3) the hand exits the reading area while holding the object are used as positive example teacher data. In this case, videos in which an operation where the object is not held is recorded in any of 1) to 3) are treated as negative example teacher data (i.e., videos in which the scanning operation is not recorded).

[0048] Also, only operations performed with the hand in a predetermined shape may be treated as the scanning operation. That is, videos including a series of actions where 1) a hand in a predetermined shape is positioned outside the reading area of the scanner 22, 2) the hand enters the reading area while maintaining the shape, and 3) the hand exits the reading area while maintaining the shape are used as positive example teacher data. In this case, videos in which an operation where the hand shape is not in the predetermined shape is recorded in any of 1) to 3) are treated as negative example teacher data (i.e., videos in which the scanning operation is not recorded).

[0049] Furthermore, by combining the above two examples, only the operations performed in the state of "an object being held by a hand having a predetermined shape" may be treated as scanning operations. In this case, when the object is being held but the shape of the hand is not the predetermined shape, or when the shape of the hand is the predetermined shape but the object is not being held, it is not treated as a scanning operation.

[0050] In addition, for example, only the operations that satisfy conditions such as the moving direction of the hand being a predetermined direction or the hand staying in the reading area of the scanner 22 for a predetermined time or more may be treated as scanning operations. Also, it may be arranged not to treat an operation as a scanning operation if the palm of the hand is captured.

[0051] Note that the method of detecting the scanning operation from the video 12 is not limited to the method using a learned model, and various methods of detecting a predetermined operation from the video can be used.

[0052] <Comparison between the number of scanning operations and the number of registered products: S104> The first determination unit 2040 determines whether a predetermined condition is satisfied for these relationships by comparing the registered number of scanning operations and the number of registered products for the target transaction (S104). The predetermined condition is a condition that holds when the number of registered products is equal to or greater than the number of scanning operations. For example, conditions such as "the number of registered products is equal to or greater than the number of scanning operations" or "the value obtained by subtracting the number of scanning operations from the number of registered products is 0 or greater". Note that in the case where the number of products that can be scanned in one scanning operation is limited to one (that is, the scanner 22 can scan only one product at a time), the predetermined condition is preferably a condition that holds when the number of registered products matches the number of scanning operations (conditions such as "the number of registered products matches the scanning operation" or "the value obtained by subtracting the number of scanning operations from the number of registered products is 0").

[0053] In order to determine whether or not the above-mentioned predetermined conditions are satisfied, the first determination unit 2040 acquires the number of scan operations and the number of registered products for the target transaction (i.e., the currently ongoing transaction). In other words, the first determination unit 2040 specifies, for the period from the start time of registration to the end time of registration for the target transaction, the number of scan operations performed during that period and the number of products registered during that period.

[0054] In this regard, as described above, the scan operation detection unit 2020 analyzes the video 12 from the start time of registration to the end time of registration and detects the scan operations included therein. Therefore, the first determination unit 2040 specifies the number of scan operations by counting the number of detected scan operations. Also, the number of registered products can be obtained from the product registration device 20. Therefore, the first determination unit 2040 acquires the number of registered products from the product registration device 20.

[0055] <Output when the predetermined conditions are not satisfied: S106> If the relationship between the number of scan operations and the number of registered products for the target transaction does not satisfy the predetermined conditions (S104: NO), the first output unit 2060 performs a predetermined output process (S106). As the predetermined output process, various processes can be adopted. Hereinafter, some examples of the predetermined output process will be illustrated.

[0056] <<Example 1 of the predetermined output process>> For example, the first output unit 2060 outputs, to the product registration device 20, a message for causing the user to recognize the existence of unregistered products. For example, this message is displayed on the display device of the product registration device 20 or output as sound from the speaker of the product registration device 20.

[0057] Here, the messages output to the product registration device 20 will be more specifically exemplified while explaining the screen transition in the product registration device 20. FIG. 6 is a diagram illustrating the screen transition of the product registration device 20. Note that each screen in FIG. 6 is a simplified version for the explanation of the registration confirmation device 2000, and various information not shown here can be displayed on each screen during actual operation. Also, in FIG. 6, a case where the product registration device 20 and the payment device are provided integrally is assumed.

[0058] Screen 30 is the screen displayed on the display device of the product registration device 20 before the registration of the product starts. When the user presses button 32, the screen displayed on the display device transitions to screen 40. Screen 40 is displayed on the display device of the product registration device 20 while the product is being registered. Note that both the number of detected scan operations and the number of registered products may be displayed on screen 40. In this case, the registration confirmation device 2000 counts the number of scan operations in real time. By displaying the number of scan operations and the number of registered products on the screen in this way, the user can recognize that the scan operation is being monitored, so a crime prevention effect can be expected.

[0059] After the scan operation is completed for all products, the user presses button 42. In response to button 42 being pressed, the registration confirmation device 2000 determines whether the relationship between the number of scan operations and the number of registered products satisfies a predetermined condition.

[0060] If the predetermined condition is satisfied, the target transaction transitions from product registration to payment. Therefore, the screen displayed on the display device transitions to the payment screen (a screen for making payment). Here, the configuration of the payment screen is arbitrary.

[0061] On the other hand, when the specified conditions are not met, the screen displayed on the display device transitions to screen 50. Screen 50 contains a message prompting the user to re-register the product. When re-registering the product, the user presses button 52. When button 52 is pressed, the state of the product registration device 20 transitions to a state where it accepts product registration again. Therefore, the screen displayed on the display device transitions to screen 40.

[0062] However, the screen when re-registering the product may be a screen different from screen 40. For example, as will be described later, in the case of re-registering only the products that were not registered, it is necessary for the user to be able to identify the products that were not registered. Therefore, information for identifying the products that were not registered is displayed on the screen. Information for identifying the products that were not registered will be described later.

[0063] When the user does not re-register the product, the user presses button 54. When button 54 is pressed, the target transaction transitions from product registration to settlement. Therefore, the screen displayed on the display device transitions to the settlement screen. Note that if button 54 is not displayed and the specified conditions are not met, it may be required that the product must be re-registered.

[0064] Here, "re-registering the product" may mean re-registering all the products, or it may mean re-registering only the products that were not registered (in other words, adding products for registration). In the former case, when button 52 is pressed, the product registration device 20 clears all the information of the products that have already been registered and transitions to a state where it accepts product registration. Then the user performs the scanning operation again for all the products.

[0065] On the other hand, when re-registering only for unregistered products, if button 52 is pressed, the product registration device 20 transitions to a state where it accepts product registration without clearing the information of already registered products. Therefore, the user performs the scanning operation again only for unregistered products.

[0066] When re-registering only for unregistered products, the user needs to identify the unregistered products. Therefore, the first output unit 2060 outputs information for the user to identify the unregistered products. For example, the first output unit 2060 presents an image in which the scanning operations performed on unregistered products are recorded to the user. By viewing this image, the user can identify which products were not registered.

[0067] The image in which the scanning operations of unregistered products are recorded can be identified by comparing the time when each product was registered and the period during which each scanning operation was performed. Specifically, when a product is registered by a certain scanning operation, the registration of that product is performed within the period from the start time to the end time of that scanning operation (hereinafter referred to as the period of the scanning operation). Therefore, if the time when a certain product was registered is within the period of a certain scanning operation, it can be understood that the product was registered by that scanning operation.

[0068] Therefore, the first output unit 2060 associates a scanning operation with the product registered by the scanning operation by comparing the period of each scanning operation with the time when each product was registered. More specifically, the first output unit 2060 associates a scanning operation with the product registered during the period of the scanning operation. According to this process, a product is associated with a scanning operation in which the product was registered, while a product is not associated with a scanning operation in which the product was not registered. Therefore, the first output unit 2060 identifies a scanning operation in which a product was not associated as a scanning operation in which the product was not registered. Note that the period of each scanning operation can be specified, for example, as the period from the generation time of the video frame detected as the start point of the scanning operation to the generation time of the video frame detected as the end point of the scanning operation.

[0069] FIG. 7 is a diagram conceptually showing the association between a scanning operation and a product. In FIG. 7, three scanning operations, namely, scanning operations X to Z, are performed. On the other hand, there are two registered products, A and B. Therefore, there is one scanning operation in which no product was registered.

[0070] Here, products A and B are registered within the periods of scanning operations X and Z, respectively. Therefore, the first output unit 2060 associates products A and B with scanning operations X and Z, respectively. And no product is associated with scanning operation Y. Therefore, the first output unit 2060 identifies scanning operation Y as a scanning operation in which no product was registered.

[0071] The first output unit 2060 outputs information about a scanning operation in which no product was registered. For example, this information is an image in which the scanning operation was recorded. This image may be a plurality of video frames (i.e., a video) from the start point to the end point of the scanning operation, or may be a part of the video frames (e.g., one video frame) included in the video.

[0072] In the latter case, it is preferable that an image that is easy for the user to understand the product is output. Therefore, for example, the first output unit 2060 identifies, from among each video frame constituting the video in which the scan operation in which the product was not registered is recorded, the one with the largest size of the image area representing the product, and outputs that video frame.

[0073] Here, when the scanner 22 is realized by a camera, instead of the image in which the scan operation is recorded, or together with the image in which the scan operation is recorded, an image generated by the scanner 22 may be output. Compared with the image in which the scan operation is recorded, in the image generated by the scanner 22, it is considered that the product is imaged large, so it becomes easier to understand the product.

[0074] For example, the first output unit 2060 outputs, among a plurality of images (videos) generated by the scanner 22, those generated during the period of the scan operation in which the product was not registered. In addition, for example, the first output unit 2060 may analyze the scan operation recorded in the video 12 to identify the time point when the product passed through the center of the reading range of the scanner 22 during the scan operation, and output the image generated by the scanner 22 at that time point.

[0075] In addition, each output image may be not the entire image generated by the camera, but only a partial image area including the product cut out from it. Here, existing technologies can be used for the technology of cutting out only a partial image area including the product from the image. For example, the image area including the product can be obtained as an image area of a predetermined size including the hand. In addition, for example, the image area including the product can be obtained by using a model learned to detect only the image area representing the product from the input image.

[0076] The information for grasping the unregistered products is not limited to the information regarding the scanning operations in which the products were not registered. For example, the first output unit 2060 may present a list of the registered products to the user. In this case, the user can grasp which products were not registered by comparing the presented list with the products that the user intends to purchase. For example, assume that products A and B are shown in the presented list, and the products that the user intends to purchase are A, B, and C. In this case, since the user can grasp that product C was not registered, the user can re-register product C.

[0077] In addition, for example, the first output unit 2060 may identify the unregistered products by analyzing the image in which the scanning operations in which the products were not registered were recorded, and output the identification information (such as the name) of those products. Here, existing technologies can be used for the technology of identifying the products included in the image by image analysis.

[0078] <<Example 2 of Predetermined Output Processing>> For example, the first output unit 2060 may perform an output that can grasp that a predetermined condition was not satisfied regarding the relationship between the number of scanning operations and the number of registered products (that is, an output that can grasp that there are scanning operations in which the products were not registered) to the relevant persons in the store, such as store clerks and security guards. For example, the first output unit 2060 transmits the predetermined output information to the mobile terminals held by the store clerks and security guards. In addition, for example, when the product registration device 20 in which the scanning operation in which the product was not registered was detected is the product registration device 20 operated by the customer, the first output unit 2060 may output the above-described output information to the product registration device 20 operated by the store clerk.

[0079] The information included in the output information is various. For example, the output information includes the identification information of the merchandise registration device 20 in which a scanning operation for which a product was not registered was performed, and an image (particularly, a face image) of the user who performed the scanning operation. For example, the user's image can be obtained from a camera installed so that the user's face or the like can be imaged, such as a surveillance camera. Further, when the camera 10 is provided so that it can image not only the user's hand but also the user's face or the like, the first output unit 2060 may obtain the user's image from the camera 10.

[0080] The predetermined output information may include an image in which a scanning operation for which a product was not registered was recorded. By viewing this image, a store clerk or the like can somewhat grasp whether the scanning operation for which a product was not registered was an operation that may have been performed deliberately so that the product would not be registered or just a scanning mistake. Note that the method for specifying the image in which the scanning operation for which a product was not registered was recorded is as described above.

[0081] Further, the first output unit 2060 may use sound, light, or the like so that a store clerk or the like can grasp that a predetermined condition has not been satisfied. For example, when a lamp that is visible from the surroundings is provided in the merchandise registration device 20, the first output unit 2060 turns on or blinks the lamp in a predetermined manner to perform an output indicating that a predetermined condition has not been satisfied in the target transaction in the merchandise registration device 20. Additionally, for example, a predetermined voice may be output from a speaker provided in the merchandise registration device 20.

[0082] <<Change of Output Processing According to Characteristics of Scanning Operation>> For scan operations where a product is not registered, there may be cases where the product was not registered unintentionally (even though the scan operation was performed with the intention of registering the product, the product was not registered) and cases where the product was intentionally not registered (where there was no intention to register the product). Hereinafter, the former will be referred to as a non-illegal scan operation, and the latter will be referred to as an illegal scan operation. The registration confirmation device 2000 may determine whether a scan operation in which a product was not registered is a non-illegal scan operation or an illegal scan operation, and perform different output processes according to the determination result.

[0083] For example, when the scan operation in which a product was not registered is a non-illegal scan operation, the first output unit 2060 performs the output process of Example 1 described above. On the other hand, when it is an illegal scan operation, the first output unit 2060 performs the output process of Example 2 described above, or both the output processes of Example 1 and Example 2. By doing so, when the user has not performed an illegal operation, it is only necessary to prompt the user to re-register the product, and when the user has performed an illegal operation, countermeasures such as notifying the store clerk, etc. can be taken to improve the security system.

[0084] When realizing such a process, the scan operation detection unit 2020 is configured to be able to distinguish between a non-illegal scan operation and an illegal scan operation. For example, in addition to detecting the scan operation, the detection model for detecting the scan operation is provided with a function of determining whether the scan operation is a non-illegal scan operation or an illegal scan operation. In this case, the learning of the detection model is performed using both the video in which the non-illegal scan operation is recorded and the video in which the illegal scan operation is recorded.

[0085] Operations to be treated as improper scanning operations are determined, for example, by the administrator of the registration confirmation device 2000 or the like. For example, if the scanning operation is performed with the barcode hidden by hand even though the user has confirmed the position of the barcode of the product, this operation is considered an improper scanning operation. In addition, for example, a scanning operation in which the product is moved so as to avoid the reading area of the scanner 22, or a scanning operation in which the products are stacked and held in the hand and the product shown in the camera 10 is moved so as to be different from the product to be read by the scanner 22 is considered an improper scanning operation.

[0086] In addition, for example, whether an operation is an improper scanning operation or not can also be determined by how much information necessary for product registration is obtained. For example, a scanning operation in which the barcode of the product is slightly hidden can be predicted to be a non-improper scanning operation. On the other hand, a scanning operation in which most or all of the barcode of the product is hidden can be predicted to be an improper scanning operation. When the scanner 22 is a camera that images the product, it is possible to predict whether it is an improper scanning operation based on the degree to which the product is hidden.

[0087] Therefore, the degree to which the product or the barcode of the product is hidden is changed between the video for learning the non-improper scanning operation and the video for learning the improper scanning operation. By doing so, the detection model can distinguish between the non-improper scanning operation and the improper scanning operation according to the degree to which the product or the barcode of the product is hidden.

[0088] <Processing when a predetermined condition is satisfied> When a predetermined condition is satisfied regarding the relationship between the number of scan operations and the number of registered products, the process performed by the registration confirmation device 2000 is arbitrary. For example, the registration confirmation device 2000 notifies the product registration device 20 that the product registration has been completed normally. In response to receiving this notification, the product registration device 20 performs a process to complete the product registration. Here, various existing processes can be adopted for the process performed by the product registration device 20 upon receiving the completion of the product registration. For example, when the product registration device 20 and the settlement device are provided integrally, the state of the product registration device 20 transitions to a state where it accepts payment. On the other hand, when the product registration device 20 and the settlement device are provided separately, for example, the product registration device 20 causes a message prompting payment at the settlement device to be displayed on the display device. Note that when the product registration device 20 and the registration confirmation device 2000 are provided integrally, the notification from the registration confirmation device 2000 to the product registration device 20 becomes unnecessary.

[0089] <Regarding monitoring other than the scan operation> In the store where the registration confirmation device 2000 is operated, monitoring other than the scan operation may be further performed. For example, operations such as a customer putting an unregistered product into a bag or a store clerk putting the money taken out from the drawer of the settlement device into a pocket are monitored. This operation monitoring is performed by analyzing the video obtained from a camera that captures the actions of the customer and the store clerk. This camera may be the camera 10 used for monitoring the scan operation, or other cameras.

[0090] [Embodiment 2] <Overview> FIG. 8 is a diagram for explaining the overview of the registration confirmation device 2000 of this embodiment. Note that FIG. 8 is an illustration for facilitating the understanding of the registration confirmation device 2000, and the functions of the registration confirmation device 2000 are not limited to those shown in FIG. 8. Also, unless otherwise specified, the registration confirmation device 2000 of Embodiment 2 has the same functions as the registration confirmation device 2000 of Embodiment 1.

[0091] The registration confirmation device 2000 of Embodiment 2 checks whether the registration of a product is performed normally according to each scanning operation, rather than each transaction. Specifically, the registration confirmation device 2000 detects a scanning operation from the video 12, and further determines whether the product has been registered as a settlement target according to the scanning operation. Then, when the product is not registered according to the scanning operation (that is, when the scanning operation is detected but the product is not registered), the registration confirmation device 2000 performs a predetermined output process. For example, the predetermined output process is a notification for making the user recognize that the product has not been registered.

[0092] <An example of the effect> According to the registration confirmation device 2000 of the present embodiment, it is determined whether the product has been registered as a settlement target according to the scanning operation, and when it is not registered, a predetermined output process is performed. Here, the registration confirmation device 2000 detects the scanning operation by analyzing the video 12 in which the scanning operation is recorded. Therefore, compared with the case where the scanning operation is detected by using a sensor that senses the passage of an object, the scanning operation can be detected with high accuracy. Therefore, it is possible to accurately check whether the product has been registered as a settlement target by the scanning operation.

[0093] <An example of the functional configuration of the registration confirmation device 2000> FIG. 9 is a diagram illustrating the functional configuration of the registration confirmation device 2000 of Embodiment 2. The registration confirmation device 2000 of Embodiment 2 includes a scanning operation detection unit 2020, a second determination unit 2080, and a second output unit 2100. The scanning operation detection unit 2020 of Embodiment 2 is the same as the scanning operation detection unit 2020 of Embodiment 1. The second determination unit 2080 determines whether the product has been registered according to the detected scanning operation. The second output unit 2100 performs a predetermined output process when the product is not registered according to the detected scanning operation.

[0094] <An example of the hardware configuration> The hardware configuration of the registration confirmation device 2000 according to this embodiment is represented by FIG. 3, for example, in the same manner as the hardware configuration of the registration confirmation device 2000 according to Embodiment 1. However, the storage device 1080 according to this embodiment stores a program for realizing the registration confirmation device 2000 according to this embodiment.

[0095] <Flow of processing> FIG. 10 is a flowchart exemplifying the flow of processing executed by the registration confirmation device 2000 according to Embodiment 2. The scan operation detection unit 2020 detects a scan operation from the video 12 (S302). The second determination unit 2080 determines whether a product has been registered according to the scan operation (S304). When a product has not been registered according to the scan operation (S304: NO), the second output unit 2100 performs a predetermined output process. The processing in FIG. 10 is repeatedly executed from the start to the end of the registration of the product for the target transaction.

[0096] <Determination as to whether a product has been registered: S304> The second determination unit 2080 determines whether a product has been registered according to the scan operation (S304). For example, when the scan operation detection unit 2020 detects a scan operation, the second determination unit 2080 determines whether a new product has been registered in the product registration device 20 after the time when the previous determination was made in the second determination unit 2080. If a new product has been registered after the previous determination, the second determination unit 2080 determines that the product has been registered according to the scan operation. On the other hand, if no new product has been registered after the previous determination, the second determination unit 2080 determines that the product has not been registered according to the scan operation. Note that information about the registration of the product can be obtained from the product registration device 20.

[0097] For example, the second determination unit 2080 may also determine whether there is a product registered during the period of the scanning operation detected by the scanning operation detection unit 2020. If there is a product registered during the period of the scanning operation, it may be determined that a product has been registered in response to the scanning operation. If there is no product registered during the period of the scanning operation, it is determined that no product has been registered in response to the scanning operation.

[0098] <Output processing by the second output unit 2100: S306> When no product is registered in response to the scanning operation (S304: NO), the second output unit 2100 performs a predetermined output process. Various processes can be adopted as the predetermined output process. Hereinafter, several examples of the predetermined output process will be illustrated.

[0099] <<Example 1 of the predetermined output process>> For example, the second output unit 2100 outputs a message for the user to recognize that no product has been registered to the product registration device 20. For example, this message may be displayed on the display device of the product registration device 20 or output as sound from the speaker of the product registration device 20.

[0100] Here, the message output to the product registration device 20 will be illustrated more specifically while explaining the screen transition in the product registration device 20. FIG. 11 is a diagram illustrating the screen transition of the product registration device 20. Each screen in FIG. 11 is simplified for the explanation of the registration confirmation device 2000, and various information not shown here may be displayed on each screen during actual operation. Also, in FIG. 11, a case where the product registration device 20 and the settlement device are provided integrally is assumed.

[0101] The screen 30 and the screen 40 are the same as those described in Embodiment 1 (the screen 30 and the screen 40 in FIG. 6). However, in this embodiment, the screen transition after the screen 40 is displayed is different from the case of Embodiment 1. When the screen 40 is displayed on the display device, the user performs a scanning operation to register a product. The registration confirmation device 2000 analyzes the video 12 to detect the scanning operation, and determines whether a product has been registered according to the scanning operation. When a product is registered according to the scanning operation, the product registration device 20 continues to accept the scanning of the product. Therefore, the screen 40 continues to be displayed on the display device of the product registration device 20.

[0102] On the other hand, when a product is not detected according to the scanning operation, the screen displayed on the display device transitions to the screen 60. The screen 60 includes a message prompting the user to re-register the product. Therefore, the user performs a scanning operation again on the product that the user tried to scan on the product registration device 20 in the previous scanning operation, and tries to register the product.

[0103] The registration confirmation device 2000 analyzes the video 12 to detect the scanning operation, and determines whether a product has been registered according to the scanning operation. When a product is registered according to the scanning operation, the screen displayed on the display device transitions to the screen 70. On the other hand, when a product is not registered according to the scanning operation, the screen 60 continues to be displayed on the display device.

[0104] The screen 70 displays a message indicating that the product has been registered normally. When the user finishes registering the product, the user presses the button 72. When the button 72 is pressed, the screen displayed on the display device transitions to the settlement screen. When the user wants to register another product, the user performs a scanning operation to try to register the product. If a product is registered according to the scanning operation, the screen displayed on the display device transitions to the screen 40. On the other hand, if a product is not registered according to the scanning operation, the screen displayed on the display device transitions to the screen 60.

[0105] In the example of FIG. 11, a message indicating that the registration of the product has been successfully performed is displayed only when the registration of the product is redone. However, this message may be displayed according to whether the product has been registered, regardless of whether the registration of the product is redone.

[0106] Note that in this embodiment, since it is determined whether a product has been registered each time a scanning operation is performed, in the "redo of product registration", it is only necessary to redo the scanning operation for the product that was attempted to be registered in the previous scanning operation.

[0107] <<Example 2 of Predetermined Output Processing>> For example, the second output unit 2100 may perform an output that enables a person related to the store, such as a store clerk or a security guard, to grasp that a product has not been registered in response to the scanning operation. For example, the second output unit 2100 transmits predetermined output information to a mobile terminal held by a store clerk or a security guard. In addition, for example, when the product registration device 20 in which a product has not been registered in response to the scanning operation is the product registration device 20 operated by a customer, the second output unit 2100 may output the above-described output information to the product registration device 20 operated by a store clerk.

[0108] The information included in the output information varies. For example, the output information includes identification information of the product registration device 20 in which a product has not been registered in response to the scanning operation and an image of the user who performed the scanning operation. The method of acquiring the image of the user is as described above. Further, the predetermined output information may include an image in which the scanning operation in which the product has not been registered is recorded.

[0109] Here, the output of the output information to a store clerk or the like may be limited to only when, in the target transaction, a situation where a product is not registered in response to a scan operation occurs a predetermined number of times or more (two or more times). For example, when an abnormality occurs in the scanner 22, a situation where registration cannot be performed normally may occur even if the user repeatedly tries to register a product. By notifying a store clerk or the like when a situation where a product is not registered in response to a scan operation occurs multiple times, the store clerk or the like can easily grasp such a situation that requires attention.

[0110] Further, the second output unit 2100 may use sound, light, etc. to enable a store clerk or the like to grasp that a product has not been registered in response to a scan operation. For example, when a lamp visible from the surroundings is provided in the product registration device 20, the second output unit 2100 lights or blinks the lamp in a predetermined manner to output a signal indicating that a product has not been registered in response to a scan operation. Additionally, for example, a predetermined voice may be output from a speaker provided in the product registration device 20.

[0111] Note that, as described in Embodiment 1, the registration confirmation device 2000 may determine whether the scan operation is a non-illegal scan operation or a scan operation, and perform different output processes according to the determination result. For example, similar to Embodiment 1, when the scan operation in which a product is not registered is a non-illegal scan operation, the second output unit 2100 performs the output process of Example 1. On the other hand, when it is an illegal scan operation, the second output unit 2100 performs the output process of Example 2, or both the output processes of Example 1 and Example 2.

[0112] As described above, embodiments of the present invention have been described with reference to the drawings, but these are examples of the present invention, and combinations of the above embodiments or various other configurations may also be adopted.

[0113] Some or all of the above embodiments may also be described as follows in the appended claims, but are not limited thereto. 1. A scan operation detection unit that detects a scan operation by analyzing a video including a user's hand, A determination unit that determines whether or not the relationship between the number of times the scanning operation has been performed and the number of items registered as settlement targets satisfies a predetermined condition for the target transaction; An output unit that performs a predetermined output process when the relationship does not satisfy the predetermined condition, and has: The registration confirmation device, wherein the predetermined condition is a condition satisfied when the number of the registered items is greater than or equal to the number of the detected scanning operations. 2. The registration confirmation device according to 1., wherein in the video, when a hand enters the reading area of the scanner and then exits the reading area, a scanning operation is detected. 3. The registration confirmation device according to 1. or 2., wherein the output unit identifies a scanning operation in which an item has not been registered by comparing the period during which each scanning operation has been performed with the time when each item has been registered, and outputs an image including the scanning operation. 4. A scanning operation detection unit that detects a scanning operation by analyzing a video including a user's hand, A determination unit that determines whether or not an item has been registered as a settlement target according to the scanning operation, An output unit that performs a predetermined output process when an item has not been registered as a settlement target according to the scanning operation, and has: The registration confirmation device, wherein in the video, when a hand enters the reading area of the scanner and then exits the reading area, a scanning operation is detected. 5. The registration confirmation device according to any one of 2. to 4., wherein in the video, when a hand holding an item enters the reading area of the scanner and then exits the reading area, a scanning operation is detected. 6. The registration confirmation device according to any one of 2. to 5., wherein in the video, when a hand in a form satisfying a predetermined criterion enters the reading area of the scanner and then exits the reading area, a scanning operation is detected. 7. The scanning operation detection unit detects the scanning operation using a detection model that detects the scanning operation from the input video, The registration confirmation device according to any one of 1. to 6., wherein the detection model is a model learned using a video obtained by imaging a scanning operation with a camera. 8. The registration confirmation device according to any one of 1. to 7., wherein the output unit outputs a message prompting a retry of scanning the product. 9. The scan operation detection unit determines whether the scan operation is an illegal scan operation or a non-illegal scan operation, When the scan operation in which the product is not registered is an illegal scan operation and when the scan operation in which the product is not registered is a non-illegal scan operation, the output unit performs different output processes. The registration confirmation device according to any one of 1. to 8. 10. A control method executed by a computer, A scan operation detection step of detecting a scan operation by analyzing a video including the user's hand, A determination step of determining whether or not a relationship between the number of times the scan operation is performed and the number of products registered as settlement targets satisfies a predetermined condition for a target transaction, An output step of performing a predetermined output process when the relationship does not satisfy the predetermined condition, and The predetermined condition is a condition satisfied when the number of registered products is greater than or equal to the number of detected scan operations. Control method. 11. In the video, when the hand enters the reading area of the scanner and then exits the reading area, a scan operation is detected. The control method according to 10. 12. In the output step, by comparing the period during which each scan operation is performed with the time when each product is registered, a scan operation in which the product is not registered is specified, and an image including the scan operation is output. The control method according to 10. or 11. 13. A control method executed by a computer, A scan operation detection step of detecting a scan operation by analyzing a video including the user's hand, A determination step of determining whether a product is registered as an item to be settled according to the scanning operation; An output step of performing a predetermined output process when the product is not registered as an item to be settled according to the scanning operation, and having: In the video, when a hand enters the reading area of the scanner and then exits the reading area, a control method in which a scanning operation is detected. 14. The control method according to any one of 11. to 13., wherein in the video, when a hand holding a product enters the reading area of the scanner and then exits the reading area, a scanning operation is detected. 15. The control method according to any one of 11. to 14., wherein in the video, when a hand in a form satisfying a predetermined criterion enters the reading area of the scanner and then exits the reading area, a scanning operation is detected. 16. In the scanning operation detection step, the scanning operation is detected using a detection model that detects the scanning operation from the input video. The control method according to any one of 10. to 15., wherein the detection model is a model learned using a video obtained by imaging a scanning operation with a camera. 17. The control method according to any one of 10. to 16., wherein in the output step, a message prompting the user to rescan the product is output. 18. In the scanning operation detection step, it is determined whether the scanning operation is an illegal scanning operation or a non-illegal scanning operation. In the output step, different output processes are performed depending on whether the scanning operation in which the product is not registered is an illegal scanning operation or a non-illegal scanning operation. The control method according to any one of 10. to 17. 19. A program for causing a computer to execute the control method according to any one of 10. to 18.

[0114] This application claims priority based on Japanese Patent Application No. 2020-035052 filed on March 2, 2020, and incorporates the entire disclosure thereof herein.

Explanation of Signs

[0115] 10 Camera 12 Video 20 Product Registration Device 22 Scanner 30 Screen 32 Button 40 Screen 42 Button 50 Screen 52 Button 54 Button 60 Screen 70 Screen 72 Button 1000 Computer 1000 Computer 1020 Bus 1040 Processor 1060 Memory 1080 Storage Device 1100 Input / Output Interface 1120 Network Interface 2000 Registration Confirmation Device 2020 Scan Operation Detection Unit 2040 First Judgment Unit 2060 First Output Unit 2080 Second Judgment Unit 2100 Second Output Unit

Claims

1. a scanning action detection unit that detects a scanning action for causing a scanner of the product registration device to scan the product by analyzing a video including a user's hand; a determination unit that determines whether or not there is a scan operation for which no product has been registered based on the total number of times the scan operation has been performed and the number of products registered as items to be settled; and an output unit that outputs information for identifying the unregistered product to the product registration device when a scan operation in which the product is not registered exists. Information processing device.

2. The information for identifying the unregistered product is at least one of an image in which a scanning operation performed on the unregistered product is recorded, an image generated by the scanner, and identification information of the unregistered product. The information processing device according to claim 1 .

3. The process of outputting the information for identifying the unregistered product to the product registration device is executed after an operation indicating completion of product registration is performed on the product registration device.

3. The information processing device according to claim 1 or 2.

4. The information processing device according to claim 1 , wherein the output unit is configured to identify a scanning operation for which no product was registered by comparing a period during which each scanning operation was performed with a time point at which each product was registered.

5. The information processing device according to claim 1 , wherein a scanning motion is detected when, in the video, a hand enters a reading area of ​​the scanner and then moves out of the reading area.

6. The scanning motion detection unit detects the scanning motion using a detection model that detects a scanning motion from an input video; The information processing device according to claim 1 , wherein the detection model is a model trained using a video obtained by capturing a scanning operation with a camera.

7. The information processing apparatus according to claim 1 , wherein the output unit further outputs a message prompting the user to rescan the product when there is a scan operation in which the product is not registered.

8. The scan operation detection unit determines whether a scan operation is an unauthorized scan operation or a non-unauthorized scan operation, The information processing device according to claim 1 , wherein the output unit performs different output processing when the scanning operation for which a product was not registered is an unauthorized scanning operation and when the scanning operation for which a product was not registered is a non-invalid scanning operation.

9. The information processing device according to claim 1 , wherein the information processing device is integrated with the product registration device.

10. 1. A computer-implemented control method comprising: a scanning action detection step of detecting a scanning action of causing a scanner of the product registration device to scan the product by analyzing a video including the user's hands; a determination step of determining whether or not there is a scan operation for which no product has been registered based on the total number of times the scan operation has been performed and the number of products registered as items to be checked out; and an output step of outputting information for identifying the unregistered product to the product registration device when there is a scan operation in which the product is not registered. Control methods.

11. A program for causing a computer to execute a control method, The control method includes: a scanning action detection step of detecting a scanning action of causing a scanner of the product registration device to scan the product by analyzing a video including the user's hands; a determination step of determining whether or not there is a scan operation for which no product has been registered based on the total number of times the scan operation has been performed and the number of products registered as items to be checked out; and an output step of outputting information for identifying the unregistered product to the product registration device when there is a scan operation in which the product is not registered. program.

Images