Commodity registration terminal, commodity registration method, and program

Abstract

To provide a technique capable of notifying a user that processing of commodity registration is in progress.SOLUTION: A commodity registration terminal includes object detection means, commodity specification means, and display processing means. The object detection means detects an object based on an image captured by an imaging unit. The commodity identification means identifies a commodity based on an image including the object detected by the object detection means. The display processing means displays a notification indicating that specification processing is in progress on a display unit during the specification processing by the commodity specification means.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] An embodiment of the present invention relates to a product registration terminal, a product registration method, and a program. [Background technology]

[0002] Among cart POS (Point Of Sales) systems in which a shopping cart is provided with an information terminal equipped with a touch panel, scanner, etc., there are cart POS systems that utilize AI (artificial intelligence) and are called frictionless carts, AI carts, smart carts, intelligent carts, etc. In these cart POS systems, when a user simply places a product in the cart, the AI ​​automatically identifies and specifies the type of product placed in the cart through image recognition of the product package, and automatically registers the product (for example, see Patent Document 1).

[0003] However, in a cart POS system, it may take some time for the product to be registered after the user inserts the product, making it difficult for the user to determine whether the product has been registered or not. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2021-87578 Summary of the Invention [Problem to be solved by the invention]

[0005] The problem to be solved by the embodiments of the present invention is to provide a technique capable of notifying a user that a product registration process is in progress. [Means for solving the problem]

[0006] In one embodiment, the product registration terminal includes an object detection means, a product identification means, and a display processing means. The object detection means detects an object based on an image captured by an imaging unit. The product identification means identifies a product based on an image including the object detected by the object detection means. The display processing means displays a notification indicating that the identification process is in progress on a display unit during identification processing by the product identification means. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram showing the overall configuration of a cart POS system according to an embodiment. [Diagram 2] FIG. 2 is a schematic diagram showing a main data structure of the information terminal table of the cart controller according to the embodiment. [Diagram 3] FIG. 3 is a block diagram showing a main circuit configuration of the information terminal according to the embodiment. [Figure 4] FIG. 4 is a schematic diagram showing the mounting position of an information terminal on a shopping cart according to the embodiment. [Diagram 5] FIG. 5 is a flowchart showing a procedure of main information processing executed by a processor of an information processing device in an information terminal according to the terminal program. [Figure 6] FIG. 6 is a diagram showing a display example of the information terminal according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] (Embodiment) Hereinafter, the embodiments will be described with reference to the drawings. The scale of each part in each drawing used in the following description of the embodiments may be changed as appropriate. In addition, the configuration of each drawing used in the following description of the embodiments may be omitted for the sake of explanation. The following embodiments are applied to a cart POS system.

[0009] Fig. 1 is an overall configuration diagram of a cart POS system 1 according to an embodiment. The cart POS system 1 includes a store server 2, a cart controller 3, an access point (indicated as AP in Fig. 1) 4, and a payment machine 5. The store server 2, the cart controller 3, the access point 4, and the payment machine 5 are connected to a network 6 such as a LAN (Local Area Network) or a wireless LAN. The cart POS system 1 includes a plurality of information terminals 7 connected to the network 6 via the access point 4.

[0010] The information terminal 7 is a device that enables a customer, who is a purchaser, to input data related to the registration of purchased items by himself / herself. The information terminal 7 is provided in a shopping cart 8 that the customer uses when shopping in the store. Hereinafter, the shopping cart 8 will be simply referred to as a cart 8. Each of the multiple carts 8 is provided with an information terminal 7. The cart 8 is an example of a carrier that transports purchased items by a customer using the cart 8.

[0011] The information terminal 7 is provided on the cart 8. The information terminal 7 is an information processing terminal that functions as a user interface for operations related to registering purchased items. The information terminal 7 is equipped with a wireless unit. The information terminal 7 performs wireless communication with the access point 4. The access point 4 relays communication between the information terminal 7 and each device connected to the network 6, i.e., the store server 2, the cart controller 3, and the payment device 5. Although only one access point 4 is shown in FIG. 1, there may be two or more access points 4 depending on the size of the store, etc.

[0012] The accounting machine 5 is a device that enables a store clerk or a customer to make a payment for purchased items. The accounting machine 5 is a well-known self-service POS terminal. The accounting machine 5 may be a well-known face-to-face POS terminal. There is no particular limit to the number of accounting machines 5.

[0013] The store server 2 supports store operations. To support the operations, the store server 2 manages a product database (DB) 21, a member database (DB) 22, and the like.

[0014] The product database 21 stores a product record that describes product data for each product sold in the store. The product record includes a product code, price, product name, etc. The product code is an identification code for the product. The sales database stores sales records that describe sales data for each item, such as product, department, time period, date, week, month, etc. The sales data includes sales units, sales amount, etc.

[0015] In the present embodiment, the product database 21 stores pre-trained learning data 211 for recognizing products stored in the product database 21 by AI. The learning data 211 is a collection of learning data for each product sold in a store, and the learning data for each product is incorporated into the learning data 211 in association with, for example, a product code. The learning data 211 can update the learning data for each product in accordance with an update of the product database 21 in response to an addition or replacement of a product sold in the store. The learning data for each product may be created by a provider of the product and provided to the store, or may be created independently by the store. In addition to the above learning data, the learning data 211 may learn data other than products, such as data of a person's hand.

[0016] Member database 22 stores a member record that describes member data for each member. The member data includes a member ID, name, gender, email address, etc. The member ID is a code such as a unique member number that is set for each member to individually identify each member. Members own a recording medium on which the member ID is recorded. The recording medium may be, for example, a magnetic card, a contact type IC (Integrated Circuit) card, a contactless IC card, a smartphone, etc.

[0017] Although not shown in the figure, the store server 2 may further manage a member payment database. The member payment database stores a member payment record that describes data required for payment for each member. In addition to the member ID, the member payment record includes an electronic money identification code and an electronic money user ID as data required for payment. The cart POS system 1 according to this embodiment limits the payment method to one type of electronic money when the member does not use the accounting machine 5. The electronic money identification code is an identification code of electronic money that is registered in advance as the electronic money to be used for payment by the member identified by the member ID. The electronic money user ID is an identification code that the electronic money center EMC uses to identify the user of the electronic money, and is assigned by the electronic money center EMC. In order to enable payment by electronic money, the member registers the electronic money identification code and electronic money ID in advance, for example, when registering as a member. If the electronic money has not been registered in advance, NULL or 0000 is stored in the electronic money identification code and electronic money ID of the member payment record. Furthermore, when electronic money is allowed to be used in this way, the store server 2 has a communication server function, or the cart POS system 1 includes a communication server. The store server 2 or the communication server supports data communication processing between the accounting machine 5 or the cart controller 3 and an electronic money center that supports payments with electronic money, via an external communication network such as the Internet or a network dedicated to electronic money.

[0018] The cart controller 3 cooperates with the information terminal 7 to support the information terminal 7 so that the information terminal 7 appears to function as a POS terminal. The cart controller 3 has an information terminal table 31 corresponding to each information terminal 7. FIG. 2 is a schematic diagram showing an example of the information terminal table 31. As shown in FIG. 2, the information terminal table 31 has a terminal ID area 311, a member ID area 312, one or more purchased product data areas 313, and a subtotal area 314. The terminal ID area 311 stores a terminal ID, which is a code such as a terminal number for uniquely identifying the information terminal 7 corresponding to the information terminal table 31. The member ID area 312 stores a member ID. The purchased product data area 313 stores purchased product data for the product every time a customer registers a product to be purchased on the information terminal 7. The purchased product data includes a product code, a unit price, a sales point (number), a product name, a sales amount, a tax amount, and the like of the purchased product purchased by the customer. The subtotal area 314 is amount data showing the total amount of the sales price and tax for all the purchased items purchased by the customer, calculated based on the purchased item data stored in the purchased item data area 313. This can be updated every time the customer performs the registration operation for the item he or she wishes to purchase.

[0019] FIG. 3 is a block diagram showing the main circuit configuration of the information terminal 7. FIG. 4 is a schematic diagram showing the mounting position of the information terminal 7 on the cart 8. The information terminal 7 includes an information processing device 71, a reader 72, and a camera 73. The information processing device 71 may be a dedicated device or a general-purpose device such as a tablet computer. The information processing device 71 includes a processor 711, a main memory 712, an auxiliary storage device 713, a wireless unit 714, a touch panel 715, a speaker 716, and a system transmission path 717. The system transmission path 717 includes an address bus, a data bus, a control signal line, and the like. The information processing device 71 connects the processor 711, the main memory 712, the auxiliary storage device 713, the wireless unit 714, the touch panel 715, and the speaker 716 to the system transmission path 717. The reader 72 and the camera 73 are connected to the system transmission path 717 via an interface not shown. In the information processing device 71, a computer is configured with a processor 711, a main memory 712, an auxiliary storage device 713, and a system transmission line 717 that connects these together. The information terminal 7 is an example of a product registration terminal.

[0020] The processor 711 corresponds to the central part of the computer. The processor 711 controls each part to realize various functions as the information processing device 71 according to an operating system or an application program. The processor 711 is, for example, a CPU (Central Processing Unit), but is not limited thereto. The processor 711 may be a multi-core / multi-threaded one and can execute multiple processes in parallel. The processor 711 may include an AI chip specialized for AI. Furthermore, the processor 711 may be realized in various other forms including integrated circuits such as a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), an FPGA (field-programmable gate array), and a DSP (Digital Signal processor). The processor 711 is an example of a processing circuit.

[0021] The main memory 712 corresponds to the main storage portion of the computer. The main memory 712 includes a nonvolatile memory area and a volatile memory area. The main memory 712 stores an operating system or an application program in the nonvolatile memory area. The main memory 712 may store data required for the processor 711 to execute processes for controlling each part in the nonvolatile or volatile memory area. The main memory 712 uses the volatile memory area as a work area where data is appropriately rewritten by the processor 711. The nonvolatile memory area is, for example, a ROM (Read Only Memory). The volatile memory area is, for example, a RAM (Random Access Memory).

[0022] The auxiliary storage device 713 corresponds to the auxiliary storage part of the computer. For example, an EEPROM (registered trademark) (Electric Erasable Programmable Read-Only Memory), a HDD (Hard Disc Drive), or an SSD (Solid State Drive) can be the auxiliary storage device 713. The auxiliary storage device 713 stores data used by the processor 711 in performing various processes, or data created by the processes in the processor 711. The auxiliary storage device 713 may store the above application program. In this embodiment, as the application program, a terminal program 7131 for causing the information processing device 71, the reader 72, and the camera 73 to function as the information terminal 7 according to the embodiment is stored. The terminal program 7131 may be stored in the main memory 712. In addition, the auxiliary storage device 713 stores learning data 7132 for commodity recognition by AI for each commodity sold in the store stored in the commodity database 21 of the store server 2. This learning data 7132 may be a copy of the learning data 211 included in the product database 21 of the store server 2. The timing of copying this learning data 211 as the learning data 7132 may be a specific timing such as each time the information processing device 71 is started up, or may be an arbitrary timing such as when the learning data 211 is distributed from the store server 2 to each information processing device 71 when the learning data 211 is updated.

[0023] The wireless unit 714 wirelessly communicates data with the access point 4 in accordance with a wireless communication protocol.

[0024] The touch panel 715 is a device that combines an input device and a display device for the information terminal 7. The touch panel 715 displays an image based on image data output from the processor 711, detects a touch position on the displayed image, and outputs the touch position information to the processor 711. As shown in Fig. 4, the information processing device 71 is attached to the cart 8 at a position and orientation such that a customer who is moving while pushing the cart 8 can see the display of the touch panel 715 and perform touch operations. The touch panel 715 is an example of a display unit.

[0025] The speaker 716 reproduces sounds based on the sound data output from the processor 711. The reproduced sounds include confirmation sounds, alert warning sounds, and the like.

[0026] The reader 72 reads data recorded on the recording medium and outputs the read data to the processor 711. The reader 72 is a magnetic card reader if the recording medium is a magnetic card, and is an IC card reader if the recording medium is a contact IC card. In the case of a recording medium that uses RFID (Radio Frequency Identification) such as a non-contact IC card or a smartphone, an RFID reader is used as the reader 72. The position at which the reader 72 is attached to the cart 8 is not particularly specified. For example, it can be attached next to the information processing device 71.

[0027] As shown in FIG. 4, the camera 73 is provided on the cart 8 so as to photograph the basket receiving section 81 of the cart 8 from above. The camera 73 is for monitoring the insertion or return of products into or from the basket receiving section 81 of the cart 8 or the basket placed in the basket receiving section 81. An LED (Light Emitting Diode) lamp 731 for illumination can be installed on the camera 73. The operating states of the camera 73 and the LED lamp 731 can be controlled by the processor 711 via the system transmission path 717. Although only one camera 73 is shown in FIG. 3 and FIG. 4, two or more cameras may be provided. For example, four cameras 73 may be arranged on the four sides or four corners of the basket receiving section 81 of the cart 8. Depending on the type of camera 73, sufficient brightness may be ensured, so operation may be performed without installing the LED lamp 731. The camera 73 is an example of an imaging section.

[0028] 3, the information terminal 7 may further include a patrol lamp 74 that is connected to the system transmission line 717 via an interface (not shown) and whose operation state is controlled by the processor 711. The patrol lamp 74 is a lamp for issuing a warning to a customer using the cart 8 and nearby store staff.

[0029] FIG. 5 is a flowchart showing the procedure of main information processing executed by the processor 711 of the information processing device 71 in the information terminal 7 according to the terminal program 7131.

[0030] Below, using this flowchart, the main operations of the cart POS system 1 when a customer uses the information terminal 7 of the cart 8 to perform the process from registering purchased items to paying for them by himself / herself will be explained. Note that the operations explained below are just an example. As long as similar results are obtained, the procedure is not particularly limited. Also, it is assumed here that the reader 72 has read the data recorded on the recording medium and that login has already been performed. In other words, the member ID area 312 of the information terminal table 31 of the cart controller 3 corresponding to the information terminal 7 has already stored the member ID read from the customer's recording medium.

[0031] In ACT11, the processor 711 of the information processing device 71 turns on the LED lamp 731 of the camera 73 and causes the camera 73 to capture an image. Image data of the captured image is stored in the main memory 712. Note that an area for storing multiple frames of this image data is secured in the main memory 712, and when the processor 711 stores new image data in that area, it erases the oldest image data and stores the new image data in the secured area. After that, the processor 711 proceeds to ACT12.

[0032] In ACT12, the processor 711 compares the object and detection position of the latest image data stored in the main memory 712 with the object and detection position of the old image data to determine whether or not there is a change. The processing of ACT12 may be processing by an object detection means. The processor 711 detects an object based on the fact that a product is put into the cart 8. Specifically, when a product is put into the basket receiving section 81 of the cart 8 or a basket placed in the basket receiving section 81, or when a product is taken out of the basket receiving section 81 or a basket, that is, when a product is returned, the product being moved for the putting in or returning is photographed by the camera 73. Therefore, when a product is put in or returned, a difference occurs between the object and detection position of the latest image data and the object and detection position of the old image data, so that the processor 711 can determine that there is a change. If it is determined that there is no change, the processor 711 determines NO in ACT12 and proceeds to the above ACT11. Moreover, if it is determined that there has been a change, the processor 711 determines YES in ACT12 and proceeds to ACT13.

[0033] In ACT13, the processor 711 detects an object in the image based on the latest image data that is determined to have changed. The processing in ACT13 may be processing by an object detection means. Specifically, the processor 711 performs object detection on the object in the latest image, sets a rectangular area called a bounding box, and detects the object in the image. The detected object and the detection position of the object in the image data are stored in the main memory 712 in association with the image data stored in the main memory 712. After that, the processor 711 proceeds to ACT14.

[0034] The processor 711 displays a notification on the touch panel 715 as ACT14. The processing of ACT14 may be processing by a display processing means. Specifically, the processor 711 displays a notification on the touch panel 715 indicating that a specification process for specifying a product corresponding to an object is in progress based on an image including the detected object. The notification includes, for example, a text message such as "recognizing", "specifying", or "processing". The specification process indicates that an object included in an image is specified as a product. The specification process includes specifying which product the object is. The specification process includes, for example, specifying an object included in an image by information that can identify the product, such as a product name. The specification process includes a process of specifying a product in order to register the product. The processor 711 may, for example, superimpose the notification on a registered product screen displayed on the touch panel 715. The registered product screen is a screen for notifying registered products based on purchased product data stored in the purchased product data area 313. The processor 711 may, for example, transition from the registered product screen displayed on the touch panel 715 to a notification display screen including the notification. The processor 711 may turn on the LED lamp 731 of the camera 73. For example, the processor 711 may change the color or blinking pattern of the LED lamp 731 to indicate that the identification process is in progress. The processor 711 may play a specific alarm sound through the speaker 716. If the cart 8 is equipped with a patrol lamp 74, the processor 711 may turn on the patrol lamp 74 instead of or in addition to the patrol lamp 74. The processor 711 may perform the process of ACT14 after the product identification process shown in ACT15 is started. The processor 711 may display a notification on the touch panel 715 during the product identification process.

[0035] In ACT15, the processor 711 identifies a product based on an image including the detected object. The process in ACT15 may be a process by a product identification means. Specifically, the processor 711 identifies a product moving in the image. Based on the learning data 7132 stored in the auxiliary storage device 713, the processor 711 performs image recognition on the object in the bounding box detected in ACT13 and stored in the main memory 712, and identifies which product the object is. After that, the processor 711 proceeds to ACT16. Note that the process performed by the processor 711 is not limited to the above process, and the processor 711 may perform a product identification process using a known technology.

[0036] In ACT16, the processor 711 determines whether or not the product has been identified. For example, if only a small portion of the product is shown in the image, it is not possible to identify what the product is. If it is determined that the product has not been identified, the processor 711 determines NO in ACT16 and proceeds to ACT11 above. If it is determined that the product has been identified, the processor 711 determines YES in ACT16 and proceeds to ACT17.

[0037] In addition, when the processor 711 determines that the product could not be identified, the processor 711 may display an error notification on the touch panel 715. The error notification includes, for example, a text message indicating that the product could not be identified. Specifically, the error notification may include messages such as "The product could not be identified," "Please insert the product again," and "Please let me read the product code." The processor 711 may superimpose the error notification on the registered product screen. In this case, the customer performs an operation such as inserting the product again. The processor 711 performs processing according to the customer's operation. For example, when the customer inserts the product again, the processor 711 proceeds to ACT11 and takes an image. When the customer performs an operation to read the product code via the camera 73 or a scanner (not shown), the processor 711 may identify the product based on the product code.

[0038] In ACT17, the processor 711 determines whether a product has been put into the basket receiving section 81 of the cart 8 or into the basket placed in the basket receiving section 81. For example, the processor 711 can determine whether a product has been put into or taken out based on the movement direction of the same product shown in the latest image and the past image. Alternatively, it is possible to detect the putting in / taking out of a product by placing a weight scale (not shown) on the bottom surface of the basket receiving section 81 of the shopping cart 8 and determining whether the weight has increased or decreased. If the processor 711 determines that a product has not been put in, that is, that a product has been taken out, it determines NO in ACT17 and proceeds to ACT22. If the processor 711 determines that a product has been put in, it determines YES in ACT17 and proceeds to ACT18.

[0039] In ACT18, the processor 711 transmits a product input notification to the cart controller 3 via the access point 4 by the wireless unit 714. The product input notification may include a terminal ID assigned to the information terminal 7 including the information processing device 71, and a product code indicating the product identified in ACT16. After that, the processor 711 proceeds to ACT19.

[0040] The cart controller 3 that has received the product input notification registers the input product. That is, the cart controller 3 queries the store server 2 for product information using the product code included in the product input notification, and acquires product information stored in the product database 21 managed by the store server 2. The cart controller 3 then additionally stores the acquired product information as new purchased product data in the purchased product data area 313 of the information terminal table 31 having the terminal ID included in the product input notification. The cart controller 3 then creates a registered product screen for notifying the registered product based on the purchased product data stored in the purchased product data area 313, and transmits the registered product screen to the sender of the product input notification.

[0041] In ACT19, the processor 711 receives, by the wireless unit 714, the registered product screen transmitted from the cart controller 3 via the access point 4. After that, the processor 711 proceeds to ACT20.

[0042] In ACT20, the processor 711 presents the received registered product screen to the customer using the cart 8 via the touch panel 715. After that, the processor 711 proceeds to the above-mentioned ACT21.

[0043] The processor 711, as ACT21, hides the notification displayed on the touch panel 715 based on the fact that the specified product has been registered. The processor 711 may hide the notification based on the fact that the registered product screen has been presented. The processing of ACT21 may be processing by a display processing means. Specifically, the processor 711 erases the notification indicating that the specification processing is in progress. The processor 711 may erase the notification superimposed on the registered product screen displayed on the touch panel 715. The processor 711 may transition from the notification display screen to the registered product screen. The processor 711 may end the illumination of the LED lamp 731 of the camera 73. The processor 711 may end the reproduction of the specific notification sound by the speaker 716. In addition, in the case where the cart 8 is equipped with the patrol lamp 74, the illumination of the patrol lamp 74 may be ended.

[0044] In this way, when an item is added to the basket receiving section 81 of the cart 8 or a basket placed in the basket receiving section 81, the information terminal 7 identifies the item through image recognition and sends an item addition notification including an item code indicating the identified item to the cart controller 3, thereby causing the cart controller 3 to register the item as a purchased item.

[0045] Furthermore, if the processor 711 determines in ACT 17 that the product has been removed, in ACT 22, the processor 711 transmits a product return notification to the cart controller 3 via the access point 4 by the wireless unit 714. The product return notification may include a terminal ID assigned to the information terminal 7 including the information processing device 71, and a product code indicating the product recognized and identified in ACT 16. After that, the processor 711 proceeds to ACT 23.

[0046] The cart controller 3 that has received the product return notice deletes the returned product from the registered purchased products. That is, the cart controller 3 deletes the purchased product data of the product code included in the product return notice, which is stored in the purchased product data area 313 of the information terminal table 31 having the terminal ID included in the product return notice, from the purchased product data area 313. The cart controller 3 then creates a registered product screen for notifying the registered product based on the purchased product data stored in the purchased product data area 313, and transmits the registered product screen to the sender of the product input notice.

[0047] In ACT23, the processor 711 receives the registered product screen transmitted from the cart controller 3 via the access point 4, by the wireless unit 714. After that, the processor 711 proceeds to ACT24.

[0048] The processor 711 presents the received registered product screen to the customer using the cart 8 via the touch panel 715 as ACT24. The processor 711 then proceeds to the above-mentioned ACT21. In this case, the processor 711 hides the notification that was displayed on the touch panel 715 as ACT21 based on the fact that the identified product has been deleted from purchased products. The processor 711 may hide the notification based on the presentation of the registered product screen.

[0049] The product identification process may be executed by the store server 2. In this case, the store server 2 performs the processes from ACT15 to ACT17. The processor 711 outputs the latest image and the past image determined to have changed to the store server 2. The store server 2 identifies the product that is moving in the image. Specifically, the store server 2 performs image recognition on the object in the bounding box set in the latest image based on the learning data 211 stored in the product database 21, and identifies which product the object is. Product identification can be realized by a known technology. The store server 2 creates a registered product screen based on the product identification result and the determination result of whether the product has been inserted, and transmits the registered product screen to the sender of the image. The processor 711 receives the registered product screen as ACT19 or ACT23.

[0050] According to this example, the information terminal 7 can provide the user of the cart 8 with a notification indicating that the product identification process is in progress from the time when the information terminal 7 detects an object based on an image put into the cart 8 to the time when the product is identified and registered. Therefore, the user of the cart 8 can recognize that the product identification process is in progress. For example, in a frictionless cart POS system or the like in which a product is automatically registered by a customer putting a product into the cart, the customer does not perform an operation such as scanning the product, so that if it takes a long time from the time when the product is put into the cart to the time when the result of the product registration is displayed, it is difficult for the customer to determine whether the product is registered normally or is not detected and needs to be retried. With the above-mentioned function, even in such a system, the customer can recognize that the product registration process is in progress by receiving a notification indicating that the product identification process is in progress until the result of the product registration is displayed.

[0051] Explain notifications. FIG. 6 is a diagram showing a display example of the information terminal 7 according to the embodiment. FIG. 6 shows a notification image Ima displayed on the touch panel 715.

[0052] The notification image Ima includes a message such as "Recognizing". For example, the touch panel 715 may display the notification image Ima superimposed on the registered product screen. The touch panel 715 displays the notification image Ima while the processor 711 is performing a product identification process.

[0053] Instead of or in addition to displaying the notification image Ima, the processor 711 may turn on the LED lamp 731 of the camera 73. Instead of or in addition to displaying the notification image Ima, the processor 711 may play a specific notification sound through the speaker 716.

[0054] According to this example, the user of the cart 8 can receive a notification indicating that the product identification process is in progress. Therefore, the user of the cart 8 can recognize that the product identification process is in progress during the period from when the product is added to the cart 8 to when the product identification process is being performed. Furthermore, when the processor 711 performs a notification using the LED lamp 731, the user of the cart 8 can recognize the notification from all angles. Therefore, the user of the cart 8 can easily recognize that the product identification process is in progress during the period from when the product is added to the cart 8 to when the product identification process is being performed. When the processor 711 performs a notification using the speaker 716, the user of the cart 8 can recognize the notification by voice. Therefore, the user of the cart 8 can easily recognize that the product identification process is in progress during the period from when the product is added to the cart 8 to when the product identification process is being performed.

[0055] (Other embodiments) In the above embodiment, the trigger for starting object detection is when there is a change in the image, but this is not limited to this. For example, at least one sensor that optically detects an object present within the field of view of the camera 73 may be disposed at any position on the cart 8, and product recognition may be started in response to the sensor detecting an object being put into or taken out of the cart 8.

[0056] Furthermore, the judgment of whether a product has been put in or taken out is not limited to being judged from an image. For example, it is possible to place a weighing scale on the bottom surface of the basket receiving portion 81 of the shopping cart 8 and judge whether a product has been put in or taken out based on whether the weight has increased or decreased.

[0057] The product registration terminal may be realized as one device as explained by taking the information terminal 7 as an example, or may be realized as a plurality of devices with distributed functions.

[0058] The above-described embodiment may be applied to a method executed by the device. The above-described embodiment may be applied to a program capable of causing a computer of the device to execute each function. The above-described embodiment may be applied to a recording medium storing the program.

[0059] Each of the one or more circuits constituting the processing circuit executes one or more processes among the multiple processes. When the processing circuit is composed of a single circuit, the single circuit executes all of the multiple processes. When the processing circuit is composed of multiple circuits, each of the multiple circuits executes a part of the multiple processes. The part of the multiple processes may be one of the multiple processes, or two or more of the multiple processes. When the processing circuit is composed of multiple circuits, the multiple circuits may be included in one device, or may be distributed across multiple devices.

[0060] The program may be transferred in a state where it is stored in the device according to the embodiment, or in a state where it is not stored in the device. In the latter case, the program may be transferred via a network, or in a state where it is recorded on a recording medium. The recording medium is a non-transitory tangible medium. The recording medium is a computer-readable medium. The recording medium may be in any form, such as a CD-ROM or a memory card, as long as it is capable of storing the program and is readable by a computer.

[0061] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims.

[0062] The above embodiment may be expressed as follows. (1) an object detection means for detecting an object based on an image captured by an imaging unit; a product identification means for identifying a product based on an image including the object detected by the object detection means; a display processing means for displaying, on a display unit, a notice indicating that the product identification process is in progress during the product identification process by the product identification means; A product registration terminal comprising: (2) The imaging unit is provided on a cart, The object detection means detects an object based on the fact that a product has been put into the cart. (1) A product registration terminal as described above. (3) the display processing means hides the notification based on the product being identified by the product identification means; (1) A product registration terminal as described above. (4) detecting an object based on an image captured by the imaging unit; identifying a product based on an image including the detected object; During the specific processing, displaying a notification indicating that the specific processing is in progress on a display unit; A product registration method comprising: (5) To the computer: A function of detecting an object based on an image captured by an imaging unit; A function of identifying a product based on an image including the detected object; a function of displaying, on a display unit, a notification indicating that the specific process is in progress during the specific process; A program that can execute the following. [Explanation of symbols]

[0063] 1...Cart POS system, 2...Store server, 3...Cart controller, 4...Access point, 5...Accounting machine, 6...Network, 7...Information terminal, 8...Shopping cart, 8...Cart, 21...Product database, 22...Member database, 31...Information terminal table, 71...Information processing device, 72...Reader, 73...Camera, 74...Patrol lamp, 81...Basket receiving section, 211...Learning data, 311...Terminal ID area, 312...Member ID area, 313...Purchased product data area, 314...Subtotal area, 711...Processor, 712...Main memory, 713...Auxiliary storage device, 714...Wireless unit, 715...Touch panel, 716...Speaker, 717...System transmission path, 731...LED lamp, 7131...Terminal program, 7132...Learning data, Ima...Notification image.

Claims

1. an object detection means for detecting an object based on an image captured by the imaging unit; a product identification means for identifying a product based on an image including the object detected by the object detection means; a display processing means for displaying, on a display unit, a notice indicating that the product identification process is in progress during the product identification process by the product identification means; A product registration terminal comprising:

2. The imaging unit is provided on a cart, the object detection means detects an object based on the fact that a product has been put into the cart; The product registration terminal according to claim 1.

3. The display processing means, based on the product being identified by the product identification means, hides the notification. The product registration terminal according to claim 1.

4. Detecting an object based on an image captured by an imaging unit; identifying a product based on an image including the detected object; During the specific processing, displaying a notification indicating that the specific processing is in progress on a display unit; A product registration method comprising:

5. On the computer, A function of detecting an object based on an image captured by an imaging unit; A function of identifying a product based on an image including the detected object; a function of displaying, on a display unit, a notification indicating that the specific process is in progress during the specific process; A program that can execute the following.

Images