Commodity registration system

The product registration system addresses recognition inconsistencies by using multiple imaging units and a registration process that allows for continuous product registration and selection, improving efficiency and convenience.

JP2024166987AActive Publication Date: 2024-11-29TOSHIBA TEC KK
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
JP2023083468
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-19
Publication Date
2024-11-29
Estimated Expiration
2043-05-19

AI Technical Summary

Technical Problem

Existing product registration systems using imaging devices in shopping carts face inconsistencies in recognition results due to varying imaging conditions, leading to inefficiencies in product registration and requiring manual confirmation of stored products.

Method used

A product registration system with an imaging section, specifying section, and registration section that uses multiple imaging units to capture product images, identifies products, and registers them efficiently by distinguishing between single and multiple product types, allowing for continuous registration and selection of candidate products until completion.

Benefits of technology

Enables efficient and continuous product registration despite varying recognition results, improving convenience by allowing product selection operations at any time during the process, thus enhancing the overall registration efficiency and customer experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024166987000001_ABST
    Figure 2024166987000001_ABST
Patent Text Reader

Abstract

To provide a commodity registration system that can efficiently register a commodity to be purchased.SOLUTION: A commodity registration system comprises: an imaging unit configured to image a commodity to be housed in a housing unit; a specifying unit configured to recognize a commodity from an image captured by the imaging unit and specify the type of the commodity to be housed in the housing unit on the basis of a result of the recognition; and a registration unit configured to, when a single type of the commodity is specified by the specifying unit, register the type of the commodity, as a commodity to be purchased, in first registration information, or when multiple types of commodities are specified by the specifying unit, register a set of the types of the commodity, as candidate commodities, in second registration information. The registration unit is configured to receive an operation to select a type of one commodity from each of the candidate commodities registered in the second registration information, until the commodities to be purchased are completely registered, and register the selected type of the commodity in the first registration information.SELECTED DRAWING: Figure 7
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] An embodiment of the present invention relates to a product registration system. [Background technology]

[0002] Conventionally, there exists a technology in which an imaging device is provided in a shopping cart or basket (hereinafter also referred to as a storage unit) used in a store, and products stored in the storage unit are recognized from images captured by the imaging device. With such a technology, a customer can register a product as a purchase target by performing an action of storing the product in the storage unit.

[0003] However, in the above-mentioned technology, even if the product to be recognized is the same, the recognition result may differ depending on the conditions such as the imaging device and the recognition method. For example, when multiple imaging devices are used, the recognition result may differ for each imaging device because the conditions such as the angle at which the product is imaged differ. In this case, since the recognized product cannot be identified, the sales registration of the next product cannot be performed continuously, and it is necessary to confirm the product stored in the storage unit with the customer each time. Summary of the Invention [Problem to be solved by the invention]

[0004] An object of the present invention is to provide a product registration system that can efficiently register products to be purchased. [Means for solving the problem]

[0005] The product registration system of the embodiment includes an imaging unit, an identification unit, and a registration unit. The imaging unit images a product stored in a storage unit. The identification unit recognizes the product from the image captured by the imaging unit, and identifies the type of the product stored in the storage unit based on the recognition result. When the identification unit identifies a single type of product, the registration unit registers the type of the product in the first registration information as a product to be purchased, and when the identification unit identifies a plurality of types of products, the registration unit registers a set of the product types as candidate products in the second registration information. Furthermore, the registration unit accepts an operation to select one type of product from each of the candidate products registered in the second registration information until the registration of the products to be purchased is completed, and registers the selected type of product in the first registration information. [Brief description of the drawings]

[0006] [Figure 1] FIG. 1 is a diagram illustrating an example of an overall configuration of a system according to an embodiment. [Diagram 2] FIG. 2 is a side view showing an example of a general configuration of the shopping cart according to the embodiment. [Diagram 3] FIG. 3 is a diagram illustrating an example of a hardware configuration of the tablet terminal according to the embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of a data configuration of the product master according to the embodiment. [Diagram 5] FIG. 5 is a diagram illustrating an example of a data configuration of a sales registration table according to the embodiment. [Figure 6] FIG. 6 is a diagram illustrating an example of a data configuration of a candidate product table according to the embodiment. [Figure 7] FIG. 7 is a diagram illustrating an example of a functional configuration of the tablet terminal according to the embodiment. [Figure 8] FIG. 8 is a diagram illustrating an example of a screen displayed on the tablet terminal according to the embodiment. [Figure 9] FIG. 9 is a diagram illustrating an example of a screen displayed on the tablet terminal according to the embodiment. [Figure 10] FIG. 10 is a diagram illustrating an example of a screen displayed on the tablet terminal according to the embodiment. [Figure 11] FIG. 11 is a diagram illustrating an example of a screen displayed on the tablet terminal according to the embodiment. [Figure 12] FIG. 12 is a flowchart illustrating an example of a product registration process performed by the tablet terminal according to the embodiment. [Figure 13] FIG. 13 is a flowchart illustrating an example of a candidate product selection process performed by the tablet terminal according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] The product registration system according to the present embodiment will be described in detail below with reference to the accompanying drawings. In the embodiment, a terminal device (hereinafter also referred to as a tablet terminal) attached to a cart (hereinafter also referred to as a shopping cart) for transporting products used in a store will be described as an example of the product registration device. Note that the invention is not limited to this application example.

[0008] Fig. 1 is a diagram showing an example of the overall configuration of a system 1 according to an embodiment. The system 1 is an example of a product registration system, and as shown in Fig. 1, includes a tablet terminal 2, a server 4, and a payment machine 8. The server 4 and the payment machine 8 are communicatively connected via a network Na.

[0009] The network Na may be the Internet, a virtual private network (VPN), a local area network (LAN), a public communication network, a mobile communication network, or the like, either alone or in appropriate combination. As an example, the network Na may be a combination of the Internet and a mobile communication network.

[0010] In addition, a wireless access point 7 is connected to the network Na. The wireless access point 7 is connected to the tablet terminal 2 via a network Nb such as a wireless LAN. Therefore, the tablet terminal 2 can communicate with the server 4 and the payment machine 8.

[0011] The shopping cart 3 has a storage section 31 (see FIG. 2) for storing products displayed in the store. The top of the storage section 31 is open, and products are stored in the storage section 31 from the top. A customer borrows a shopping cart 3 from a store, and while moving around the store with the shopping cart 3, stores the products he or she wishes to purchase from the products displayed in the storage section 31.

[0012] The tablet terminal 2 is provided so as to be attachable to the shopping cart 3. The tablet terminal 2 recognizes products stored in the storage unit 31 from images captured by the imaging unit 34 described below, and executes sales registration processing related to the products. Here, the sales registration processing refers to processing for making the products identified by the tablet terminal 2 the target for purchase. Specifically, the sales registration processing refers to processing for registering information related to the recognized products (hereinafter also referred to as sales registration data) in the sales registration table 242 described below. For example, in the sales registration processing, sales registration data including a product code capable of identifying the product type, etc., and the number of purchases of the product, etc., are registered in the sales registration table 242.

[0013] Furthermore, when the tablet terminal 2 accepts an operation by a customer to perform a transaction on the accounting machine 8, it executes a process to transfer the sales registration data for one transaction to the accounting machine 8. For example, the tablet terminal 2 encodes and displays the sales registration data as a code symbol such as a two-dimensional code. In this case, the accounting machine 8 reads the code symbol displayed on the tablet terminal 2 to obtain the sales registration data for one transaction registered on the tablet terminal 2, and executes accounting processing based on the sales registration data.

[0014] The server 4 performs information processing for providing a service for carrying out buying and selling of goods in a store where the tablet terminal 2 is used. The server 4 may be, for example, a store server provided in the store, or a server device provided outside the store. Note that in this embodiment, the server 4 is realized by a single device, but it may also be a cloud server (or a cloud system) realized by multiple devices connected to a network.

[0015] The server 4 is an information processing device configured as a computer equipped with a processor, a main storage device, an auxiliary storage device, etc. The server 4 stores a product master in which the names and prices of products sold in a store are registered in a database (not shown) and provides the product master to the tablet terminal 2 and the cashier 8.

[0016] The accounting machine 8 is an accounting device installed in a store. The accounting machine 8 executes payment processing for products based on sales registration data. The payment processing includes displaying the total price and tax amount of products related to one transaction, calculating and displaying change based on the deposit received from the customer, and issuing a receipt with product information and payment information (total price, deposit amount, change amount, etc.) printed on it.

[0017] Next, the shopping cart 3 will be described. Fig. 2 is a side view showing an example of the overall configuration of the shopping cart 3. The shopping cart 3 is equipped with a basket-shaped storage section 31 with an open top, rotating casters 32 arranged below the storage section 31, and a handle section 33 connected to the storage section 31 and casters 32 and held by the customer when moving the shopping cart 3.

[0018] The storage section 31 has a generally rectangular side and bottom section surrounded by a mesh-like lattice 311 with an open top, allowing the products stored in the storage section 31 to be viewed from the outside. The side and bottom sections may be formed of walls instead of the mesh-like lattice 311.

[0019] The shopping cart 3 also includes a detachable tablet terminal 2 near the handle 33. The shopping cart 3 also includes a plurality of imaging units 34 (341, 342) near the edge of the opening of the storage unit 31. The imaging unit 34 is an example of an imaging unit, and is realized by a digital camera or the like. Note that while FIG. 2 shows an example in which two imaging units 341, 342 are provided, the number is not limited to this, and a configuration in which three or more units are provided may also be used. Hereinafter, when there is no need to distinguish between the imaging units 341, 342, they will be referred to as imaging unit 34.

[0020] The imaging units 34 are provided at different positions from each other so that they can capture images of the products stored in the storage unit 31. For example, the imaging direction of the imaging unit 34 is set so that at least a part of the imaging range includes the inside of the storage unit 31. The imaging unit 34 captures images of the products stored in the storage unit 31 and outputs the captured images to the tablet terminal 2. Here, the image means a still image or a video in frame units.

[0021] Next, a description will be given of the hardware configuration of the tablet terminal 2. FIG.

[0022] As shown in FIG. 3, the tablet terminal 2 includes a CPU (Central Processing Unit) 21, a ROM (Read Only Memory) 22, a RAM (Random Access Memory) 23, a storage unit 24, a display unit 25, an operation unit 26, a communication unit 27, and a connection unit 28.

[0023] The CPU 21 is an example of a processor, and controls the overall operation of the tablet terminal 2. The ROM 22 stores various programs. The RAM 23 is a workspace in which programs and various data are expanded.

[0024] The CPU 21, the ROM 22, and the RAM 23 are connected via a bus or the like to constitute a computer-configured control unit 200. In the control unit 200, the CPU 21 operates in accordance with a program stored in the ROM 22 or the storage unit 24 and loaded in the RAM 23, thereby executing various types of processing.

[0025] The storage unit 24 has a storage medium such as a HDD or a flash memory, and maintains the stored contents even when the power is cut off. The storage unit 24 stores programs that can be executed by the CPU 21 and various setting information. The storage unit 24 also stores a product master 241, a sales registration table 242, a candidate product table 243, and the like.

[0026] The product master 241 is a data table for managing products sold in a store. The sales registration table 242 is an example of first registration information. The sales registration table 242 is a data table for managing products that have been registered as sales using one's own tablet terminal 2. The candidate product table 243 is an example of second registration information. The candidate product table 243 is a data table for managing products that are candidates for sales registration.

[0027] The display unit 25 is a display device such as an LCD (Liquid Crystal Display). The display unit 25 displays various information under the control of the CPU 21. The operation unit 26 is an input device such as a keyboard or a pointing device. The operation unit 26 outputs operation contents input via the input device to the CPU 21. The operation unit 26 may be a touch panel provided on the display unit 25.

[0028] The communication unit 27 is a wired or wireless communication interface connectable to the network Nb. The communication unit 27 is communicatively connected to the wireless access point 7 via the network Nb. The control unit 200 communicates with the server 4 and the payment machine 8 via the communication unit 27.

[0029] The connection unit 28 is an interface for connecting a peripheral device. The connection unit 28 is a wired or wireless interface conforming to standards such as USB (Universal Serial Bus) and Bluetooth (registered trademark). In this embodiment, the connection unit 28 is connected to the imaging unit 34.

[0030] FIG. 4 is a diagram showing an example of the data configuration of the product master 241. As shown in FIG. 4, the product master 241 stores product codes and product information in association with each other. The product code is an example of identification information capable of identifying the type of product sold in a store. The product code may be, for example, a JAN code. The product information is information related to a product corresponding to the product code. For example, the product name, price, product image, etc. are registered in the product information. Here, the product name is information indicating the name and product name of the product. The price is information indicating the unit price of the product. The product image is image information representing the appearance, etc. of the product.

[0031] It should be noted that the data configuration of the product master 241 is not limited to that shown in FIG. 4. For example, it may be configured to register, in association with a product code, feature information indicating the external features of a product corresponding to the product code. Such feature information can be used as information for matching when recognizing a product from an image. It should be noted that the feature information may be the above-mentioned product image, or may be information (e.g., feature amount) that quantitatively indicates the external features of the product.

[0032] In addition, in this embodiment, the tablet terminal 2 is configured to hold the product master 241, but this is not limiting, and the server 4 may be configured to hold the product master 241. In this case, each tablet terminal 2 can refer to the product master 241 held by the server 4 and handle the product master 241 in the same way as if it were held locally.

[0033] Fig. 5 is a diagram showing an example of the data configuration of sales registration table 242. As shown in Fig. 5, sales registration table 242 stores registration numbers and product codes in association with each other. The registration number is information indicating the order in which sales registration was performed. The product code is the product code of the product registered as a sales item, and corresponds to the product code in product master 241.

[0034] Specifically, when a single type of product (product code) is identified from the image of the imaging unit 34, the product code of the product is associated with a newly issued registration number and registered in the sales registration table 242. Also, as described below, when multiple types of products are identified from the image of the imaging unit 34, the product code of the product is not registered in the sales registration table 242 but is registered in the candidate product table 243. However, in this embodiment, in order to maintain the order of product sales registration, a new registration number is issued and registered in the sales registration table 242 even when multiple types of products are identified from the image of the imaging unit 34.

[0035] The data configuration of the sales registration table 242 is not limited to that shown in Fig. 5. For example, the sales registration table 242 may be configured to register product information corresponding to a product code in association with the product code. Also, the sales registration table 242 may be configured to register date and time information indicating the date and time when the sales registration was performed in association with the product code.

[0036] Fig. 6 is a diagram showing an example of the data configuration of the candidate product table 243. As shown in Fig. 6, the candidate product table 243 stores a management ID, a registration number, and a candidate product (candidate product 1, 2) in association with each other. The management ID is identification information for managing data registered in the candidate product table 243. The registration number is a registration number related to the sales registration of the product managed by the management ID, and corresponds to the registration number in the sales registration table 242. The product code of a product that is a candidate for sales registration is registered as a candidate product.

[0037] In this embodiment, the configuration is provided with two storage areas for candidate products (candidate products 1 and 2) so that product codes of products identified from the images of the imaging units 341 and 342 can be stored, but the number of storage areas is not limited to this.

[0038] In addition, in this embodiment, the tablet terminal 2 is configured to hold the sales registration table 242 and the candidate product table 243, but this is not limiting, and the server 4 may be configured to hold the sales registration table 242 and the candidate product table 243. In this case, the server 4 can manage the sales registration table 242 and the candidate product table 243 for each transaction by, for example, issuing a transaction ID that can identify each transaction performed on each tablet terminal 2, and holding the sales registration table 242 and the candidate product table 243 in association with the transaction ID.

[0039] Specifically, the server 4 notifies the tablet terminal 2 of the issued transaction ID, thereby causing the tablet terminal 2 to hold the transaction ID while the transaction of the transaction ID is being carried out. The tablet terminal 2 then transmits the product code of the product to be registered for sales to the server 4 together with the transaction ID, and instructs the server 4 to register the product in the sales registration table 242 corresponding to the transaction ID. This allows the server 4 to centrally manage the products registered for sales on each of the tablet terminals 2. The tablet terminal 2 also transmits the product code of a candidate product that is a candidate for sales registration to the server 4 together with the transaction ID, and instructs the server 4 to register the product in the candidate product table 243 corresponding to the transaction ID. This allows the server 4 to centrally manage the products that are candidates for sales registration on each of the tablet terminals 2. It is assumed that the tablet terminal 2 is capable of referring to the sales registration table 242 and the candidate product table 243 of the transaction related to the tablet terminal itself.

[0040] Next, a description will be given of the functional configuration of the tablet terminal 2. FIG.

[0041] As shown in FIG. 7, the tablet terminal 2 includes an image acquisition unit 201, a product recognition unit 202, a registration processing unit 203, and a payment processing unit 204 as functional components.

[0042] The functional configuration of the tablet terminal 2 is a software configuration that is realized by the processor of the tablet terminal 2 operating in accordance with a program stored in a ROM or memory unit, but is not limited to this. For example, some or all of the functional configuration of the tablet terminal 2 may be realized by hardware such as a dedicated circuit.

[0043] The image acquisition unit 201 acquires the image captured by the imaging unit 34 via the connection unit 28. The product recognition unit 202 is an example of an identification unit. The product recognition unit 202 identifies the product imaged by the imaging unit 34 based on the image acquired by the image acquisition unit 201. Specifically, the product recognition unit 202 recognizes the product from the image captured by the imaging unit 34 by using a known object recognition technology or the like, and identifies the type of the product, that is, the product code, based on the external characteristics of the recognized product.

[0044] Specifically, the commodity recognition unit 202 processes the images captured by the imaging units 341, 342, respectively, and recognizes the commodities stored in the storage unit 31 from the images captured by the imaging units 341, 342 at the same timing. Note that the commodity recognition unit 202 may be configured to identify the commodity code held in the code symbol by decoding the code symbol attached to the commodity.

[0045] Note that the accuracy of recognizing products by the product recognition unit 202 is not 100%, and the recognition result (hereinafter also referred to as the identification result) may differ depending on the image capturing conditions of the product, etc. For example, even if the product to be recognized is the same, the image capturing unit 341 and the image capturing unit 342 have different conditions such as the angle at which the product is captured, so that the product identification result may differ between the image from the image capturing unit 341 and the image from the image capturing unit 342. In such a case, the product recognition unit 202 will identify two different product codes from the images from the image capturing unit 341 and the image capturing unit 342.

[0046] The registration processing unit 203 is an example of a registration unit, a display control unit, and a reception unit. The registration processing unit 203 executes sales registration processing of the product identified by the product recognition unit 202. Specifically, the registration processing unit 203 associates the product code of the product identified by the product recognition unit 202 with a registration number and registers it in the sales registration table 242. In addition, the registration processing unit 203 reads out the product name and price of the product corresponding to the registered product code from the product master 241, and displays them on the display unit 25, thereby displaying a list of the sales-registered products in the order of their registration numbers.

[0047] As described above, depending on the processing result of the commodity recognition unit 202, two different commodity codes may be identified from the image from the imaging unit 341 and the image from the imaging unit 342. In this case, the registration processing unit 203 registers the different commodity codes identified by the commodity recognition unit 202 as candidate commodities in the candidate commodity table 243. More specifically, the registration processing unit 203 records the order of sales registration by registering a registration number in the sales registration table 242, and then associates the registration number with the different commodity codes (candidate commodities) identified by the commodity recognition unit 202 and registers them in the candidate commodity table 243.

[0048] Furthermore, when the registration processing unit 203 registers a candidate product in the candidate product table 243, it causes the display unit 25 to display information notifying the presence of the candidate product. Here, the method of notification is not particularly limited, and for example, the notification may be made in the form of an operator as shown in FIG.

[0049] Fig. 8 is a diagram showing an example of a screen (hereinafter also referred to as a product registration screen) displayed on the tablet terminal 2. As shown in Fig. 8, the product registration screen A has a product display area Aa, a total display area Ab, and an operation area Ac.

[0050] In the product display area Aa, the details of the registered products, such as the product name, number of items, and price of each product registered for sales, are displayed in the order of registration. Specifically, the registration processing unit 203 displays the product name, number of items, price, etc. of each registered product in the product display area Aa in the order of registration number, based on the information stored in the product master 241 and the sales registration table 242. In addition, based on the registration number stored in the candidate product table 243, the registration processing unit 203 displays an operator Aaa with a message prompting the selection of a product in the display area of ​​the product display area Aa corresponding to the registration number.

[0051] Here, the operator Aaa is an example of information notifying the presence of a candidate product. The operator Aaa is associated with a corresponding registration number (or management ID), and upon receiving an operation of the operator Aaa, the registration processing unit 203 displays, for example, a screen shown in FIG. 9 on the display unit 25 based on the registration number (or management ID) of the operator Aaa.

[0052] Fig. 9 is a diagram showing an example of a screen (hereinafter also referred to as a product selection screen) displayed on the tablet terminal 2. As shown in Fig. 9, the product selection screen B has a first display area Ba and a second display area Bb for displaying candidate products.

[0053] Specifically, when the registration processing unit 203 receives an operation of the operator Aaa, it reads out the product codes registered as candidate products from the candidate product table 243 based on the registration number (or management ID) associated with the operator Aaa. Then, the registration processing unit 203 displays the product names and product images corresponding to each of the product codes read out as candidate products in the first display area Ba and the second display area Bb, respectively, based on the information stored in the product master 241. Fig. 9 shows an example in which "Product EF" is displayed in the first display area Ba, and "Product GH" is displayed in the second display area Bb.

[0054] Here, the first display area Ba and the second display area Bb also function as operators, and when a customer selects one of the products, i.e., the product to be purchased, the registration processing unit 203 registers the product code of the candidate product corresponding to the selected display area in the sales registration table 242. More specifically, the registration processing unit 203 registers the product code of the candidate product corresponding to the selected display area in the sales registration table 242 in association with the relevant registration number. The registration processing unit 203 also deletes the data entry related to the registration number registered in the sales registration table 242 from the candidate product table 243. Then, the registration processing unit 203 erases the product selection screen B and returns the display to the product registration screen A.

[0055] For example, when "product EF" is selected on the product selection screen B in Fig. 9, the registration processing unit 203 displays the product name (product EF), quantity, price, etc. of the product registered for sales in the product display area Aa instead of the operation button Aaa, as shown in the product registration screen A in Fig. 10. Here, Fig. 10 is a diagram showing an example of a screen displayed on the display unit 25 of the tablet terminal 2.

[0056] It is assumed that sales registration of a new commodity is possible in either the state of Fig. 8 or Fig. 10. That is, in the tablet terminal 2 of this embodiment, even if a candidate commodity is registered in the candidate commodity table 243, a new commodity can be registered in the sales registration table 242.

[0057] In this way, even if the identification results of the commodity recognition unit 202 obtained from images of the same commodity differ, the tablet terminal 2 can continue the sales registration operation without stopping the sales registration operation of the new commodity. Also, the tablet terminal 2 can select the commodity registered in the candidate commodity table 243 as a candidate commodity at any timing, thereby improving convenience.

[0058] Continuing with the explanation of the product registration screen A, the points and purchase amount (total amount) of the registered products are displayed in the total display area Ab. Specifically, the registration processing unit 203 calculates the total points and total amount based on the prices and points of the products registered as sales, and displays them in the total display area Ab. Note that the points and prices of the candidate products are excluded from the calculation.

[0059] Various operation buttons are displayed in the operation area Ac. For example, a checkout button Aca is displayed in the operation area Ac. The checkout button Aca is an operator for ending the sales registration of a product and instructing the checkout of the product registered in the sales registration table 242.

[0060] When the registration processing unit 203 receives the operation of the checkout button Aca, the registration processing unit 203 confirms the products registered in the sales registration table 242 as products for one transaction by terminating the process related to product registration, provided that the candidate product table 243 is empty. Here, if the candidate product table 243 is not empty, that is, if a candidate product is registered, the registration processing unit 203 notifies the customer that there is an unconfirmed product and displays a screen for selecting a product to be registered for sales from the candidate products. For example, the registration processing unit 203 prompts the customer to select a product to be purchased by sequentially displaying the product selection screen B shown in FIG. 9 in the order of the management IDs (registration numbers) registered in the candidate product table 243. Then, when the candidate product table 243 becomes empty, the registration processing unit 203 confirms the products registered in the sales registration table 242 as products for one transaction by terminating the process related to product registration. After operating the checkout button Aca, the customer can make a payment for the products registered in the sales registration table 242 for one transaction by moving the shopping cart 3 to the checkout machine 8.

[0061] In this way, the registration processing unit 203 suppresses the transaction and continues the sales registration process until the candidate product table 243 becomes empty. This allows the customer to select each of the candidate products registered in the candidate product table 243 at any time before the transaction is made. For example, the customer can process multiple candidate products collectively just before the transaction, or process any candidate product during the sales registration operation. Therefore, the system 1 of this embodiment can improve the degree of freedom when registering product sales.

[0062] In the above example, after the operator Aaa on the product registration screen A is operated, the product selection screen B is used to select a product to be registered as a sales target from among candidate products, but this is not limiting. For example, as shown in Fig. 11, the operator corresponding to the first display area Ba and the second display area Bb of the product selection screen B may be displayed on the product registration screen A, allowing the candidate product to be selected within the product registration screen A.

[0063] FIG. 11 is a diagram showing an example of a screen displayed on the tablet terminal 2, and shows another display form of the product registration screen A. In FIG.

[0064] The product registration screen A shown in Fig. 11 corresponds to the product registration screen A in Fig. 8, and shows an example in which the presence of candidate products, which was represented by the operator Aaa, is represented by operators Aab and Aac representing product names, etc. Here, the operators Aab and Aac are operators corresponding to the first display area Ba and the second display area Bb of the product selection screen B, and represent product names, etc. corresponding to the product codes of the candidate products.

[0065] 11, a product to be purchased can be selected by operating any one of the operators Aab and Aac. Specifically, when any one of the operators Aab and Aac is selected, the registration processing unit 203 operates in the same manner as the product selection screen B to register the product code of the candidate product corresponding to the selected operator in the sales registration table 242.

[0066] This allows the customer to check the products that have been registered for sale and the products that have been designated as candidate products by looking at the product registration screen A, and to select a product to be registered for sale from among the displayed candidate products. Therefore, the tablet terminal 2 can improve the convenience of the candidate product selection operation.

[0067] 7, the payment processor 204 executes a process related to the payment of the registered sales items in response to the customer's operation. For example, when the items for one transaction are confirmed by operating the checkout button Aca, the payment processor 204 performs the checkout for the items registered in the sales registration table 242.

[0068] Specifically, the payment processing unit 204 inputs data capable of identifying the sales-registered products into the payment machine 8, thereby enabling the payment machine 8 to make a payment. For example, the payment processing unit 204 encodes the sales registration data registered in the sales registration table 242 into a code symbol such as a two-dimensional code and displays it on the display unit 25. In this case, the payment machine 8 obtains the sales registration data by reading the code symbol displayed on the display unit 25 of the tablet terminal 2, and performs payment processing for the products for one transaction that have been registered as sales, based on the obtained sales registration data.

[0069] Furthermore, for example, the payment processing unit 204 may transmit the sales registration data registered in the sales registration table 242 to the payment machine 8 by short-range wireless communication such as Bluetooth (registered trademark). In this case, the payment machine 8 acquires the sales registration data transmitted from the tablet terminal 2, and performs payment processing for the products for one transaction that have been registered as sales, based on the acquired sales registration data.

[0070] Also, for example, in a configuration in which the sales registration table 242 is stored in the server 4, the payment processing unit 204 encodes the transaction ID of the transaction into a code symbol such as a barcode or two-dimensional code, and displays it on the display unit 25. In this case, the payment machine 8 obtains the transaction ID by reading the code symbol displayed on the display unit 25 of the tablet terminal 2. The payment machine 8 then obtains sales registration data related to the obtained transaction ID from the server 4, and performs payment processing for the products for one transaction that have been registered as sales, based on the obtained sales registration data.

[0071] The payment for the products is not limited to the payment using the cashier 8, but may be made on the tablet terminal 2. For example, when the payment processing unit 204 receives an instruction to start payment via the operation unit 26, the payment processing unit 204 displays a screen on the display unit 25 that allows the user to select a payment method such as electronic money payment or credit card payment. When the payment method is selected, the payment processing unit 204 receives input of payment information for making the payment using the selected payment method, and executes a payment process to settle the total amount of the products registered as sales based on the input payment information by cooperating with an external payment server. Such payment information may be, for example, a credit card number or a two-dimensional code for code payment. In the case of a two-dimensional code for code payment, the input may be received via the imaging unit 34.

[0072] An example of the operation of the tablet terminal 2 will be described below with reference to Fig. 12. Fig. 12 is a flowchart showing an example of a product registration process performed by the tablet terminal 2.

[0073] First, when the control unit 200 of the tablet terminal 2 starts this process in response to an operation via the operation unit 26, the control unit 200 initializes the sales registration table 242 and the candidate product table 243 (step S11).

[0074] In addition, when the sales registration table 242 and the candidate product table 243 are configured to be held by the server 4, the control unit 200 requests the server 4 to issue a transaction ID, thereby causing the server 4 to hold the sales registration table 242 and the candidate product table 243 related to the transaction ID of the transaction. Also, the control unit 200 may obtain the issued transaction ID from the server 4 and hold it in the RAM 23 or the like.

[0075] Next, the image acquisition unit 201 acquires an image from the imaging unit 34 (341, 342) (step S12). Next, the commodity recognition unit 202 executes a process of recognizing a commodity from the image acquired in step S12 (step S13). Next, the registration processing unit 203 determines whether or not a commodity has been identified from the image captured by the imaging unit 34 as a result of the process of step S13 (step S14). Here, if a commodity has not been identified (step S14; No), the process proceeds to step S20.

[0076] Furthermore, if it is determined in step S14 that the product has been identified (step S14; Yes), the registration processing unit 203 determines whether or not the product has been identified from both images captured by the imaging units 341 and 342 (step S15). If the product has been identified from either one of the images captured by the imaging units 341 and 342 (step S15; No), that is, if the identified product is of a single type, the registration processing unit 203 registers the product code of the identified product in the sales registration table 242 (step S17) and proceeds to step S20.

[0077] Furthermore, in step S15, if a product is identified from images from both the imaging unit 341 and the imaging unit 342 (step S15; Yes), the registration processing unit 203 determines whether the identified products match (step S16). If the identified products match (step S16; Yes), that is, if the identified products are of a single type, the registration processing unit 203 registers the product code of the product in the sales registration table 242 (step S17) and proceeds to step S20.

[0078] Furthermore, in step S16, if it is determined that the identified products are not a match (step S16; No), that is, if there are multiple types of identified products, the registration processing unit 203 registers the product codes of those products as candidate products in the candidate product table 243 (step S18). Next, the registration processing unit 203 notifies the presence of candidate products by displaying the operation button Aaa, etc. on the above-mentioned product registration screen A (step S19), and proceeds to step S20.

[0079] Next, the registration processing unit 203 determines whether or not a selection operation for selecting a target product for sales registration from the candidate products has been received (step S20). For example, the registration processing unit 203 determines whether or not an operation for the operator Aaa on the product registration screen A has been received.

[0080] If no selection operation is performed (step S20; No), the process proceeds to step S22. If it is determined that a selection operation has been received (step S20; Yes), the registration processing unit 203 executes a candidate product selection process (step S21) and proceeds to step S22. The candidate product selection process will be described later.

[0081] Next, the registration processing unit 203 determines whether or not an operation to end the sales registration has been performed (step S22). For example, the registration processing unit 203 determines whether or not an operation on the checkout button Aca on the product registration screen A has been accepted.

[0082] If the end operation is not performed (step S22; No), the registration processing unit 203 continues the sales registration by returning to step S12. If it is determined that the end operation has been accepted (step S22; Yes), the registration processing unit 203 determines whether the candidate product table 243 is empty (step S23). If a candidate product is registered in the candidate product table 243 (step S23; No), the registration processing unit 203 executes a candidate product selection process (step S24) and returns the process to step S23.

[0083] On the other hand, if it is determined in step S23 that the candidate product table 243 is empty (step S23; Yes), the registration processing unit 203 determines the products registered in the sales registration table 242 as products for one transaction, and ends this process.

[0084] Next, a candidate product selection process performed by the tablet terminal 2 will be described with reference to Fig. 13. Fig. 13 is a flowchart showing an example of the candidate product selection process performed by the tablet terminal 2. Note that in this process, a form in which a product selection screen B is used to select a target product for sales registration will be described.

[0085] First, the registration processing unit 203 displays the product selection screen B on the display unit 25 based on the candidate products registered in the candidate product table 243 (step S31).

[0086] For example, when a registration number is designated by operating the operator Aaa, the registration processing unit 203 displays the product selection screen B of the candidate products related to the designated registration number on the display unit 25. In addition, when a registration number is not designated, for example, when the checkout button Aca is operated, the registration processing unit 203 sets the candidate products to be displayed in ascending order of the management ID (registration number), and displays the product selection screen B of the candidate products on the display unit 25.

[0087] Next, the registration processing unit 203 waits until a product to be registered as a sales target is selected from the candidate products (step S32; No). When a product to be registered as a sales target is selected (step S32; Yes), the registration processing unit 203 registers the code symbol of the selected product in the sales registration table 242 (step S33). Next, the registration processing unit 203 deletes the data entry related to the product registered in step S33 from the candidate product table 243 (step S34), and returns to the product registration process of FIG.

[0088] As described above, the system 1 of this embodiment includes an imaging unit 34 that captures images of products to be purchased stored in a storage unit, a product recognition unit 202 that recognizes products from images captured by the imaging unit 34 and identifies the type of product stored in the storage unit based on the recognition result, and a registration processing unit 203 that, when a single type of product is identified, registers the type of product as a product to be purchased in the sales registration table 242, and, when multiple types of products are identified, registers a set of the product types as candidate products in the candidate product table 243. Furthermore, the registration processing unit 203 accepts an operation to select one product from each of the candidate products registered in the candidate product table 243 until the registration of products to be purchased is completed, and registers the product code of the selected product in the sales registration table 242.

[0089] As a result, in the system 1 of this embodiment, even if the type of product identified from the captured image of the same product is different, it is not necessary to confirm the product each time, and new product sales registration can be continuously performed. Furthermore, in the system 1 of this embodiment, the selection operation of the product registered in the candidate product table 243 as a candidate product can be performed at any timing until the sales registration is completed. Therefore, the system 1 of this embodiment can efficiently perform product sales registration. Furthermore, the system 1 of this embodiment can improve the convenience when performing product sales registration.

[0090] The above-described embodiment can be modified as appropriate by changing a part of the configuration or function of each of the above-described devices. Therefore, some modified examples of the above-described embodiment will be described below as other embodiments. Note that the following mainly describes the differences from the above-described embodiment, and a detailed description of the common points with the contents already described will be omitted. The modified examples described below may be implemented individually or in appropriate combination.

[0091] (Variation 1) In the above embodiment, the shopping cart 3 is provided with a plurality of imaging units 34, but the present invention is not limited to this, and may be provided with a single imaging unit 34. In this case, the product recognition unit 202 performs recognition processing of different types (recognition methods) on the image captured by the single imaging unit 34, and outputs product identification results for each type of recognition processing.

[0092] For example, the commodity recognition unit 202 performs a first recognition process for identifying a commodity by reading a code symbol and a second recognition process for identifying a commodity by general object recognition on an image captured by the single imaging unit 34. Then, the commodity recognition unit 202 outputs the processing results of the first recognition process and the second recognition process, that is, the commodity identification results.

[0093] Furthermore, the registration processing unit 203 performs the same processing as in the above-described embodiment based on the identification result output by the commodity recognition unit 202. As a result, in the tablet terminal 2 according to this modification, even when a single imaging unit 34 is used, it is possible to achieve the same effect as in the above-described embodiment.

[0094] (Variation 2) In the above-described embodiment, a consistency check of the product identification results is performed when storing a product in the storage section 31, but this is not limited to the above, and a consistency check of the product identification results may also be performed when removing a product from the storage section 31.

[0095] Specifically, when the product recognition unit 202 of this embodiment recognizes a product from an image, it detects the direction in which the product moves from temporally consecutive images. Specifically, the product recognition unit 202 detects the movement of the product relative to the storage unit 31 (e.g., a motion vector). In other words, the product recognition unit 202 detects information that can determine whether the product has been stored in the storage unit 31 or removed from the storage unit 31. Note that the product recognition unit 202 identifies the product code once for a product recognized from a series of images.

[0096] Furthermore, when the product recognition unit 202 recognizes a product and detects that the product is moving downward, the registration processing unit 203 determines that the product has been stored in the storage unit 31. In this case, the registration processing unit 203 registers the product code of the product identified by the product recognition unit 202 in the sales registration table 242 in association with the registration number.

[0097] Furthermore, when the product recognition unit 202 recognizes a product and detects that the product is moving upward, the registration processing unit 203 determines that the product has been removed from the storage unit 31.

[0098] Then, the registration processing unit 203 performs a match determination for the product identified by the product recognition unit 202 in each of the cases where it is determined that the product has been stored in the storage unit 31 and where it is determined that the product has been removed from the storage unit 31. Then, when the registration processing unit 203 determines that the identified product does not match, it causes the display unit 25 to display a screen on which a product to be purchased or canceled can be selected, as in the above-described embodiment.

[0099] In addition, when a product is removed from the storage unit 31, the product code of the product is already registered in the sales registration table 242 or the candidate product table 243, so the registration processing unit 203 does not make a new registration, but deletes the registered data entry.

[0100] Specifically, when the product codes of the products identified by the product recognition unit 202 are the same or the same, the registration processing unit 203 deletes the data entry including the product code from the sales registration table 242 or the candidate product table 243. Note that when the product code of the identified product exists in both the sales registration table 242 and the candidate product table 243, it is preferable that the registration processing unit 203 preferentially deletes it from the sales registration table 242 in which confirmed products are registered.

[0101] Furthermore, if the product codes of the products identified by the product recognition unit 202 do not match, the registration processing unit 203 displays on the display unit 25 a screen (e.g., product selection screen B) on which the product corresponding to the product code can be selected, and prompts the customer to make a selection. Then, the registration processing unit 203 deletes the data entry including the product code of the selected product from the sales registration table 242 or the candidate product table 243. In this case, since the products to be deleted are products registered in the sales registration table 242 or the candidate product table 243, if the product recognition unit 202 identifies a product that is not registered in the table, the product may be excluded from the target of the match determination.

[0102] It should be noted that when a product is removed from the storage section 31, if the product identification results are inconsistent, it is preferable to display the product selection screen B each time such a case occurs, but as in the above-described embodiment, the configuration may be such that this can be done at any time.

[0103] In this case, for example, the product code of the identified product may be stored in the candidate product table 243 as a candidate product to be deleted, and the candidate product to be deleted may be selected at any timing. As an example, flag information or the like may be used to distinguish whether the product is a candidate product for sales registration or a candidate product to be deleted. In this case, it is preferable to select the candidate product to be deleted before selecting the candidate product for sales registration, in order to prevent the same product from being deleted after being registered as a sales product.

[0104] (Variation 3) In the above embodiment, the embodiment in which products are stored in the storage section 31 of the shopping cart 3 has been described, but the present invention is not limited to this, and products may be stored in other types of storage section 31. For example, a shopping basket used for carrying products in a store may be an example of the storage section 31. In this case, the imaging section 34 is installed near the edge of the opening of the shopping basket, etc., and is installed so that at least a part of the imaging range includes the inside of the shopping basket.

[0105] In this case, the tablet terminal 2 may be replaced with a mobile terminal such as a smartphone having the same functions as the tablet terminal 2. The shopping basket may be configured to be carried by the customer or may be configured to be carried in the storage section 31 of the shopping cart 3.

[0106] (Variation 4) In the above-described embodiment, the tablet terminal 2 recognizes (and identifies) the product from the image captured by the imaging unit 34, but this is not limiting and an external device such as the server 4 may perform this function. For example, the server 4 may be equipped with the product recognition unit 202. In this case, the image acquisition unit 201 of the tablet terminal 2 transmits the acquired image to the server 4. The product recognition unit 202 of the server 4 recognizes (and identifies) the product from the image transmitted from the tablet terminal 2, and returns the processing result to the tablet terminal 2 that transmitted the image.

[0107] Furthermore, the registration processing unit 203 of the tablet terminal 2 displays the above-mentioned product registration screen A and product selection screen B on the display unit 25 based on the processing result returned from the server 4. Then, the registration processing unit 203 of the tablet terminal 2 may register the product code of the product selected from the candidate products via the screen in the sales registration table 242, or delete the data entry related to the product code from the candidate product table 243.

[0108] (Variation 5) In the above embodiment, the sales registration table 242 and the candidate product table 243 are separate tables, but the present invention is not limited to this. The candidate products may be managed using only the sales registration table 242.

[0109] In this case, when there are multiple types of products identified by the product recognition unit 202, the registration processing unit 203 may register the product codes of these products in the sales registration table 242. Furthermore, the registration processing unit 203 may handle products for which multiple different product codes are registered for one registration number as candidate products, and execute a process to prompt the selection as in the above-described embodiment. Specifically, when a product to be registered as a sales product is selected from the candidate products, the registration processing unit 203 deletes from the sales registration table 242 the product codes of other products that are registered in association with the same registration number as the selected product.

[0110] In this modified example, the first registration information and the second registration information refer to the sales registration table 242.

[0111] The programs executed by each device in the above-described embodiments (and the modified examples) are provided in a state where they are pre-installed in a ROM, a storage unit, etc. The programs executed by each device in the above-described embodiments may be provided by being recorded in a computer-readable recording medium such as a CD-ROM, a flexible disk (FD), a CD-R, or a digital versatile disk (DVD) in the form of an installable or executable file.

[0112] Furthermore, the programs executed by each device of the above-described embodiments may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Also, the programs executed by each device of the above-described embodiments may be provided or distributed via a network such as the Internet.

[0113] Although the embodiment of the present invention has been described above, this embodiment is presented as an example and is not intended to limit the scope of the invention. These new embodiments and their modifications can be implemented in various other forms, and various omissions, substitutions, changes, and combinations can be made without departing from the gist of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]

[0114] 1 System 2. Tablet devices 3. Shopping Cart 4 Server 8. Cashier 200 Control section 201 Image Acquisition Unit 202 Product recognition department 203 Registration processing unit 204 Payment Processing Unit 241 Product Master 242 Sales registration table 243 Candidate Product Table [Prior art documents] [Patent documents]

[0115] [Patent Document 1] JP 2014-235530 A

Claims

1. an imaging unit that captures an image of a product to be purchased stored in the storage unit; an identification unit that recognizes a commodity from an image captured by the imaging unit and identifies a type of the commodity stored in the storage unit based on the recognition result; a registration unit that, when a single type of product is identified by the identification unit, registers the type of product in the first registration information as a product to be purchased, and, when a plurality of types of products are identified by the identification unit, registers a set of the types of products in the second registration information as candidate products; having A product registration system in which the registration unit accepts an operation to select a type of product from each of the candidate products registered in the second registration information until registration of the products to be purchased is completed, and registers the selected type of product in the first registration information.

2. a display control unit that displays, on a display device, a screen on which a type of one product can be selected from the candidate products for each of the candidate products registered in the second registration information; The registration unit registers a type of the product selected based on the screen in the first registration information. The product registration system according to claim 1.

3. a reception unit for receiving an instruction to end registration of a product to be purchased, The display control unit displays the screen when the candidate product is registered in the second registration information at the time when the reception unit receives the instruction. The product registration system according to claim 2.

4. the display control unit displays the screen including an operator capable of selecting one type of product from the candidate products at a timing when the candidate products are registered in the second registration information.

4. The product registration system according to claim 2 or 3.

5. A plurality of the imaging units are provided, The identification unit identifies a product type from the images captured by each of the imaging units. The product registration system according to claim 1.

6. the identification unit performs different types of recognition processing on the images captured by the imaging unit, and outputs product identification results for each type of recognition processing. The product registration system according to claim 1.

Citation Information

Patent Citations

  • Information processing apparatus, information processing system, and information processing method

    JP2014149863A

  • Information processing apparatus and program

    JP2017211881A

  • Object recognition device, object recognition system, and object recognition method

    JP2021056803A

  • Commodity pickup apparatus

    JP2022089455A

  • Login control device and program

    JP2022140497A