Product reading device, store payment system, and product reading method

Abstract

To provide a commodity reader, etc. capable of accurately registering special commodities such as set commodities and campaign commodities.SOLUTION: A commodity reader 1 includes: an image acquisition unit 151 that acquires a commodity image; a commodity identification unit 152 that identifies individual commodities; a commodity identification unit 153 that determines whether or not one, two, or more of the commodities included in the commodity image correspond to a special commodity with a price different from a standard price so as to identify the special commodities; a display control unit 154 that displays information on the special commodities identified by the commodity identification unit together with the commodity image on a display unit; and a reading information transmission unit 156 that transmits, to the outside, first code information associated with the special commodity when the special commodity is selected by the operator, or otherwise, transmits, to the outside, second code information associated with the individual commodity when completion of confirmation of the commodity is entered by the operator with no special commodities selected.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a commodity reading device, a store payment system, and a commodity reading method. [Background technology]

[0002] Conventionally, a technology has been disclosed in which a product image is linked to a product ID, the product image is compared with an object contained in a captured image, and a candidate product corresponding to the object in the image is identified based on the degree of similarity (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2013-182323 A Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional techniques have a problem in that they cannot handle cases where a product included in a captured image corresponds to a predetermined set of products or corresponds to a predetermined campaign product.

[0005] Therefore, the present invention has been made in consideration of these points, and aims to provide a product reading device, a store payment system, and a product reading method that are capable of accurately registering special products such as set products and campaign products. [Means for solving the problem]

[0006] The product reading device of the present invention comprises an image acquisition unit that acquires a product image showing multiple products purchased by a customer; a product identification unit that identifies each of the products included in the product image; a product identification unit that determines whether one or more of the products included in the product image are special products set at a price different from a standard price and identifies the special products corresponding to the one or more products; a display control unit that displays information about the special products identified by the product identification unit on a display unit together with the product image; and a read information transmission unit that, when the special product is selected by an operator, transmits first code information associated with the special product to the outside, and, when the special product is not selected and the operator inputs completion of product confirmation, transmits second code information associated with each of the products to the outside.

[0007] The product reading device of the present invention may further include a memory unit that stores, in association with the product, whether or not the product may be a special product, and the product identification unit may determine whether or not the one or more products are the special products by referring to the memory unit.

[0008] The special product includes a component product included in a specified set product, and the memory unit stores the first code information and a plurality of the component products included in the set product in association with the set product, and the read information transmission unit may transmit the first code information stored in the memory unit in association with the set product when the product image includes images of the plurality of component products.

[0009] The special products include constituent products included in a specified set product and a specified campaign product, and if the product identification unit determines that the one or more products correspond to both the set product and the campaign product, the display control unit may display on the display unit a first icon for the operator to select the set product and a second icon for the operator to select the campaign product.

[0010] The display control unit may continue to display at least the other of the first icon and the second icon on the display unit after either the first icon or the second icon is selected by the operator.

[0011] The display control unit may, when the size of the product identified by the product identification unit in the product image is equal to or larger than a predetermined standard size, superimpose text information indicating the type or name of the product on the product image, and when the size of the product identified by the product identification unit in the product image is less than a predetermined standard size, superimpose alphanumeric characters or symbols with a shorter number of characters than the text information on the product image, and display text information indicating the type or name of the product corresponding to the alphanumeric characters or symbols in an area that does not overlap the product image.

[0012] The display control unit may display a graphic image superimposed on each of the multiple products in a product display area in which the products are displayed, and display the product name in an area different from the product display area in association with a graphic of the same color as the graphic image superimposed on the product.

[0013] The product identification unit may identify products that are missing to satisfy the conditions of the set product by comparing the one or more products in the product image with the special product, and the display control unit may display the missing products identified by the product identification unit on the display unit.

[0014] The store payment system of the present invention includes the product reading device described above, and a store terminal that is communicatively connected to the product reading device and executes payment processing based on the first code information or the second code information received from the product reading device.

[0015] The product reading method of the present invention includes the steps of acquiring a product image showing multiple products purchased by a customer, identifying each of the products included in the product image, determining whether one or more of the products included in the product image are special products set at a price different from a standard price and identifying the special products corresponding to the one or more products, displaying information about the special products together with the product image on a display unit, and, if the special product is selected by an operator, transmitting first code information associated with the special product to the outside, and if the special product is not selected and the operator inputs completion of product confirmation, transmitting second code information associated with each of the products to the outside, and each of the steps is executed by a computer. Effect of the Invention

[0016] According to the present invention, it is possible to provide a product recognition device, a store payment system, and a product recognition method that are capable of accurately registering special products such as set products and campaign products. [Brief description of the drawings]

[0017] [Figure 1] FIG. 1 is a diagram showing an overview of a store payment system. [Diagram 2] FIG. 2 is a block diagram showing the configuration of a store payment system. [Diagram 3] 2 is a perspective view showing the appearance of a camera stand included in the commodity reading device; FIG. [Figure 4] 1 is an example of a product image generated by an imaging device. [Diagram 5] This is information about a set product. [Figure 6] This is information about campaign products. [Figure 7] 13 is an example of a result display screen that displays a result of identification of a commodity by a commodity reading device. [Figure 8] 4 is a flowchart for explaining an operation sequence of the present embodiment. [Figure 9]FIG. 8 is a diagram showing the state after the salad set icon is pressed on the result display screen of FIG. 7. [Figure 10] FIG. 10 is a diagram showing the state after the icon of the doughnut CP is further pressed on the result display screen of FIG. 9. [Figure 11] 13 is another example of a result display screen showing a product identification result. [Figure 12] 13 is yet another example of a result display screen showing a product identification result. [Figure 13] 13 is yet another example of a result display screen showing a product identification result. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] <First embodiment> (Overview of the store payment system S100) 1 is a diagram showing an overview of the store payment system S100. In the following, an example is shown in which the store payment system S100 is used in a bakery shop and a store clerk registers products.

[0019] The store payment system S100 is a system installed in a bakery shop, and includes a product reading device 1 and a store terminal 2. Products such as bread, salad, and donuts are sold in the bakery shop. A customer places the products he or she wants to purchase on a tray, takes it to the cash register, and pays for the products at the cash register.

[0020] As shown in FIG. 1, the commodity reading device 1 takes an image of the commodities P1 and P2 (also simply referred to as "commodities P") placed on a tray T to generate a commodity image ((1) in FIG. 1), and analyzes the commodity image to identify each of the commodities P1 and P2. As an example, the commodity P1 is a "round bread" and the commodity P2 is a "donut". The commodity reading device 1 judges whether the identified one or more commodities P correspond to a "special commodity" which is a predetermined set commodity or a campaign commodity. If one or more commodities P correspond to a special commodity, the commodity reading device 1 displays information on the commodities P1 and P2 and information on the special commodity on the display unit 12.

[0021] The operator, a store clerk, checks the display unit 12 and selects a "special product" on the screen, which may be a set product or a campaign product ((2) in FIG. 1). The product reading device 1 then transmits first code information associated with the special product to the store terminal 2. On the other hand, if the store clerk does not select a "special product" but instead selects "check results" for the product on the screen, the product reading device 1 transmits second code information associated with each of the products P1 and P2 to the store terminal 2 ((3) in FIG. 1).

[0022] According to the store payment system S100 of this embodiment, information on special products such as set products and campaign products identified by the product reading device 1 is displayed on the display unit 12, and when a store clerk checks and selects the special product, first code information associated with the special product is transmitted to the store terminal 2.

[0023] According to this configuration, the commodity reading device 1 can photograph the commodity P that the customer is about to purchase and identify it as a special commodity such as a set commodity or a campaign commodity, so that the store clerk does not need to manually register the set commodity or the campaign commodity, for example. Even if the commodity reading device 1 erroneously identifies a special commodity that is a set commodity or a campaign commodity, the code information of the special commodity is not transmitted to the store terminal 2 unless the store clerk checks the contents displayed on the display unit 12 and selects the special commodity. Therefore, it is possible to prevent the settlement process from being executed based on the erroneous information of the special commodity.

[0024] (Specific configuration of the store payment system S100) Fig. 2 is a block diagram showing the configuration of the store payment system S100. Fig. 3 is a perspective view showing the appearance of a camera stand 17 provided in the commodity reading device 1. As shown in Fig. 2, the commodity reading device 1 has a communication unit 11, a display unit 12, an input device 13, an imaging device 14, a control unit 15, and a storage unit 16. As an example, the commodity reading device 1 is configured by a tablet computer.

[0025] The communication unit 11 is a communication interface for communicatively connecting the commodity reading device 1 to the store terminal 2. The display unit 12 is one or more displays on which various information is displayed. The display unit 12 may be a touch panel display.

[0026] The input device 13 is a device for receiving input from an operator. The input device 13 is, for example, a touch panel provided as a part of a touch panel display. The input device 13 may be any user interface other than a touch panel. The arbitrary user interface may be one that receives input from the operator via a physical button or the like, or may be a graphical user interface that receives input from the operator via an icon on a screen or the like.

[0027] The imaging device 14 has an imaging element (not shown) such as a CCD or a CMOS. The imaging device 14 captures an image of the product P placed on the tray T and generates a product image showing one or more products P. The imaging device 14 may be supported by any configuration, but for example, a camera stand 17 as shown in FIG. 3 may support the imaging device 14.

[0028] The camera stand 17 includes a base 18 and a stand 19. The base 18 has a flat shape, and is provided with a mounting portion 18a on the upper surface of which a tray T is placed. The stand 19 is a member that holds the imaging device 14. The stand 19 is provided so as to extend upward from the base 18, and holds the imaging device 14 at a position above the mounting portion 18a.

[0029] The imaging device 14 may have an auxiliary light module (not shown). The auxiliary light module is a module that irradiates light onto the product P when photographing the product P. The auxiliary light module has a light-emitting element such as a PL (photoluminescence) or an LED (light-emitting diode). By providing the auxiliary light module, the captured image becomes clearer. As a result, for example, the unevenness of the surface of bread is photographed more precisely, improving the accuracy of image recognition of the product P, which will be described later.

[0030] The control unit 15 is, for example, a CPU (Central Processing Unit). The control unit 15 executes a computer program stored in the storage unit 16 to function as an image acquisition unit 151, a product identification unit 152, a product identification unit 153, a display control unit 154, an input reception unit 155, and a read information transmission unit 156.

[0031] The image acquisition unit 151 acquires a product image in which the product P is captured, generated by the imaging device 14. FIG. 4 is an example of a product image 301 generated by the imaging device 14. This product image 301 includes round bread, donuts, salad, and cup yogurt as products P1 to P4 that a customer can purchase. The round bread, donuts, and salad are products without barcodes. The cup yogurt may also be a product without a barcode. The cup yogurt is placed sideways so that the product reading device 1 can identify the product.

[0032] Referring again to FIG. 2, the product identification unit 152 analyzes the product image 301 acquired by the image acquisition unit 151, identifies each product P included in the product image 301, and specifies a product code associated with each product P. Specifically, the product identification unit 152 may identify each product P using a machine learning model. As the machine learning model, for example, a model including a neural network trained by deep learning can be used. This machine learning model has, for example, an input layer, an intermediate layer, and an output layer. Data of the product image 301 is input to the input layer, a calculation is performed based on the weighting coefficient in the neural network, and information indicating the identification result of the product P is output from the output layer.

[0033] The product identification unit 153 determines whether or not one or more products P included in the product image 301 correspond to special products for which a price different from the standard price is set. Specifically, the product identification unit 153 determines whether or not one or more products P correspond to a predetermined special product by referring to the storage unit 16, and identifies the predetermined special product corresponding to the one or more products P. Here, examples of "special products for which a price different from the standard price is set" include set products and campaign products.

[0034] Fig. 5 shows information on set products. As shown in Fig. 5, the storage unit 16 stores the first code information, which is the product code of each set product, the product name of the set product, the target products constituting the set product, and the price of the set product, in association with each other. A "set product" is a product that combines two or more products P, and is priced lower than the combined price of the individual products P. A set product has a unique product code that is different from the product codes associated with the individual products P.

[0035] A specific example of a set product is a "salad set," which is a set consisting of a round bread and a salad. The data in FIG. 5 includes price information for each set product, but the storage unit 16 does not need to have price information for the set products.

[0036] Fig. 6 shows information on campaign products. As shown in Fig. 6, the storage unit 16 stores the first code information, which is the product code of each campaign product, the name of the campaign, the product covered by the campaign, and the price of the campaign product, in association with each other. A "campaign product" is, for example, a product that is priced lower than the regular price of the product. The campaign product also has a unique product code that is different from the product code of each product P.

[0037] Specific examples of campaign items are a "round bread campaign" and a "donut campaign," in which round bread and donuts are priced lower than their normal prices. The data in FIG. 6 includes price information for items covered by each campaign, but the storage unit 16 does not need to have the price information.

[0038] The display control unit 154 causes various pieces of information to be displayed on the display unit 12. The display control unit 154 causes information on each of the products P identified by the product identification unit 152, and information on set products and campaign products identified by the product identification unit 153, to be displayed on the display unit 12 together with product images 301. Fig. 7 is an example of a result display screen 300 that displays the identification results of the products P by the product reading device 1, and the display control unit 154 causes the display unit 12 to display such a result display screen 300.

[0039] The result display screen 300 in FIG. 7 includes a product image 301 , a product information display section 302 , a special product information display section 303 , and an icon display section 304 .

[0040] In the product image 301, in the product display area where each product P is displayed, a graphic image 301a is displayed superimposed on the product P. The graphic image 301a may have any shape, but in the example of Fig. 7, a rectangular frame-shaped image is displayed surrounding the product P. The graphic image 301a is displayed in a different color for each type of product P, for example.

[0041] The product information display unit 302 displays the product name, which is information on each product P identified by the product identification unit 153. The product information display unit 302 displays an indicator 302a near the product name of each product P. The indicator 302a is displayed in an area different from the product display area of ​​each product P, and is, for example, a figure of the same color as the graphic image 301a displayed superimposed on the product P. For example, for round bread, the indicator 302a of the same color as the graphic image 301a of round bread is displayed in association with the product name "round bread". Note that the same color and the same color are not limited to being completely the same color, and may be similar colors.

[0042] The special product information display unit 303 displays information on the special product identified by the product identification unit 153. In the example of FIG. 7, a first icon 303a and a second icon 303b are displayed, and the first icon 303a displays "Salad Set", which is the product name of the special product, and the second icon 303b displays "Donuts CP", which indicates a donut campaign. The first icon 303a is an icon that allows the store clerk to select the salad set, which is a set product. The second icon 303b is an icon that allows the store clerk to select the donut campaign, which is a campaign product.

[0043] When one or more products P included in the product image 301 correspond to both a set product and a campaign product, the display control unit 154 displays both a first icon 303a for selecting the set product and a second icon 303b for selecting the campaign product. Although Fig. 7 shows an example in which products P1 and P3 correspond to a salad set, product P2 corresponds to a donut set, and a single product P does not correspond to multiple special products, the present invention is also applicable to a case in which a single product P corresponds to a predetermined set product and also corresponds to a predetermined campaign product.

[0044] The icon display section 304 includes an icon 304a displayed as "Confirm result" and an icon 304b displayed as "Re-recognize." The icon 304a is an icon that is pressed by the store clerk to input that the store clerk has confirmed the recognition result of the commodity P. The icon 304b is an icon that is pressed by the store clerk to cause the commodity recognition device 1 to recognize the commodity again.

[0045] Referring again to Fig. 2, the input receiving unit 155 receives inputs of the store clerk pressing various icons on the result display screen 300.

[0046] When a set product or a campaign product is selected by a store clerk by pressing the first icon 303a or the second icon 303b, the read information transmission unit 156 transmits first code information associated with the selected special product to the store terminal 2. When a set product or a campaign product is not selected and the store clerk presses the "Confirm result" icon 304a to input completion of product confirmation, the read information transmission unit 156 transmits second code information associated with each product P to the store terminal 2.

[0047] As illustrated in FIG. 5 and FIG. 6, the first code information is a unique character string associated with each set product or campaign product. The first code information may include price information of each set product or campaign product in addition to or instead of the unique character string. The first code information may also include information such as a character string indicating that the product is a set product or a campaign product (in other words, a character string indicating that the product is a special product). The second code information may also include at least one of the product name and price of each product. When the first code information or the second code information includes a character string indicating that the product is a special product or a character string indicating that the product is a regular product, an external device that receives the code information from the product reading device 1 can advantageously identify whether the target product is a regular product or a special product based on the presence or absence of such information.

[0048] The storage unit 16 is a storage medium for storing various data, and includes a ROM (Read Only Memory), a RAM (Random Access Memory), and a hard disk. The storage unit 16 stores an information processing program executed by the control unit 15. The storage unit 16 stores each product P sold at a store where the store payment system S100 is used, in association with a product code of each product P. As described above, the storage unit 16 stores whether or not one or more predetermined target products may become a special product, in association with the product P. Specifically, the storage unit 16 also stores data on set products and campaign products, as shown in FIG. 5 and FIG. 6.

[0049] Referring again to FIG. 2, the store terminal 2 is, for example, a POS terminal arranged in a store, and has a display unit 21, a control unit 22, and a storage unit 23. The display unit 21 is a display that displays the amount to be paid for the product purchased by the customer. The control unit 22 identifies the product corresponding to the code information of each product P and special product based on the code information of the product P and special product transmitted from the product reading device 1. Specifically, the control unit 22 refers to the product master stored in the storage unit 23 to identify the product P and special product. The storage unit 23 stores, as product master data, the code information of each product P and special product and product name and price data corresponding to each code information. The control unit 22 refers to the storage unit 23 and reads out the product name and price data corresponding to the code information of the product P and special product. The control unit 22 causes the display unit 21 to display the total price of the products.

[0050] The control unit 22 also has a function of executing a payment process, for example, by cash, credit card, or smartphone payment. For the payment process by cash, credit card, or smartphone payment, a known technique can be used, and a detailed description thereof will be omitted.

[0051] (Operation sequence) Fig. 8 is a flow chart for explaining the operation sequence of S101 in this embodiment. Fig. 9 is a diagram showing the state after the icon for salad set is pressed on the result display screen 300 in Fig. 7. Fig. 10 is a diagram showing the state after the icon for donuts CP is further pressed on the result display screen 300 in Fig. 9. In the following, an example will be given in which a store clerk operates the commodity reading device 1 and the store terminal 2 as an operator.

[0052] First, in step S1, a customer places the product P he or she wishes to purchase on a tray T and places it on the camera stand 17. Here, it is assumed that products P1, P2, P3, and P4 are placed on the tray T. The store clerk operates the product reading device 1 to cause the product reading device 1 to photograph the product P. As a result, a product image 301 is generated in which the four products P1, P2, P3, and P4 are captured, as shown in FIG.

[0053] Next, in step S2, the product identification unit 152 identifies the individual products P1, P2, P3, and P4. The product image 301 includes four products P1, P2, P3, and P4, and the product identification unit 152 identifies each product as "round bread," "donut," "salad," and "cup yogurt," respectively.

[0054] The product specification unit 153 uses the identification result of the product identification unit 152 and refers to the storage unit 16 to determine whether the products P1, P2, P3, and P4 included in the product image 301 correspond to a predetermined set product and campaign product. For example, the product specification unit 153 specifies that the products P1 and P3 are a "salad set" which is a set product. The product specification unit 153 also specifies that the product P2 is a target product of a donut campaign.

[0055] Next, in step S3, the display control unit 154 causes the display unit 12 to display information on each product P identified and specified as described above and information on the special products as a result display screen 300 as shown in Fig. 7. The result display screen 300 displays a product image 301, information on each product P shown in the product image 301, and information on the special products. The information on the special products is displayed as a first icon 303a of "salad set" and a second icon 303b of "donuts CP" indicating the donuts campaign.

[0056] Next, in step S4, the store clerk compares the actual product P on the tray T (see FIG. 1) with the content displayed on the result display screen 300 to confirm whether the product P has been correctly identified. If the product P has not been correctly identified, the store clerk presses icon 304b to input "re-recognize." In this case, the product reading device 1 executes the process from step S1 to step S3 again. If the store clerk does not press icon 304b in step S4, the process proceeds to step S5.

[0057] Next, in step S5, the store clerk compares the actual product P on the tray T with the contents of the result display screen 300, and after confirming that the product P1 "round bread" and the product P2 "salad" on the tray T correspond to the "salad set", the store clerk selects the first icon 303a of the "salad set". When the first icon 303a is selected, the display control unit 154 causes the display unit 12 to display the result display screen 300 as shown in Fig. 9. On this screen, the product information display unit 302 displays the text 302b of "salad set", which is information about the set product, and the first icon 303a is grayed out.

[0058] In the state of FIG. 9, the second icon 303b of the doughnut campaign remains displayed. After the store clerk compares the actual product P on the tray T with the contents of the result display screen 300 and confirms that the product P2 "Doughnut" is a target product of the doughnut campaign, the store clerk selects the second icon 303b of "Doughnut CP". When the second icon 303b is selected, the display control unit 154 causes the display unit 12 to display the result display screen 300 as shown in FIG. 10. On this screen, the product information display unit 302 displays the characters 302c of "Doughnut CP", which is information about the campaign product, and the second icon 303b is grayed out.

[0059] In this way, a set product or a campaign product is selected by the store clerk pressing the first icon 303a and the second icon 303b on the result display screen 300. On the other hand, the selection of the set product or the campaign product may be cancelled by, for example, pressing a deselection icon (not shown) or pressing the grayed-out first icon 303a or the second icon 303b again.

[0060] In step S5, if a special product is selected and the "result confirmation" icon 304a is pressed by the store clerk in step S6-1, the read information transmission unit 156 transmits the first code information associated with the special product to the store terminal 2 in step S7. For example, as shown in FIG. 9, if the product image 301 includes products P1, P2, P3, and P4, and a "salad set" is selected as the special product, the read information transmission unit 156 transmits the first code information associated with the salad set to the store terminal 2. The read information transmission unit 156 also transmits the second code information, which is the product code of each of the products P2 and P4 other than the special product, to the store terminal 2. Note that the product code information may be transmitted in only one step of selecting the special product, instead of the two steps of selecting the special product and inputting the result confirmation.

[0061] If a special product is not selected in step S5 and the store clerk presses the “Confirm Results” icon 304a in step S6-2, in step S7, only the second code information, which is the product code associated with each of the products P1, P2, P3, and P4, is sent to the store terminal 2.

[0062] Next, in step S8, the store terminal 2 receives the first code information or the second code information transmitted by the product reading device 1, refers to the product master stored in the memory unit 23, reads the price of the product associated with the first code information or the second code information, and displays the total of these prices on the store terminal 2 as the payment amount.

[0063] Next, in step S9, the store clerk operates the store terminal 2 and performs payment processing using any payment method, such as cash, credit card, or smartphone payment, based on instructions from the customer.

[0064] Through the above series of steps, the store payment system S100 of this embodiment and the product reading method using the same identify the product P, such as bread, salad, and other products without barcodes, by image analysis, and if the product P is a set product or a campaign product, information about the set product or the campaign product is displayed on the display unit 12, and after confirmation by a store clerk, the payment process can be executed at the store terminal 2.

[0065] (Action and effect) According to the commodity reading device 1 of the present embodiment configured as above, when it is determined that one or more commodities P included in the commodity image 301 correspond to a special commodity such as a set commodity or a campaign commodity, information of the specified special commodity is displayed on the display unit 12 together with the commodity image 301. Then, when the store clerk selects the special commodity, the first code information associated with the selected special commodity is transmitted to the store terminal 2. On the other hand, when the commodity reading device 1 receives an input of completion of confirmation without selecting a special commodity, the second code information associated with each commodity P is transmitted to the store terminal 2. In the configuration of the present embodiment, even if the commodity reading device 1 erroneously identifies a special commodity that is a set commodity or a campaign commodity, the code information of the special commodity is not transmitted to the store terminal 2 unless the store clerk checks the contents displayed on the display unit 12 and selects the special commodity. Therefore, it is possible to prevent the store terminal 2 from executing a settlement process based on erroneous information of a special commodity such as a set commodity or a campaign commodity.

[0066] In this embodiment, when the product identification unit 153 determines that one or more specified products correspond to both the set product and the campaign product, both a first icon 303a for selecting the set product and a second icon 303b for selecting the campaign product are displayed. example For example, even if "round bread" is a component product of the "salad set" and is also a target product of the "round bread campaign," the first icon for selecting the set product is And , the second icon for selecting campaign products (not shown in FIG. 7 etc.) This allows the store clerk to freely select either one. With this configuration, it becomes possible to flexibly respond to the operation of the store in which the commodity reading device 1 is used.

[0067] In this embodiment, as shown in Fig. 9, a mode has been described in which the first icon 303a of the salad set is selected and the second icon 303b of the donut campaign remains displayed. As described above, when a given product corresponds to both a set product and a campaign product, after one of the icons is selected by the store clerk, at least the other icon that was not selected among the first icon 303a and the second icon 303b may continue to be displayed on the display unit 12. This configuration has the advantage that it is easy for the store clerk to change the selection, for example, from the set product to the campaign product.

[0068] <Variation 1> Although the embodiment of the present invention has been described above with reference to the drawings, the present invention can be modified as appropriate without departing from the spirit of the present invention. Fig. 11 is an example of a result display screen showing the result of identifying a commodity. The commodity reading device 1 may photograph a commodity P and display the result of identifying the commodity P on the display unit 12 as a result display screen 300A as shown in Fig. 11.

[0069] As a result, on the display screen 300A, for product P1 (round bread) which is shown in a relatively large size in the product image 301, the words “round bread”, which are text information indicating the type or name of the product, are displayed superimposed on the image of product P1 within the graphic image 301a.

[0070] On the other hand, for product P5 (chocolate), which is shown in a relatively small size, a circled "1" is displayed as an alphanumeric character or symbol with a shorter number of characters than the text information "chocolate" indicating product P5, superimposed on the image of product P5 inside the graphic image 301a. Then, in the product information display section 302, the text information indicating the type or name of product P5, "chocolate", is displayed in association with the "1" in an area that does not overlap the product image.

[0071] 11, when the sizes of the products P1 and P5 identified by the product identification unit 152 in the product image 301 are equal to or larger than a predetermined reference size, the display control unit 154 displays text information indicating the type of product, etc., superimposed on the image of the product P1, as with the product P1. On the other hand, when the sizes of the products P1 and P5 in the product image 301 are smaller than the predetermined reference size, the display control unit 154 displays a display of a shorter number of characters than the text information indicating the type of product, etc., superimposed on the image of the product P5, as with the product P5.

[0072] When the size of the product P shown in the product image 301 is small, there is a risk that information indicating the product name, etc., may be difficult to see if it is displayed superimposed on the image of the product P, but with this configuration, if the size of the product P is less than a predetermined standard size, alphanumeric characters, etc. with a shorter number of characters are displayed superimposed on the image of the product P, making it easier to confirm the product in the product image 301. On the other hand, if the size of the product P is equal to or larger than the predetermined standard size, text information indicating the product is displayed superimposed on the product P, making it easier for the store clerk to confirm the identification result of the product P.

[0073] <Variation 2> Fig. 12 is another example of a result display screen showing the product identification result. Fig. 13 is another example of a result display screen showing the product identification result. The product reading device 1 may photograph a product and display the result of identifying the product on the display unit 12 in a format like the result display screen 300B in Figs. 12 and 13. The result display screen 300B is an example of a case where the product reading device 1 is used in, for example, a sushi restaurant where customers can take products home.

[0074] In this example, the imaging device 14 of the commodity reading device 1 captures six pieces of "Tekkamaki" as the commodity P6 to generate the commodity image 301, and the commodity identification unit 152 individually identifies each commodity P6. The commodity information display unit 302 displays information about the commodity P6, such as "Tekkamaki 6 pieces."

[0075] The product identification unit 153 determines whether the multiple products P6 correspond to a predetermined set product by referring to the storage unit 16. When the product identification unit 153 determines that the multiple products P6 correspond to, for example, a "tekkamaki set" which is a predetermined set product, the display control unit 154 displays an icon 303c indicating the "tekkamaki set" as shown in FIG.

[0076] When the store clerk selects the icon 303c, the display control unit 154 treats the six products P6 as one set product, as shown in Fig. 13, and displays one graphic image 301a together with the product P6 on the display unit 12. The display control unit 154 also displays information such as "Tekkamaki Set 1 item" on the product information display unit 302.

[0077] In this way, when the product image 301 contains multiple constituent products that make up a specified set product, the multiple products may be displayed in a grouped state on the condition that the store clerk selects the icon 303c.

[0078] <Variation 3> For example, if a salad set is composed of salad, bread, and a drink, and the products that the customer is trying to purchase are only salad and bread, the product reading device 1 may be configured to present information on the missing products to the operator. For example, the product identification unit 153 compares one or more products P shown in the product image 301 with the constituent products of the set product stored in the storage unit 16, and identifies products that are missing to satisfy the conditions of the set product. As a specific example, the product identification unit 153 identifies that a specific product, "drink," is missing to satisfy the conditions of the salad set. The display control unit 154 causes the display unit 12 to display, as an example, information on the missing products identified by the product identification unit 153.

[0079] According to this configuration, the store clerk can look at the display unit 12 and check the item that is missing from the set product, and can recommend the customer to purchase a drink. Also, if the customer can see the display unit 12, the store clerk can directly urge the customer to purchase the "drink." The product reading device 1 may output the information on the missing product as a voice.

[0080] Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. For example, all or part of the device can be configured by distributing or integrating functionally or physically in any unit. In addition, new embodiments resulting from any combination of multiple embodiments are also included in the embodiments of the present invention. The effect of the new embodiment resulting from the combination combines the effect of the original embodiment. [Explanation of symbols]

[0081] 1 Product reader 2. Store terminal 11 Communications Department 12 Display section 13 Input Devices 14 Imaging Device 15 Control section 16 Memory section 17 Camera Stand 18 Pedestal 18a Placement section 19 Stand 21 Display section 22 Control section 23 Memory section 151 Image acquisition unit 152 Product Identification Department 153 Product Specification Department 154 Display control section 155 Input reception section 156 Read information transmission unit 300 Results display screen 300A result display screen 300B Result display screen 301 Product Images 301a Graphic images 302 Product information display section 302a Indicator 302b character 302c characters 303 Special product information display section 303a First Icon 303b Second Icon 303c Icon 304 Icon display section 304a Icon 304b Icon S100 Store payment system T-Tray

Claims

1. an image acquisition unit that acquires product images showing a plurality of products purchased by a customer; A product identification unit that identifies each of the products included in the product image; a product identification unit that determines whether or not one or more of the products included in the product image correspond to a special product having a price set different from a standard price, and identifies the special product corresponding to the one or more of the products; A display control unit that displays information about the special product identified by the product identification unit as a selectable image together with the product image on a display unit; a read information transmitting unit which transmits to the outside first code information associated with the special product for products that are identified by the product identification unit as being the special product and that are selected by the operator to be treated as special products, and which transmits to the outside second code information associated with each of the products for products other than the special products and for products that are identified by the product identification unit as being the special product but that are not selected by the operator to be treated as special products when the operator inputs completion of confirmation of the product; A product reading device comprising:

2. A storage unit is further provided for storing whether the product may become the special product or not in association with the product, The product identification unit determines whether the one or more products correspond to the special product by referring to the storage unit. The product recognition device according to claim 1 .

3. The special product includes a component product included in a predetermined set product, the storage unit stores the first code information and a plurality of the component products included in the set product in association with the set product; the read information transmitting unit transmits the first code information stored in the storage unit in association with the set product when the product image includes images of the plurality of component products and the set product associated with the plurality of component products is selected by an operator to be treated as the special product. The product reading device according to claim 2 .

4. The special products include components included in a predetermined set product and a predetermined campaign product, When the product identification unit determines that the one or more products correspond to both the set product and the campaign product, the display control unit causes the display unit to display a first icon for the operator to select the set product and a second icon for the operator to select the campaign product. The product reading device according to any one of claims 1 to 3.

5. the display control unit continues to display at least the other of the first icon and the second icon on the display unit after one of the first icon and the second icon is selected by the operator. The product reading device according to claim 4.

6. When a size of the product identified by the product identification unit in the product image is equal to or larger than a predetermined reference size, the display control unit displays text information indicating a type or a name of the product superimposed on the image of the product, When the size of the product identified by the product identification unit in the product image is less than a predetermined reference size, alphanumeric characters or symbols having a shorter number of characters than the text information are displayed superimposed on the product image, and text information indicating the type or name of the product corresponding to the alphanumeric characters or symbols is displayed in an area not superimposed on the product image. The product reading device according to claim 1 .

7. the display control unit displays a graphic image superimposed on each of the plurality of products in a product display area in which the products are displayed, and displays a product name in an area different from the product display area in association with a graphic of the same color as the graphic image superimposed on the product. The product reading device according to any one of claims 1 to 6.

8. The product identification unit identifies a product that is missing to satisfy a condition of the set product by comparing the one or more products in the product image with a set product that is the special product, The display control unit causes the display unit to display the missing product identified by the product identification unit. The product reading device according to any one of claims 1 to 7.

9. The product recognition device according to any one of claims 1 to 8, a store terminal that is communicably connected to the commodity reading device and that executes a payment process based on the first code information or the second code information received from the commodity reading device; A store payment system equipped with the above.

10. acquiring a product image showing a plurality of products to be purchased by a customer; identifying each of the products included in the product image; determining whether or not one or more of the products included in the product image correspond to a special product with a price set different from a standard price, and identifying the special product corresponding to the one or more products; displaying information about the special product as a selectable image on a display unit together with the product image; For the products identified as the special products and selected by the operator to be treated as special products, first code information associated with the special products is transmitted to the outside, and for the products other than the special products and the products identified as the special products but not selected by the operator to be treated as special products, when the operator inputs completion of confirmation of the products, second code information associated with each of the products is transmitted to the outside; wherein each of the steps is executed by a computer.

Images