Article registration device and program

Abstract

To provide an article registration device and a program that provide discounts for articles photographed at angles that do not reflect the surface on which a discount sticker is affixed.SOLUTION: An article registration device according to the embodiment includes an imaging unit, a memory unit, and a control unit. The imaging unit captures images of an article and generates image data. The memory unit stores information about the article and a flag for determining the application of a price discount to the article associated with the information about the article. The control unit recognizes the article from the image data generated by the imaging unit, checks the media related to the price discount of the article, and if the media related to the price discount of the article is confirmed, checks whether information related to the same article as the recognized article is stored in the memory unit. If the information about the same article as the recognized article is stored in the memory unit, the control unit activates the flag.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] An embodiment of the present invention relates to a product registration device and a program. [Background technology]

[0002] Conventionally, self-service POS systems, in which customers register products, are used as POS (Point of Sales) systems in retail stores and the like. One method of registering products in a self-service POS system is to register products by capturing images of products with a camera connected to a product registration device and recognizing the images. In addition, when applying a discount in this product registration method, there is a method that applies when a discount sticker is recognized in the image. However, in such a product registration device, in order to apply a discount, the discount sticker needs to be included in the image captured by the camera. Therefore, it has been difficult to apply a discount to a product that is captured at an angle that does not capture the surface with the discount sticker attached. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-187971 A Summary of the Invention [Problem to be solved by the invention]

[0004] The problem to be solved by the present invention is to provide a product registration device and a program that can give a discount to a product that is imaged at an angle that does not capture the surface to which a discount sticker is affixed. [Means for solving the problem]

[0005] In order to achieve the above object, the product registration device of this embodiment includes an imaging unit, a storage unit, and a control unit. The imaging unit captures an image of a product and generates image data. The storage unit stores information about the product and a flag for determining whether a discount on the product associated with the information about the product is applied. The control unit recognizes the product from the image data generated by the imaging unit, checks the medium related to the discount on the product, and if the medium related to the discount is confirmed, checks whether the storage unit stores information about the same product as the recognized product, and if information about the same product as the recognized product is stored, controls to enable the flag. [Brief description of the drawings]

[0006] [Figure 1] FIG. 1 is a block diagram showing an example of a configuration of a product registration device according to a first embodiment. [Diagram 2] FIG. 2 is a schematic diagram showing an example of a product master according to the first embodiment. [Diagram 3] FIG. 2 is a schematic diagram showing an example of a registration table according to the first embodiment. [Figure 4] FIG. 2 is a view showing an example of an input image according to the first embodiment. [Diagram 5] FIG. 2 is a view showing an example of a region recognition image according to the first embodiment. [Figure 6] A flowchart showing an example of a product registration process according to the first embodiment. [Figure 7] FIG. 11 is a block diagram showing an example of the configuration of a product registration device according to a second embodiment. [Figure 8] A flowchart showing an example of a product registration process according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] (First embodiment) Hereinafter, the present embodiment will be described with reference to the drawings. FIG. 1 is a block diagram showing an example of the configuration of a product registration device 10 according to the first embodiment. The product registration device 10 is composed of, for example, a control unit 100, a memory unit 101, a display unit 102, a camera 103, a scanner 104, a printer 105, a speaker 106, a card reader / writer 107, and a communication I / F 108, and each unit is connected via a bus.

[0008] The product registration device 10 is a terminal for registering products. The product registration device 10 is installed in a retail store such as a supermarket, and registers products by acquiring information about the products. Furthermore, multiple product registration devices 10 may be installed in a store, and these may be connected to a network such as a wired or wireless LAN (Local Area Network). Furthermore, the product registration device 10 may be connected to an accounting device for performing accounting, a computer for managing the store, or a store computer for managing the entire system.

[0009] The control unit 100 controls the entire product registration device 10. The control unit 100 includes a CPU (Central Processing Unit) 1001, a ROM (Read Only Memory) 1002, and a RAM (Random Access Memory) 1003. The CPU 1001 controls each unit of the product registration device 10. The ROM 1002 stores various programs, software, and data that cooperate with the CPU 1001 to execute various functions of the product registration device 10. The RAM 1003 is used as a work area for the CPU 1001, and loads various software and data stored in the ROM 1002 and the storage unit 101, data acquired from each unit, and the like. The control unit 100 operates according to the programs loaded in the RAM 1003 to recognize acquired images, generate images for product recognition, and execute processes related to discounts. In addition, the image recognition performed by the CPU 1001 may be performed by any method, such as deep learning using a CNN (Convolutional Neural Network) or graph cuts.

[0010] The storage unit 101 cooperates with the control unit 100 to function as a storage unit that stores various programs, software, and data that execute various functions of the product registration device 10. The storage unit 101 is configured with a storage medium such as a non-volatile memory, for example, a hard disk drive (HDD) or a flash memory. The storage unit 101 stores a product master 1011 and a registration table 1012 as shown in Figs. 3 and 4. The storage unit 101 may also store information to be used for image recognition, such as product feature amounts.

[0011] The product master 1011 is a table for storing information about products sold in a store and for managing products. The product master 1011 is composed of areas such as product code, product name, price, and discount.

[0012] The product code is an area that stores a product code, which is a unique identification code given to each type of product. The product code is, for example, a JAN (Japanese Article Number) code or a PLU (price look-up) code. The product name is an area that stores the name of the product associated with the product code. The price is an area that stores the price of the product associated with the product. In addition to the above areas, the product master 1011 may also have an area that stores a flag indicating whether the product is an age-restricted product, such as alcoholic beverages, whose purchase is restricted. The price discount is an area that stores the discount amount and discount rate applied when a price discount is implemented.

[0013] The registration table 1012 is a table that stores the products that a shopper purchases in one transaction. In other words, the registration table 1012 is a list of registered products. As shown in FIG. 3, the registration table 1012 is composed of areas such as a product code, a product name, a price, and a discount flag. The product code is, for example, a JAN (Japanese Article Number) code or a PLU (price look-up) code. The product name is an area that stores the name of the product associated with the product code. The price is an area that stores the price of the product associated with the product. The transaction price is an area that stores the price paid by the customer that was determined when the product was registered. The discount flag is an area that stores a flag for determining whether or not to apply the discount amount and discount rate stored in the discount area stored in the product master 1011. In this embodiment, when "1" is stored in the discount flag area, the discount amount and discount rate stored in the discount area stored in product master 1011 are applied, and when "0" is stored, the discount amount and discount rate stored in the discount area stored in product master 1011 are not applied. In other words, when the discount flag area is in the "1" state, the discount flag is enabled and a discount is applied, and when it is in the "0" state, the discount flag is disabled and no discount is applied.

[0014] In the registration table 1012 shown in Fig. 3, product names "aaa", "bbb", and "ccc" are registered. For example, for the product name "aaa" for which "0" is stored in one discount flag area, no discount is applied and the transaction price remains at 100 yen. On the other hand, for the product name "ccc" for which "1" is stored in the discount flag area, the discount area stored in the product master 1011 is referenced and a discount of 100 yen is applied, so the transaction price is stored as 200 yen. Although a discount is set for the product name "aaa" in the product master 1011, the discount flag is set to "0", which is invalid, indicating that the product registration device 10 captured an image without recognizing the discount sticker.

[0015] 1, the display unit 102 displays various information related to the product registration device 10. The display unit 102 has a liquid crystal display or an organic EL (Electro Luminescence) display device, and displays information for the user to operate. The display unit 102 may also be a touch panel.

[0016] Camera 103 captures an image of a product. Camera 103 captures an image of a preset range and generates image data. The generated image data is, for example, an input image S1 as shown in FIG. 4. Note that although only one product is captured in input image S1, multiple products may be captured as long as they are within a preset capture range.

[0017] The scanner 104 optically reads the one-dimensional or two-dimensional code to obtain information.

[0018] The printer 105 prints transaction-related information, coupons, advertisements, and other information.

[0019] The speaker 106 outputs audio notifications and warnings to the customer.

[0020] The card reader / writer 107 reads payment media such as credit cards and prepaid cards to obtain information and rewrite information. The card reader / writer 107 reads and rewrites information in the card by short-distance wireless communication or by reading a magnetic tape.

[0021] The communication I / F 108 transmits and receives information to and from other product registration devices 10, computers for store management, store computers, and the like.

[0022] Next, a description will be given of images for product recognition executed by the control unit 100. In this embodiment, when registering a product, the control unit 100 uses a product confirmation image S10, which is two images: an input image S1, which is an image for recognizing a discount sticker and a product without a discount sticker, and an image for recognizing a product with a discount sticker attached.

[0023] The input image S1 is an image for recognizing discount labels and products with no discount labels attached. The input image S1 is an image captured by the camera 103 of the product registration device 10 within a range that was set in advance when the product was registered. As a result of capturing the image, the camera 103 generates the input image S1 as image data, as shown in Fig. 4. The input image S1 includes an area S2 showing the product and an area S3 showing the discount label.

[0024] The control unit 100 recognizes the product and the discount sticker from the input image S1. The control unit 100 uses image recognition technology. For example, features are used for image recognition. The control unit 100 digitizes the color, pattern, shape, etc. of the product and the discount sticker by using deep learning etc. from multiple image data of the product and the discount sticker stored in the memory unit 101 etc., which are photographed under various conditions, and stores it as a feature for identifying the product. The control unit 100 extracts the outline of the object reflected in the input image S1 (i.e., the outline of the areas S2 and S3) and outputs the feature. After outputting the feature, the control unit 100 recognizes what kind of product or discount sticker the object reflected in the input image S1 is by comparing it with the stored feature. In addition, an object that is not a product, such as a customer's bag, in the input image S1 is not included in the feature stored in the memory unit 101 etc., and is therefore recognized as not being a product.

[0025] Furthermore, the control unit 100 recognizes the recognized product area and discount sticker area as being divided into area S2 and area S3, respectively.

[0026] After recognizing areas S2 and S3, the control unit 100 generates a product recognition image S10, which is image data including area S4, which is the area obtained by removing area S3 from area S2 shown in Fig. 5. In this embodiment, the image data including area S4 is referred to as product recognition image S10. When the product recognition image S10 is recognized using an image recognition technique that uses features, the features included in the removed area are excluded from the recognition. By excluding area S3, which corresponds to the discount sticker, it becomes possible to recognize the product without being influenced by the patterns and characters printed on the discount sticker.

[0027] Next, the process of product registration by the product registration device 10 will be described. 6 is a flowchart showing an example of the product registration process according to the first embodiment. In this embodiment, the purpose of the user having the product registration device 10 acquire the input image S1 is to register an unregistered product or to rescan a product to which a discount has not been applied, i.e., to recognize a discount sticker.

[0028] The control unit 100 starts product registration (ACT101). The control unit 100 starts product registration when it receives a command to start product registration from a virtual key or the like displayed on the display unit 102 that also has a touch panel. When the control unit 100 receives a command to start product registration (YES in ACT101), the control unit 100 executes processes required for product registration, such as switching the screen and making the camera 103 ready to capture images.

[0029] If the command to start product registration has not been received (NO in ACT101), the control unit 100 executes a loop process until the command to start product registration is received.

[0030] When a command to start product registration is received (YES in ACT101), the control unit 100 acquires an image of the product. The control unit 100 develops the input image S1 received from the camera 103 in the RAM 1003.

[0031] The control unit 100 checks whether a discount sticker is present (ACT103). The control unit 100 checks whether a discount sticker is present by comparing the feature amount stored in the storage unit 101 or the like with the feature amount extracted from the input image S1.

[0032] If the check result shows that there is no discount sticker (NO in ACT103), the control unit 100 recognizes the product (ACT104). The control unit 100 identifies the type of product without a discount sticker by comparing the feature amount of the product without a discount sticker (hereinafter, a product without a discount sticker) included in the input image S1 with the feature amount stored in the memory unit 101 or the like.

[0033] After recognizing the product without a discount label, the control unit 100 registers the product without a discount label (ACT 110). The control unit 100 accesses the product master 1011, extracts information about the recognized product without a discount label, and develops the extracted information in the registration table 1012.

[0034] The control unit 100 checks whether images of other products have been acquired (ACT111). If the check shows that other input images S1 of other products have been acquired (YES in ACT111), the control unit 100 returns the process to ACT103. If the check shows that other input images S1 of other products have not been acquired (NO in ACT111), the control unit 100 ends the process. Note that in this embodiment, the process ends if other input images S1 have not been acquired, but a loop process returning to ACT102 may be performed unless a subtotal button or the like is selected.

[0035] The process returns to ACT103, where the control unit 100 checks whether a discount sticker is present (ACT103). The control unit 100 checks whether an area S3 indicating a discount sticker is present by comparing the feature amount stored in the memory unit 101 or the like with the feature amount extracted from the input image S1. If the check shows that a discount sticker is present (YES in ACT103), the control unit 100 generates a product recognition image S10 (ACT105). The control unit 100 recognizes an area S4, which is the area obtained by excluding the area S3 indicating the discount sticker from the area S2 indicating the product included in the input image S10, and generates a product recognition image S10, which is image data including the area S4.

[0036] The control unit 100 recognizes the product with the discount sticker (ACT106). The control unit 100 performs image recognition on the area S4 included in the product recognition screen S10. The control unit 100 compares the feature amount of the area S4 included in the product recognition screen S10 with the feature amount stored in the memory unit 101 or the like to recognize what kind of product the product with the discount sticker is.

[0037] After recognizing the product with a discount sticker, the control unit 100 checks whether a product identical to the recognized product with a discount sticker has already been registered in this transaction (ACT107). The control unit 100 accesses the registration table 1012 and checks whether a product identical to the product with a discount sticker exists. After checking, if the same product is already stored in the registration table 1012 (YES in ACT107), the control unit 100 activates the discount flag for the product that is already registered (ACT108). The control unit 100 accesses the registration table 1012 and changes the discount flag area of ​​the product identical to the already registered product with a discount sticker to "1".

[0038] The control unit 100 performs product registration by inputting information about products with discount labels into the registration table 1012 (ACT 110). The control unit 100 extracts information about products with discount labels from the product master 1011 and expands it into the registration table 1012.

[0039] After registering the product with the discount sticker, the control unit 100 checks whether an image of the product has been acquired (ACT111).

[0040] In ACT107, the control unit 100 checks whether an identical product to the product with the discount sticker has been registered, and if the result of the check is that the identical product is not stored in the registration table 1012 (NO in ACT107), the control unit 100 activates the discount flag for the product with the discount sticker and registers the product (ACT108 and ACT109). After registering the product with the discount sticker, the control unit 100 checks whether an image of the product has been acquired (ACT110).

[0041] According to the above embodiment, the product registration device 10 can provide discounts even for products for which the discount stickers cannot be recognized.

[0042] Second Embodiment Next, a second embodiment will be described. In this embodiment, the same reference numerals as in the first embodiment denote the same elements.

[0043] In the second embodiment, when the control unit 100 checks whether an identical product to the recognized product has already been registered, it also checks whether the interval between the previous product registration time of the same product and the time when the product with a discount label was recognized was within a preset time limit. That is, the product registration device 10 in the second embodiment not only checks whether a product with a discount label has already been registered within one transaction, but also does not consider it to be the same product unless it can confirm that the time from the current product recognition to the previous product registration was within a preset time limit.

[0044] The configuration of the product registration device 10 in the second embodiment will be described. Fig. 7 shows an example of the configuration of the product registration device 10 according to the second embodiment. The product registration device 10 of the second embodiment has a clock 201 in addition to the configuration of the first embodiment. The clock 201 is connected to each unit via a bus. The clock 201 keeps track of time. The memory unit 101 also stores the number of seconds within the time limit.

[0045] Next, the product registration process in the second embodiment will be described. As a premise, when a product is registered in ACT110, the control unit 100 measures the registration time and expands it in the RAM 1003. When the control unit 100 acquires the input image S1, which is an image of the product, it also acquires the current time (ACT201). When the control unit 100 acquires the input image S1, it accesses the clock 201, acquires the current time, and expands it in the RAM 1003. The control unit 100 recognizes the discount sticker from the input image S1, which is an image of the product (YES in ACT103), generates a product recognition image S10 (ACT105), and checks whether the same product as the product with the discount sticker has already been registered in this transaction (ACT106). If the result of the check shows that the product with the discount sticker has already been registered (YES in ACT106), the control unit 100 checks whether the product with the discount sticker has been registered within a certain time (ACT202). The control unit 100 compares the time when the input image S1, which is expanded in the RAM 1003, was acquired with the registration time of an already registered product, and outputs the difference. After outputting the difference, the control unit 100 expands the time limit stored in the memory unit 101 in the RAM 1003. After expansion, the control unit 100 compares the difference with the time limit. If the difference is within the time limit (YES in ACT202), the control unit 100 activates the discount flag of the previously registered product (ACT108). If the difference exceeds the time limit (NO in ACT202), the control unit 100 turns on the discount flag (ACT109). The control unit 100 determines that the product recognized this time is not the same as the product recognized last time, and registers the product as a different product (ACT110).

[0046] Similarly, in the second embodiment, the product registration device 10 can provide discounts on products whose discount stickers could not be recognized. Also, by setting a time limit, the possibility of providing discounts on products with the same name but not the same product is reduced.

[0047] This embodiment is not limited to the above example, and various modifications are possible. For example, the process of enabling the discount flag in the product registration of the first and second embodiments may be performed manually. Specifically, the control unit 100 displays an image on the display unit 102 inquiring whether to apply the discount before the process of enabling the discount flag in ACT 108. Two virtual keys, one for allowing the application and one for not allowing the application, are displayed on this image.

[0048] Furthermore, as a modified example of the first and second embodiments, in the process of checking in ACT103 whether a discount sticker is attached to a product during product registration, even if a flag sticker indicating a discount is recognized in addition to a discount sticker, it may be determined that a discount sticker is attached.

[0049] In the above modification, the product registration device 10 can also apply discounts to products for which discount stickers could not be recognized. Also, applying discounts manually prevents discounts from being applied to the wrong products.

[0050] Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]

[0051] 10...Product registration device 100...Control unit 101...Storage section 1011...Product master 1012…Registration table 103...Camera

Claims

1. an imaging unit that captures an image of a commodity and generates image data; A storage unit that stores information about the product and a flag associated with the information about the product for determining whether or not a discount is applied to the product; a control unit that recognizes the product from the image data generated by the imaging unit, checks for a medium related to a discount on the product, and if the medium related to the discount is confirmed, checks whether information related to a product identical to the recognized product is stored in the storage unit, and if information related to a product identical to the recognized product is stored, controls to enable the flag; A product registration device comprising:

2. A product registration device as described in claim 1, wherein when the control unit is able to confirm one of two or more different discount-related media attached to the product, the control unit checks whether information regarding the same product as the recognized product is stored in the memory unit.

3. an imaging unit that captures an image of a commodity and generates image data; a storage unit that stores information about the product, a flag for determining whether a discount is applied to the product, which is associated with the information about the product, and a time when the product was registered, which is associated with the information about the product; a control unit that acquires image data and an image capture time from the imaging unit, recognizes a product from the image data and checks whether a medium related to a discount is present, and if the medium related to the discount is confirmed on the product, checks whether information related to a product identical to the recognized product is stored in the storage unit, and if information related to a product identical to the recognized product is stored, compares the image capture time with the product registration time and, if the time is within a preset time, enables the flag; A product registration device comprising:

4. A product registration device as described in claim 3, wherein when the control unit is able to confirm one of two or more different discount-related media attached to the product, the control unit checks whether information regarding the same product as the recognized product is stored in the memory unit.

5. a computer as a product registration device, comprising an imaging unit for imaging a product and generating image data, a control unit for executing product registration, and a storage unit for storing information about the registered product and a flag for determining whether a discount is applied, in association with the information about the product; image data acquisition means for acquiring image data from the imaging unit; A product recognition means for recognizing a product from the image data; a medium recognition means for recognizing a medium related to a discount from the image data; a product confirmation means for confirming, when the discount-related medium is recognized, whether information on a product identical to the recognized product is stored in the storage unit; A flag management that controls to enable the flag when information related to the same product as the product is stored; A program that can be used to

Images