Product registration system and program

Abstract

To provide a product registration system and a program that can enhance reliability and enables practical operation when the range of products to be identified by image recognition is expanded.SOLUTION: A product registration system includes: an image capturing unit for obtaining a plurality of images by capturing an image of an article per one entry of the article into a predetermined opening; a feature information outputting unit for outputting feature information indicating appearance features of the article reflected in the image; a similarity output unit that compares the feature information with matching data associated with information on a product including at least a price to output a degree of similarity indicating the degree of similarity between them; a product identification unit that identifies information on the product matching the feature information based on the degree of similarity; and a product registration unit that registers information on the product identified by the product identification unit. When there are multiple matching data whose similarity exceeds a first threshold in multiple images for the same input, the product identification unit assumes that the one with the higher price among the information of the product associated with those matching data is the information on the product that matches the feature information.SELECTED DRAWING: Figure 1

Description

【Technical Field】 【0001】 Embodiments of the present invention relate to a product registration system and a program. 【Background Art】 【0002】 In the processing of sales data, a technique for identifying a product based on information indicating the external features (such as shape and color) of an article obtained by image recognition is used (for example, Patent Document 1). Currently, this technique is mainly used for identifying products that are not attached with code symbols such as fruits and vegetables. For other products with code symbols displayed on the package, product identification is conventionally performed based on the code information obtained by reading the code symbols. 【0003】 Here, in recent years, attempts have been made to expand the target of product identification by image recognition to products with code symbols, and product identification has been attempted from the external features of the package without relying on code symbol reading. However, there are problems in terms of reliability (accuracy) when performing product identification for registration as a transaction target by image recognition of the package, and improvements are being made towards practical application. 【Summary of the Invention】 【Problems to be Solved by the Invention】 【0004】 The problem to be solved by the present invention is to provide a product registration system and a program that can be used practically by enhancing the reliability when the range of products identified by image recognition is expanded. 【Means for Solving the Problems】 【0005】 The product registration system of the embodiment includes: an imaging unit that images an article passing through an opening which is an entrance to a predetermined area and obtains multiple images with each image taken for each entry into the area; a feature information output unit that receives the input of the images and outputs feature information which is information indicating the external features of the article captured in the images; a similarity output unit that compares the feature information with matching data which is a representative value of information indicating the external features of the product associated with product information including at least the price and outputs a similarity score indicating the degree of similarity between the two; a function unit that identifies the product information that matches the feature information based on the similarity score, wherein if there are multiple matching data in multiple images related to the same entry where the similarity score exceeds a first threshold, the product identification unit that identifies the product information with the higher price among the product information associated with those matching data is the product information that matches the feature information; and a product registration unit that registers the product information identified by the product identification unit. [Brief explanation of the drawing] 【0006】 [Figure 1] Figure 1 shows an example of the configuration of the product registration system according to the first embodiment. [Figure 2] Figure 2 is a block diagram showing an example of the configuration of a portable terminal. [Figure 3] Figure 3 is a block diagram showing an example of a server configuration. [Figure 4] Figure 4 shows an example of the items included in the product master. [Figure 5] Figure 5 is a block diagram showing an example of the functional configuration of a portable terminal and a server. [Figure 6] Figure 6 shows the flow of processing performed by the server's control unit. [Figure 7] Figure 7 is a block diagram showing an example of the configuration of a POS terminal according to the sixth embodiment. [Modes for carrying out the invention] 【0007】 (First Embodiment) The embodiments will be described with reference to the drawings. Figure 1 is a diagram showing an example of the configuration of the product registration system 100 of the embodiment. 【0008】 The product registration system 100 includes a server 1, a portable terminal 5 used by being attached to a cart 3, and an imaging unit 6. 【0009】 The product registration system 100 handles items such as bundled products 90 and individual products 91. Individual products 91 are an example of a first product. Bundle products 90 are an example of a second product, which is a set of multiple first products. In other words, bundled products 90 are multiple identical individual products 91 bundled together. An example of a product in which bundled products 90 and individual products 91 coexist in a store is canned beer. 【0010】 Individual product 91 and bundled product 90 are assigned different product codes. A product code is an example of information that can identify a product, such as a JAN code. Often, a barcode encoding the product code is displayed on the product's packaging. Here, a barcode is an example of a code symbol. The code symbol may also take other forms, such as a two-dimensional code. The packaging material that bundles individual product 91 into bundled product 90 displays a different code symbol than that of individual product 91. 【0011】 Cart 3 consists of a cargo basket 31 supported by a frame 34 with wheels 32 and a handle 33. The cargo basket 31 is a container having an opening 301 on its top surface. Here, the area inside the cargo basket 31 is an example of a predetermined area, and the opening 301 is an example of an entrance to or exit from the predetermined area. 【0012】 The imaging unit 6 is a camera that captures images of items passing through the opening 301. The imaging unit 6 has an image sensor such as a CCD (Charge Coupled Device) or CMOS (Complementary MOS). The imaging unit 6 transmits (outputs) an image (captured image) that captures the appearance of the items (bundle products 90 or individual products 91) to the portable terminal 5. The imaging unit 6 obtains multiple images with each capture when items are placed into the receiving basket 31. 【0013】 One or more imaging units 6 can be attached to each cart 3. In Figure 1, there is one imaging unit 6 attached to each cart 3, but multiple imaging units 6 may be provided on each cart 3 as needed. Also, if there is an imaging unit built into the portable terminal 5, that may be used as the imaging unit 6. 【0014】 One portable terminal 5 is attached to each cart 3. Additionally, one or more portable terminals 5 communicate with each server 1. 【0015】 In a product registration system 100 with this configuration, when products (bundle products 90 or individual products 91) are placed in the receiving basket 31, the products passing through the opening 301 are captured by the imaging unit 6, and the captured images are transmitted to the server 1. The server 1 identifies the products captured in the received image based on information indicating the external characteristics of the items (shape, color, pattern, etc.) obtained through image recognition, and automatically registers the identified products as purchase targets (transaction targets). 【0016】 In this case, if the product being entered is bundle product 90, it is possible that it may be recognized not only as bundle product 90 but also as a single product 91. Thus, when multiple products are candidates in the imaging process for a single entry, given the correspondence between bundle product 90 and single product 91, the one with the higher price is considered to be the correct product to register, i.e., the "correct" product. 【0017】 In this embodiment, even when a plurality of products are candidates in an imaging for one input, the "correct" product is automatically identified and registered. Hereinafter, the configuration and processing therefor will be described. 【0018】 FIG. 2 is a block diagram showing an example of the configuration of the portable terminal 5. The portable terminal 5 is, for example, a tablet-type terminal device, and includes a CPU (Central Processing Unit) 51, a ROM (Read Only Memory) 52, a RAM (Random Access Memory) 53, a communication unit 54, a display unit 55, an operation unit 56, an audio output unit 57, a storage unit 59, and the like. 【0019】 The CPU 51 is an example of a processor and comprehensively controls each part of the portable terminal 5. The ROM 52 stores various programs. The RAM 53 is a workspace for developing programs and various data. The CPU 51, the ROM 52, and the RAM 53 are connected via a bus or the like, and constitute a control unit 50 of a computer configuration. 【0020】 The communication unit 54 is a communication interface that communicably connects the control unit 50 and an external device (for example, the server 1) via a network. Further, the communication unit 54 receives the input of the captured image output by the imaging unit 6. 【0021】 The display unit 55 has a display device such as an LCD (Liquid Crystal Display), and displays various information under the control of the CPU 51. 【0022】 The operation unit 56 has an input device such as a touch panel provided on the surface of the display unit 55, and outputs the operation content input via the input device to the CPU 51. 【0023】 The audio output unit 57 has a speaker and a buzzer, and outputs audio under the control of the CPU 51. 【0024】 The memory unit 59 has a storage medium such as an SSD (Solid State Drive) or flash memory, and retains its contents even when the power is cut off. The memory unit 59 stores programs 591 that the CPU 51 can execute, as well as various setting information. The memory unit 59 (or RAM 53) also stores a registered product list 592, which is a list of registered products, from the start to the completion of a single transaction. 【0025】 Program 591 is application software that enables the portable terminal 5 to function as a mobile POS. A mobile POS is a portable POS terminal. POS is an abbreviation for "Point of Sale," meaning "point-of-sale information management." A POS terminal is a device that manages point-of-sale information. 【0026】 The control unit 50 performs various processes by having the CPU 51 operate according to the programs stored in the ROM 52 and memory unit 59 and loaded into the RAM 53. 【0027】 Furthermore, upon execution of program 591, the control unit 50 provides the following various functional units. These various functional units include, for example, the image transmission unit 501 (described later, see Figure 5), the product information acquisition unit 502, the display control unit 505, the operation reception unit 506, and the voice control unit 507. 【0028】 Figure 3 is a block diagram showing an example of the configuration of Server 1. Server 1 includes a CPU 11, ROM 12, RAM 13, communication unit 14, storage unit 19, etc. 【0029】 The CPU 11 is an example of a processor and comprehensively controls all parts of the server 1. The ROM 12 stores various programs. The RAM 13 is a workspace for displaying programs and various data. The CPU 11, ROM 12, and RAM 13 are connected via a bus or the like and constitute the control unit 10 of the computer configuration. 【0030】 The communication unit 14 is a communication interface that connects the control unit 10 and an external device (for example, a portable terminal 5) so that they can communicate via a network. 【0031】 The storage unit 19 has a storage medium such as an HDD (Hard Disk Drive) or flash memory, and retains its contents even when the power is cut off. The storage unit 19 stores programs 191 that the CPU 11 can execute, product master 192, product registration information 193, and various setting information used when executing programs 191. The product registration information 193 is information about products that customers have registered to purchase, in other words, information about the products being traded. 【0032】 The control unit 10 performs various processes by operating according to the program 191 stored in the ROM 12 and memory unit 19 and loaded into the RAM 13, which the CPU 11 has configured. In addition, the control unit 10 provides various functional units. These functional units include, for example, the feature information output unit 101 (described later, see Figure 5), the similarity output unit 102, the product identification unit 103, the correspondence confirmation unit 104, the candidate suggestion unit 105, and the product registration unit 106. 【0033】 Figure 4 shows an example of the items included in the product master 192. The product master 192 is an example of product information, where information about the products handled by a store is compiled, for example, in a table format. Examples of product information items include "product code," "product name," "unit price," "matching data," "color," and "corresponding product." 【0034】 A product code is an example of information that can identify a product, such as a JAN code. Other information (unit price, matching data, color, corresponding products, etc.) is stored in association with the product code. 【0035】 The "Product Name" field records the name of the product. The "Unit Price" field records the price for one product. The "Color" field records information indicating the color and pattern (design) of the product. 【0036】 The matching data represents representative values ​​of the feature information. Feature information is information that describes the external characteristics of an item. Feature information is a value output (obtained) by applying predetermined processing to an image (captured image) of the product's appearance, and is, for example, a quantity with magnitude and direction, such as a vector. The matching data represents standard values ​​of the feature information, which can vary depending on imaging conditions, etc. 【0037】 The "corresponding product" field records "information about the corresponding product." The "corresponding product" is the product that corresponds to the product indicated by the product code of the relevant record. In this embodiment, "information about the corresponding product" is an example of correspondence information, and is information that associates the other product when the product is either a single product 91 or a bundle product 90. 【0038】 Specifically, if the product code of the record in question indicates a single item 91, the "Corresponding Product" field will record information indicating a bundled product 90 (for example, the product code). Conversely, if the product code of the record in question indicates a bundled product 90, the "Corresponding Product" field will record information indicating a single item 91 (for example, the product code). 【0039】 Figure 5 is a block diagram showing an example of the functional configuration of the portable terminal 5 and the server 1. The control unit 50 of the portable terminal 5 includes functional units such as an image transmission unit 501, a product information acquisition unit 502, a display control unit 505, an operation reception unit 506, and a voice control unit 507. 【0040】 The image transmission unit 501 transmits (transfers, outputs) the captured image received from the imaging unit 6 to the server 1. When the product information acquisition unit 502 receives information about the registered product from the server 1, it adds the information to the registered product list 592 stored in the storage unit 59. 【0041】 The display control unit 505, the operation reception unit 506, and the voice control unit 507 operate according to signals received from the server 1. The signals received from the server 1 are signals transmitted by the server 1 under the control of the control unit 10, and indicate instructions from the server 1. The display control unit 505 controls the display content of the display unit 55 according to the instructions from the server 1. The voice control unit 507 controls the sound output by the voice output unit 57 according to the instructions from the server 1. The operation reception unit 506 receives operations via the operation unit 56 according to the instructions from the server 1, and transmits the operations received by the operation unit 56 to the server 1. 【0042】 The control unit 10 of server 1 includes functional units such as a feature information output unit 101, a similarity output unit 102, a product identification unit 103, a correspondence confirmation unit 104, a candidate suggestion unit 105, and a product registration unit 106. Note that the correspondence confirmation unit 104 and the candidate suggestion unit 105 are part of the product identification unit 103. 【0043】 When the feature information output unit 101 receives an input of an image captured from the portable terminal 5, it performs a first process on the image to acquire feature information of the object captured in the image and outputs it. 【0044】 The similarity output unit 102 compares the matching data with the feature information and performs a second process based on both to output the similarity score. The similarity score is a value that indicates the degree of similarity between the two (matching data and feature information), and is shown as a number from "0" to "100", with a larger value indicating a higher degree of similarity (greater similarity). The similarity output unit 102, for example, brute-forces the matching data stored in the product master 192 and outputs the similarity score between each matching data and the feature information output by the feature information output unit 101. The output destination for the similarity score is the product identification unit 103. 【0045】 The product identification unit 103 identifies product information that matches the feature information based on the similarity output by the similarity output unit 102. In the above example, for example, a similarity of "100" indicates a perfect match, and a similarity of "0" indicates a complete difference. In this embodiment, the product identification unit 103 identifies products whose similarity exceeds a first threshold (for example, "80") as items captured in the image. If there are no products whose similarity exceeds the first threshold, the product identification unit 103 is unable to identify which product the item captured in the image is. 【0046】 Furthermore, if there are multiple matching data (for example, two) whose similarity exceeds the first threshold among multiple captured images that are grouped together as being related to the same input operation, the product identification unit 103 performs processing by the correspondence confirmation unit 104. 【0047】 The correspondence confirmation unit 104 determines whether the information of a product associated with multiple (e.g., two) matching data sets whose similarity exceeds a first threshold (e.g., "80") includes each other's information as correspondence information. If it does, it determines that the product information with the higher price recorded in the product master 192 is the information of the product that matches the characteristic information. 【0048】 Furthermore, if the correspondence confirmation unit 104 finds that the information of a product associated with multiple (for example, two) matching data sets whose similarity exceeds the first threshold does not include each other's information as correspondence information, it outputs an alert requesting the imaging unit 6 to perform an action to re-image the product. The action requested by the alert is, for example, to temporarily remove the product from the receiving basket 31 and put it back in. This alert prompts the product to be put back in. 【0049】 Alerts are issued, for example, through a screen display by the display unit 55 of the portable terminal 5 or through audio output by the audio output unit 57. In other words, the response confirmation unit 104 outputs a signal as an alert output to instruct the display control unit 505 and the audio control unit 507 of the portable terminal 5. 【0050】 Furthermore, if the number of matching data points whose similarity exceeds the first threshold (e.g., "80") is equal to or greater than the second threshold (e.g., "3"), the product identification unit 103 outputs an alert requesting the imaging unit 6 to perform an action to re-image. The action requested by the alert is, for example, to temporarily remove the product from the receiving basket 31 and put it back in. This alert prompts the re-insertion of the product. 【0051】 Alerts are issued, for example, through a screen display on the display unit 55 of the portable terminal 5 or through audio output from the audio output unit 57. In other words, the product identification unit 103 outputs a signal as an alert output to instruct the display control unit 505 and the audio control unit 507 of the portable terminal 5. 【0052】 Furthermore, if there is no matching data with a similarity exceeding the first threshold, and the product identification unit 103 is unable to identify which product the item captured in the image is, and there is matching data with a similarity exceeding the third threshold (for example, "70"), the candidate suggestion unit 105 performs the necessary processing. 【0053】 In the process of presenting candidates, the candidate presentation unit 105 presents information on products associated with matching data that exceeds the third threshold as options. In this case, the candidate presentation unit 105 further accepts an operation to select an option as part of the operation acceptance process. 【0054】 The candidate presentation unit 105 performs processing related to candidate presentation and operation reception via the display unit 55, operation unit 56, and audio output unit 57. 【0055】 More specifically, the candidate presentation unit 105 presents candidates, for example, by displaying them on the screen using the display unit 55 of the portable terminal 5, or by outputting them as audio using the audio output unit 57. In other words, the candidate presentation unit 105 outputs signals to the display control unit 505 and the audio control unit 507 of the portable terminal 5 as output related to candidate presentation. 【0056】 Furthermore, the candidate presentation unit 105 accepts operation requests, for example, from the operation unit 56 of the portable terminal 5. In other words, the candidate presentation unit 105 receives a signal output by the operation reception unit 506 of the portable terminal 5 as input for operation requests. 【0057】 Furthermore, if the product identification unit 103 cannot identify which product an item captured in the image is because there is no matching data with a similarity exceeding the first threshold, and there is also no matching data exceeding the third threshold (for example, "70"), it outputs an alert requesting the imaging unit 6 to perform an action to retake the image. The action requested by the alert is, for example, to temporarily remove the product from the receiving basket 31 and put it back in. This alert prompts the product to be put back in. 【0058】 Alerts are issued, for example, through a screen display on the display unit 55 of the portable terminal 5 or through audio output from the audio output unit 57. In other words, the product identification unit 103 outputs a signal as an alert output to instruct the display control unit 505 and the audio control unit 507 of the portable terminal 5. 【0059】 The product registration unit 106 registers the product information identified by the product identification unit 103. "Registration" by the product registration unit 106 means adding (recording) the product information output by the product identification unit 103 to the product registration information 193. 【0060】 Figure 6 shows the processing flow executed by the control unit 10 of server 1. In this process, the control unit 10 waits for input of an captured image (step S1; No), and when it receives an captured image from the portable terminal 5 (step S1; Yes), it proceeds to the next step. 【0061】 In step S2, the control unit 10, acting as a feature information output unit 101, outputs feature information of the products captured in the image. Next, the control unit 10, acting as a similarity output unit 102, compares the feature information output by the feature information output unit 101 with various matching data and outputs the similarity to each matching data (step S3). 【0062】 Next, the control unit 10, acting as a product identification unit 103, if there is matching data whose similarity to the feature information exceeds a first threshold (e.g., "80") (step S4; Yes), lists the product codes associated with the matching data (step S5). This list (temporary list) is recorded, for example, using the free space in the storage unit 19 or RAM 13. 【0063】 If the number of product codes (i.e., the number of products) in the temporary list is 1 (step S6; 1 product), the control unit 10, as the product registration unit 106, stores the product code recorded in the temporary list and the information of the product that matches it in the registered product list 592 (step S9). 【0064】 In step S6, if the number of items (i.e., the number of item codes) in the temporary list is 2 (step S6; 2 items), the control unit 10, as a correspondence confirmation unit 104, determines whether the 2 items in the temporary list are corresponding items (step S7). That is, the correspondence confirmation unit 104 determines whether the correspondence information associated with the 2 item codes in the temporary list corresponds to each other's item codes. 【0065】 In step S7, if the two items in the temporary list are corresponding items (step S7; Yes), the control unit 10 determines that the item with the higher price recorded in the item master 192 is the item that matches the characteristic information (step S8). Then, in the following step S9, the control unit 10, acting as the item registration unit 106, stores the item code determined in step S8 and the information of the matching item in the registered item list 592. 【0066】 In step S7, if the two items included in the temporary list are not corresponding items (step S7; No), the control unit 10 prompts the system to re-enter the items (step S10). Also, in step S6, if the number of items (i.e., the number of item codes) included in the temporary list is three or more (step S6; three or more), the control unit 10 prompts the system to re-enter the items (step S10). After detecting the re-entry, the control unit 10 returns to step S1. 【0067】 Here, we will add some details about re-insertion and its detection. The action of re-insertion is, for example, to place an empty hand into the receiving basket 31, pick up the product, and put the product back in. In this case, the image captured by the imaging unit 6 will first show the hand without the product, then the hand holding the product will appear and then disappear, the hand holding the product will appear again, and then the hand without the product will appear. The control unit 10 can detect the re-insertion of the product from this series of captured images. Note that this action is just one example, and the control unit 10 may detect the re-insertion of the product by other methods. When the control unit 10 detects an action that matches this typical pattern of action related to re-insertion, it determines that re-insertion has occurred and clears the temporary list. 【0068】 In step S4, if there is no matching data with a similarity to the feature information that exceeds the first threshold (e.g., "80") (step S4; No), and the product of the item captured in the image cannot be identified, and there is matching data with a similarity to the third threshold (e.g., "70") that exceeds the third threshold (step S11; Yes), the control unit 10 performs processing as a candidate presentation unit 105. 【0069】 The control unit 10, acting as a candidate presentation unit 105, presents information on products associated with matching data exceeding the third threshold as options and accepts an operation to select an option (step S12). In step S12, the control unit 10 may accept an operation to select one product or an operation to select multiple products. Once an option is selected, the control unit 10 proceeds to step S9. 【0070】 In step S11, if there is no matching data that exceeds the third threshold (step S11; No), the control unit 10 prompts the product to be re-inserted (step S13). After detecting the re-insertion, the control unit 10 returns the process to step S1. 【0071】 As described above, in this embodiment, if multiple products are detected in a single image, and both are corresponding products, the one with the higher value is registered. By processing in this way, even if bundle product 90 is detected not only as bundle product 90 but also as individual product 91 when it is inserted, the correct one (i.e., bundle product 90) can be registered. 【0072】 Therefore, according to this embodiment, when the range of products identified by image recognition is expanded from fresh produce to packaged products, the reliability, which is difficult to maintain, can be raised to a practical level through operational improvements. 【0073】 The embodiments described above can also be modified and implemented as appropriate by changing some of the configurations or functions of each of the devices described above. Therefore, several modifications of the embodiments described above will be described below as other embodiments. In the following, we will mainly describe the differences from the embodiments described above, and the same reference numerals will be used for parts that are common with the content already described, and detailed explanations will be omitted. Furthermore, the modifications described below may be implemented individually or in combination as appropriate. 【0074】 (Second Embodiment) In the above embodiment, step S10 prompted the user to re-enter the product, but the implementation is not limited to this. In this embodiment, if the number of matching data with a similarity exceeding the first threshold (e.g., "80") is equal to or greater than the second threshold (e.g., "3"), the product identification unit 103 presents information on products associated with matching data exceeding the first threshold as options, instead of prompting re-entry in S10. 【0075】 According to this embodiment, the issue can be addressed by selecting a candidate instead of the manual process of re-inserting the product. 【0076】 (Third embodiment) In the above embodiment, the correspondence confirmation unit 104 performs processing in step S7, but the implementation is not limited to this. The product identification unit 103 in this embodiment does not perform the processing in step S7. In other words, the product identification unit 103 proceeds from Yes in step S6 to step S8. 【0077】 This embodiment allows for increased processing speed. This embodiment is particularly suitable for systems where, for example, a certain level of accuracy and reliability in image recognition can be achieved, leading to a stronger conviction that the item with the higher price among two items in a temporary list is the correct one. 【0078】 (Fourth Embodiment) In this embodiment, the product identification unit 103, when there are multiple matching data sets whose similarity exceeds a first threshold (for example, "80") for multiple captured images taken for the same input, presents information on the products associated with those matching data sets as options, in descending order of price. 【0079】 This embodiment is easier to understand if considered as a modification of the second embodiment described above. For example, when presenting candidates (options) when multiple products are temporarily recorded in a list with a single input, products that are considered to have a higher probability of being "correct" can be presented in a way that is more likely to catch the customer's eye. 【0080】 (Fifth embodiment) In this embodiment, the "information regarding corresponding products" recorded in the "Corresponding Products" field of the product master 192 is limited to information that can determine whether or not a corresponding product exists, such as a flag. In this case, it is not necessary to specify which product is the corresponding product. 【0081】 According to this embodiment, the complexity of maintaining the product master 192 can be reduced while achieving the same effects as in the third and fourth embodiments. 【0082】 (Sixth Embodiment) The embodiments will be described with reference to the drawings. Figure 7 is a block diagram showing an example of the configuration of the POS terminal 7 of the embodiment. The POS terminal 7 may be the most common type of POS terminal operated by a store employee, or it may be a fully self-service POS terminal operated by a customer. 【0083】 Figure 7 is a block diagram showing an example of the configuration of a POS terminal 7. The POS terminal 7 includes a CPU 71, ROM 72, RAM 73, communication unit 74, display unit 75, operation unit 76, audio output unit 77, storage unit 79, etc. 【0084】 The CPU 71 is an example of a processor and comprehensively controls all parts of the POS terminal 7. The ROM 72 stores various programs. The RAM 73 is a workspace for displaying programs and various data. The CPU 71, ROM 72, and RAM 73 are connected via a bus or the like and constitute the control unit 70 of the computer configuration. 【0085】 The communication unit 74 is a communication interface that connects the control unit 70 and an external device (e.g., server 1) to enable communication via a network. The communication unit 74 also receives input of captured images output by the imaging unit 6. 【0086】 The display unit 75 has a display device such as an LCD and displays various information under the control of the CPU 71. 【0087】 The operation unit 76 has a touch panel superimposed on the surface of the display unit 75, as well as input devices such as a keyboard and a pointing device, and outputs the operation content input via the input devices to the CPU 71. 【0088】 The audio output unit 77 has a speaker and a buzzer, and outputs sound under the control of the CPU 71. 【0089】 The memory unit 79 has a storage medium such as an HDD, SSD, or flash memory, and retains its contents even when the power is cut off. The memory unit 79 stores programs 791 that the CPU 71 can execute, as well as various setting information. The memory unit 79 (or RAM 73) also stores a registered product list, which is a list of registered products, from the start to the completion of a single transaction. 【0090】 Program 791 is application software for making the POS terminal 7 functional. 【0091】 The control unit 50 performs various processes by having the CPU 51 operate according to the programs stored in the ROM 52 and memory unit 59 and loaded into the RAM 53. 【0092】 Furthermore, upon execution of program 791, the control unit 70 provides the various functional units described in the first embodiment (image transmission unit 501, product information acquisition unit 502, display control unit 505, operation reception unit 506, voice control unit 507, etc. See Figure 5). 【0093】 In the conventional system, when an operator (store clerk or customer) operating the POS terminal 7 registers an item, they point (hold) the barcode at the barcode scanner. However, in this embodiment, the operator can register an item by passing it through the opening 301 after bringing it to the POS terminal 7. The opening 301 is the boundary between the area where the registered item is placed and the rest of the area. The opening 301 may be, for example, the edge of a basket into which the item is transferred, or the edge of a shopping bag (reusable bag) held by a predetermined hook. The imaging unit 6 captures an image of the item when the operator transfers it to the basket or shopping bag, and obtains the captured image. 【0094】 With this configuration, even if a POS terminal of a different type than that of the first embodiment is used, the same effects as those of the first embodiment can be obtained. 【0095】 (Seventh Embodiment) In the above embodiment, product identification is performed using characteristic information of the package's appearance. However, the implementation is not limited to this, and products may also be identified by reading code symbols such as barcodes or two-dimensional codes. 【0096】 In this embodiment, when reading code symbols by recognizing, extracting, and decoding them from the captured image output by the imaging unit 6, it is possible that multiple code symbols may be read with a single product input, similar to the above embodiment. In such cases, as in the above embodiment, automatic product registration can be enabled by processing the one with the higher price as "correct". 【0097】 (Variation 1) In each of the embodiments described above, Server 1 may consist of multiple devices rather than a single device. Also, in each embodiment, Server 1 may be a cloud server. 【0098】 (Modification 2) In each of the embodiments described above, the product master 192 and the product registration information 193 do not need to be stored in the same storage unit 19, and may be stored in storage units of different devices. For example, the storage unit 19 in the first embodiment is merely an example of a storage area that can store various types of information, and does not indicate that the product master 192 and the product registration information 193 must be stored in the same storage unit 19. 【0099】 The programs executed by each device in the above-described embodiments are provided pre-installed in ROM or the like. The programs executed by each device in the above-described embodiments may also be provided as files in an installable or executable format, recorded on a computer-readable recording medium such as a CD-ROM, flexible disk (FD), CD-R, or DVD (Digital Versatile Disk). 【0100】 Furthermore, the programs executed by each of the above-described embodiments may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Alternatively, the programs executed by each of the above-described embodiments may be provided or distributed via a network such as the Internet. 【0101】 While several embodiments of the present invention have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These novel embodiments can be carried out in a variety of other forms, and various omissions, substitutions, modifications, and combinations are possible without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims of the invention and its equivalents. [Explanation of Symbols] 【0102】 100... Product registration system, 1 ... Server, 10 ... Control unit, 101...Feature information output unit, 102...Similarity output unit, 103...Product identification unit, 104...Correspondence confirmation unit, 105...Candidate presentation unit, 106...Product registration unit, 11...CPU, 12...ROM, 13...RAM, 14...Communication section, 19...Memory unit, 191...Program, 192...Product master, 193...Product registration information, 3... Cart, 31...receiving basket, 301...opening, 32...wheels, 33...handlebars, 34...frame, 5. Portable devices, 50 ... Control unit, 501...Image transmission unit, 502...Product information acquisition unit, 505...Display control unit, 506...Operation reception unit, 507...Voice control unit, 51...CPU, 52...ROM, 53...RAM, 54...Communication section, 55...Display unit, 56...Operation unit, 57...Audio output unit, 59...Memory unit, 591...Program, 592...Registered product list, 6...Imaging Unit, 7…POS terminal, 70 ... Control unit, 71...CPU, 72...ROM, 73...RAM, 74...Communication section, 75...Display unit, 76...Operation unit, 77...Audio output unit, 79...Memory unit, 791...Program, 90... Bundle products, 91... Individual products. [Prior art documents] [Patent Documents] 【0103】 [Patent Document 1] Japanese Patent Publication No. 2013-089087

Claims

[Claim 1] An imaging unit that images an object passing through an opening which is an entrance or exit to a predetermined area, and obtains multiple images with each image taken for each insertion into the area, A feature information output unit receives the aforementioned image as input and outputs feature information which is information indicating the external features of the object captured in the aforementioned image. A similarity output unit that compares the characteristic information with representative data, which is associated with product information including at least the price and represents the characteristics of the appearance of the product, and outputs a similarity score indicating the degree of similarity between the two. A functional unit for identifying product information that matches the feature information based on the similarity, wherein if there are multiple matching data for multiple images related to the same input, the product identification unit determines that the product information with the higher price among the product information associated with those matching data is the product information that matches the feature information. A product registration unit that registers information about the product identified by the product identification unit, A product registration system equipped with the following features. [Claim 2] The information of the aforementioned product further includes correspondence information, which is information that associates the other product with either the first product or a second product which is a set of multiple first products. The product identification unit outputs an alert requesting the imaging unit to repeat the imaging process if, although there are multiple matching data sets for multiple images related to the same input that have a similarity exceeding the first threshold, the product information associated with those matching data sets does not contain each other's information as corresponding information. The product registration system according to claim 1. [Claim 3] The aforementioned product identification unit is, If there are multiple matching data sets whose similarity exceeds the first threshold for multiple images related to the same input, and the number of such matching data sets is less than a second threshold greater than 2, then the product information associated with those matching data sets, with the higher price, is determined to be the product information that matches the feature information. If the number of matching data whose similarity exceeds the first threshold is equal to or greater than the second threshold, an alert is output requesting the imaging unit to perform an action to re-image. The product registration system according to claim 1. [Claim 4] The aforementioned product identification unit is, If there are multiple matching data sets whose similarity exceeds the first threshold for multiple images related to the same input, and the number of such matching data sets is less than a second threshold greater than 2, then the product information associated with those matching data sets, with the higher price, is determined to be the product information that matches the feature information. If the number of matching data whose similarity exceeds the first threshold is equal to or greater than the second threshold, then the information of the product associated with the matching data whose similarity exceeds the first threshold is presented as an option. The product registration system according to claim 1. [Claim 5] The aforementioned product identification unit is, If there are multiple matching data sets whose similarity exceeds the first threshold for multiple images related to the same input, and the number of such matching data sets is less than a second threshold greater than 2, then the product information associated with those matching data sets, with the higher price, is determined to be the product information that matches the feature information. If the number of matching data whose similarity exceeds the first threshold is equal to or greater than the second threshold, the information of the products associated with those matching data is presented as options in descending order of price. The product registration system according to claim 1. [Claim 6] A computer for a product registration system, which includes an imaging unit that images an item passing through an opening that is an entrance or exit to a predetermined area and obtains multiple images with each image taken when an item is placed in the area, A feature information output unit receives the aforementioned image as input and outputs feature information which is information indicating the external features of the object captured in the aforementioned image. A similarity output unit that compares the characteristic information with representative data, which is associated with product information including at least the price and represents the characteristics of the appearance of the product, and outputs a similarity score indicating the degree of similarity between the two. A functional unit for identifying product information that matches the feature information based on the similarity, wherein if there are multiple matching data for multiple images related to the same input, the product identification unit determines that the product information with the higher price among the product information associated with those matching data is the product information that matches the feature information. A product registration unit that registers information about the product identified by the product identification unit, A program designed to function as such.

Images