Product registration system and program

The product registration system improves the reliability of image recognition by capturing multiple images, comparing feature information, and registering the highest-priced product, effectively addressing the challenge of identifying bundled products versus single products.

JP2024166775AActive Publication Date: 2024-11-29TOSHIBA TEC KK
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
JP2023083111
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-19
Publication Date
2024-11-29
Estimated Expiration
2043-05-19

Smart Images

  • Figure 2024166775000001_ABST
    Figure 2024166775000001_ABST
Patent Text Reader

Abstract

To provide a product registration system and a program that can enhance reliability and enables practical operation when the range of products to be identified by image recognition is expanded.SOLUTION: A product registration system includes: an image capturing unit for obtaining a plurality of images by capturing an image of an article per one entry of the article into a predetermined opening; a feature information outputting unit for outputting feature information indicating appearance features of the article reflected in the image; a similarity output unit that compares the feature information with matching data associated with information on a product including at least a price to output a degree of similarity indicating the degree of similarity between them; a product identification unit that identifies information on the product matching the feature information based on the degree of similarity; and a product registration unit that registers information on the product identified by the product identification unit. When there are multiple matching data whose similarity exceeds a first threshold in multiple images for the same input, the product identification unit assumes that the one with the higher price among the information of the product associated with those matching data is the information on the product that matches the feature information.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] An embodiment of the present invention relates to a product registration system and a program. [Background technology]

[0002] In sales data processing, a technology is used to identify products using information indicating the external characteristics (shape, color, etc.) of the product obtained by image recognition (for example, Patent Document 1). Currently, this technology is mainly used to identify products that do not have code symbols such as barcodes, such as fruits and vegetables. For other products that have code symbols on their packaging, products are identified using code information obtained by reading the code symbols, as in the past.

[0003] In recent years, attempts have been made to expand the scope of product identification using image recognition to include products with code symbols, and to identify products from the external characteristics of the package, rather than by reading the code symbol. However, there are problems with reliability (accuracy) when identifying products for registration as trade objects using image recognition of the package, and efforts are being made toward practical use. Summary of the Invention [Problem to be solved by the invention]

[0004] The problem to be solved by the present invention is to provide a product registration system and program that can be used practically and that can increase reliability when the range of products identified by image recognition is expanded. [Means for solving the problem]

[0005] The product registration system of an embodiment includes an imaging unit that images an item passing through an opening that is an entrance / exit to a specified area, and obtains multiple images by imaging each time the item is inserted into the area; a characteristic information output unit that receives the image and outputs characteristic information that is information indicating the external characteristics of the item reflected in the image; a similarity output unit that compares the characteristic information with matching data that is associated with product information including at least price and is a representative value of information indicating the external characteristics of the product, and outputs a similarity that indicates the degree of similarity between the two; a product identification unit that identifies product information that matches the characteristic information based on the similarity, where when there are multiple matching data whose similarity exceeds a first threshold value for multiple images related to the same insertion, the product identification unit identifies the product information associated with the matching data with the higher price as the product information that matches the characteristic information; and a product registration unit that registers the product information identified by the product identification unit. [Brief description of the drawings]

[0006] [Figure 1] FIG. 1 is a diagram illustrating an example of a configuration of a product registration system according to the first embodiment. [Diagram 2] FIG. 2 is a block diagram showing an example of the configuration of a portable terminal. [Diagram 3] FIG. 3 is a block diagram illustrating an example of a configuration of the server. [Figure 4] FIG. 4 is a diagram showing examples of items included in the product master. [Diagram 5] FIG. 5 is a block diagram illustrating an example of a functional configuration of the portable terminal and the server. [Figure 6] FIG. 6 is a diagram showing a flow of processing executed by the control unit of the server. [Figure 7] FIG. 7 is a block diagram illustrating an example of the configuration of a POS terminal according to the sixth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] (First embodiment) The embodiment will be described with reference to the drawings. Fig. 1 is a diagram showing an example of the configuration of a product registration system 100 according to the embodiment.

[0008] The product registration system 100 includes a server 1, a portable terminal 5 and an imaging unit 6 that are attached to a cart 3 when in use.

[0009] The product registration system 100 handles, for example, bundled products 90 and individual products 91. The individual products 91 are an example of a first product. The bundled product 90 is an example of a second product that is a set of multiple first products. In other words, the bundled product 90 is a collection of multiple identical individual products 91. An example of a product in which such bundled products 90 and individual products 91 coexist in a store is canned beer.

[0010] Different product codes are assigned to the individual products 91 and the bundled product 90. The product code is an example of information capable of identifying a product, such as a JAN code. A barcode with the product code encoded therein is often displayed on the exterior (package) of the product. Here, the barcode is an example of a code symbol. The code symbol may be in other forms, such as a two-dimensional code. A code symbol different from that of the individual products 91 is displayed on the packaging material of the bundled product 90 that bundles the individual products 91.

[0011] The cart 3 has a cargo basket 31 supported by a frame 34 with wheels 32 and a handle 33. The cargo basket 31 is a container having an opening 301 on the top surface. Here, the area inside the cargo basket 31 is an example of a predetermined area, and the opening 301 is an example of an entrance to the predetermined area.

[0012] The imaging unit 6 is a camera that captures an image of an item passing through the opening 301. The imaging unit 6 has an imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary MOS). The imaging unit 6 transmits (outputs) an image (captured image) that captures the appearance of the item (bundle product 90 or single product 91) to the portable terminal 5. The imaging unit 6 captures multiple images for each insertion into the item receiving basket 31.

[0013] One or more imaging units 6 are attached to one cart 3. In Fig. 1, one cart 3 is attached with one imaging unit 6, but one cart 3 may be provided with multiple imaging units 6 as necessary. Also, if the portable terminal 5 has an in-built imaging unit, it may be used as the imaging unit 6.

[0014] One portable terminal 5 is attached to one cart 3. Furthermore, one or more portable terminals 5 communicate with one server 1.

[0015] In the product registration system 100 configured as above, when a product (bundle product 90 or single product 91) is placed in the receiving basket 31, the product passing through the opening 301 is captured by the imaging unit 6, and the captured image is transmitted to the server 1. The server 1 identifies the product reflected in the received captured image based on information indicating the external characteristics of the item (shape, color, pattern, etc.) obtained by image recognition, and automatically registers the identified product as a purchase item (transaction item).

[0016] Here, if the inserted product is a bundle product 90, it is possible that it may be recognized not only as the bundle product 90 but also as a single product 91. In this way, when multiple products are selected as candidates in a single image taken for insertion, because there is a correspondence between the bundle product 90 and the single product 91, the product with the higher price is considered to be the correct product to be registered, that is, the "correct" product.

[0017] In this embodiment, even if multiple products are candidates for image capture in one insertion, the "correct" product is automatically identified and registered. The configuration and processing for this purpose will be described below.

[0018] 2 is a block diagram showing an example of the configuration of the portable terminal 5. The portable terminal 5 is, for example, a tablet-type terminal device, and includes a CPU (Central Processing Unit) 51, a ROM (Read Only Memory) 52, a RAM (Random Access Memory) 53, a communication unit 54, a display unit 55, an operation unit 56, an audio output unit 57, a storage unit 59, etc.

[0019] The CPU 51 is an example of a processor and controls each part of the portable terminal 5. The ROM 52 stores various programs. The RAM 53 is a workspace for developing programs and various data. The CPU 51, the ROM 52, and the RAM 53 are connected via a bus or the like to constitute a control unit 50 having a computer configuration.

[0020] The communication unit 54 is a communication interface that connects the control unit 50 and an external device (for example, the server 1) so that they can communicate with each other via a network. The communication unit 54 also receives an input of a captured image output by the imaging unit 6.

[0021] The display unit 55 has a display device such as an LCD (Liquid Crystal Display), and displays various types of information under the control of the CPU 51.

[0022] The operation unit 56 has an input device such as a touch panel that is provided over the surface of the display unit 55, and outputs the operation contents input via the input device to the CPU 51.

[0023] The audio output unit 57 has a speaker and a buzzer, and outputs audio under the control of the CPU 51.

[0024] The storage unit 59 has a storage medium such as an SSD (Solid State Drive) or a flash memory, and maintains the stored contents even when the power is cut off. The storage unit 59 stores a program 591 executable by the CPU 51 and various setting information. The storage unit 59 (or RAM 53) also stores a registered product list 592, which is a list of registered products (registered products) from the start to the completion of one transaction.

[0025] The program 591 is application software for making the portable terminal 5 function as a mobile POS. A mobile POS is a mobile POS terminal. POS is an abbreviation for "Point Of Sale" and means "point of sale information management." A POS terminal is a device that performs point of sale information management.

[0026] The control unit 50 executes various types of processing by the CPU 51 operating in accordance with programs stored in the ROM 52 or storage unit 59 and loaded into the RAM 53 .

[0027] Furthermore, by executing the program 591, the control unit 50 provides the following various functional units. The various functional units are, for example, an image transmission unit 501, a product information acquisition unit 502, a display control unit 505, an operation reception unit 506, an audio control unit 507, etc., which will be described later (see FIG. 5).

[0028] 3 is a block diagram showing an example of the configuration of the server 1. The server 1 includes a CPU 11, a ROM 12, a RAM 13, a communication unit 14, a storage unit 19, and the like.

[0029] The CPU 11 is an example of a processor and controls each part of the server 1. The ROM 12 stores various programs. The RAM 13 is a workspace for expanding the programs and various data. The CPU 11, the ROM 12, and the RAM 13 are connected via a bus or the like to constitute a control unit 10 of a computer configuration.

[0030] The communication unit 14 is a communication interface that connects the control unit 10 and an external device (eg, the portable terminal 5) so that they can communicate with each other via a network.

[0031] The storage unit 19 has a storage medium such as an HDD (Hard Disk Drive) or a flash memory, and maintains the stored contents even when the power is cut off. The storage unit 19 stores a program 191 executable by the CPU 11, a product master 192, product registration information 193, various setting information used when executing the program 191, and the like. The product registration information 193 is information on products registered by a customer to be purchased, that is, information on products that are the subject of a transaction.

[0032] The control unit 10 executes various processes by the CPU 11 operating in accordance with a program 191 stored in the ROM 12 or the storage unit 19 and expanded in the RAM 13. This also causes the control unit 10 to provide various functional units. The various functional units are, for example, a feature information output unit 101, a similarity output unit 102, a product identification unit 103, a correspondence confirmation unit 104, a candidate presentation unit 105, a product registration unit 106, and the like, which will be described later (see FIG. 5).

[0033] 4 is a diagram showing examples of items included in the product master 192. The product master 192 is an example of product information, and information on products handled by a store is compiled, for example, in the form of a table. Items of product information include, for example, "product code," "product name," "unit price," "verification data," "color," and "compatible products."

[0034] The product code is an example of information that can identify a product, such as a JAN code. Other information (unit price, verification data, color, corresponding products, etc.) is stored in association with the product code.

[0035] The "Product Name" field records the name of the product. The "Unit Price" field records the price for one product. The "Color" field records information indicating the color and design of the product.

[0036] The matching data is a representative value of the feature information. The feature information is information that indicates the external characteristics of an item. The feature information is a value that is output (obtained) by applying a predetermined process to an image (captured image) of the external appearance of a product, and is, for example, a quantity having a magnitude and direction, such as a vector. The matching data is a standard value of the feature information that varies depending on the imaging conditions, etc.

[0037] The "corresponding product" field records "information about the corresponding product." A "corresponding product" is a product that corresponds to the product indicated by the product code of the corresponding record. In this embodiment, the "information about the corresponding product" is an example of correspondence information, and is information that corresponds a product to either a single product 91 or a bundle product 90 when the product is the other.

[0038] Specifically, if the product code of the corresponding record indicates a single product 91, then what is recorded in the "Corresponding product" field is information (e.g., product code) indicating the bundle product 90. Also, if the product code of the corresponding record indicates a bundle product 90, then what is recorded in the "Corresponding product" field is information (e.g., product code) indicating the single product 91.

[0039] 5 is a block diagram showing an example of a functional configuration of the portable terminal 5 and the server 1. The control unit 50 of the portable terminal 5 includes functional units such as an image transmission unit 501, a product information acquisition unit 502, a display control unit 505, an operation reception unit 506, and an audio control unit 507.

[0040] The image transmission unit 501 transmits (transfers, outputs) the captured image received from the imaging unit 6 to the server 1. When the product information acquisition unit 502 receives information on the registered product from the server 1, it adds the information to a registered product list 592 stored in the storage unit 59.

[0041] The display control unit 505, the operation reception unit 506, and the audio control unit 507 operate according to signals received from the server 1. The signals received from the server 1 are signals transmitted by the server 1 under the control of the control unit 10, and indicate instructions from the server 1. The display control unit 505 controls the display content of the display unit 55 according to the instructions of the server 1. The audio control unit 507 controls the audio output by the audio output unit 57 according to the instructions of the server 1. The operation reception unit 506 receives operations via the operation unit 56 according to the instructions of the server 1, and transmits the operations received by the operation unit 56 to the server 1.

[0042] The control unit 10 of the server 1 includes functional units such as a feature information output unit 101, a similarity output unit 102, a product identification unit 103, a correspondence checking unit 104, a candidate presentation unit 105, and a product registration unit 106. The correspondence checking unit 104 and the candidate presentation unit 105 are parts of the product identification unit 103.

[0043] When the characteristic information output unit 101 receives an input of a captured image received from the mobile terminal 5, it performs a first process on the captured image to obtain characteristic information of an article captured in the captured image and outputs this.

[0044] The similarity output unit 102 compares the matching data with the feature information and outputs the similarity by performing a second process based on both. The similarity is a value indicating the degree of similarity between the two (matching data and feature information) and is indicated by a numerical value from "0" to "100", for example, with the value increasing when the degree of similarity is high (more similar). The similarity output unit 102 performs a brute force search of the matching data stored in the product master 192, for example, and outputs the similarity between each matching data and the feature information output by the feature information output unit 101. The similarity is output to the product identification unit 103.

[0045] The product identification unit 103 identifies information about a product that matches the feature information based on the similarity output by the similarity output unit 102. In the above example, for example, a similarity of "100" indicates a perfect match, and a similarity of "0" indicates a perfect difference. In this embodiment, the product identification unit 103 identifies a product whose similarity exceeds a first threshold (for example, "80") as an item reflected in the captured image. If there is no product whose similarity exceeds the first threshold, the product identification unit 103 is deemed to have been unable to identify which product is the item reflected in the captured image.

[0046] In addition, when there are multiple (e.g., two) pieces of matching data whose similarity exceeds a first threshold value among multiple captured images grouped as relating to the same insertion operation, the product identification unit 103 performs processing using the correspondence confirmation unit 104.

[0047] The correspondence confirmation unit 104 determines whether information on a product associated with multiple (e.g., two) matching data whose similarity exceeds a first threshold value (e.g., "80") contains each other's information as corresponding information, and if so, determines that the product information with the higher price recorded in the product master 192 is the product information that matches the characteristic information.

[0048] Furthermore, when information on a product associated with a plurality (e.g., two) pieces of matching data whose similarity exceeds the first threshold value does not include each other's information as corresponding information, the correspondence confirmation unit 104 outputs an alert requesting execution of an operation to redo imaging by the imaging unit 6. The operation requested by the alert is, for example, an operation to temporarily remove the product from the baggage basket 31 and then reinsert it. This alert prompts the user to reinsert the product.

[0049] The alert is issued, for example, by a screen display by the display unit 55 of the portable terminal 5 or a voice output by the voice output unit 57. That is, the correspondence confirmation unit 104 outputs a signal for instructing the display control unit 505 or the voice control unit 507 of the portable terminal 5 as an alert output.

[0050] Furthermore, when the number of pieces of matching data whose similarity exceeds a first threshold value (e.g., "80") is equal to or greater than a second threshold value (e.g., "3"), the product identification unit 103 outputs an alert requesting the user to perform an action to redo the imaging by the imaging unit 6. The action requested by the alert is, for example, an action to temporarily remove the product from the receiving basket 31 and then reinsert it. This alert prompts the user to reinsert the product.

[0051] The alert is issued, for example, by a screen display by the display unit 55 of the portable terminal 5 or a voice output by the voice output unit 57. That is, the product identification unit 103 outputs a signal for instructing the display control unit 505 or the voice control unit 507 of the portable terminal 5 as an alert output.

[0052] In addition, when there is no matching data whose similarity exceeds the first threshold value and the product identification unit 103 is unable to identify the product that appears in the captured image, if there is matching data whose similarity exceeds a third threshold value (e.g., "70"), the product identification unit 103 performs processing using the candidate presentation unit 105.

[0053] In the process of presenting the candidates, the candidate presentation unit 105 presents, as options, information on products associated with matching data exceeding the third threshold. In this case, the candidate presentation unit 105 further accepts an operation to select an option as a process of accepting an operation.

[0054] The candidate presentation unit 105 performs processes related to candidate presentation and operation reception via the display unit 55, the operation unit 56, and the audio output unit 57.

[0055] More specifically, the candidate presentation unit 105 presents the candidates, for example, by displaying the candidates on the display unit 55 of the portable terminal 5 or by outputting the candidates through audio by the audio output unit 57. That is, the candidate presentation unit 105 outputs signals for instructing the display control unit 505 and the audio control unit 507 of the portable terminal 5 as outputs related to the candidate presentation.

[0056] Furthermore, the candidate presentation unit 105 receives an operation from, for example, the operation unit 56 of the portable terminal 5. That is, the candidate presentation unit 105 receives a signal output by the operation reception unit 506 of the portable terminal 5 as an input related to the operation reception.

[0057] Furthermore, when there is no matching data with a similarity exceeding the first threshold value and the product identification unit 103 is unable to identify the product that appears in the captured image, and there is also no matching data exceeding a third threshold value (e.g., "70"), the product identification unit 103 outputs an alert requesting the user to perform an action to redo the image capture by the imaging unit 6. The action requested by the alert is, for example, the action of temporarily removing the product from the baggage basket 31 and then reinserting it. This alert prompts the user to reinsert the product.

[0058] The alert is issued, for example, by a screen display by the display unit 55 of the portable terminal 5 or a voice output by the voice output unit 57. That is, the product identification unit 103 outputs a signal for instructing the display control unit 505 or the voice control unit 507 of the portable terminal 5 as an alert output.

[0059] The product registration unit 106 registers information about the product identified by the product identification unit 103. The “registration” by the product registration unit 106 means adding (recording) the product information output by the product identification unit 103 to the product registration information 193.

[0060] 6 is a diagram showing the flow of processing executed by the control unit 10 of the server 1. In this processing, the control unit 10 waits for input of a captured image (step S1; No), and when receiving a captured image from the portable terminal 5 (step S1; Yes), proceeds to the next processing.

[0061] In step S2, the control unit 10 outputs feature information of the product captured in the captured image as the feature information output unit 101. Next, the control unit 10, as the similarity output unit 102, compares the feature information output by the feature information output unit 101 with various pieces of matching data and outputs the similarity for each piece of matching data (step S3).

[0062] Next, when there is matching data whose similarity with the feature information exceeds a first threshold value (e.g., "80") (step S4; Yes), the control unit 10, functioning as the product identification unit 103, makes a list of product codes associated with the matching data (step S5). This list (temporary list) is recorded, for example, using free space in the memory unit 19 or RAM 13.

[0063] If the number of product codes contained in the temporary list (i.e., the number of products) is 1 (step S6; 1 product), the control unit 10, as the product registration unit 106, stores the product code recorded in the temporary list and information on the products that match it in the registered product list 592 (step S9).

[0064] In step S6, if the number of products (i.e., the number of product codes) included in the temporary list is two (step S6; two), the control unit 10, functioning as the correspondence checking unit 104, determines whether the two products included in the temporary list correspond to each other (step S7). That is, the correspondence checking unit 104 determines whether the correspondence information associated with the two product codes included in the temporary list are each other's product codes.

[0065] In step S7, if the two products included in the temporary list correspond to each other (step S7; Yes), the control unit 10 determines that the product with the higher price recorded in the product master 192 is the product that matches the characteristic information (step S8). Then, in the following step S9, the control unit 10, as the product registration unit 106, stores the product code determined in step S8 and the information of the product that matches the product code in the registered product list 592.

[0066] In step S7, if the two products included in the temporary list are not corresponding to each other (step S7; No), the control unit 10 prompts the user to re-input the products (step S10). In addition, in step S6, if the number of products (i.e., the number of product codes) included in the temporary list is three or more (step S6; three or more), the control unit 10 prompts the user to re-input the products (step S10). After detecting the re-input, the control unit 10 returns the process to step S1.

[0067] Here, we will provide further details on re-insertion and its detection. The re-insertion action is, for example, placing an empty hand into the baggage receiving basket 31, grabbing the product, and then replacing the product. In this case, the hand without the product is first captured in the image captured by the imaging unit 6, then the hand with the product is captured and then leaves, the hand with the product is captured again, and then the hand without the product is captured. This series of captured images allows the control unit 10 to detect the re-insertion of the product. Note that this action is only an example, and the control unit 10 may detect the re-insertion of the product by other methods. When an action that matches the pattern of such a typical action related to re-insertion is detected, the control unit 10 determines that re-insertion has been performed, and clears the temporary list.

[0068] In step S4, if there is no matching data whose similarity with the feature information exceeds a first threshold value (e.g., "80") (step S4; No), and the item captured in the captured image cannot be identified, but if there is matching data whose similarity exceeds a third threshold value (e.g., "70") (step S11; Yes), the control unit 10 performs processing as the candidate presentation unit 105.

[0069] The control unit 10 as the candidate presentation unit 105 presents, as options, information on products associated with matching data exceeding the third threshold, and accepts an operation to select an option (step S12). In step S12, the control unit 10 may accept an operation to select any one of the products, or may accept an operation to select multiple products. When an option is selected, the control unit 10 advances the process to step S9.

[0070] In step S11, if there is no matching data exceeding the third threshold (step S11; No), the control unit 10 prompts the user to re-insert the product (step S13). After detecting the re-insertion, the control unit 10 returns the process to step S1.

[0071] As described above, in this embodiment, when multiple products are detected in one image capture, if the two products are associated, the one with the higher price is registered. By processing in this manner, even if a bundled product 90 is inserted and detected as not only a bundled product 90 but also a single product 91, the correct one (i.e., bundled product 90) can be registered.

[0072] Therefore, according to this embodiment, when the range of products identified by image recognition is expanded from fruits and vegetables to packaged products, reliability, which is difficult to maintain, can be raised to a practical level by ingenuity in operation.

[0073] The above-described embodiment can be modified as appropriate by changing a part of the configuration or function of each of the above-described devices. Therefore, some modified examples of the above-described embodiment will be described below as other embodiments. Note that the following mainly describes the differences from the above-described embodiment, and the same reference numerals are used for the common points with the contents already described, and detailed description is omitted. The modified examples described below may be implemented individually or in appropriate combination.

[0074] Second embodiment In the above embodiment, in step S10, re-input of the product is prompted, but the implementation is not limited to this. In the present embodiment, when the number of matching data whose similarity exceeds a first threshold value (e.g., "80") is equal to or greater than a second threshold value (e.g., "3"), the product specification unit 103 presents, as an option, information on products associated with matching data that exceeds the first threshold value, instead of re-inputting in S10.

[0075] According to this embodiment, instead of the trouble of re-inputting, it is possible to deal with the problem by selecting a candidate.

[0076] Third embodiment In the above embodiment, the correspondence confirmation unit 104 performs the process in step S7, but the present invention is not limited to this. The product identification unit 103 in this embodiment does not perform the process in step S7. That is, the product identification unit 103 advances the process from Yes in step S6 to step S8.

[0077] According to this embodiment, the processing speed can be increased. This embodiment is suitable for use in a system in which, for example, a certain degree of accuracy and reliability of image recognition can be obtained, so that it is more certain that the higher priced item of two items in a temporary list is the "correct" item.

[0078] (Fourth embodiment) In this embodiment, when there are multiple pieces of matching data whose similarity exceeds a first threshold value (e.g., "80") between multiple captured images related to the same input, the product identification unit 103 presents information on products associated with those matching data as options in order of highest price.

[0079] This embodiment can be easily understood as a modified example of the second embodiment described above. For example, when multiple products are recorded in a temporary list by one input, candidates (options) can be presented in order of the product that is considered to have a higher probability of being "correct" so that it is more noticeable to the customer.

[0080] Fifth embodiment In this embodiment, the "information about the corresponding product" recorded in the "corresponding product" field in the product master 192 is information such as a flag that is sufficient to determine whether or not a corresponding product exists. In this case, it does not matter which product is the corresponding product.

[0081] According to this embodiment, the same effects as those of the third and fourth embodiments can be obtained while reducing the complexity of maintaining the product master 192.

[0082] Sixth embodiment An embodiment will be described with reference to the drawings. Fig. 7 is a block diagram showing an example of the configuration of a POS terminal 7 according to the embodiment. The POS terminal 7 may be a POS terminal operated by a store clerk, which is the most common type, or a fully self-service POS terminal operated by a customer.

[0083] 7 is a block diagram showing an example of the configuration of the POS terminal 7. The POS terminal 7 includes a CPU 71, a ROM 72, a RAM 73, a communication unit 74, a display unit 75, an operation unit 76, a voice output unit 77, a storage unit 79, and the like.

[0084] The CPU 71 is an example of a processor and controls each part of the POS terminal 7. The ROM 72 stores various programs. The RAM 73 is a workspace for developing programs and various data. The CPU 71, the ROM 72, and the RAM 73 are connected via a bus or the like to constitute a control unit 70 having a computer configuration.

[0085] The communication unit 74 is a communication interface that connects the control unit 70 and an external device (for example, the server 1) so that they can communicate with each other via a network. The communication unit 74 also receives an input of a captured image output by the imaging unit 6.

[0086] The display unit 75 has a display device such as an LCD, and displays various information under the control of the CPU 71.

[0087] The operation unit 76 has input devices such as a touch panel provided over the surface of the display unit 75, a keyboard, and a pointing device, and outputs operation contents input via the input devices to the CPU 71.

[0088] The audio output unit 77 has a speaker and a buzzer, and outputs audio under the control of the CPU 71.

[0089] The storage unit 79 has a storage medium such as a HDD, SSD, or flash memory, and maintains the stored contents even when the power is cut off. The storage unit 79 stores a program 791 executable by the CPU 71 and various setting information. The storage unit 79 (or RAM 73) also stores a registered product list, which is a list of registered products (registered products) from the start to the completion of one transaction.

[0090] The program 791 is application software for causing the POS terminal 7 to function.

[0091] The control unit 50 executes various types of processing by the CPU 51 operating in accordance with programs stored in the ROM 52 or storage unit 59 and loaded into the RAM 53 .

[0092] In addition, by executing the program 791, the control unit 70 provides various functional units (image transmission unit 501, product information acquisition unit 502, display control unit 505, operation reception unit 506, audio control unit 507, etc.; see FIG. 5) described in the first embodiment.

[0093] Conventionally, when registering a product, an operator (a store clerk or a customer) who operates the POS terminal 7 points (holds) a barcode over a barcode scanner, but in this embodiment, the operator can register the product by passing the product that he or she has brought to the POS terminal 7 through the opening 301. The opening 301 is the boundary between the area where the registered product is placed and the rest of the area. The opening 301 may be, for example, the edge of a basket into which the product is transferred, or the edge of a shopping bag (my bag) held by a predetermined hook. The imaging unit 6 captures an image of the product when the operator transfers the product to the basket or shopping bag, and obtains the captured image.

[0094] According to this configuration, even if a different type of POS terminal is used, the same effects as those of the first embodiment can be obtained.

[0095] Seventh embodiment In the above embodiment, the product is identified by characteristic information on the exterior of the package, but the present invention is not limited to this. The product may be identified by reading a code symbol such as a barcode or a two-dimensional code.

[0096] According to this embodiment, when the code symbol is read by recognizing, extracting and decoding the code symbol from the captured image output by the imaging unit 6, it is possible that multiple code symbols may be read when a product is inserted at one time, as in the above embodiment. Even in such a case, it is possible to automatically register the product by processing the higher priced item as the "correct" item, as in the above embodiment.

[0097] (Variation 1) In each of the above-described embodiments, the server 1 may be configured by a plurality of devices rather than a single device. Also, in each of the embodiments, the server 1 may be a cloud server.

[0098] (Variation 2) In each of the above-described embodiments, the product master 192 and the product registration information 193 do not need to be stored in the same storage unit 19, and may be stored in storage units of different devices. For example, the storage unit 19 in the first embodiment is an example that simply provides a storage area capable of storing various information, and does not indicate that the product master 192 and the product registration information 193 are required to be stored in the same storage unit 19.

[0099] The programs executed by each device of the above-described embodiments are provided by being pre-installed in a ROM, etc. The programs executed by each device of the above-described embodiments may be provided by being recorded in an installable or executable format on a computer-readable recording medium such as a CD-ROM, a flexible disk (FD), a CD-R, or a digital versatile disk (DVD).

[0100] Furthermore, the programs executed by each device of the above-described embodiments may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Also, the programs executed by each device of the above-described embodiments may be provided or distributed via a network such as the Internet.

[0101] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, modifications, and combinations can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]

[0102] 100...Product registration system, 1 ... server, 10 ... control unit, 101: characteristic information output unit; 102: similarity output unit; 103: product identification unit; 104: correspondence confirmation unit, 105: candidate presentation unit, 106: product registration unit, 11...CPU, 12...ROM, 13...RAM, 14...Communication section, 19 ... storage unit, 191 ... program, 192 ... product master, 193 ... product registration information, 3 … Cart, 31 ... receiving basket, 301 ... opening, 32 ...wheel, 33 ...handle, 34 ...frame, 5 …Mobile terminals, 50 ... control unit, 501: image transmission unit; 502: product information acquisition unit; 505: display control unit; 506: operation reception unit; 507: audio control unit; 51...CPU, 52...ROM, 53...RAM, 54...Communication section, 55 ... display unit, 56 ... operation unit, 57 ... audio output unit, 59 ... memory unit, 591 ... program, 592 ... registered product list, 6 ... imaging unit, 7 … POS terminals, 70 ... control unit, 71...CPU, 72...ROM, 73...RAM, 74...Communication section, 75 ... display unit, 76 ... operation unit, 77 ... audio output unit, 79 ... memory unit, 791 ... program, 90...Bundle products, 91...Single products. [Prior art documents] [Patent documents]

[0103] [Patent Document 1] JP 2013-089087 A

Claims

1. an imaging unit that captures an image of an article passing through an opening that is an entrance / exit to a predetermined area, and obtains a plurality of images for each image taken for each insertion into the area; a feature information output unit that receives the image and outputs feature information that indicates the features of the appearance of the article reflected in the image; a similarity output unit that compares the feature information with matching data that is a representative value of information indicating the appearance characteristics of the product and is associated with product information including at least the price, and outputs a similarity that indicates the degree of similarity between the two; a product identification unit that identifies, based on the similarity, information on the product that matches the characteristic information, and that, when there are a plurality of pieces of matching data whose similarity exceeds a first threshold value for the plurality of images involved in the same input, identifies the product information associated with the matching data, the product with a higher price, as the product information that matches the characteristic information; a product registration unit that registers information on the product identified by the product identification unit; A product registration system comprising:

2. The information on the product further includes correspondence information that corresponds the product to the other product when the product is either a first product or a second product that is a set of a plurality of the first products, When there are a plurality of pieces of matching data whose similarity exceeds the first threshold value in the plurality of images related to the same input, but the information of the products associated with the matching data does not include each other's information as the corresponding information, the product identification unit outputs an alert requesting execution of an operation to redo the imaging by the imaging unit. The product registration system according to claim 1.

3. When the number of the pieces of matching data whose similarity exceeds the first threshold is equal to or greater than a second threshold, the product identification unit outputs an alert requesting execution of an operation to redo the imaging by the imaging unit. The product registration system according to claim 1.

4. When the number of the matching data whose similarity exceeds the first threshold is equal to or greater than a second threshold, the product identification unit presents, as an option, information on the products associated with the matching data whose similarity exceeds the first threshold. The product registration system according to claim 1.

5. When the plurality of images related to the same input have a plurality of pieces of matching data whose similarity exceeds the first threshold value, the product identification unit presents information on the products associated with the matching data as options in descending order of price. The product registration system according to claim 1.

6. A computer of a product registration system including an imaging unit that captures an image of an article passing through an opening that is an entrance and exit to a predetermined area and obtains a plurality of images for each image taken for each insertion into the area, a feature information output unit that receives the image and outputs feature information that indicates the features of the appearance of the article reflected in the image; a similarity output unit that compares the feature information with matching data that is a representative value of information indicating the appearance characteristics of the product and is associated with product information including at least the price, and outputs a similarity that indicates the degree of similarity between the two; a product identification unit that identifies, based on the similarity, information on the product that matches the characteristic information, and that, when there are a plurality of pieces of matching data whose similarity exceeds a first threshold value for the plurality of images involved in the same input, identifies the product information associated with the matching data, the product with a higher price, as the product information that matches the characteristic information; a product registration unit that registers information on the product identified by the product identification unit; A program to function as a

Citation Information

Patent Citations

  • Commodity reader and program

    JP2013089087A