Registration device, registration method, recognition device, recognition method, and program

Abstract

To support highly-accurate determination of objects.SOLUTION: A registration device includes: a processor for performing registration processing of an article; and a database for storing information on the article. The processor is configured to: acquire first image data of a to-be-registered article which is to be registered in the database; calculate, for each of one or more registered articles already stored in the database, a degree of similarity of second image data of the registered article with the first image data of the to-be-registered article; determine whether or not there exists a registered article having the degree of similarity equal to or greater than a threshold; and register in the database, if it determines that there exists a registered article having the degree of similarity equal to or greater than the threshold, detailed information indicating a detailed classification of the to-be-registered article in association with the first image data.SELECTED DRAWING: Figure 8

Description

[Technical field]

[0001] The present disclosure relates to a registration device, a registration method, a recognition device, a recognition method, and a program. [Background technology]

[0002] Patent Document 1 discloses an online shopping system including an image DB for storing product images, and a product information DB for storing size information for each size category of products and stock status in association with the product images stored in the image DB. The online shopping system has an image determination means for extracting an image that matches a product image stored in the image DB based on a received product image, and extracting product information associated with the image by referring to the product information DB. The online shopping system further includes a suitable size determination means for determining a size category suitable for the product by comparing the size of the product extracted by the image determination means with the size of the customer, and an information transmission means for transmitting the product information extracted by the image determination means, the size category determined by the suitable size determination means, and the stock status to a mobile terminal. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2009-294909 A Summary of the Invention [Problem to be solved by the invention]

[0004] In Patent Document 1, the online shopping system associates image data of a product with a product ID and stores the product ID in a database. Furthermore, the online shopping system can register information about the product by linking the image data with the product ID. However, if there is a product with a very similar appearance to a product previously registered in the database, there is a possibility that the product will be mistakenly recognized as a different product when the target object is recognized based on the database. One example of a product with a very similar appearance is a product such as shampoo and treatment that has the same design.

[0005] The present disclosure has been devised in consideration of the above-described conventional situation, and aims to support highly accurate discrimination of objects. [Means for solving the problem]

[0006] The present disclosure provides a registration device comprising a processor that performs registration processing for an item, and a database that stores information about the item, wherein the processor acquires first image data of a registered item to be registered in the database, calculates a similarity between second image data of at least one registered item already stored in the database and the first image data of the registered item for each of the registered items, determines whether or not there is a registered item whose similarity is greater than or equal to a threshold, and if it is determined that there is a registered item whose similarity is greater than or equal to the threshold, links detailed information indicating a detailed classification of the registered item to the first image data and registers it in the database.

[0007] The present disclosure also provides a registration method, which includes obtaining first image data of a registered item to be registered in a database, calculating a similarity between second image data of at least one registered item already stored in the database and the first image data of the registered item for each of the registered items, determining whether or not a registered item exists whose similarity is greater than or equal to a threshold, and if it is determined that a registered item exists whose similarity is greater than or equal to the threshold, linking detailed information indicating a detailed classification of the registered item to the first image data and registering it in the database.

[0008] The present disclosure also provides a recognition device that includes a processor that is communicatively connected to a database in which information about an object is registered in advance and that authenticates an object, wherein the processor determines whether or not there is an object similar to the object in the database, and if it is determined that there is a similar object similar to the object in the database, determines whether or not the registration information of the similar object stored in the database includes classification presence / absence information indicating whether or not a detailed classification of the similar object is included, and if it is determined that the registration information includes the classification presence / absence information indicating that the detailed classification is included, detects detailed information indicating the characteristics of the object from a captured image of the object.

[0009] The present disclosure also provides a recognition method which determines whether or not there is an article similar to an object to be authenticated in a database in which information about the article is registered in advance, and if it is determined that there is an article similar to the object in the database, determines whether or not the registration information of the similar article stored in the database includes classification presence / absence information indicating whether the similar article includes a detailed classification, and if it is determined that the registration information includes the classification presence / absence information indicating the detailed classification, detects detailed information indicating the characteristics of the object from a captured image of the object.

[0010] The present disclosure also provides a program that causes a computer to execute the following processes: determining whether or not there is an article similar to an object to be authenticated in a database in which information about the article is pre-registered; if it is determined that there is a similar article similar to the object in the database, determining whether or not the registration information of the similar article stored in the database includes classification presence / absence information indicating whether or not a detailed classification of the similar article is included; and if it is determined that the registration information includes the classification presence / absence information indicating that the detailed classification is included, detecting detailed information indicating the characteristics of the object from a captured image of the object.

[0011] These comprehensive or specific aspects may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a recording medium, or may be realized as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium. Effect of the Invention

[0012] According to the present disclosure, it is possible to assist in highly accurate discrimination of objects. [Brief description of the drawings]

[0013] [Figure 1] 1A and 1B are diagrams showing examples of articles having similar appearances. [Diagram 2] 1 is a system configuration diagram of a recognition system according to an embodiment of the present invention. [Diagram 3] 13 is a flowchart showing an example of an article registration process. [Figure 4] 13 is a flowchart illustrating an example of detailed information position information calculation processing. [Diagram 5] 1 is an example of a first screen displayed on a display device. [Figure 6] 11 is a second example of a screen displayed on a display device. [Figure 7] 13 is a third example of a screen displayed on a display device. [Figure 8] 4 is a fourth example of a screen displayed on a display device. [Figure 9] FIG. 11 is a diagram showing an example of data stored in an item DB. [Figure 10] 11 is a flowchart illustrating an example of a process for recognizing an object. [Figure 11] 13 is a flowchart showing detailed information recognition processing. [Figure 12] FIG. 11 is a diagram showing an example of a screen displayed in the authentication process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, with reference to the drawings as appropriate, a detailed description will be given of an embodiment specifically disclosing a registration device, a registration method, a recognition device, a recognition method, and a program according to the present disclosure. However, more detailed description than necessary may be omitted. For example, detailed description of already well-known matters and duplicated description of substantially the same configuration may be omitted. This is to avoid the following description becoming unnecessarily redundant and to facilitate understanding by those skilled in the art. Note that the attached drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

[0015] First, examples of articles having similar appearances will be described with reference to Fig. 1. Fig. 1 is a diagram showing examples of articles having similar appearances.

[0016] In recent years, there is a need to capture an image of an object with a camera and recognize the object based on the captured image. For example, in cases where a product is displayed (i.e., shelf layout is recognized) to automatically detect out-of-stock items, or where a product is recognized without reading a barcode or the like to automatically settle a payment, if the object can be accurately recognized, the convenience of the user (e.g., a person who manages the object) can be improved.

[0017] For example, there is a method of recognizing an object using an image recognition technology such as deep distance learning, using a database in which images of the appearance of an object are registered in advance. However, when there are multiple objects with similar appearances, it is difficult to accurately recognize what object the object is, and there is a risk of misrecognizing a similar object as the object.

[0018] For example, as shown in Figure 1, there are products whose contents are the same but whose sizes or volumes are different, but whose appearances show different text indicating the volume of the product ("190ml" and "380ml"), but whose other parts are almost identical in design. Since the object is cut out from the captured image and resized before image recognition processing is performed, information about the size of the object is lost. In other words, it is difficult to accurately determine the type of object from its actual size.

[0019] As shown in FIG. 1, there are products that have similar appearances but different types, textures, or flavors (i.e., scents or tastes) of contents. Examples of such products include shampoo and treatment, damage care shampoo and scalp care shampoo, or apple-flavored black tea and strawberry-flavored black tea. In the case of such products, some text (e.g., "SHAMPOO" and "TREATMENT") or pictures (e.g., pictures of apples and strawberries) are different, and other parts have almost the same design. In addition, even clothing such as underwear is often sold in packages with the same design, except for characters indicating size and figures indicating shape (sleeve length, etc.). In image recognition using a database in which images of the appearance of products are registered in advance, it is difficult to accurately distinguish small differences in text or pictures, and there is a possibility that the target object will be mistaken for a different product.

[0020] In addition, for example, there are products such as shampoos and treatments that have some different colors, and although the color differences can be distinguished using deep distance learning technology, it may not be possible to accurately distinguish the products from the color differences when the lighting conditions are different.

[0021] In view of this situation, there is a need to correctly recognize an object even when there are objects with similar appearances.

[0022] Next, a system configuration of the recognition system 1 according to the present embodiment will be described with reference to Fig. 2. Fig. 2 is a system configuration diagram of the recognition system 1 according to the present embodiment.

[0023] The recognition system 1 includes an imaging device 10, a display device 11, an input device 12, a recognition terminal 20, a registration device 30, an imaging device 40, a display device 41, and an input device .

[0024] The imaging device 10 is a device that captures an image when recognizing an object. The imaging device 40 is a device that captures an image when registering an image of an object in the object DB 34 of the registration device 30. The imaging devices 10 and 40 are configured to have at least a lens (not shown) and an image sensor (not shown) as optical elements. The lens receives light reflected by the imaging object from within the angle of view of the imaging area of ​​the imaging devices 10 and 40, and forms an optical image of the imaging object on the light receiving surface (in other words, the imaging surface) of the image sensor. The image sensor is a solid-state imaging element such as a Charged Coupled Device or a Complementary Metal Oxide Semiconductor. The image sensor converts the optical image formed on the imaging surface through the lens into an electrical signal every predetermined time (for example, 1 / 30 (second)). For example, when the predetermined time is 1 / 30 (second), the frame rate of the imaging devices 10 and 40 is 30 fps. Furthermore, the imaging devices 10 and 40 may generate image data (video data) by performing a predetermined signal processing on the electrical signal at the above-mentioned predetermined time intervals. Note that the image data is a still image, and the video data is a video. Hereinafter, the image data and the video data will be referred to as a captured image. The imaging device 10 outputs the captured image to the recognition terminal 20. The imaging device 40 outputs the captured image to the registration device 30.

[0025] The display device 11 is a device that displays the authentication result, the progress, or a confirmation screen of the object. The display device 41 is a screen that displays the registration information of the item DB 34, the progress and results of the registration of the item by the item registration device 30 to the item DB 34, etc. The display devices 11 and 41 are, for example, displays, etc. The display device 11 may be configured integrally with the recognition terminal 20. The display device 41 may also be configured integrally with the registration device 30. In such a case, the display devices 11 and 41 are, for example, the screens of a personal computer.

[0026] The input device 12 is a device that accepts input when authenticating an object. The input device 42 is a device that accepts input of various information about an item from a user when registering the item. The input device 42 may also read data about the item distributed by a company that sells the item and output it to the registration device 30. The input devices 12 and 42 are, for example, a keyboard, a mouse, a touch pen, or the like. The input device 12 may be formed integrally with the display device 11, and the input device 42 may be formed integrally with the display device 41. In this case, the input devices 12 and 42 are touch panel displays, or the like.

[0027] The recognition terminal 20 is a terminal device used when authenticating an object. The recognition terminal 20 includes a memory 21, a communication I / F 22, and a processor 23.

[0028] The memory 21 is configured using, for example, a Random Access Memory (hereinafter referred to as "RAM") and a Read Only Memory (hereinafter referred to as "ROM"), and temporarily stores programs necessary for the operation of the recognition terminal 20, and further data generated during the operation. The RAM is, for example, a work memory used during the operation of the recognition terminal 20. The ROM stores and holds, for example, a program for controlling the recognition terminal 20 in advance.

[0029] The communication I / F 22 is an interface circuit that performs wireless or wired communication between the imaging device 10, the display device 11, the input device 12, the processor 23, and the communication I / F 31. The I / F is an interface. The communication with the communication I / F 31 may be via a network. The communication method by the communication I / F 22 is wired communication or wireless communication. Specifically, the wireless communication may be a Wide Area Network (hereinafter referred to as "WAN"), a Local Area Network (hereinafter referred to as "LAN"), Long Term Evolution (hereinafter referred to as "LTE"), mobile communication such as 4G and 5G, power line communication, short-range wireless communication (e.g., Bluetooth (registered trademark) communication), or communication for mobile phones. The wireless communication method is not limited to the above-mentioned examples.

[0030] The processor 23 is, for example, a central processing unit (hereinafter referred to as "CPU"), a digital signal processor (hereinafter referred to as "DSP"), a graphical processing unit (hereinafter referred to as "GPU"), or a field programmable gate array (hereinafter referred to as "FPGA"). The processor 23 functions as a controller that manages the overall operation of the recognition terminal 20. The processor 23 performs control processing for managing the operation of each part of the recognition terminal 20, input / output processing of data between each part of the recognition terminal 20, calculation processing of data, and storage processing of data. The processor 23 operates according to a program stored in the memory 21. The processor 23 uses the memory 21 during operation, and temporarily stores data generated or acquired by the processor 23 in the memory 21. The processor 23 realizes the functions of the control unit 231, the object recognition calling unit 232, the detailed information acquisition unit 233, and the detailed information recognition unit 234 by using the program and data stored in the memory 21.

[0031] The control unit 231 controls the imaging device 10, the display device 11, and the input device 12 via the communication I / F 22. In addition, the control unit 231 performs processing to call an article recognition unit 333 of the registration device 30 based on an instruction received from an object recognition calling unit 232 when recognizing an object.

[0032] When recognizing an object, the object recognition calling unit 232 outputs an instruction to the control unit 231 to cause the article recognition unit 333 to perform recognition processing of the object.

[0033] The detailed information acquisition unit 233 acquires detailed information about the object from the article DB 34 of the registration device 30 based on the recognition result of the object output from the control unit 231 .

[0034] The detailed information recognition unit 234 recognizes detailed information from the captured image of the object based on the detailed information acquired by the detailed information acquisition unit 233 from the article DB 34.

[0035] The processor 23 may further include an object recognition unit (not shown). The object recognition unit recognizes an object and calculates a score of similarity between the object and a captured image of an object registered in the object DB 34. When the object recognition unit is omitted from the processor 23, the object recognition and the similarity score are calculated by the object recognition unit 333 and the similar object determination unit 334 of the registration device 30.

[0036] The registration device 30 is a device that registers captured images and various information of an item in the item DB 34. The various information is, for example, the item name, the item type, the item identification (hereinafter referred to as "ID"), the item inventory, the item size and capacity, or the item release date. Here, the item refers to a product sold in a store or an item managed in a warehouse. The registration device 30 includes a communication I / F 31, a memory 32, a processor 33, and an item DB 34.

[0037] The communication I / F 31 is an interface circuit that performs wireless or wired communication between the communication I / F 22, the imaging device 40, the display device 41, the input device 42, and the processor 43. Note that communication with the communication I / F 22 may be via a network. The communication method by the communication I / F 31 is wired communication or wireless communication. Specifically, the wireless communication may be mobile communication such as WAN, LAN, LTE, 4G, 5G, power line communication, short-range wireless communication (e.g., Bluetooth (registered trademark) communication), or communication for mobile phones. Note that the wireless communication method is not limited to the above-mentioned examples.

[0038] The memory 32 is configured using, for example, a RAM and a ROM, and temporarily stores programs necessary for the operation of the registration device 30, as well as data generated during the operation. The RAM is, for example, a work memory used during the operation of the registration device 30. The ROM stores and holds, for example, a program for controlling the registration device 30 in advance.

[0039] The processor 33 is, for example, a CPU, a DSP, a GPU, or an FPGA. The processor 33 functions as a controller that manages the overall operation of the registration device 30. The processor 33 performs control processing for managing the operation of each part of the registration device 30, data input / output processing between each part of the registration device 30, data arithmetic processing, and data storage processing. The processor 33 operates according to a program stored in the memory 32. The processor 33 uses the memory 32 during operation, and temporarily stores data generated or acquired by the processor 33 in the memory 32. The processor 33 realizes the functions of the control unit 331, the product registration unit 332, the product recognition unit 333, the similar product determination unit 334, the detailed information acquisition unit 335, and the detailed information registration unit 336 by using the program and data stored in the memory 32.

[0040] The control unit 331 controls the imaging device 40, the display device 41, and the input device 42. The control unit 331 outputs the captured image received from the imaging device 40 to the item registration unit 332. The control unit 331 outputs the item information received from the input device 42 to the detailed information acquisition unit 335.

[0041] The product registration unit 332 registers the captured image of the product in the product DB 34. The product registration unit 332 also registers product information received from the input device 42 in association with the captured image. The product information includes the product name, product type, product ID, product inventory, product size / capacity, product release date, and the like.

[0042] The item recognition unit 333 recognizes the item using known techniques such as image recognition technology, and calculates the similarity to other items already stored in the item DB 34 as a score. The higher the similarity, the higher the score. The score is calculated as a numerical value such as a percentage. The score may be a three-level evaluation such as low, medium, and high.

[0043] The similar product determination unit 334 determines whether the score of the product stored in the product DB 34 calculated by the product recognition unit 333 is equal to or greater than a predetermined threshold, and selects products having a score equal to or greater than the predetermined threshold from among the products stored in the product DB 34. The predetermined threshold is, for example, a value determined by a person or the like who constructs the recognition system 1.

[0044] If an article similar to the article to be registered is stored in the article DB 34 based on the determination result of the similar article determination unit 334, the detailed information acquisition unit 335 extracts detailed information from the captured image using a known character recognition technique such as Optical Character Recognition. The detailed information is a design such as text or a picture that indicates the difference from the similar article. For example, the detailed information is a text indicating the capacity, a text indicating the type of contents such as shampoo or treatment, a design indicating the flavor such as a picture of an apple, etc. Note that the detailed information described above is an example and is not limited.

[0045] The detailed information registration unit 336 links the detailed information acquired by the detailed information acquisition unit 335 to the captured image and information on the item output to the item DB 34 by the item registration unit 332, and registers the information in the item DB 34. The item registration unit 332 and the detailed information registration unit 336 may be configured as a single registration unit.

[0046] The item DB 34 is a database that links and registers captured images, information, and detailed information of items. The item DB 34 is a non-volatile storage device, such as a hard disk drive, a solid state drive, a universal serial bus memory, an optical disk, or a magnetic tape.

[0047] (Registration process) First, a method for registering information about an item in a database will be described. The item registration process will be described with reference to Fig. 3. Fig. 3 is a flowchart showing an example of the item registration process. Each process in the flowchart of Fig. 3 is executed by a processor 33.

[0048] The control unit 331 controls the imaging device 40 to capture an image of the article. The control unit 331 acquires an image of the article from the imaging device 40 via the communication I / F 31 (step St100). Note that instead of the captured image of the article, the control unit 331 may acquire an image of the exterior of the article from distribution data distributed by a company that sells the article input to the input device 42. Hereinafter, the captured image of the exterior of the article to be stored in the article DB 34 and the distribution data already stored in the article DB 34 are referred to as image data.

[0049] The control unit 331 displays a screen on the display device 41 to prompt the user to input the name of the item. The control unit 331 acquires the name of the item input by the user to the input device 42 based on the instruction on the screen displayed on the display device 41 from the input device 42 (step St101). Note that the user may select the item to be registered from data on information related to the item distributed in advance by a company selling the item, etc., and input the selected item to the input device 42.

[0050] The control unit 331 outputs the image data acquired in the process of step St100 and the name of the item acquired in the process of step St101 to the item registration unit 332. In addition, the control unit 331 outputs the image data acquired in the process of step St100 to the item recognition unit 333.

[0051] The item recognition unit 333 calculates a similarity score between the image data of the items already stored in the item DB 34 and the image data acquired from the control unit 331 (step St102). The item recognition unit 333 calculates the similarity using a known technique. For example, the item recognition unit 333 calculates the feature amount of each captured image using a known technique, and calculates the similarity by comparing the feature amounts. The item recognition unit 333 outputs the calculated similarity score of each image data to the similar item determination unit 334.

[0052] The similar product determination unit 334 determines whether the score of each image data is equal to or greater than a predetermined threshold value (step St103).

[0053] The similar product determination unit 334 determines whether the score of each image data is equal to or greater than a predetermined threshold, and determines whether there is image data with a score equal to or greater than the predetermined threshold (that is, whether there is a similar image) (step St104).

[0054] When the similar product determination unit 334 determines that there is no similar image (step St104, NO), it executes a registration process (step St105). As the registration process, the similar product determination unit 334 outputs to the product registration unit 332 an instruction to register the image data and the name of the product in the product DB 34 and information that there is no similar product (i.e., information that the detailed recognition flag is off). The product registration unit 332 saves the image data and the name of the product in the product DB 34. At this time, the product registration unit 332 may also save the feature amount calculated by the object recognition 333 in the product DB 34. Note that since the similarity of products can be determined using the feature amount without using the image data, when the feature amount is saved, the saving of the image data may be omitted. However, since the feature amount may change depending on the environment in which the image data is captured or the algorithm used for the calculation, the product registration unit 332 can respond to more diverse situations by saving both the image data and the feature amount in the product DB 34. Furthermore, when the item registration unit 332 stores the image data and the name of the item in the item DB 34, it registers them with the detail recognition flag turned off. The detail recognition flag is a flag that indicates whether or not an item similar to the item to be registered exists in the item DB 34. If an item similar to the item to be registered exists in the item DB 34, the detail recognition flag is turned on. If an item similar to the item to be registered does not exist in the item DB 34, the detail recognition flag is turned off. The detail recognition flag can be read as classification presence / absence information.

[0055] When it is determined that a similar image is present (step St104, YES), the similar product determination unit 334 outputs to the detailed information acquisition unit 335 a signal indicating that a similar image is present.

[0056] The detailed information acquisition unit 335 acquires text corresponding to detailed information indicating the characteristics from the name of the item (step St106). The detailed information indicating the characteristics of the item is text indicating the type, texture, color, shape, use, or flavor of the item. For example, when the item name is "XXXYYY Treatment Refill 425ml", the detailed information acquisition unit 335 extracts "Treatment" which is text corresponding to the detailed information. Also, when the item name is "Apple-flavored black tea", the detailed information acquisition unit 335 extracts "Apple" which is text corresponding to the detailed information. Also, when there are multiple words indicating the characteristics of an item such as "XXXYYY Treatment Scalp Care Refill 425ml", the detailed information acquisition unit 335 may extract multiple words such as "Treatment Scalp Care" as detailed information. Also, the detailed information does not necessarily have to be a meaningful word or a set of words. For example, when the item name is "Apple-flavored black tea", even if the detailed information is "Rin", which is a part of "Apple", it can be sufficiently distinguished from "Strawberry-flavored black tea". In addition, as detailed information, the detailed information acquisition unit 335 may extract information indicating features different from the features of similar items from among the information indicating the features of the item. For example, in the above example, "treatment" is also information indicating the features of the item, but since it is difficult to distinguish it from similar items and is not suitable as item information, it may be excluded from the target to be extracted as detailed information. As an example of a method for extracting the corresponding text from the name of the item, for example, the processor 33 automatically displays text candidates on the display device 41 and the user selects one, or the user inputs the corresponding text into the input device 42. In addition, when the processor 33 automatically displays text candidates on the display device 41, the detailed information acquisition unit 335 may extract text that is not used in the item name of a similar item as a candidate. For example, between "XXXYYY Treatment Refill 425ml" and "XXXYYY Treatment Scalp Care Refill 425ml", the detailed information acquisition unit 335 extracts "scalp care" as a candidate. This is because the text other than "scalp care" is common to both items and cannot be used as detailed information to distinguish between the items.Hereinafter, detailed information is defined as text or character strings indicating the characteristics of an item extracted from the name of the item, and images of text, character strings, designs, etc., indicating the characteristics of an item extracted from the image data of the item. Both text and character strings are character information consisting of one or more characters. In this embodiment, character information extracted from image data and processed as necessary is called text, and character information included in image data is called character strings. However, this distinction is not strict, and depending on the context, the expressions text and character strings may be used without distinction. Examples of character information of one character suitable for use as detailed information include "S", "M", and "L" indicating sizes. Text or character strings indicating the characteristics of an item extracted from the name of the item are read as first information. Furthermore, images of text, character strings, designs, etc., indicating the characteristics of an item extracted from the image data of the item are read as second information. Furthermore, the first information and the second information are each read as detailed information.

[0057] The detailed information acquisition unit 335 performs character recognition on the image data and searches for a character string whose content is indicated by the text acquired in the process of step St106 (step St107). The detailed information acquisition unit 335 may search for a picture whose content is indicated by the text acquired in the process of step St106 using a known technique such as image recognition. For example, if the text acquired in the process of step St106 is "apple", the detailed information acquisition unit 335 searches for a picture of an apple. The registration device 30 may hold, for example, a dictionary of paraphrases of the text acquired in the process of step St106 (a dictionary that stores hiragana, katakana, English, and synonyms of the text) in the memory 32, and the detailed information acquisition unit 335 may search for a character string based on the dictionary. The detailed information acquisition unit 335 may also search for a character string based on synonyms of the text acquired in the process of step St106.

[0058] The detailed information acquiring unit 335 judges whether or not a character string corresponding to the detailed information is detected (step St108).

[0059] When the detailed information acquisition unit 335 determines that a character string corresponding to the detailed information has not been detected (step St108, NO), the detailed information acquisition unit 335 displays a screen on the display device 41 to prompt the user to manually select an area of ​​the corresponding character string from the image data. The detailed information acquisition unit 335 performs character recognition on the area based on the user's input operation from the input device 42 to acquire detailed information (step St109). After the process of step St109, the processor 33 proceeds to the process of step St110.

[0060] When the detailed information acquisition unit 335 determines that a character string corresponding to the detailed information has been detected (step St108, YES), it executes a process of calculating the position of the detailed information in the image data (calculation of detailed information position information) (step St110). The detailed information position information calculation process will be described in detail with reference to FIG. 4. When detailed information is detected, the detailed information acquisition unit 335 sets a rectangular frame that includes all of the corresponding character strings. The processes from step St106 to step St110 are performed on both the image data of the item to be registered and the image data of similar items.

[0061] The detailed information acquisition unit 335 outputs the detailed information and the position information of the detailed information to the detailed information registration unit 336. The detailed information registration unit 336 links the image data, the name of the item, the detailed information, and the position information of the detailed information, and registers them in the item DB 34 (step St111). When executing the process of step St111, the detailed information acquisition unit 335 turns on the detailed recognition flag of the item. In addition, the detailed information registration unit 336 also turns on the detailed recognition flag for similar items registered in advance in the item DB 34, overwrites and saves the detailed information and the detailed information position information. The detailed recognition flag makes it possible to determine whether or not the item is an item that requires detailed classification. Here, detailed classification of the item refers to the fact that, when recognizing an object, the recognition terminal 20 extracts detailed information from the captured image of the object using the detailed information and the detailed information position information, and classifies the object based on the extracted detailed information.

[0062] Next, detailed information position information calculation processing will be described with reference to Fig. 4. Fig. 4 is a flowchart showing an example of detailed information position information calculation processing.

[0063] The detailed information acquisition unit 335 detects a common area from an image to be registered in the item DB 34 (i.e., a registered image) (step St200). The common area is an area of ​​a design common between the appearance of the item to be registered and the appearance of an item that has already been registered in the item DB 34 and is similar to the item to be registered. For example, it is a logotype or a figure indicating the brand name of the item. In other words, the detailed information acquisition unit 335 detects an area of ​​a design common to the similar image from the registered image. The common area is specified by a rectangular frame that includes the corresponding design inside. Note that the frame that specifies the common area is not limited to a rectangle and may be other shapes such as a circle or a triangle, but for convenience of the following explanation, it is assumed to be a rectangle. The size of the frame is, for example, a size that is several percent larger than the minimum size that includes the corresponding design inside. The size of the frame may be set automatically or may be arbitrarily set by the user. The detailed information acquisition unit 335 detects the design and size of the common area.

[0064] The detailed information acquisition unit 335 detects a common area between similar images (step St201).

[0065] The detailed information acquisition unit 335 calculates position information of the detailed information between the registered image and the similar image (hereinafter, referred to as detailed information position information) (step St202). The position of the detailed information is indicated by a relative position with respect to the common area. The relative position is expressed, for example, by relative coordinates based on a specified coordinate of the common area. The position of the detailed information is indicated, for example, with the center of the common area as the starting point, by the vertical and horizontal sizes of the common area, and by the angle and distance to the center position within the frame specified in the detailed information. Note that the position of the detailed information may not start from the center of the common area, but may start from any point (for example, any end point) of the frame. The position of the detailed information may also be indicated by the angle and distance from the starting point of the common area to any point (for example, an end point of the frame) within the frame specified in the detailed information.

[0066] The detailed information acquisition unit 335 determines whether or not an error (that is, a positional deviation) in the detailed information position information between the registered image and the similar image is within a predetermined threshold value (step St203).

[0067] When the detailed information acquiring unit 335 determines that the error in the detailed information position information between the registered image and the similar image is not within a predetermined threshold (step St203, NO), it displays a screen on the display device 41 to prompt the user for confirmation. The user refers to the information on the common area and the detailed information position information displayed on the display device 41, checks whether the position of the detailed information is correct, and if it is correct, inputs that fact into the input device 42. If the position of the detailed information is incorrect, the user specifies the correct position and inputs it into the input device 42. The detailed information acquiring unit 335 determines the detailed information position information based on the information input by the user to the input device 42 (step St205).

[0068] When it is determined that the error between the detailed information position information of the registered image and the similar image is within a predetermined threshold (step St203, YES), the detailed information acquiring unit 335 confirms the detailed information position information (step St205).

[0069] The detailed information acquisition unit 335 may store, as information on the detected common area, the design and size of the common area in the item DB 34 in association with information on the corresponding item.

[0070] As described above, when storing image data of an item and the name of the item in the item DB 34, if an item with a similar appearance exists, the registration device 30 can link and register the detailed information in the image data that indicates the characteristics of the item. In addition, the registration device 30 can link the position of the detailed information indicating the characteristics of the item from the image data of the item to the registration information of the item and store it in the item DB 34. As a result, when recognizing an object using the item DB 34, if an object with a very similar appearance exists, the recognition terminal 20 can perform character recognition or image recognition on the position of the detailed information, so that the detailed information can be extracted with high accuracy. In other words, the recognition terminal 20 can recognize the object based on the extracted detailed information, so that erroneous recognition can be prevented. In this way, the registration device 30 can generate a database that supports high-accuracy discrimination of objects.

[0071] Next, the flow of example screens displayed on the display device 41 will be described with reference to Figs. 5, 6, 7, and 8. Fig. 5 is a first example screen displayed on the display device 41. Fig. 6 is a second example screen displayed on the display device 41. Fig. 7 is a third example screen displayed on the display device 41. Fig. 8 is a fourth example screen displayed on the display device 41.

[0072] The screen MN1 is an example of a screen showing input information of an article to be registered in the article DB 34.

[0073] The frame FR1 displays a captured image output from the imaging device 40 to the registration apparatus 30 or an image input from the input device 42. The frame FR2 displays the name of the item output from the input device 42. In the example shown in FIG. 5, "XXXYYY Shampoo Refill" is displayed in the frame FR2. The frame FR3 is a frame in which, if there is an item similar to the item to be registered currently among the items stored in the item DB 34, image data of the similar item is displayed. The frame FR4 is a frame in which the name of the similar item is displayed.

[0074] As shown in MN2, if it is determined that there is an article similar to the article to be registered among the articles registered in the article DB34, image data of the similar article is displayed in frame FR3. Also, a score indicating the similarity is displayed in a predetermined position in frame FR3. In the example shown in FIG. 6, it is displayed at the edge of frame FR3, but the position where it is displayed may be any position. In frame FR4, "XXXYYY Treatment Refill" is displayed as the name of the similar article.

[0075] When detailed information of the registered image and the similar image is detected, frames SP1 and SP2 are displayed for the detailed information of the image data as shown on the screen MN3. In this way, in the frames FR1 and FR3, a mark such as a rectangular frame is superimposed on the area showing the detailed information of the image data. When using a mark and a frame, the shape of the frame may be a shape other than a rectangle, for example, a circle or an ellipse. Furthermore, the mark is not limited to a frame, and may be a semi-transparent filled figure or an arrow pointing to the area of ​​the detailed information. In other words, the mark may be any figure that attracts the user's attention to the area of ​​the detailed information. Furthermore, text regarding the detailed information is displayed in the frames FR5 and FR6.

[0076] Finally, when a common area between the registered image and the similar image is detected, frames CO1 and CO2 are displayed for the common area as shown on the screen MN4.

[0077] As described above, the registration device 30 can display to the user the detailed information, common area, and detailed information location information to be registered in the item DB 34. This allows the user to correct any errors in the information to be registered, so the registration device 30 can prevent erroneous information from being registered in the item DB 34 and generate a highly accurate item DB 34. Furthermore, the registration device 30 can improve user convenience.

[0078] Next, an example of data stored in the item DB 34 will be described with reference to Fig. 9. Fig. 9 is a diagram showing an example of data stored in the item DB 34. In Fig. 9, a treatment and shampoo are used as examples of items to be registered in the item DB 34. Note that the data example shown in Fig. 9 is merely an example and is not limiting.

[0079] Table T1 is an example of data when a treatment is added to the item DB 34. The captured image of the item is linked to the item code, item name, manufacturer name, size, and detailed recognition flag. The detailed recognition flag is represented, for example, as "1" when the flag is set, and "0" when the flag is not set. When a treatment is registered in the item DB 34, there are no similar items, so the detailed recognition flag is 0, and detailed information and detailed information location information are not registered.

[0080] Table T2 is an example of data when a shampoo, which is a similar product to a treatment, is added after the treatment is registered in the product DB 34.

[0081] Since shampoo is a similar product to the already registered treatment, the detailed recognition flags for the treatment and shampoo are set to "1." Detailed information and detailed information location information are further added to the registered information for the treatment. The registered information for shampoo is linked to the captured image, with the product code, product name, manufacturer name, size, detailed recognition flag, detailed information, and detailed information location information.

[0082] The detailed information stores text extracted from the item name and text extracted from the image data. For example, in table T2, "Treatment" extracted from the item name and "TREATMENT" extracted from the image data are stored as detailed information. Note that the detailed information is not limited to text, and image data of a design showing the characteristics of the item may also be stored. For example, if the item is apple-flavored black tea, an image of an apple may be stored as detailed information.

[0083] (Recognition processing) Next, a method for authenticating an object using the generated article DB 34 will be described. First, a process for recognizing an object will be described with reference to Fig. 10. Fig. 10 is a flow chart showing an example of the process for recognizing an object.

[0084] The control unit 231 causes the imaging device 10 to capture an image of the object to be authenticated this time. The captured image of the object is acquired from the imaging device 10 (step St300). The control unit 231 outputs the captured image to the object recognition calling unit 232.

[0085] The object recognition call unit 232 transmits the captured image to the item recognition unit 333 via the control unit 231 and outputs an instruction to calculate the feature amount. The item recognition unit 333 calculates the feature amount of the captured image and the feature amount of the item stored in the item DB 34 using a known technique. If the feature amount of the item is stored in the item DB 34, the item recognition unit 333 may read out the stored feature amount without calculating the feature amount of the item again. The item recognition unit 333 calculates the similarity between the calculated object and the object stored in the item DB 34 as a score based on the feature amount of the object and the item stored in the item DB 34. The item recognition unit 333 outputs the score to the control unit 231. The control unit 231 acquires the score of the item stored in the item DB 34 as the recognition result of the object (step St301).

[0086] The control unit 231 determines whether or not there is a candidate with a score equal to or higher than a threshold value among the scores of the items stored in the item DB 34 acquired in the process of step St301 (step St302).

[0087] When the control unit 231 determines that there is no candidate whose score is equal to or greater than the threshold value in the process of step St302 (step St302, NO), it determines that it is not possible to determine what kind of item the object is from the item DB 34 (step St303). When the control unit 231 is unable to determine what kind of item the object is, the control unit 231 may cause the display device 11 to display a screen indicating that the object could not be determined.

[0088] When it is determined in the process of step St302 that there is a candidate whose score is equal to or greater than the threshold value (YES in step St302), the control unit 231 determines whether there is a candidate whose detailed recognition flag is on (step St304).

[0089] When it is determined that there is no candidate with the detailed recognition flag turned on (step St304, NO), the control unit 231 determines that the article with the highest score (referred to as Top1) is the target object (step St305).

[0090] When the control unit 231 determines that there is a candidate whose detailed recognition flag is on (step St304, YES), it checks whether or not there is detailed information for all the candidates (i.e., whether or not there is a detailed recognition flag) and performs a recognition process for the detailed information (step St306). N shown in step St306 is the number of candidates and indicates an integer of 1 or more. The control unit 231 executes the process of step St306 in descending order of score.

[0091] The control unit 231 judges whether the detailed recognition flag of the candidate is on or not (step St307). When the control unit 231 judges that the detailed recognition flag of the candidate is not on (step St307, NO), if there is no candidate list, the control unit 231 creates and adds a candidate list. The candidate list is a list for storing items whose score is equal to or greater than a threshold but whose detailed recognition flag is off. This candidate list is used to determine the recognition result from among items whose detailed information is not registered when none of the items whose detailed information is registered is determined as the recognition result. Note that, if there is already a candidate list, the control unit 231 may or may not add the information of the corresponding item to the candidate list. In this embodiment, even if the information of the corresponding item is added to the candidate list, the corresponding item is unlikely to be finally determined as the recognition result. The reason for this will be explained below. Since the process of step St306 is performed in descending order of score, if there is already a candidate list, items with a higher score than the corresponding item are already recorded in the candidate list. Here, the candidate list is information used when determining an item without using detailed information, and the recognition result is determined, for example, based on the high or low score. That is, an article with a lower score than the articles already recorded in the candidate list is unlikely to be determined as the recognition result. That is, the control unit 231 may add to the candidate list only the information of the article with the highest score among the articles whose candidate detailed recognition flags are not on. Note that, if the process of step St306 is not performed in descending order of score, the score of the article in question may be compared with the scores of the articles already included in the candidate list, and only the higher article may be stored in the candidate list.

[0092] When the control unit 231 determines that the detailed information recognition flag of the candidate is on (step St307, YES), the control unit 231 executes detailed information recognition processing (step St309). The detailed information recognition processing will be described in detail with reference to FIG.

[0093] Based on the processing result of step St309, the detailed information recognition unit 234 judges whether or not the result of character recognition of the image of the area showing the detailed information matches the detailed information of the candidate item stored in the item DB 34 (step St310). Note that in the processing of step St310, the detailed information recognition unit 234 may also determine that there is a match not only when the results of character recognition are a perfect match, but also when the notations are different, such as hiragana, katakana, and roman letters, when the languages ​​are different, such as Japanese and English, or when the words are synonyms.

[0094] If the detailed information recognition unit 234 determines that the result of character recognition of the image of the area indicating the detailed information matches the detailed information of the candidate item stored in the item DB34 (step St310, YES), it determines that the target object is the item used in the processing of step St310 (step St311).

[0095] If the detailed information recognition unit 234 determines that the result of character recognition of the image of the area indicating the detailed information does not match the detailed information of the candidate item stored in the item DB34 (step St310, NO), it processes the item with the next highest score (step St312).

[0096] When the detailed information recognition unit 234 determines that the character recognition results of the images of the areas showing the detailed information for all of the N candidates do not match the detailed information of the candidate items stored in the item DB 34, it determines the candidate in the candidate list as the recognition result (step St313). When the candidate list includes information on multiple items, the detailed information recognition unit 234 may determine the item with the highest score as the recognition result on the display device 11, or may display a screen that prompts the user to select which item to determine as the recognition result.

[0097] Next, the detailed information recognition process will be described with reference to Fig. 11. Fig. 11 is a flowchart showing the detailed information recognition process.

[0098] The detailed information acquiring unit 233 acquires detailed information position information of the candidate item from the item DB 34 via the control unit 231 (step St400). The detailed information acquiring unit 233 outputs the detailed information position information acquired in the process of step St400 to the detailed information recognizing unit 234.

[0099] The detailed information recognition unit 234 detects a common area of ​​the objects (step St401). When image data of the common area of ​​the candidate objects is stored in the object DB 34, the detailed information recognition unit 234 may detect the common area of ​​the objects by referring to the image data.

[0100] The detailed information recognition unit 234 detects and acquires the width and height of the common area (step St402).

[0101] The detailed information recognition unit 234 calculates the position of the detailed information area (referred to as detailed information area) from the captured image based on the detailed information position information of the candidate item and the width and height information of the common area calculated in the processing of step St402 (step St403). For example, in the example shown in Fig. 12, the detailed information recognition unit 234 detects the detailed information by calculating the distance R1 from the common area to the detailed information based on the width H1 and height W1 of the common area in the captured image displayed in the frame FR7. Note that the object calculated by the detailed information recognition unit 234 is not limited to the position of the detailed information area, and may include other information. For example, information such as the width and height of the detailed information area may be included.

[0102] The detailed information recognition unit 234 acquires an image of the detailed information region calculated in the process of step St403 (step St404). The detailed information recognition unit 234 may cut out the image of the detailed information region from the captured image of the object, or may add a mark such as a rectangular frame to the detailed information region.

[0103] The detailed information recognition unit 234 performs character recognition on the detailed information area to detect a character string written in the detailed information area (step St405).

[0104] As described above, the recognition terminal 20 can perform character recognition by focusing on a portion of the captured image of the object that is likely to have text indicating the object's characteristics written therein, using the detailed information and detailed information position information stored in advance in the item DB 34. In other words, even if there is another item whose appearance is very similar to the object, the recognition terminal 20 can perform character recognition by focusing on a portion of the object's appearance where text indicating the object's characteristics is written therein, thereby enabling the recognition terminal 20 to distinguish the object from similar items with high accuracy. This allows the recognition terminal 20 to recognize the object with high accuracy by using the item DB 34.

[0105] In the above, the detailed information is described as text information. If the detailed information is a design such as a picture, the recognition terminal 20 may use image recognition instead of text recognition to determine whether the image in the detailed information area matches the detailed information of the candidate item stored in the item DB 34. Generally, image recognition outputs information indicating the similarity between images. Therefore, for example, the user may adjust a threshold value for determining that the detailed information matches, so that it is possible to set how strictly the recognition terminal 20 determines the degree of match of the detailed information.

[0106] Next, an example of a screen displayed in the authentication process will be described with reference to Fig. 12. Fig. 12 is a diagram showing an example of a screen displayed in the authentication process.

[0107] Screen MN5 is an example of a screen that displays items similar to the target object (that is, items with scores equal to or greater than a threshold).

[0108] Frame FR7 displays a captured image of the object. Frames FR8, FR9, and FR10 display items with scores equal to or greater than a threshold value from among the items registered in the item DB34. Areas C1, C2, and C3 display the scores of each item. In the example shown on screen MN5, the item displayed in frame FR8 has a score of "0.98", the highest of the three items, and the item displayed in frame FR10 has a score of "0.65", the lowest of the three items. In this manner, the items may be displayed in order of score.

[0109] In the captured images displayed in the frames FR7, FR8, FR9, and FR10, a rectangular frame is superimposed on the detected common area and detailed information. For example, a rectangular frame is superimposed on the common area and detailed information when the user presses the button BT. Also, the user may be able to switch between an image with a rectangular frame superimposed and an image without a rectangular frame by pressing the button BT. This allows the recognition terminal 20 to assist the user in checking whether there are any errors in the detection results of the common area and detailed information.

[0110] (Summary of this embodiment) The above description of the present embodiment discloses the following techniques.

[0111] <Technology 1> The registration device of this embodiment includes a processor that performs item registration processing and a database that stores information about the items, and the processor acquires first image data of the registered items to be registered in the database, calculates the similarity between second image data of at least one registered item already stored in the database and the first image data of the registered item for each registered item, determines whether there is a registered item whose similarity is greater than or equal to a threshold, and if it is determined that there is a registered item whose similarity is greater than or equal to the threshold, links detailed information indicating a detailed classification of the registered item to the first image data and registers it in the database.

[0112] As a result, when there is another article whose appearance is similar to that of the registered article, the registration device according to the present embodiment can register the detailed information in the database by linking the detailed information to the first image data. In other words, the registration device can generate a database that supports accurate classification by distinguishing the registered article from other registered articles whose appearance is similar to that of the registered article with high accuracy based on the detailed information when recognizing the object. As a result, the registration device can support high-accuracy discrimination of the object.

[0113] <Technology 2> In the registration device described in Technology 1, the processor further obtains the name of the registered item, and if it determines that a registered item exists whose similarity is equal to or greater than a threshold, it extracts first information indicating the characteristics of the registered item from the name, extracts second information indicating the same meaning as the first information from the first image data, and registers the second information in the database, linking it to the first image data as detailed information.

[0114] As a result, the registration device according to the present embodiment can register in advance in a database the second information indicating the features of the registered article contained in the first image data, which is data on the appearance of the registered article. As a result, the registration device can generate a database that supports highly accurate extraction of the features of an object when the object is recognized using the database.

[0115] <Technology 3> In the registration device described in Technology 1 or Technology 2, a processor detects a common area of ​​a design common to the second image data from the first image data, calculates a relative position between the common area and the second information, and further links the relative position information to the first image data and registers it in a database.

[0116] As a result, the registration device according to the present embodiment can store the position information of the second information in a database in advance. That is, the registration device can generate a database that supports character recognition or image recognition at a position where information indicating the characteristics of an object is present when the object is recognized using the database. As a result, the registration device can generate a database that supports highly accurate recognition of information indicating the characteristics of an object.

[0117] <Technology 4> In the registration device according to any one of the first to third aspects, the second information is character information or an image.

[0118] This allows the registration device according to the present embodiment to store in advance in a database text, character strings, or images representing the characteristics of an article contained in the first image data.

[0119] <Technology 5> In a registration device described in any one of Techniques 1 to 4, a processor displays a screen including first image data with a mark superimposed on the position of the second information of the first image data, the name of the registered item, and the second information on a display device communicatively connected to the processor.

[0120] In this way, the registration device according to the present embodiment allows the user to check the information to be registered in the database. Since the user can check whether the information to be registered in the database is correct, the registration device can improve the convenience for the user.

[0121] <Technology 6> In the registration device according to any one of Techniques 1 to 5, the detailed information is information indicating features of a detailed classification of registered items in the first image data that are different from features of registered items having a similarity equal to or greater than a threshold value.

[0122] As a result, the registration device according to this embodiment can use the detailed information to distinguish the target object from registered articles that have a similarity degree equal to or greater than a threshold and have a similar appearance with a high degree of accuracy.

[0123] <Technology 7> In the registration device described in any one of Techniques 1 to 6, the processor further registers classification presence / absence information indicating whether the detailed information is linked to the first image data and registered in the database, in the database, linked to the first image data.

[0124] As a result, the registration device of this embodiment can determine whether or not more detailed classification is necessary when recognizing an object based on the classification presence / absence information, and can generate a database that assists in making more detailed classification as necessary.

[0125] <Technology 8> In the registration device according to any one of Techniques 1 to 7, the first image data is an image of the registration item captured by an imaging device communicatively connected to the processor.

[0126] As a result, the registration device of this embodiment can create image data to be registered in the database by capturing an image of the actual registered article with an imaging device, even if there is no image data of the appearance of the registered article provided by a company selling the registered article, etc.

[0127] <Technology 9> The recognition device of this embodiment is communicatively connected to a database in which information about items is pre-registered, and includes a processor that authenticates the object, and the processor determines whether there is an item in the database that is similar to the object, and if it is determined that there is a similar item in the database that is similar to the object, it determines whether the registration information of the similar item stored in the database includes classification presence / absence information indicating whether detailed classification of the similar item is included, and if it is determined that the registration information includes classification presence / absence information indicating detailed classification, it detects detailed information representative of the characteristics of the object from the captured image of the object.

[0128] As a result, the recognition device according to the present embodiment can perform detailed recognition processing of the object based on classification information by recognizing the object using a database in which information on the object is registered in advance. As a result, when there is an object that has a similar appearance to the object, the recognition device can perform detailed recognition processing, making it possible to distinguish the object with high accuracy.

[0129] <Technology 10> In the recognition device described in Technology 9, a processor detects common areas of a design that are common to image data of a similar item from an image of the object, and detects detailed information from the image of the object based on information on the relative positions of the common areas and the detailed information included in the registration information of the similar item.

[0130] As a result, the recognition device according to the present embodiment can detect detailed information of the object from the captured image with high accuracy, thereby suppressing erroneous recognition of the object.

[0131] <Technology 11> In the recognition device described in Technology 9 or Technology 10, if the detected detailed information matches the detailed information of a similar item, the target object is determined to be an item similar to the similar item.

[0132] As a result, the recognition device according to this embodiment can recognize the object with high accuracy based on the detailed information.

[0133] <Technology 12> In the recognition device described in any one of Techniques 9 to 11, when the processor determines that there are multiple similar items to the target object in the database, it determines whether the detected detailed information matches the detailed information of the similar items in order of the degree of similarity between the target object and the similar items.

[0134] As a result, the recognition device according to this embodiment can determine with high accuracy which similar article the target object corresponds to, even when there are multiple similar articles to the target object in the database.

[0135] Although the embodiments have been described above with reference to the accompanying drawings, the present disclosure is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications, corrections, substitutions, additions, deletions, and equivalents within the scope of the claims, and it is understood that these also belong to the technical scope of the present disclosure. In addition, the components in the above-mentioned embodiments may be arbitrarily combined within the scope of the invention. [Industrial Applicability]

[0136] The technology disclosed herein is useful as a registration device, a registration method, a recognition device, a recognition method, and a program that assist in highly accurate discrimination of objects. [Explanation of symbols]

[0137] 1 Recognition System 10 Imaging Device 11 Display Devices 12 Input Devices 20 Recognition terminal 21 Memory 22 Communication I / F 23 Processors 231 Control Unit 232 Object Recognition Call Unit 233 Detailed information acquisition section 234 Detailed information recognition unit 30 Registration device 31 Communication I / F 32 Memory 33 Processors 331 Control Unit 332 Product Registration Department 333 Article recognition section 334 Similar article determination department 335 Detailed information acquisition section 336 Detailed Information Registration Department 34 Goods DB 40 Imaging Device 41 Display Devices 42 Input Devices

Claims

1. A processor that performs registration processing of an item; A database for storing information about the item; The processor, Acquire first image data of a registration item to be registered in the database; Calculating a similarity between second image data of at least one registered article already stored in the database and the first image data of the registered article for each of the registered articles; determining whether or not there is a registered article whose similarity is equal to or greater than a threshold value; When it is determined that there is a registered article whose similarity is equal to or greater than a threshold, detailed information indicating a detailed classification of the registered article is linked to the first image data and registered in the database. Registration device.

2. The processor, Further obtaining the name of the registered article; When it is determined that a registered article exists in which the degree of similarity is equal to or greater than a threshold value, first information indicating a characteristic of the registered article is extracted from the name; extracting second information indicating the same meaning as the first information from the first image data; The second information is linked to the first image data as the detailed information and registered in the database. The registration device according to claim 1 .

3. The processor, Detecting a common area of ​​a design common to the first image data and the second image data; Calculating a relative position between the common area and the second information; The information on the relative position is further linked to the first image data and registered in the database. The registration device according to claim 2.

4. The second information is character information or an image. The registration device according to claim 2.

5. The processor displays a screen including the first image data in which a mark is superimposed on the position of the second information of the first image data, the name of the registered article, and the second information on a display device communicatively connected to the processor. The registration device according to claim 2.

6. The detailed information is information indicating a feature that is different from a feature of a registered article having a similarity degree equal to or greater than a threshold value among detailed classifications of the registered article of the first image data. The registration device according to claim 1 .

7. The processor further registers, in the database, classification presence / absence information indicating whether the detailed information is associated with the first image data and registered in the database, in association with the first image data. The registration device according to claim 1 .

8. The first image data is an image of the registered item captured by an imaging device communicatively connected to the processor. The registration device according to claim 1 .

9. Acquire first image data of a registration article to be registered in a database; Calculating a similarity between second image data of at least one registered article already stored in the database and the first image data of the registered article for each of the registered articles; determining whether or not there is a registered article whose similarity is equal to or greater than a threshold value; When it is determined that there is a registered article whose similarity is equal to or greater than a threshold, detailed information indicating a detailed classification of the registered article is linked to the first image data and registered in the database. How to register.

10. a processor communicably connected to a database in which information about the item is registered in advance and which performs authentication of the object; The processor, determining whether there is an article in the database similar to the object; When it is determined that there is a similar article similar to the target object in the database, it is determined whether or not the registration information of the similar article stored in the database includes classification presence / absence information indicating whether or not a detailed classification of the similar article is included; when it is determined that the classification presence / absence information indicating that the detailed classification is included is included, detailed information indicating a feature of the object is detected from the captured image of the object. recognition device.

11. The processor, Detecting a common area of ​​a design common to the image data of the similar article from the captured image of the object; detecting the detailed information from the captured image of the object based on information on a relative position between the common area and the detailed information included in the registration information of the similar article; The recognition device according to claim 10.

12. The processor determines that the object is a similar item to the similar item if the detected detailed information matches the detailed information of the similar item. The recognition device according to claim 11.

13. When the processor determines that there are a plurality of similar items to the target object in the database, the processor determines whether the detected detailed information matches the detailed information of the similar items in order of the degree of similarity between the target object and the similar items. The recognition device according to claim 12.

14. Determine whether or not there is an article similar to the object to be authenticated in a database in which information about the article is registered in advance; When it is determined that there is a similar article similar to the target object in the database, it is determined whether or not the registration information of the similar article stored in the database includes classification presence / absence information indicating whether or not a detailed classification of the similar article is included; when it is determined that the classification presence / absence information indicating that the detailed classification is included is included, detailed information indicating a feature of the object is detected from the captured image of the object. Recognition method.

15. A process of determining whether or not there is an article similar to the object to be authenticated in a database in which information about the article is registered in advance; When it is determined that there is a similar article similar to the target object in the database, a process of determining whether or not the registration information of the similar article stored in the database includes classification presence / absence information indicating whether or not a detailed classification of the similar article is included; when it is determined that the classification presence / absence information indicating that the detailed classification is included is included, detecting detailed information indicating a feature of the object from the captured image of the object; program.

Images