Object recognition system, object recognition program, and object recognition method

Abstract

To provide a novel object recognition system capable of further appropriately identifying an object even when it is assumed that there are similar objects therein.SOLUTION: An object recognition system for estimating which one of registered objects registered in a database is a target object included in an object image captured by the user, includes acquisition means for acquiring the object image, registration means for registering a plurality of external appearance images representing external appearances of one object when viewed from different feature angles, in the database for every registered object, and estimation means for estimating which one of the registered objects is the target object based on the object images and the external appearance images.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an object recognition system, an object recognition program, and an object recognition method for recognizing, from a photographed image of an object, whether the object is one of pre-registered objects. [Background technology]

[0002] For example, there are cases where an industrial product or its parts needs to be reordered due to damage or loss. However, especially with specialized products or parts, even if the actual product is available, it is often the case that the product name or model number is unknown, and since there are many similar products and parts, the consumer is unable to identify the product they need. In particular, while many products are now sold over the Internet, if the consumer is unable to identify the product they are looking for, they end up having to go to a store with the actual product and consult individually, or they end up having to send in pictures and exchange information multiple times, which does not ensure sufficient convenience.

[0003] To address this issue, there is known a technology for identifying products registered in a dictionary in advance from an external appearance image of the product. For example, Patent Document 1 discloses a technology for recognizing product candidates by capturing information indicating the external appearance of the product, extracting multiple feature quantities from the information, and comparing the feature quantities with the feature quantities of each product. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2018-142358 Summary of the Invention [Problem to be solved by the invention]

[0005] For example, as in Patent Document 1, in cases where candidates only need to be selected from among the target products handled, product candidates can be relatively easily identified from the appearance of the item, and even when there are multiple candidates, the product can be appropriately identified by having the user select one. However, when identifying products or parts that often have similar shapes, there is a problem in that there are many similar objects and it may not be possible to identify them simply by registering the appearance as viewed from one direction.

[0006] Therefore, an object of the present invention is to provide a novel object recognition system that can more appropriately identify an object even when similar objects are assumed. [Means for solving the problem]

[0007] In order to solve the above problems, the present invention provides an object recognition system that estimates which of the registered objects registered in a database a target object included in an object image photographed by a user is, and includes an acquisition means that acquires the object image, a registration means that registers a plurality of appearance images that represent the appearance of an object when viewed from different characteristic angles, or their feature amounts, in the database as appearance information for each of the registered objects, and an estimation means that estimates which of the registered objects the target object is based on the object image and the appearance information.

[0008] With this configuration, objects can be recognized more accurately based on multi-angle exterior images of each object.

[0009] In a preferred embodiment of the present invention, the registration means includes means for generating a plurality of appearance images based on information indicating the three-dimensional shape of the registered object, and registering the appearance information based on the generated appearance images in the database for each registered object.

[0010] With this configuration, appearance images can be registered based on three-dimensional shape information, making it possible to easily create a database of registered objects without the need to take a large number of appearance images for each registered object.

[0011] In a preferred embodiment of the present invention, the registration means includes means for acquiring an image of the registered object, and registering the image as the appearance image for each registered object in the database.

[0012] With this configuration, an image obtained by actually photographing an object can be registered as an appearance image, and the object image acquired by the user can be matched with an appearance image similar to the object image.

[0013] In a preferred embodiment of the present invention, the registration means generates or acquires the appearance image of the registered object having an attachment on its appearance, and registers the appearance information based on the generated appearance image in the database.

[0014] When a user actually photographs an object, the subject may be an object with dirt, scratches, rust, or other attachments. By registering an appearance image with attachments on its exterior in a database, it is expected that the effect of supporting appropriate object recognition can be expected even when an attachment is present.

[0015] In a preferred embodiment of the present invention, the registration means registers the appearance information in association with information indicating the characteristic angle.

[0016] With this configuration, for example, if multiple object images are obtained from a user along with information regarding the shooting angle, it becomes possible to identify the object based on changes in appearance depending on the angle, thereby further improving accuracy.

[0017] In a preferred form of the present invention, the registration means is configured to be able to register, in correspondence with the registered object, a characteristic image containing information for distinguishing the registered object from other objects, and the estimation means identifies candidates for the registered object that may be the target object, and generates photographing instruction information for instructing the user to photograph the target object based on the characteristic image of the registered object.

[0018] With this configuration, for example, by registering an image that includes a characteristic shape not found in other registered objects or a part with a unique marking, etc. as a characteristic image, it becomes possible to instruct the user to capture that part in the image. This makes it possible to more accurately identify the target object even when there are multiple similar objects.

[0019] In a preferred embodiment of the present invention, when the estimation means is unable to identify a candidate registered object that may be the target object, it generates shooting instruction information to instruct the user to remove any attachments from the target object.

[0020] With this configuration, even if the appearance differs from the registered appearance image due to adhesions, it is possible to obtain an object image that can accurately recognize the object by giving appropriate instructions to the user.

[0021] In a preferred embodiment of the present invention, the estimation means estimates which of the registered objects the target object is, using an estimation model trained with training data that takes the object image as input and outputs a registered object that is identical to the target object included in the object image.

[0022] With this configuration, it is possible to improve the estimation accuracy by appropriately selecting the data set.

[0023] The present invention is an object recognition program for estimating which of the registered objects registered in a database a target object included in an object image photographed by a user is, and causes a computer to function as an acquisition means for acquiring the object image, a registration means for registering in the database a plurality of appearance images representing the appearance of a single object when viewed from different characteristic angles, the appearance images or their features as appearance information for each of the registered objects, and an estimation means for estimating which of the registered objects the target object is based on the object image and the appearance information.

[0024] The present invention is an object recognition method for estimating which of the registered objects registered in a database a target object included in an object image photographed by a user is, and includes a computer that executes the following steps: an acquisition step for acquiring the object image; a registration step for registering, in the database, a plurality of appearance images each representing the appearance of an object when viewed from a different characteristic angle, the appearance images or their features as appearance information for each of the registered objects; and an estimation step for estimating which of the registered objects the target object is based on the object image and the appearance information. [Effects of the Invention]

[0025] According to the present invention, it is possible to provide a novel object recognition system that can more appropriately identify an object even when similar objects are assumed. [Brief explanation of the drawings]

[0026] [Figure 1] 1 is a functional block diagram of an object recognition system according to an embodiment of the present invention. [Figure 2] 1A and 1B are diagrams illustrating an example of an object according to an embodiment of the present invention. [Figure 3] 1 is a diagram showing a direction in which an exterior image of an object recognition system according to an embodiment of the present invention is captured; [Figure 4] FIG. 2 is a diagram showing an example of an exterior image of the object recognition system according to the embodiment of the present invention. [Figure 5]FIG. 2 is a diagram showing an example of object information stored in a database in the object recognition system according to the embodiment of the present invention. [Figure 6] 1 is a process flowchart relating to object recognition by an object recognition system according to an embodiment of the present invention. [Figure 7] 10 is a diagram illustrating an example of a display of a photographing instruction of the object recognition system according to the embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0027] The object recognition system of the present invention will be described below with reference to the drawings. Note that the embodiment shown below is an example of the present invention, and the present invention is not limited to the embodiment below, and various configurations can be adopted.

[0028] For example, although the configuration, operation, etc. of an object recognition system are described in this embodiment, methods, devices, computer programs, etc. with similar configurations can also achieve similar effects. Furthermore, the program may be stored on a recording medium. Using this recording medium, the program can be installed on, for example, a computer. Here, the recording medium storing the program may be a non-transitory recording medium, such as a CD-ROM.

[0029] In the present invention, the term "object" refers to any object that exists as a real object, and the object is not limited to this. In this embodiment, the object is assumed to be an industrial product, a component that constitutes the product, a subassembly that combines multiple components, or the like.

[0030] In this invention, multiple objects are registered in advance along with their appearances, and these registered objects are called “registered objects.” The target object photographed by the user is then recognized as one of the registered objects based on its appearance.

[0031] In the present invention, an image showing the appearance of a registered object when viewed from a specific angle is called an appearance image. This specific angle is hereinafter referred to as a "characteristic angle." In this embodiment, appearance information showing the appearance image or the feature amount obtained from the appearance image is stored in a database in association with the characteristic angle for each appearance image. By storing appearance information for multiple appearance images, each associated with a different characteristic angle, for each registered object, it becomes possible to recognize objects based on their appearance.

[0032] In the following, in this embodiment, the appearance information is assumed to be the appearance image itself, and the registration of the appearance information is described as "registering an appearance image" for simplicity's sake. However, the feature of the appearance image can also be used instead of the appearance image. Here, the feature can be a vector value obtained by calculation based on the pixel values ​​of the appearance image. It is preferable to use information that combines local features and global features of the image as the feature. The feature to be used can also be determined by machine learning. For example, it is conceivable that the task of identifying the type of object from a photographed image of the object is trained in a neural network model, and the model extracts feature values, and the feature to be used is determined to increase the identification accuracy.

[0033] It is also possible to register multiple appearance images for the same registered object at the same characteristic angle. For example, in this embodiment, multiple appearance images with attachments such as rust, dirt, scratches, etc. and appearance images without such attachments are registered as appearance images associated with the same characteristic angle, linked to object information of the same registered object. Furthermore, in the present invention, appearance information does not necessarily need to be registered in association with the characteristic angle, and object information and appearance images may be registered in association with each other.

[0034] Next, the functional configuration of the object recognition system of this embodiment will be described with reference to Fig. 1. In the object recognition system of this embodiment, an object recognition device 1 and a terminal device 2 are configured to be able to communicate with each other via a network NW, and the object recognition device 1 is connected to a database DB via a wired or wireless connection. The object recognition device 1 receives an object image captured by a user from the terminal device 2 and performs a recognition process to recognize which of the registered objects registered in the database DB a target object included in the object image is.

[0035] A general computer device such as a server device equipped with an arithmetic unit such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), a main storage device such as a RAM (Random Access Memory), an auxiliary storage device such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), or a flash memory, and various input / output devices including a means for connecting to a network NW can be used as the object recognition device 1. The object recognition device 1 may also be configured using multiple computers.

[0036] Any computer device, such as a smartphone or tablet terminal, equipped with various input / output devices including a camera, an arithmetic unit, a storage device, and a means for connecting to the network NW can be used as the terminal device 2. A dedicated application for transmitting an object image to the object recognition device 1, a browser application for accessing a dedicated web page, etc. are stored in the storage device, and the arithmetic unit executes various processes, so that any computer device functions as the terminal device 2 of the present invention.

[0037] In this embodiment, a computer equipped with a camera is used as the terminal device 2 to transmit an object image to the object recognition device 1 via the network NW, but it may also be configured such that, for example, an object image is transmitted directly from a digital camera or the like to the object recognition device 1 via a wired or wireless connection. Alternatively, it may be configured such that the object recognition device 1 acquires an object image that has been stored in advance in an arbitrary database or the like.

[0038] The object recognition device 1 includes a registration means 11 that registers an appearance image for each registered object, an acquisition means 12 that acquires an object image transmitted from the terminal device 2, and an estimation means 13 that estimates which registered object is the target object included in the object image. Note that these means may be distributed among multiple computers.

[0039] The registration means 11 registers object information for each registered object and registers a plurality of appearance images linked to the object information. Each appearance image shows the appearance of the object when viewed from a specific characteristic angle, and is registered in association with the characteristic angle. An example of registering appearance images will be described below with reference to FIGS. 2 to 4.

[0040] To explain an example of registering an external image, an example of an object is shown in Figure 2. As mentioned above, the type of object is not limited, but here we will assume a differential pressure transmitter as shown in Figure 2 as an example. This differential pressure transmitter has a restriction between the upstream and downstream pipes, and measures the difference between the upstream and downstream pressures to measure the flow rate of fluid passing through the pipe. A differential pressure transmitter can be divided into multiple parts as shown in Figure 2, and in this example, we will assume that a conduit, which is one of the parts that make up the differential pressure transmitter, will be registered as a registered object.

[0041] In this example, the conduit is distinguished into a low-pressure side and a high-pressure side, but the shapes are very similar and it is difficult to distinguish between them. In this way, when similar parts or other objects are assumed, appearance images are registered in association with multiple characteristic angles so that each object can be distinguished. Note that while it is preferable to register multiple appearance images at multiple characteristic angles, association with the characteristic angles is not essential, and multiple appearance images at different characteristic angles may simply be registered.

[0042] Appearance images can be obtained by taking images from above, horizontally, and below the object while shifting the observation angle (characteristic angle) around the vertical axis, as shown in Figure 3. Here, when obtaining appearance images, the object or camera may be actually moved to take images of the actual object for each characteristic angle, or computer graphics (CG) technology may be used to obtain information indicating the three-dimensional shape, and appearance images as viewed from each characteristic angle may be generated based on that information.

[0043] FIG. 4 shows an example of an appearance image for each characteristic angle thus obtained. In this embodiment, the characteristic angles are registered in association with the horizontal and vertical angles from a reference position, and images showing the respective appearances are registered. Note that here, the vertical angle is set to three levels: upward, horizontal, and downward. However, this angle may also be changed in a range from -90 degrees to 90 degrees at predetermined intervals (e.g., 30 degrees) with horizontal being set to 0 degrees. Furthermore, the characteristic angles do not necessarily have to be at constant intervals like this.

[0044] Furthermore, the registration means 11 of this embodiment registers, separately from the appearance image, or a part of the appearance image, as a feature image containing information for distinguishing the registered object from other objects, linked to the object information. For example, in the example of the conduit in FIG. 3, an image of the part where different characters are engraved on the low-pressure side and the high-pressure side can be registered as a feature image. In this embodiment, the registration means 11 also registers the feature image in association with its observation angle, similar to the feature angle of the appearance image. Here, the observation angle of the feature image is an angle at which the registered object can be easily distinguished from other similar objects.

[0045] The acquisition means 12 acquires object images captured by the terminal device 2. There are no limitations on the method of acquiring the object images, and for example, an image captured in advance may be uploaded as the object image. However, in this embodiment, it is assumed that the camera provided in the terminal device 2 is pointed at the target object, object images are captured as needed, and the captured images are transmitted to the object recognition device 1 each time or at predetermined intervals, and then acquired by the acquisition means 12. In this embodiment, the acquisition means 12 also acquires information related to the image capturing angle along with the object image.

[0046] The estimation means 13 estimates which registered object the target object included in the object image is, based on the object image acquired by the acquisition means 12 and the appearance image of the registered object registered in the database DB. In this embodiment, the estimation means 13 estimates which registered object the target object is, using an estimation model trained with training data that takes the object image as input and outputs the same registered object as the target object included in the object image. That is, the estimation means 13 estimates which registered object the target object is, by inputting the object image into an estimation model that has trained the task of taking the appearance image as input and estimating the registered object associated with it.

[0047] The database DB stores various information such as object information for each registered object and an appearance image linked to the object information. FIG. 5 is a diagram showing an example of object information for each registered object registered in the database DB. The object information stores an object ID that uniquely identifies the registered object, a model number, an object name, a manufacturer name, a characteristic image ID, and a shooting instruction. The characteristic image ID is information that specifies a characteristic image that shows the appearance of an important part, particularly for distinguishing the registered object from other similar registered objects. In this manner, in this embodiment, a characteristic image is registered for each registered object.

[0048] A characteristic image is an image showing the appearance of an object from a shooting direction, such as a characteristic shape, marking, or printing of the object, which is useful for distinguishing it from other objects. For example, an image obtained by enlarging a characteristic part of the object may be used as the characteristic image. Any of the appearance images associated with the characteristic image can be used as the characteristic image, but an image other than the appearance image, such as an enlarged image of a specific part rather than the entire object, may also be used. In other words, the characteristic image may or may not be associated with information indicating the observation angle of the object. The shooting instruction is a message that prompts the user to shoot the part indicated by the characteristic image, and is registered as a character string. The shooting instruction may also be in other forms, such as an illustration (image) of an arrow indicating the shooting direction.

[0049] In this way, appearance images and feature images for each feature angle are registered in the database DB for each registered object. Then, based on the registered information and the object image captured by the user, the present invention infers which of the registered objects is the target object included in the object image. The procedure for object recognition processing will be described below using a flowchart.

[0050] 6 is a flowchart of the recognition process in this embodiment. In this embodiment, the imaging means 21 in the terminal device 2 captures an image of a target object and generates an object image. In the object recognition device 1, first, in step S1, the acquisition means 12 acquires the object image transmitted from the terminal device 2. Here, in this embodiment, together with the object image, information on the imaging angle obtained by an acceleration sensor, a gyro sensor, etc. provided in the terminal device 2 is also acquired.

[0051] Here, when capturing object images, it is possible to accept a capture instruction and capture each image one by one, but it is preferable for the user to point the camera at the object while the imaging means 21 continuously captures object images at any time and transmits them to the object recognition device 1. In this way, the acquisition means 12 continuously captures multiple object images, and an estimation process is performed based on these images as described below, which is expected to improve recognition accuracy. Also, the shooting angle does not need to be acquired as an absolute value each time an object image is captured, and it may be acquired as a relative change in the direction and amount of movement based on the shooting angle of a certain object image.

[0052] Next, in step S2, the estimation means 13 identifies candidates for the target object based on the object image and the information registered in the database DB. Specifically, for example, a method is envisaged in which candidates for the target object are identified by inputting information on the object image and the shooting angle using a neural network model that has learned the task of identifying objects based on the appearance image and its characteristic angle registered in the database DB.

[0053] In step S3, the estimation means 13 determines whether or not the candidate has been successfully identified. For example, in step S2, the probability that the target object is each registered object is calculated for each registered object, and the success or failure of the candidate identification can be determined based on whether or not there is a registered object for which the calculated probability is equal to or greater than a predetermined value. In this case, if there is no registered object that satisfies a numerical value equal to or greater than the predetermined value, it is determined that the candidate has not been identified, and the process proceeds to step S4.

[0054] If a candidate cannot be identified, it is assumed that there may be a problem with the object image. For example, this may be the case when the appearance, such as the shape or color, of the object has changed due to dirt, rust, or other attachments. Therefore, in this embodiment, in step S4, the estimation means 13 generates photographing instruction information for instructing the user to remove the attachments from the target object and transmits it to the terminal device 2. The terminal device 2 then outputs based on the photographing instruction information, thereby instructing the user to remove the attachments. In addition, if a large movement is detected during photographing of the object image, it is assumed that there will be problems with the object image due to blurring. Therefore, other photographing instruction information may be generated, such as photographing instruction information instructing the user not to move the object for a certain period of time, such as "Please do not move."

[0055] If the candidate is successfully identified in step S3, the process proceeds to step S5, where the estimation means 13 checks whether there are multiple candidates. If there are multiple candidates, the process proceeds to step S6, where the estimation means 13 generates shooting instruction information for instructing the user to photograph the target object based on a feature image of a registered object of any of the identified candidates. The estimation means 13 transmits the generated shooting instruction information to the terminal device 2, and the output means 23 performs output based on the shooting instruction information.

[0056] FIG. 7 is an example of an instruction screen displayed based on the photographing instruction information. In this embodiment, the photographing instruction information is generated based on the characteristic image and photographing instruction in the object information of one of the candidates identified by the estimation means 13. Here, the characteristic image is displayed, and the photographing instruction, "Please photograph the marking as shown in the image," is also displayed. Alternatively, for example, the characteristic image may be superimposed on a preview of the photographed image, and instructions such as "Please align the object with the guide" may be displayed. The characteristic image and photographing instruction of which registered object among the candidates to use may be determined arbitrarily. For example, a method of using the object information of the candidate with the higher probability calculated in step S3 is envisioned.

[0057] In addition, although an example in which the characteristic image and the shooting instructions are displayed as they are has been shown here, it is also possible to generate shooting instruction information that specifically instructs the shooting angle, etc. For example, it is assumed that the observation angle is registered in the database DB in association with the characteristic image, and shooting instruction information is generated based on this.

[0058] More specifically, the acquisition means 12 first acquires the object image and its shooting angle. Then, the estimation means 13 compares the shooting angle with the observation angle associated with the characteristic image, and generates shooting instruction information that specifically indicates the amount and direction of movement, i.e., the direction and amount of movement from the current shooting angle. In this case, the shooting instruction output by the output means 23 can be, for example, a statement such as "Please move 30 degrees to the right and take the picture," or an arrow indicating the direction in which the camera should be moved.

[0059] In this embodiment, the shooting instruction information is generated based on the characteristic images of the identified candidates. However, it is also possible to generate shooting instruction information that simply instructs the user to change the shooting angle, for example, by saying "Take the photo from a different direction" or "Rotate the object," without using the characteristic images or their observation angles.

[0060] As described above, in this embodiment, object images are continuously acquired, so that recognition is performed again when the user moves the camera according to instructions. Here, since the candidates have already been narrowed down in step S3, the estimation means 13 identifies a more appropriate registered object from among the candidates based on the object images and feature images acquired according to instructions in step S6, and outputs the identified registered object as a candidate to the terminal device 2 in step S7, and receives approval of the recognition result from the user. Note that, here too, the same processes as in steps S2 to S5 may be performed on all registered objects without narrowing down the options.

[0061] If the user approves the displayed candidate (Yes in step S8), the process proceeds to step S9, where the object is identified and the process ends. On the other hand, if the displayed candidate and the photographed object are considered to be different, the user may choose not to approve them (No in step S8), and the process returns to step S1, where the recognition process is repeated again.

[0062] As described above, the object recognition system of this embodiment allows a user to take a photograph of a target object using the terminal device 2, and can appropriately recognize the object even when there is a similar object. This makes it possible, for example, to display a sales page for a specified registered object or to make an inquiry based on the registered object, and even if the user does not know the model number of the target object, the user can easily perform procedures such as ordering the object, repairing it, or inquiring about the manufacturer.

[0063] Specifically, the present invention may be applied to a mail-order product page, where after an object is identified, a purchase button is displayed to add the identified registered object to a shopping cart. This allows a user to easily purchase the correct product even when the model number of the object at hand is unknown. Furthermore, the present invention may be applied to a website provided by a product manufacturer, where, after an object is identified, a manual display button is displayed to display links to a group of manuals for the identified registered object. The group of manuals may include repair procedures, instruction manuals, installation manuals, specifications, etc. for the object.

[0064] In the above application example, if the object is a part of a product and cannot be purchased separately, or if there is no manual for the part itself, the object information can be linked to the product, and a purchase button or manual button for the product corresponding to the linked product can be displayed. [Explanation of symbols]

[0065] 1:Object recognition device 2: Terminal device 11: Registration method 12: Acquisition means 13: Estimation means 21: Imaging means 22: Input method 23: Output means DB: Database NW: Network

Claims

1. An object recognition system that estimates which of registered objects registered in a database a target object included in an object image captured by a user is, an acquisition means for acquiring the object image; a registration means for rotating a three-dimensional model of an object, generating a plurality of appearance images observed from each characteristic angle, extracting feature values ​​from each of the appearance images, and registering the plurality of appearance images or each of the feature values ​​in the database as appearance information of the registered object; an estimation means for estimating which of the registered objects the target object is based on the object image and the appearance information, The registration means uses a photographed image of an object and features extracted from the photographed image as training data, and inputs the appearance image into an object identification model that has learned which of local features, global features, or a combination thereof to use as the features so as to increase the accuracy of object identification, and extracts the features from the appearance image.

2. 2. The object recognition system according to claim 1, wherein the registration means has means for acquiring an image of the registered object and registering the image as the appearance image for each registered object in the database.

3. 3. The object recognition system according to claim 1, wherein the registration means generates or acquires the appearance image of the registered object having an attachment on its appearance, and registers the appearance information based on the generated appearance image in the database.

4. 4. The object recognition system according to claim 1, wherein the registration means registers the appearance information in association with information indicating the characteristic angle.

5. the registration means is configured to be able to register a feature image including information for distinguishing the registered object from other objects in association with the registered object; 5. The object recognition system according to claim 1, wherein the estimation means identifies candidates of the registered object that may be the target object, and generates photographing instruction information for instructing a user to photograph the target object based on the feature image of the registered object.

6. 6. The object recognition system according to claim 1, wherein when the estimation means cannot identify a candidate registered object that may be the target object, the estimation means determines that the target object has rust, dirt, or scratches attached thereto, and generates photographing instruction information to instruct a user to remove the attachment from the target object.

7. 7. The object recognition system according to claim 1, wherein the estimation means estimates which of the registered objects the target object is, using an estimation model trained with training data that receives the object image as an input and outputs a registered object that is identical to the target object included in the object image.

8. An object recognition program for estimating which of registered objects registered in a database a target object included in an object image photographed by a user is, an acquisition means for acquiring the object image; a registration means for rotating a three-dimensional model of an object, generating a plurality of appearance images observed from each characteristic angle, extracting feature values ​​from each of the appearance images, and registering the plurality of appearance images or each of the feature values ​​in the database as appearance information of the registered object; and causing the computer to function as an estimation means for estimating which of the registered objects the target object is based on the object image and the appearance information; The registration means is an object recognition program that uses a photographed image of an object and features extracted from the photographed image as training data, inputs the appearance image into an object identification model that has learned which of local features, global features, or a combination thereof to use as the features so as to increase the accuracy of object identification, and extracts the features from the appearance image.

9. An object recognition method for estimating which of registered objects registered in a database a target object included in an object image captured by a user is, comprising: an acquisition step of acquiring the object image; a registration step of rotating a three-dimensional model of the object to generate a plurality of appearance images observed from each characteristic angle, extracting feature amounts from each of the appearance images, and registering the plurality of appearance images or each of the feature amounts in the database as appearance information of the registered object; an estimation step of estimating which of the registered objects the target object is based on the object image and the appearance information; In the registration step, the appearance image is input to an object identification model that has learned, using a photographed image of the object and feature amounts extracted from the photographed image as training data, which of local features, global features, and a combination thereof should be used as the feature amounts so as to increase the accuracy of object identification, and the feature amounts of the appearance image are extracted.

Images