Appraisal system, method for appraisal, and appraisal program

Abstract

To enable a viewer of an appraisal result to easily understand grounds for determination of authenticity of an appraisal item.SOLUTION: According to one embodiment, an appraisal system includes: an appraisal unit for appraising whether an appraisal item matches a registered item on the basis of a registered item image registered in advance and an appraisal item image captured by a user; an image generation unit for generating at least one of a matching-point image indicating matching points between the registered item image and the appraisal item image and a non-matching-point image indicating non-matching points between the registered item image and the appraisal item image; and a display unit for displaying the result of the appraisal and at least one of the matching-point image and the non-matching-point image.SELECTED DRAWING: Figure 12

Description

[Technical Field]

[0001] The present invention relates to an authentication system, an authentication method, and an authentication program. [Background technology]

[0002] Conventionally, methods for authenticating an item based on an image of the item have been proposed. For example, Cited Document 1 discloses a method for authenticating an item, which includes acquiring a brand name of the item to be authenticated input via a user terminal, acquiring an image of a portion of the surface of the item by the user terminal, extracting features of the image, determining the authenticity of the item based on the brand name, the features, and the image features of the item registered in a database, and outputting the determination result to the user terminal, wherein the portion of the surface of the item includes at least one of a trademark, a logo, a brand name, a product name, and a product number. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2020-161149 Summary of the Invention [Problem to be solved by the invention]

[0004] The technology described in Cited Document 1 only outputs an appraisal result indicating whether the item is genuine or fake. This means that the person requesting the appraisal or the person viewing the appraisal result cannot know why the appraisal result was reached, which makes it difficult for them to feel satisfied with the appraisal result.

[0005] The present invention has been made in consideration of the above-mentioned problems, and aims to enable viewers of the appraisal results to easily understand the basis for the appraisal of the authenticity of an appraised item. [Means for solving the problem]

[0006] According to one embodiment, the appraisal system comprises an appraisal unit that appraises whether an appraised product matches a registered product based on a pre-registered registered product image and an appraised product image taken by a user, an image generation unit that generates at least one of a matching point image showing matching points between the registered product image and the appraised product image and a mismatch point image showing mismatch points between the registered product image and the appraised product image, and a display unit that displays the appraisal result and at least one of the matching point image and the mismatch point image. [Effects of the Invention]

[0007] According to one embodiment, the viewer of the appraisal result can easily understand the reason for the appraisal of the authenticity of the appraised item. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 illustrates an example of the configuration of an appraisal system 1000. [Figure 2] 2 is a diagram illustrating an example of a hardware configuration of an authentication device 1. FIG. [Figure 3] FIG. 2 is a diagram illustrating an example of a functional configuration of an appraisal device 1. [Figure 4] FIG. 2 is a diagram illustrating an example of the functional configuration of a user terminal 2. [Figure 5] FIG. 2 is a diagram illustrating an example of a functional configuration of a registrant terminal 3. [Figure 6] 10 is a flowchart illustrating an example of a registration method. [Figure 7] FIG. 2 is a diagram showing an example of a shooting screen sc1. [Figure 8] 10 is a flowchart showing an example of an appraisal method. [Figure 9] FIG. 10 is a diagram showing an example of a shooting screen sc2. [Figure 10] FIG. 10 is a diagram showing an example of a screen sc3 during shooting. [Figure 11] FIG. 10 is a diagram illustrating an example of an appraisal condition. [Figure 12] FIG. 10 is a diagram showing an example of an appraisal result screen sc4. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, each embodiment of the present invention will be described with reference to the accompanying drawings. Note that, in the description of the specification and drawings relating to each embodiment, components having substantially the same functional configuration are designated by the same reference numerals, and redundant description will be omitted.

[0010] <System configuration> First, an overview of the appraisal system 1000 according to this embodiment will be described. The appraisal system 1000 is an information processing system that appraises whether an appraised product matches a registered product based on an image of a registered product and an image of an appraised product, and displays an image that explains the appraisal together with the appraisal result.

[0011] The authenticated item is an item that is the subject of authentication. The authenticated item image is a photograph of the authenticated item.

[0012] A registered item is an item that has been registered as genuine (authentic). A registered item can be any item for which authentication is performed. Registered items include, but are not limited to, fine art (paintings, objects, sculptures, ceramics, calligraphy, ukiyo-e prints, textiles, photographs, etc.) or luxury goods (bags, shoes, watches, jewelry, alcohol, replicas of fine art, etc.). A registered item image is a photograph of the registered item.

[0013] Appraisal refers to determining whether the appraised product matches the registered product, that is, determining whether the appraised product is a genuine registered product.

[0014] FIG. 1 is a diagram showing an example of the configuration of an authentication system 1000. As shown in FIG. 1, the authentication system 1000 includes an authentication device 1, a user terminal 2, and a registrant terminal 3, which are communicably connected to each other via a network N. The network N is, for example, a wired local area network (LAN), a wireless LAN, the Internet, a public line network, a mobile data communication network, or a combination of these. In the example of FIG. 1, the authentication system 1000 includes one each of the authentication device 1, the user terminal 2, and the registrant terminal 3, but may include multiple of each.

[0015] The authentication device 1 is an information processing device that authenticates whether an authenticated product matches a registered product based on a registered product image and an authenticated product image. The authentication device 1 is, for example, but is not limited to, a PC (Personal Computer), a smartphone, a tablet terminal, a server device, or a microcomputer. In the example of FIG. 1, the authentication device 1 is a single information processing device, but may also be realized as a system consisting of multiple information processing devices connected via a network N.

[0016] The user terminal 2 is an information processing device used by a user. A user is a person who requests the appraisal of an item using the appraisal system 1000. The user may be, but is not limited to, the owner or seller of the item to be appraised. The user terminal 2 is equipped with a photographing device, photographs the item to be appraised, and transmits the image of the item to the appraisal device 1. The user terminal 2 also has a display device, which displays the appraisal results from the appraisal device 1 and an image that is the reason for the appraisal. The user terminal 2 is, for example, but is not limited to, a PC, a smartphone, or a tablet terminal.

[0017] The registrant terminal 3 is an information processing device used by a registrant. A registrant is a person who registers a registered item in the appraisal system 1000. A registrant may be, but is not limited to, the creator, owner, or seller of the registered item. The registrant terminal 3 is equipped with a photographing device, photographs the registered item, and transmits the registered item image to the appraisal device 1. The registrant terminal 3 may be, for example, but is not limited to, a PC, a smartphone, or a tablet terminal.

[0018] <Hardware configuration of information processing device 100> Next, a description will be given of the hardware configuration of the information processing device 100. Fig. 2 is a diagram showing an example of the hardware configuration of the information processing device 100. As shown in Fig. 2, the information processing device 100 includes a processor 101, a memory 102, a storage 103, a communication I / F 104, an input device 105, an output device 106, and a drive device 107, which are connected to each other via a bus B.

[0019] The processor 101 controls each component of the information processing device 100 and realizes the functions of the information processing device 100 by loading various programs including an OS (Operating System) stored in the storage 103 into the memory 102 and executing the programs. The processor 101 is, for example, but not limited to, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), or a DSP (Digital Signal Processor).

[0020] The memory 102 is, for example, a read-only memory (ROM), a random access memory (RAM), or a combination thereof. The ROM is, for example, a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a combination thereof. The RAM is, for example, but not limited to, a dynamic random access memory (DRAM) or a static random access memory (SRAM).

[0021] The storage 103 stores various programs including an OS and data. The storage 103 is, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or storage class memories (SCM), but is not limited to these.

[0022] The communication I / F 104 is an interface for connecting the information processing device 100 to an external device via the network N and controlling communication. The communication I / F 104 is, for example, Bluetooth (registered trademark), Wi-Fi (registered trademark), ZigBee (registered trademark), or Ethernet (registered trademark), but is not limited to these.

[0023] The input device 105 is a device for inputting information to the information processing device 100. The input device 105 is, for example, a mouse, a keyboard, a touch panel, a microphone, a scanner, an imaging device (camera), various sensors, or an operation button, but is not limited to these.

[0024] The output device 106 is a device for outputting information from the information processing device 100. The output device 106 is, for example, a display device, a projector, a printer, a speaker, or a vibrator, but is not limited to these.

[0025] The drive device 107 is a device that reads and writes data from and to the recording medium 108. The drive device 107 is, for example, but not limited to, a magnetic disk drive, an optical disk drive, a magneto-optical disk drive, or an SD card reader. The recording medium 108 is, for example, but not limited to, a CD (Compact Disc), a DVD (Digital Versatile Disc), an FD (Floppy Disk), an MO (Magneto-Optical disk), a BD (Blu-ray (registered trademark) Disc), a USB (registered trademark) memory, or an SD card.

[0026] In this embodiment, the program may be written to the memory 102 or the storage 103 during the manufacturing stage of the information processing device 100, or may be provided to the information processing device 100 via the network N, or may be provided to the information processing device 100 via a non-transitory computer-readable recording medium such as the recording medium 108.

[0027] <Functional configuration of the appraisal device 1> Next, a description will be given of the functional configuration of the verification device 1. Fig. 3 is a diagram showing an example of the functional configuration of the verification device 1. As shown in Fig. 3, the verification device 1 includes a communication unit 11, a storage unit 12, and a control unit 13.

[0028] The communication unit 11 is realized by the communication I / F 104. The communication unit 11 transmits and receives information to and from the user terminal 2 and the registrant terminal 3 via the network N.

[0029] The storage unit 12 is realized by the memory 102 and the storage 103. The storage unit 12 stores registered product information 121, authenticated product information 122, and authenticated information 123.

[0030] Registered product information 121 is information about a registered product. The registered product information 121 includes one or more registered product images and bibliographic information. The bibliographic information is information that indicates the bibliographic items of a registered product. The bibliographic information includes information that indicates at least one of the registered product's identification information, registrant, work type, work name, artist, production date, production technique, material, size, weight, registration date, owner, seller, and price.

[0031] The authenticated product information 122 is information related to the authenticated product. The authenticated product information 122 includes one or more authenticated product images and bibliographic information. The bibliographic information is information indicating the bibliographic items of the authenticated product. The bibliographic information includes information indicating at least one of the identification information of the authenticated product, the person (user) requesting the appraisal, the date of the request for the appraisal, the repair history of the authenticated product, and the identification information of the registered product to be authenticated.

[0032] The verification information 123 is information relating to verification, and includes information indicating the verification result, the verification reason, the match point image, the mismatch point image, and the mismatch reason.

[0033] The appraisal result is information indicating whether the appraised item matches the registered item, i.e., whether the appraised item is a genuine registered item. The appraisal results may be "Match (genuine)", "Mismatch (forgery)", "Highly likely to match", or "Highly likely to mismatch".

[0034] The reason for appraisal is text that indicates the reason for the appraisal. For example, the reason for appraisal is text that explains why the appraisal result was determined as it was, based on the similarities and differences between the registered item and the appraised item. For example, the reason for appraisal could be text such as, "The signatures on the registered item and the appraised item match. Although there is some change in the edges, this is likely due to aging, so we have determined that the appraised item matches the registered item."

[0035] The matching point image is an image showing the matching points between the registered product image and the authenticated product image. The matching points are portions of the registered product image and the authenticated product image that are determined to match. The matching point image may be an image including the matching points in the registered product image, or an image including the matching points in the authenticated product image. The matching point image may also be the entire registered product image or authenticated product image including the matching points, or an image obtained by cutting out a portion including the matching points from the registered product image or authenticated product image.

[0036] The matching point image is preferably an image in which the matching points in the registered product image or the authenticated product image are highlighted. For example, the matching point image is an image in which the similarity of the matching points is superimposed on the registered product image or the authenticated product image, but is not limited to this. Matching points and similarity will be described in detail later.

[0037] The mismatch point image is an image that shows the mismatch points between the registered product image and the authenticated product image. The mismatch points are parts of the registered product image and the authenticated product image that are determined to not match (are mismatched). The mismatch point image may be an image that includes the mismatch points in the registered product image, or an image that includes the mismatch points in the authenticated product image. The mismatch point image may also be the entire registered product image or authenticated product image that includes the mismatch points, or an image that has been cut out of the registered product image or authenticated product image to include the mismatch points.

[0038] The mismatch point image is preferably an image in which the mismatch points in the registered product image or the authenticated product image are highlighted. For example, the mismatch point image is an image in which the similarity of the mismatch points is superimposed on the registered product image or the authenticated product image, but is not limited to this. The mismatch points and similarity will be described in detail later.

[0039] The reason for the mismatch is the reason for the mismatch in the mismatch image, and includes at least one of "aging," "restoration," "difference in the shooting environment," "flaws in the shooting," and "difference in the item."

[0040] "Aging" means that although the authenticated item is an authentic work (registered item), changes have occurred in the registered item over time, resulting in discrepancies between the image of the registered item and the image of the authenticated item.

[0041] "Restoration" means that the appraised item is an authentic work (registered item), but as a result of restoring the registered item, discrepancies have arisen between the image of the registered item and the image of the appraised item.

[0042] "Differences in the photography environment" means that although the appraised item is a genuine product (registered item), there are differences between the photography environment when the registered item was photographed and the photography environment when the appraised item was photographed, resulting in discrepancies between the registered item image and the appraised item image. Photography environments include, but are not limited to, the brightness, direction, and color of the lighting at the time of photography.

[0043] "Photography defects" means that although the appraised item is an authentic product (registered item), there was a defect in the photographing of the registered item image or the appraised item image, resulting in a discrepancy between the registered item image and the appraised item image. Photography defects include, but are not limited to, camera shake, glare (shadows, reflected light, etc.), and differences in the settings of the photographing device (exposure time, aperture value, zoom, etc.).

[0044] "Difference in item" means that the authenticated item is a counterfeit (not a registered item), and therefore there are discrepancies between the images of the registered item and the authenticated item.

[0045] The verification information 123 may include only one of the match point image and the mismatch point image, or may include multiple match point images and multiple mismatch point images. If the verification information 123 does not include any mismatch point images, the verification information 123 does not need to include the reason for the mismatch.

[0046] The control unit 13 is realized by the processor 101 reading and executing a program from the memory 102 and working in cooperation with other hardware components. The control unit 13 controls the overall operation of the authentication device 1. The control unit 13 includes an acquisition unit 131, an authentication unit 132, an authentication reason generation unit 133, an image generation unit 134, and a mismatch reason generation unit 135.

[0047] The acquisition unit 131 acquires information from the user terminal 2 and the registrant terminal 3 and stores it in the storage unit 12.

[0048] The appraisal unit 132 appraises whether the appraised product matches the registered product based on a registered product image that has been registered in advance and an appraised product image taken by the user.

[0049] The verification reason generating unit 133 generates a verification reason that explains the verification result by the verifying unit 132 .

[0050] The image generating unit 134 generates at least one of a matching point image showing matching points between the registered product image and the authenticated product image, and a mismatching point image showing mismatching points between the registered product image and the authenticated product image.

[0051] The mismatch reason generation unit 135 generates a mismatch reason, which is text explaining why a mismatch point occurred in the mismatch image. The mismatch reason generation unit 135 may generate the mismatch reason using a machine learning model such as a large-scale language model, or may select a mismatch reason from a plurality of pre-set mismatch reasons using a machine learning model such as a classification model.

[0052] The functional configuration of the authentication device 1 is not limited to the above example. For example, the authentication device 1 may have some of the above functional configurations, and the user terminal 2 may have the rest. The authentication device 1 may also have functional configurations other than those described above. Each functional configuration of the authentication device 1 may be realized by software, as described above, or by hardware such as an IC chip, a SoC (System on Chip), an LSI (Large Scale Integration), or a microcomputer.

[0053] <Functional configuration of user terminal 2> Next, we will explain the functional configuration of the user terminal 2. Fig. 4 is a diagram showing an example of the functional configuration of the user terminal 2. As shown in Fig. 4, the user terminal 2 includes a communication unit 21, a storage unit 22, and a control unit 23.

[0054] The communication unit 21 is realized by the communication I / F 104. The communication unit 21 transmits and receives information to and from the verification device 1 via the network N.

[0055] The storage unit 22 is realized by the memory 102 and the storage 103. The storage unit 22 stores the authenticated product information 221 and the authenticated product information 222. The authenticated product information 221 and the authenticated product information 222 are similar to the authenticated product information 122 and the authenticated product information 123, and therefore a description thereof will be omitted.

[0056] The control unit 23 is realized by the processor 101 reading and executing a program from the memory 102 and working in cooperation with other hardware components. The control unit 23 controls the overall operation of the user terminal 2. The control unit 23 includes an imaging unit 231, an appraisal requesting unit 232, and a display unit 233.

[0057] The photographing unit 231 controls the photographing device in response to a user's operation, photographs the appraised item, and stores the obtained image of the appraised item in the storage unit 22 as the appraised item information 221.

[0058] The appraisal request unit 232 accepts input of appraisal product information 221 (bibliographic information) from the user, stores it in the memory unit 22 as appraisal product information 221, transmits the stored appraisal product information 221 to the appraisal device 1, and requests the appraisal device 1 to appraise the appraisal product.

[0059] The display unit 233 controls the display device in response to user operations to display various screens, and displays an authentication result screen that displays the authentication information 222.

[0060] The functional configuration of the user terminal 2 is not limited to the above example. For example, the user terminal 2 may have some of the above functional configuration, with the remaining functional configuration being provided by the authentication device 1. The user terminal 2 may also have functional configurations other than those described above. Each functional configuration of the user terminal 2 may be realized by software, as described above, or by hardware such as an IC chip, SoC, LSI, or microcomputer.

[0061] <Functional configuration of registrant terminal 3> Next, we will explain the functional configuration of the registrant terminal 3. Fig. 5 is a diagram showing an example of the functional configuration of the registrant terminal 3. As shown in Fig. 5, the registrant terminal 3 includes a communication unit 31, a storage unit 32, and a control unit 33.

[0062] The communication unit 31 is realized by the communication I / F 104. The communication unit 31 transmits and receives information to and from the verification device 1 via the network N.

[0063] The storage unit 32 is realized by the memory 102 and the storage 103. The storage unit 32 stores registered product information 321. The registered product information 321 is similar to the registered product information 121, and therefore a description thereof will be omitted.

[0064] The control unit 33 is realized by the processor 101 reading and executing a program from the memory 102 and working in cooperation with other hardware components. The control unit 33 controls the overall operation of the registrant terminal 3. The control unit 33 includes an imaging unit 331, a registration request unit 332, and a display unit 333.

[0065] The photographing unit 331 controls the photographing device in response to the operation of the registrant, photographs the registered product, and stores the obtained registered product image in the storage unit 32 as registered product information 321.

[0066] The registration request unit 332 accepts input of registration product information 321 (bibliographic information) from the user, stores it in the memory unit 32 as appraisal product information 321, transmits the stored registration product information 321 to the appraisal device 1, and requests the appraisal device 1 to register the registration product.

[0067] The display unit 333 controls the display device in response to the operation of the registrant and displays various screens. The display unit 333 displays a screen that displays the registered product information 321.

[0068] The functional configuration of the registrant terminal 3 is not limited to the above example. For example, the registrant terminal 3 may have some of the above functional configuration, with the remaining functional configuration being provided by the verification device 1. The registrant terminal 3 may also have functional configurations other than those described above. Each functional configuration of the registrant terminal 3 may be realized by software, as described above, or by hardware such as an IC chip, SoC, LSI, or microcomputer.

[0069] <How to register a registered product> Next, a description will be given of a registration method for a registered product executed by the authentication system 1000. Fig. 6 is a flowchart showing an example of the registration method executed by the authentication system 1000.

[0070] (Step S101) The registrant operates the registrant terminal 3 to display a registration screen for the registered item on the registrant terminal 3. The display unit 333 of the registrant terminal 3 displays the registration screen on the display device in response to the registrant's operation. The registration screen is a screen for registering the registered item.

[0071] (Step S102) The registrant operates the registrant terminal 3 to input bibliographic information of the registered product from the registration screen. The display unit 333 of the registrant terminal 3 stores the bibliographic information input by the registrant in the storage unit 32 as registered product information 321.

[0072] (Step S103) The registrant operates the registrant terminal 3 to display a photographing screen sc1 of the registered product on the registrant terminal 3, and photographs the registered product using the photographing device. The display unit 333 displays the photographing screen sc1 in response to the registrant's operation. The photographing unit 332 of the registrant terminal 3 controls the photographing device in response to the registrant's operation, photographs the registered product, and stores the obtained registered product image in the memory unit 32 as registered product information 321 in association with the bibliographic information.

[0073] The photographing screen sc1 is a screen for photographing a registered product. Fig. 7 is a diagram showing an example of the photographing screen sc1. The photographing screen sc1 in Fig. 7 includes eight registered product image display areas a1 and a back button b1. The left diagram in Fig. 7 shows the photographing screen sc1 before photographing the registered product, and the right diagram in Fig. 7 shows the photographing screen sc1 after photographing the registered product.

[0074] The registered product image display area a1 is an area for displaying the registered product image. In the example of Fig. 7, by selecting the registered product image display area a1, the photographing device is started, and the registered product image obtained by photographing is displayed in the registered product image display area a1.

[0075] On the photographing screen sc1, it is possible to photograph one registered item from multiple viewpoints. In the example of FIG. 7, the registered item can be photographed from eight viewpoints: "front," "back," "left side," "right side," "flat," "bottom," "signature," and "pattern," and a registered item image display area a1 is provided for photographing the registered item from each viewpoint. The registrant photographs the registered item from one or more viewpoints depending on the prepared viewpoints. In the right diagram of FIG. 7, the registered item is photographed from seven viewpoints: "front," "back," "left side," "right side," "flat," "bottom," and "signature," and the registered item images obtained by photographing are displayed in each registered item image display area a1.

[0076] Note that the viewpoints from which images can be captured are not limited to the above examples. Furthermore, the viewpoints from which images can be captured may be set according to the type of registered product. Furthermore, the viewpoints from which images can be captured may be set by the registrant.

[0077] The back button b1 is a button for returning from the photographing screen sc1 to the registration screen. When the registrant selects the back button b1, the registration product image obtained by photographing is stored in the storage unit 32 as the registration product information 321, and the registration screen is displayed.

[0078] (Step S104) The registrant requests the registration of the registered product from the registration screen. In response to the registrant's operation, the registration request unit 332 transmits the registered product information 321 stored in the storage unit 32 to the appraisal device 1 and requests the appraisal device 1 to register the registered product.

[0079] (Step S105) The acquisition unit 131 of the appraisal device 1 acquires the registered product information 321 from the registrant terminal 3 and stores it in the storage unit 12 as the registered product information 121. In this way, the registered product is registered in the appraisal device 1.

[0080] <How to authenticate items> Next, a description will be given of an authentication method for an authenticated item executed by the authentication system 1000. Fig. 8 is a flowchart showing an example of the authentication method executed by the authentication system 1000.

[0081] (Step S201) The user operates the user terminal 2 to display a registration screen for the authenticated product on the user terminal 2. In response to the user's operation, the display unit 233 of the user terminal 2 displays the registration screen on the display device. The registration screen is a screen for registering the authenticated product.

[0082] (Step S202) The user operates the user terminal 2 to input bibliographic information of the inspected product from the registration screen. The bibliographic information includes identification information of the registered product to be inspected. The display unit 233 of the user terminal 2 stores the bibliographic information input by the user in the memory unit 22 as inspected product information 221.

[0083] (Step S203) The user operates the user terminal 2 to display the photographing screen sc2 of the appraised item on the user terminal 2, and photographs the appraised item using the photographing device. It is preferable that the user photographs the appraised item from a perspective corresponding to the registered item image. This allows the user terminal 2 to obtain an appraised item image that corresponds to (is from the same perspective as) the registered item image. The display unit 233 displays the photographing screen sc2 in response to the user's operation. The photographing unit 231 of the user terminal 2 controls the photographing device in response to the user's operation to photograph the appraised item, and stores the obtained appraised item image in the memory unit 22 as appraised item information 221, associated with bibliographic information.

[0084] The photographing screen sc2 is a screen for photographing the item to be appraised. Fig. 9 is a diagram showing an example of the photographing screen sc2. The photographing screen sc2 in Fig. 9 includes eight appraisal item image display areas a2 and a back button b2. The left diagram in Fig. 9 shows the photographing screen sc2 before the appraisal item is photographed, and the right diagram in Fig. 9 shows the photographing screen sc2 after the appraisal item is photographed.

[0085] The appraisal product image display area a2 is an area where the appraisal product image is displayed. In the example of FIG. 9, by selecting the appraisal product image display area a2, the photographing device is activated, making it possible to photograph the appraisal product, and the appraisal product image obtained by photographing is displayed in the appraisal product image display area a2. Also, as shown in the left diagram of FIG. 9, the appraisal product image display area a2 for the viewpoint before photographing the appraisal product displays the registered product image corresponding to that viewpoint. This allows the user to easily understand from which viewpoint the appraisal product should be photographed in order to obtain an appraisal product image corresponding to the registered product image.

[0086] On the photographing screen sc2, it is possible to photograph one item for appraisal from multiple viewpoints. In the example of FIG. 9, the item for appraisal can be photographed from eight viewpoints: "front," "back," "left side," "right side," "flat," "bottom," "signature," and "pattern," and an appraisal item image display area a2 is provided for photographing the item for appraisal from each viewpoint. The user photographs the item for appraisal from one or more viewpoints depending on the prepared viewpoints. In the right diagram of FIG. 9, the item for appraisal is photographed from seven viewpoints: "front," "back," "left side," "right side," "flat," "bottom," and "signature," and the photographed images of the item for appraisal are displayed in each of the appraisal item image display areas a2.

[0087] Note that the viewpoints that can be photographed are not limited to the above example. The viewpoints that can be photographed may be set according to the type of item to be authenticated. The viewpoints that can be photographed may also be set by the user. The viewpoints that can be photographed may also be limited to viewpoints for which registered item images have been registered. In this case, in the example of FIG. 9, the authenticated item image display area a2 for "pattern" is not displayed. Furthermore, the user may not photograph the authenticated item from some of the viewpoints for which registered item images have been registered. For example, in the example of FIG. 9, the authenticated item may be photographed from six viewpoints: "front," "back," "left side," "right side," "bottom," and "signature," and the authenticated item may not be photographed from the "flat" viewpoint. In this case, the appraisal, described below, is performed using the authenticated item images and registered item images from the six viewpoints.

[0088] The back button b2 is a button for returning from the photographing screen sc2 to the registration screen. When the user selects the back button b2, the photographed image of the appraised product is stored in the storage unit 22 as the appraised product information 221, and the registration screen is displayed.

[0089] 10 is a diagram showing an example of the during-photographing screen sc3. The during-photographing screen sc3 is a screen that is displayed when the user selects the appraisal item image display area a2 and the photography device starts up. The during-photographing screen sc3 in FIG. 10 includes the during-photographing image display area a3, a photography button b3, and a back button b4.

[0090] The image-in-capture display area a3 is an area that displays the image being captured by the image capture device. The registered product image im1 from the viewpoint selected by the user is transparently displayed in the image-in-capture display area a3. In the example of Figure 10, the user selects the "front" image display area a2 for the authenticated product, and the "front" registered product image im1 is transparently displayed in the image-in-capture display area a3. The user captures the authenticated product aw so that the image im2 of the authenticated product aw being captured by the image capture device overlaps with the registered product image im1. In this way, by displaying the registered product image im1 as a guide in the image-in-capture display area a3 when capturing the authenticated product, an authenticated product image that is closer to the registered product image im1 can be obtained. As a result, the accuracy of the appraisal, as described below, can be improved.

[0091] The photographing button b3 is a button for executing photographing. When the user selects the photographing button b3, the image being photographed is acquired as an image of the appraisal item and is saved in the storage unit 22 as the appraisal item information 221.

[0092] The back button b4 is a button for returning from the shooting screen sc3 to the shooting screen sc2. When the user selects the back button b4, the shooting screen sc2 is displayed.

[0093] (Step S204) The user requests the appraisal of an appraised product from the registration screen. In response to the user's operation, the appraisal requesting unit 232 transmits the appraised product information 221 stored in the storage unit 22 to the appraisal device 1 and requests the appraisal device 1 to appraise the appraised product.

[0094] (Step S205) The acquisition unit 131 of the appraisal device 1 acquires the appraised product information 221 from the user terminal 2 and stores it in the storage unit 12 as the appraised product information 122. In this way, the appraised product is registered in the appraisal device 1.

[0095] (Step S206) When requested to perform appraisal, the appraisal unit 132 refers to the identification information of the registered product included in the appraised product information 122, acquires the registered product information 121 of the registered product to be appraised, and stores it in the storage unit 12 as appraisal information 123. The appraisal unit 132 appraises whether the appraised product matches the registered product based on the acquired registered product image and the appraised product image included in the appraised product information 122, and stores the appraisal result in the storage unit 12 as appraisal information 123. Here, a specific example of an appraisal method will be described.

[0096] ·Appraisal method (1) In the first authentication method, the authentication unit 132 calculates the similarity between the registered product image and the authenticated product image, and then authenticates whether the authenticated product matches the registered product based on the similarity. To this end, the authentication unit 132 first calculates the similarity between corresponding registered product images and authenticated product images (images from the same perspective). In the examples of FIGS. 7 and 9, the authentication unit 132 calculates seven similarities: the similarity between the registered product image and the authenticated product image for the "front," the similarity between the registered product image and the authenticated product image for the "back," the similarity between the registered product image and the authenticated product image for the "left side," the similarity between the registered product image and the authenticated product image for the "right side," the similarity between the registered product image and the authenticated product image for the "flat surface," the similarity between the registered product image and the authenticated product image for the "bottom," and the similarity between the registered product image and the authenticated product image for the "signature."

[0097] The appraisal unit 132 may input the corresponding registered product image and appraised product image into a machine learning model that determines whether the subjects of the two images are identical, and obtain the degree of certainty of match output by the machine learning model as the similarity between the registered product image and the appraised product image.

[0098] Alternatively, the appraisal unit 132 may extract feature points from the corresponding registered product image and the authenticated product image, and calculate the similarity based on the coordinates and number of feature points. The appraisal unit 132 may extract feature points using an algorithm such as SIFT (Scale-Invariant Feature Transform), or may extract feature points by inputting the registered product image and the authenticated product image into a machine learning model that extracts feature points.

[0099] The appraisal unit 132 may calculate the similarity between the registered product image and the appraised product image from one perspective using multiple methods. For example, when three calculation methods are used, a similarity of 21 is calculated in the examples of Figures 7 and 9. The appraisal unit 132 may also calculate the similarity between the registered product image and the appraised product image from one perspective using multiple methods, calculate one similarity from the multiple similarities obtained, and use this as the similarity between the registered product image and the appraised product image from that perspective. For example, it is possible to use a weighted average of the multiple similarities calculated using multiple calculation methods as the similarity for that perspective.

[0100] Next, the appraisal unit 132 appraises whether the appraised product matches the registered product based on the similarity of each aspect.

[0101] ·Appraisal method (1-1) The appraisal unit 132, for example, inputs the similarity of each aspect into a machine learning model that determines whether the appraised product matches the registered product based on the similarity of each aspect, and obtains the judgment result output by the machine learning model as the appraisal result.

[0102] ·Appraisal method (1-2) Alternatively, the assessment unit 132 may calculate an overall similarity based on the similarity of each viewpoint, and assess the overall similarity as a "match" if the overall similarity is equal to or greater than a threshold, and assess the overall similarity as a "mismatch" if the overall similarity is less than the threshold. The overall similarity may be, for example, a weighted average of the similarities of each viewpoint, but is not limited to this.

[0103] ·Appraisal method (1-3) Furthermore, the appraisal unit 132 may rule-based appraise whether the appraised product matches the registered product in accordance with preset appraisal conditions based on the similarity of each viewpoint.

[0104] Here, Fig. 11 is a flowchart showing an example of the verification conditions. In the example of Fig. 11, the similarity is a real number between 0 and 1, and the larger the value, the higher the similarity. Under the verification conditions of Fig. 11, if the similarity of all aspects is 0.8 or higher (step S301: YES), it is determined that there is a match (step S302). If the similarity of at least some of the aspects is less than 0.8 (step S301: NO), the similarity of "signature" is 0.8 or higher (step S303: YES), and all of the remaining similarities are 0.7 or higher (step S304: YES), it is determined that there is a match (step S302). Otherwise (step S303: NO, step S304: NO), it is determined that there is a mismatch (step S305).

[0105] ·Appraisal method (2) In the appraisal method (2), the appraisal unit 132 directly appraises whether the appraised product matches the registered product from the registered product image and the appraised product image. Specifically, the appraisal unit 132 inputs all the registered product images and the appraised product images into a machine learning model that determines whether the subjects of two pairs of images match, and obtains the determination result output by the machine learning model as the appraisal result.

[0106] The authentication method and authentication conditions are not limited to the above examples, and the authentication unit 132 may use a combination of the above authentication methods.

[0107] (Step S207) The authentication reason generation unit 133 generates an authentication reason according to the authentication method and authentication result of the authentication unit 132, and stores the authentication reason in the memory unit 12 as authentication information 123. If the authentication result is "match", the authentication reason generation unit 133 explains the authentication reason explaining why there is a match. Also, if the authentication result is "mismatch", the authentication reason generation unit 133 explains the authentication reason explaining why there is a mismatch.

[0108] When the appraisal unit 132 uses appraisal method (1-1), the appraisal reason generation unit 132 generates text such as "As a result of appraisal using AI based on the similarity of images from each perspective, we have determined that the appraised product matches the registered product" or "As a result of appraisal using AI based on the similarity of images from each perspective, we have determined that the appraised product does not match the registered product." When the appraisal unit 132 uses appraisal method (1-1), the appraisal unit 132 may use a machine learning model that outputs the reason for the judgment along with the judgment result. In this case, the appraisal reason generation unit 133 may acquire the reason for the judgment output by the machine learning model as the reason for the appraisal. As such a machine learning model, a multimodal large-scale language model that is trained to input similarity and output the judgment result and the reason for the judgment can be used.

[0109] When the appraisal unit 132 uses the appraisal method (1-2), the appraisal reason generation unit 132 generates text such as "The appraised product is appraised to match the registered product because the overall similarity exceeds the standard" or "The appraised product is appraised to not match the registered product because the overall similarity falls below the standard" as the appraisal reason.

[0110] When the appraisal unit 132 uses the appraisal method (1-3), the appraisal reason generation unit 132 generates an appraisal reason based on the appraisal conditions. In the example of Fig. 11, the appraisal reason generation unit 132 generates text such as "Since all images were determined to match (similarity is 0.8 or more), the authenticated item is authenticated as a match to the registered item" or "Since the signature was determined to be a mismatch (similarity is less than 0.8), the authenticated item is authenticated as a mismatch to the registered item." Note that appraisal reasons may be preset according to the appraisal results and the appraisal conditions.

[0111] When the appraisal unit 132 uses appraisal method (2), the appraisal reason generation unit 132 generates text such as "As a result of comprehensively assessing the image using AI, we have determined that the appraised product matches the registered product" or "As a result of comprehensively assessing the image using AI, we have determined that the appraised product does not match the registered product" as the appraisal reason. Note that when the appraisal unit 132 uses appraisal method (2), the appraisal unit 132 may use a machine learning model that outputs the reason for the assessment along with the assessment result. In this case, the appraisal reason generation unit 133 may acquire the reason for the assessment output by the machine learning model as the appraisal reason. Such a machine learning model may be a multimodal large-scale language model that is trained to receive an image as input and output a judgment result and a reason for the assessment.

[0112] The method for generating an appraisal reason is not limited to the above example. For example, a machine learning model for generating an appraisal reason may be prepared separately from the machine learning model used in the appraisal method.

[0113] (Step S208) The image generation unit 134 generates a match point image based on the registered product image and the authenticated product image, and stores the match point image in the storage unit 12 as the authentication information 123.

[0114] First, the image generation unit 134 calculates the similarity of the appraisal image to the corresponding registered product image for each region of the appraisal image. The region may be one pixel or a rectangular region of any size including multiple pixels. The method for calculating the similarity is arbitrary. Furthermore, if the appraisal unit 132 calculates the similarity for each region in the process of calculating the similarity between the corresponding registered product image and the appraisal image, the image generation unit 134 may acquire the similarity for each region calculated by the appraisal unit 132.

[0115] Next, the image generation unit 134 determines that a portion in the authenticated product image where the similarity is equal to or greater than the first threshold is a matching point, and acquires an image including the matching point. If there is a matching point in the authenticated product image, the image generation unit 134 may acquire the entire authenticated product image, or may cut out a predetermined range in the authenticated product image that includes the matching point.

[0116] Next, the image generation unit 134 highlights the matching points in the image containing the matching points. The image generation unit 134 highlights the matching points by, for example, surrounding the matching points with a line, changing the color of the matching points, overlaying a transparent colored layer on the matching points, or overlaying a similarity on the matching points. In this way, a matching point image based on the authenticated product image, i.e., a matching point image in which the matching points in the authenticated product image are highlighted, is generated.

[0117] The method for generating a matching point image based on a registered product image is the same as that described above. Specifically, in the above, the authenticated product image is replaced with the registered product image, and the registered product image is replaced with the authenticated product image. Furthermore, if there are no matching points between the authenticated product image and the registered product image, the matching point image is not generated. Furthermore, the image generation unit 134 may generate multiple matching point images for one authenticated product.

[0118] (Step S209) The image generating unit 134 generates a mismatch point image based on the registered product image and the authenticated product image, and stores the mismatch point image in the storage unit 12 as the authentication information 123.

[0119] First, the image generation unit 134 calculates the similarity of the appraisal image to the corresponding registered product image for each region of the appraisal image. The region may be one pixel or a rectangular region of any size including multiple pixels. The method for calculating the similarity is arbitrary. Furthermore, if the appraisal unit 132 calculates the similarity for each region in the process of calculating the similarity between the corresponding registered product image and the appraisal image, the image generation unit 134 may acquire the similarity for each region calculated by the appraisal unit 132.

[0120] Next, the image generation unit 134 determines that a portion in the authenticated product image where the similarity is equal to or less than the second threshold is a mismatch point, and acquires an image including the mismatch point. When a mismatch point is found in the authenticated product image, the image generation unit 134 may acquire the entire authenticated product image, or may cut out a predetermined range in the authenticated product image that includes the mismatch point.

[0121] Next, the image generation unit 134 highlights the mismatched points in the image including the mismatched points. The image generation unit 134 highlights the mismatched points by, for example, surrounding the mismatched points with a line, changing the color of the mismatched points, overlaying a transparent colored layer on the mismatched points, or overlaying a similarity on the mismatched points. In this way, a mismatched points image based on the authenticated product image, i.e., a matching points image in which the mismatched points in the authenticated product image are highlighted, is generated.

[0122] The method for generating a mismatch image based on a registered product image is the same as that described above. Specifically, in the above, the authenticated product image is replaced with the registered product image, and the registered product image is replaced with the authenticated product image. Furthermore, if there are no mismatches between the authenticated product image and the registered product image, no mismatch image is generated. Furthermore, the image generation unit 134 may generate multiple mismatch image for one authenticated product.

[0123] (Step S210) When a mismatch image is generated in step S208, the mismatch reason generation unit 135 generates a mismatch reason for the mismatch point in the generated mismatch image and stores the mismatch reason in the storage unit 12 as appraisal information 123. For example, the mismatch reason generation unit 135 generates a mismatch reason using a machine learning model that outputs a mismatch reason when it receives the mismatch image and a portion of the authenticated product image or registered product image that corresponds to the mismatch image. Alternatively, the mismatch reason generation unit 135 may generate a mismatch reason using a machine learning model that outputs a mismatch reason when it receives the authenticated product image or registered product image that is the basis of the mismatch image and the corresponding authenticated product image or registered product image. In addition, the mismatch reason generation unit 135 may generate mismatch reasons according to conditions set for each mismatch reason, such as "aging if the mismatch point is equal to or greater than a predetermined percentage of the area of ​​the authenticated product image or the registered product image and the variation in the similarity of the mismatch point is within a predetermined range," "repair if the mismatch point matches a repair location in the repair history of the authenticated product information 122," "difference in the shooting environment if the brightness of the mismatch image and the part of the authenticated product image or the registered product image corresponding to the mismatch image differs by more than a predetermined value," "fault in shooting if the blur of the mismatch point image is greater than a predetermined value," and "difference in the item in any other cases."

[0124] (Step S211) The verification device 1 transmits the verification information 123 stored in the storage unit 12 to the user terminal 2 .

[0125] (Step S212) The user operates the user terminal 2 to display the verified information 222 on the user terminal 2. The display unit 233 displays the verified result screen sc4 in response to the user's operation.

[0126] The appraisal result screen sc4 is a screen for displaying the appraisal information 222. FIG. 12 is a diagram showing an example of the appraisal result screen sc4. The appraisal result screen sc4 in FIG. 12 displays bibliographic information sc41 (title, etc.) of the registered item that was the subject of the appraisal of the appraised item, a registered item image sc42 of the registered item (registered item image of the "left side"), the appraisal result sc43 (match), the appraisal reason sc44 (all images were determined to match (similarity is 0.8 or higher), so the appraised item is appraised as matching the registered item), a match point image sc45 (an image in which the similarity is superimposed on the appraised item image of the "signature"), a mismatch point image sc46 (an image in which the similarity is superimposed on the appraised item image of the "right side"), and a mismatch reason sc47 (changes over time).

[0127] <Summary> As described above, according to this embodiment, an appraisal system 1000 can be realized which includes an appraisal unit 132 that appraises whether an appraised product matches a registered product based on a pre-registered registered product image and an appraised product image taken by a user, an image generation unit 134 that generates at least one of a matching point image showing the matching points between the registered product image and the appraised product image and a mismatching point image showing the mismatching points between the registered product image and the appraised product image, and a display unit 223 that displays the appraisal result and at least one of the matching point image and the mismatching point image.

[0128] With this configuration, the viewer of the authentication result can easily understand the basis of the authentication (at least one of the match point image and the mismatch point image). For example, in the example of Figure 12, the viewer of the authentication result can visually understand that the signature of the authenticated item matches that of the registered item, and can therefore feel convinced of the authentication result of "match."

[0129] In addition, by displaying the reason for the discrepancy, the viewer of the appraisal result can be more convinced. For example, in the example of Figure 12, although a discrepancy is found in the image of the "right side" of the authenticated product, the viewer of the appraisal result can be convinced that this is due to the aging of the authenticated product and does not mean that the authenticated product is a counterfeit.

[0130] <Additional Notes> The present embodiment includes the following disclosure.

[0131] (Appendix 1) an appraisal unit that appraises whether an appraised product matches a registered product based on a pre-registered registered product image and an appraised product image taken by a user; an image generating unit that generates at least one of a match point image showing a match point between the registered product image and the authenticated product image, and a mismatch point image showing a mismatch point between the registered product image and the authenticated product image; a display unit that displays the appraisal result and at least one of the match point image and the mismatch point image; An appraisal system comprising:

[0132] (Appendix 2) The appraisal unit appraises whether the appraised product matches the registered product based on the similarity between the registered product image and the appraised product image. The appraisal system described in Appendix 1.

[0133] (Appendix 3) The matching point image is an image in which the matching points in the registered product image or the authenticated product image are highlighted. The appraisal system described in Appendix 1.

[0134] (Appendix 4) The match point image is an image in which the similarity is superimposed on the registered product image or the authenticated product image. The appraisal system described in Appendix 2.

[0135] (Appendix 5) The matching point is a portion in the registered product image or the authenticated product image where the similarity is equal to or greater than a first threshold value. The appraisal system described in Appendix 2.

[0136] (Appendix 6) The mismatch point image is an image in which the mismatch points in the registered product image or the authenticated product image are highlighted. The appraisal system described in Appendix 1.

[0137] (Appendix 7) The mismatch point image is an image in which the similarity is superimposed on the registered product image or the authenticated product image. The appraisal system described in Appendix 2.

[0138] (Appendix 8) The mismatch point is a portion in the registered product image or the authenticated product image where the similarity is equal to or less than a second threshold value. The appraisal system described in Appendix 2.

[0139] (Appendix 9) a mismatch reason generating unit that generates a mismatch reason that is a reason why a mismatch point occurs in the mismatch image; The display unit displays the reason for the discrepancy together with the appraisal result. The appraisal system described in Appendix 1.

[0140] (Appendix 10) The reasons for the discrepancies include at least one of the following: aging, repairs, differences in the shooting environment, imperfections in the photography, and differences in the items. The appraisal system described in Appendix 7.

[0141] (Appendix 11) further comprising an appraisal reason generation unit that generates a text explaining the reason for the appraisal; The display unit displays the reason for the appraisal together with the appraisal result. The appraisal system described in Appendix 1.

[0142] (Appendix 12) The display unit displays the registered product image as a guide when photographing the appraised product. The appraisal system described in Appendix 1.

[0143] (Appendix 13) An appraisal method executed by an appraisal system, an appraisal step of appraising whether the appraised product matches the registered product based on a pre-registered registered product image and an appraised product image taken by a user; an image generating step of generating at least one of a match point image showing a match point between the registered product image and the authenticated product image and a mismatch point image showing a mismatch point between the registered product image and the authenticated product image; a display step of displaying the identification result and at least one of the match point image and the mismatch point image; Appraisal methods including.

[0144] (Appendix 14) The appraisal system, an appraisal step of appraising whether the appraised product matches the registered product based on a pre-registered registered product image and an appraised product image taken by a user; an image generating step of generating at least one of a match point image showing a match point between the registered product image and the authenticated product image and a mismatch point image showing a mismatch point between the registered product image and the authenticated product image; a display step of displaying the identification result and at least one of the match point image and the mismatch point image; An appraisal program that executes the above.

[0145] The embodiments disclosed herein are illustrative in all respects and should not be considered limiting. The scope of the present invention is defined by the claims, not by the above meaning, and is intended to include all modifications within the meaning and scope of the claims. Furthermore, the present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention. [Explanation of symbols]

[0146] 1:Identification device 2: User terminal 3:Registrant terminal 121:Registered product information 122: Appraisal product information 123: Appraisal information 131: Acquisition Department 132: Appraisal Department 133: Appraisal Reason Generation Department 134:Image Generation Department 135: Inconsistency Reason Generation Department

Claims

1. an appraisal unit that appraises whether an appraised product matches a registered product based on a pre-registered registered product image and an appraised product image taken by a user; an image generating unit that generates at least one of a match point image showing a match point between the registered product image and the authenticated product image, and a mismatch point image showing a mismatch point between the registered product image and the authenticated product image; a display unit that displays the appraisal result and at least one of the match point image and the mismatch point image; An appraisal system comprising:

2. The appraisal unit appraises whether the appraised product matches the registered product based on the similarity between the registered product image and the appraised product image. The appraisal system according to claim 1 .

3. The matching point image is an image in which the matching points in the registered product image or the authenticated product image are highlighted. The appraisal system according to claim 1 .

4. The match point image is an image in which the similarity is superimposed on the registered product image or the authenticated product image. The appraisal system according to claim 2 .

5. The matching point is a portion in the registered product image or the authenticated product image where the similarity is equal to or greater than a first threshold value. The appraisal system according to claim 2 .

6. The mismatch point image is an image in which the mismatch points in the registered product image or the authenticated product image are highlighted. The appraisal system according to claim 1 .

7. The mismatch point image is an image in which the similarity is superimposed on the registered product image or the authenticated product image. The appraisal system according to claim 2 .

8. The mismatch point is a portion in the registered product image or the authenticated product image where the similarity is equal to or less than a second threshold value. The appraisal system according to claim 2 .

9. a mismatch reason generating unit that generates a mismatch reason that is a reason why a mismatch point occurs in the mismatch image; The display unit displays the reason for the discrepancy together with the appraisal result. The appraisal system according to claim 1 .

10. The reasons for the discrepancies include at least one of the following: aging, repair, differences in the photography environment, imperfections in photography, and differences in the items. The appraisal system according to claim 7.

11. further comprising an appraisal reason generation unit that generates a text explaining the reason for the appraisal; The display unit displays the reason for the appraisal together with the appraisal result. The appraisal system according to claim 1 .

12. The display unit displays the registered product image as a guide when photographing the appraised product. The appraisal system according to claim 1 .

13. An appraisal method executed by an appraisal system, an appraisal step of appraising whether the appraised product matches the registered product based on a pre-registered registered product image and an appraised product image taken by a user; an image generating step of generating at least one of a match point image showing a match point between the registered product image and the authenticated product image and a mismatch point image showing a mismatch point between the registered product image and the authenticated product image; a display step of displaying the identification result and at least one of the match point image and the mismatch point image; Appraisal methods including.

14. The appraisal system, an appraisal step of appraising whether the appraised product matches the registered product based on a pre-registered registered product image and an appraised product image taken by a user; an image generating step of generating at least one of a match point image showing a match point between the registered product image and the authenticated product image and a mismatch point image showing a mismatch point between the registered product image and the authenticated product image; a display step of displaying the identification result and at least one of the match point image and the mismatch point image; An appraisal program that executes the above.

Images