Information processing device, information processing method, program

Abstract

Determine whether the identification code obtained from the user is legitimate. [Solution] The information processing device includes a receiving unit that receives identification information based on an identification code read by the user's terminal device and an image of the area surrounding the identification code, which is captured by the terminal device, from the terminal device, and a determination unit that determines whether the identification information is valid based on the identification information and the image of the area surrounding the identification code, and determines that the identification information is valid if the image of the area surrounding the identification code satisfies predetermined conditions.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] Patent Document 1 describes a method of acquiring information related to a product using an image captured by a user's camera of a receipt issued at a store.

Prior Art Document

Patent Document

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In order to obtain more detailed information related to a product from a user, it may be necessary to obtain the information of the barcode displayed on the product together with the receipt.

[0005] However, a user may attempt to illegally register barcode information using a barcode that has been generated or forged (hereinafter also simply referred to as "forged") instead of the barcode actually displayed on the product.

[0006] Therefore, an object of the present invention is to determine whether an identification code such as a barcode obtained from a user is legitimate.

Means for Solving the Problems

[0007] An information processing device according to one aspect of the present invention comprises a receiving unit that receives identification information based on an identification code read by a user's terminal device and an image of the area surrounding the identification code, including the area around the identification code, taken by the terminal device, from a terminal device; and a determination unit that determines whether the identification information is valid based on the identification information and the image of the area surrounding the identification code, and determines that the identification information is valid if the image of the area surrounding the identification code satisfies predetermined conditions.

[0008] An information processing method according to one aspect of the present invention includes the steps of receiving identification information based on an identification code read by a user's terminal device and an image of the area surrounding the identification code, which is captured by the terminal device, from the terminal device, and determining whether the identification information is valid based on the identification information and the image of the area surrounding the identification code, wherein the method determines that the identification information is valid if the image of the area surrounding the identification code satisfies predetermined conditions.

[0009] A program according to one aspect of the present invention involves causing a computer to perform the following steps: receiving identification information based on an identification code read by the user's terminal device and an image of the area surrounding the identification code, which includes the area around the identification code and was captured by the terminal device; and determining whether the identification information is valid based on the identification information and the image of the area surrounding the identification code, wherein the computer determines that the identification information is valid if the image of the area surrounding the identification code satisfies predetermined conditions. [Effects of the Invention]

[0010] According to the present invention, it is possible to determine whether or not an identification code obtained from a user is legitimate. [Brief explanation of the drawing]

[0011] [Figure 1] Figure 1 shows an example of the system configuration of the information processing system in this embodiment. [Figure 2]Figure 2 shows an example of the hardware configuration of the information processing device and terminal device in this embodiment. [Figure 3] Figure 3 shows an example of the functional block configuration of the information processing device in this embodiment. [Figure 4] Figure 4 shows an example of an image information database and a reference image information database in this embodiment. [Figure 5] Figure 5 shows an example of the functional block configuration of the terminal device in this embodiment. [Figure 6] Figure 6 shows an example of a product and identification code in this embodiment. [Figure 7] Figure 7 is a schematic diagram showing an example of the processing flow of the information processing device in this embodiment. [Figure 8] Figure 8 is a schematic diagram showing another example of the processing flow of the information processing device in this embodiment. [Figure 9] Figure 9 shows an example of a screen displayed on the terminal device in this embodiment. [Figure 10] Figure 10 is a flowchart of the processing procedure of the information processing device in this embodiment. [Modes for carrying out the invention]

[0012] Embodiments of the present invention will be described below with reference to the attached drawings. In each drawing, components denoted by the same reference numerals have the same or similar configuration. Furthermore, the following embodiments are illustrative examples for explaining the present invention and are not intended to limit the present invention to these embodiments only. In addition, for the sake of ease of understanding, embodiments in which the present invention is realized using the information processing device 10 will be described as examples, but as stated above, the present invention is not limited thereto. Furthermore, those skilled in the art can adopt embodiments in which each of the elements described below is replaced with equivalent components, and such embodiments are also included within the scope of the present invention.

[0013] <System Configuration> FIG. 1 is a diagram showing an example of the system configuration of the information processing system 1 according to the present embodiment. The information processing system 1 shown in FIG. 1 includes an information processing device 10 and a plurality of terminal devices 20. The information processing device 10 and the terminal devices 20 are communicably connected to each other via a network N.

[0014] The information processing device 10 receives, from the terminal device 20, identification information based on an identification code 32 read by the user's terminal device 20 and a peripheral image (hereinafter also referred to as “identification code peripheral image 33”) including the periphery of the identification code 32 photographed by the terminal device 20. Then, the information processing device 10 determines whether the identification information is valid based on the identification information and the identification code peripheral image 33 (hereinafter also referred to as “determination process”). The information processing device 10 determines that the identification information is valid when the identification code peripheral image 33 satisfies a predetermined condition. The determination process includes, for example, determining the presence or absence of forgery of the identification code 32 received by the information processing device 10 without the intervention of a human judgment such as visual inspection, and can determine that the identification information is valid regardless of whether it is before or after the purchase of the product 30.

[0015] The terminal device 20 is, for example, a mobile phone terminal (including a smartphone), a tablet, or a personal computer. The user can execute reading of the identification code 32 according to the present embodiment, photographing of the identification code peripheral image 33, etc. by operating the terminal device 20. The terminal device 20 may notify the information processing device 10 of its own position detected using GPS (Global Positioning System) or the like.

[0016] The network N is a network for communication between the information processing device 10 and the terminal devices 20. For example, the network N may be any of the Internet, an intranet, a LAN, a mobile communication network, a dedicated line, a packet communication network, a telephone line, a corporate internal network, other communication lines, combinations thereof, etc. Also, the network N may be wired or wireless.

[0017] <Hardware Configuration> FIG. 2 is a diagram showing an example of the hardware configuration of the information processing apparatus 10 and the terminal apparatus 20 in the present embodiment. The information processing apparatus 10 and the terminal apparatus 20 include a processor 11 such as a CPU (Central Processing Unit) and a GPU (Graphical Processing Unit), a memory, a storage device 12 such as an HDD (Hard Disk Drive) and / or an SSD (Solid State Drive), a communication IF (Interface) 13 that performs wired or wireless communication, an input device 14 that receives input operations, and an output device 15 that outputs information. The input device 14 is, for example, a keyboard, a touch panel, a mouse, and / or a microphone, etc. The output device 15 is, for example, a display, a touch panel, and / or a speaker, etc.

[0018] <Functional Block Configuration> (Information Processing Apparatus 10) FIG. 3 is a diagram showing an example of the functional block configuration of the information processing apparatus 10 in the present embodiment. The information processing apparatus 10 includes a storage unit 100 and a control unit 110. The storage unit 100 can be realized by using the storage device 12 included in the information processing apparatus 10. Also, the control unit 110 can be realized by the processor 11 of the information processing apparatus 10 executing a program stored in the storage device 12. Further, the program can be stored in a storage medium. The storage medium storing the program may be a non-transitory computer readable medium (Non-transitory computer readable medium). The non-transitory storage medium is not particularly limited, but may be, for example, a storage medium such as a USB memory or a CD-ROM.

[0019] The control unit 110 provides various functions necessary for executing determination processing. The control unit 110 includes, for example, a reception unit 111, an acquisition unit 112, a determination unit 113, an output unit 114, and a guide frame display unit 115.

[0020] The receiving unit 111 receives identification information based on the identification code 32 read by the user's terminal device 20, and an image 33 of the area surrounding the identification code, which was captured by the terminal device 20, from the terminal device 20.

[0021] The acquisition unit 112 acquires a reference image corresponding to the identification information received by the receiving unit 111 from the reference image information DB 102.

[0022] The determination unit 113 determines whether the identification information is valid based on the identification information received from the receiving unit 111 and the image 33 surrounding the identification code. For example, the determination unit 113 determines that the identification information is valid if the similarity between the image 33 surrounding the identification code and the correct image satisfies predetermined conditions. The determination unit 113 may also have an external system perform the determination of whether the identification information is valid. For example, the determination unit 113 may send the identification information received by the receiving unit 111 and the image 33 surrounding the identification code to an external image determination system (not shown), and determine whether the identification information is valid based on the determination result received from the image determination system.

[0023] The output unit 114 outputs the result determined by the determination unit 113. The output unit 114 may output, for example, the score calculated by the determination unit 113.

[0024] The guide frame display unit 115 displays the identification code guide frame 40 and / or the identification code surrounding image guide frame 41 on the screen of the terminal device 20 when reading the identification code 32 and / or capturing the identification code surrounding image 33.

[0025] The storage unit 100 stores data necessary for the information processing device 10 to perform a determination process. This data includes various databases, such as an image information database (image information DB) 101 and a reference image information database (reference image information DB) 102.

[0026] Figure 4 shows an example of the image information DB 101 and the reference image information DB 102 in this embodiment. In the example shown in Figure 4, the image information DB 101 includes, for example, an image information ID that identifies a record in the image information DB 101, a user ID that identifies a user, an identification information ID that indicates acquired identification information, an identification code surrounding image ID that indicates an acquired image surrounding the identification code, reception date and time information that indicates the date and time of reception, and determination result information that indicates the result of the determination. The image information DB 101 stores information received from the user, including images received by the information processing device 10.

[0027] Furthermore, the reference image information DB102 includes, for example, a reference image information ID that identifies a record in the reference image information DB102, a reference image identification information ID that identifies reference image identification information, and a reference image ID that identifies a reference image.

[0028] A reference image is an image that is referenced when performing a judgment process. For example, a reference image is used to determine whether the received identification code surrounding image 33 has been illegally tampered with. Reference image identification information is identification information corresponding to the reference image. Note that there may be multiple reference images corresponding to the reference image identification information.

[0029] The reference images stored in the reference image information DB102 may be, for example, images obtained from actual products, or images of identification code surrounding images 33 that have not been illegally altered. If the received identification code surrounding images 33 are determined to be legitimate, the identification information read from the received identification code surrounding images 33 and the corresponding identification code 32 may be stored in the reference image information DB102 as the reference image and the corresponding reference image identification information.

[0030] Furthermore, the items in Image Information DB101 and Reference Image Information DB102 are not limited to the above items. For example, Image Information DB101 may store images of product 30 received from the user. This allows for obtaining more appropriate information from the user. Preferably, the image of product 30 is an image that uniquely identifies product 30. In addition, the user's location information at the time the image was sent, product 30 evaluation information, etc., may also be included as items in Image Information DB101.

[0031] The image information DB 101 may, for example, store information obtained by the determination unit 113 after it has analyzed the received identification code surrounding image 33. The analysis results of the identification code surrounding image 33 include, for example, color information, character information, shape information, and / or pattern information analyzed from the identification code surrounding image 33. Through the above processing, information from the identification code surrounding image 33 can be collected more accurately.

[0032] Similarly, in the reference image information DB102, for example, the determination unit 113 may analyze the reference image and store the information of the analysis results. The analysis results of the reference image include, for example, color information, character information, shape information, and / or pattern information analyzed from the reference image. This allows for more accurate collection of information about the reference image.

[0033] Furthermore, the function of analyzing the received identification code surrounding image 33 and / or reference image is not limited to the determination unit 113. For example, the information processing device 10 may cooperate with an external system, such as an image analysis system, to obtain analysis result information from the image analysis system and store it in the image information DB 101 and / or reference image information DB 102.

[0034] Furthermore, the image information DB 101 may store, for example, information about the background image obtained by the determination unit 113 removing the identification code 32 portion from the received identification code surrounding image 33. The information about the background image of the identification code surrounding image 33 includes, for example, color information, character information, shape information, and / or pattern information analyzed from the background image of the identification code surrounding image 33. Through the above processing, information about the identification code surrounding image 33 can be collected more accurately.

[0035] Similarly, the reference image information DB102 may store information about the background image obtained by, for example, the determination unit 113 removing the identification code portion from the reference image. The information about the background image of the reference image includes, for example, color information, character information, shape information, and / or pattern information analyzed from the background image of the reference image. Through the above processing, information about the reference image can be collected more accurately.

[0036] Furthermore, the function of removing the identification code portion from the received identification code surrounding image 33 and / or reference image and analyzing the background is not limited to the determination unit 113. For example, the information processing device 10 and the image analysis system may cooperate, and the analysis result information may be obtained from the image analysis system and stored in the image information DB 101 and / or reference image information DB 102.

[0037] The database is not limited to the image information DB101 and the reference image information DB102; for example, it may also include a user DB103. The user DB103 registers user IDs associated with attribute information, etc. Attribute information includes, for example, the user's age, gender, occupation, address, and / or date of birth. It is preferable that the user registers this attribute information, for example, when installing the application on the terminal device 20.

[0038] (Terminal device 20) Figure 5 shows an example of the functional block configuration of the terminal device 20 in this embodiment. The terminal device 20 includes a storage unit 200 and a control unit 210. The storage unit 200 can be implemented using a storage device 12 provided by the terminal device 20. The control unit 210 can be implemented by the processor 11 of the terminal device 20 executing a program stored in the storage device 12. This program can be stored in a storage medium. The storage medium on which the program is stored may be a non-transitory computer-readable medium. The non-transitory storage medium is not particularly limited, but may be a storage medium such as a USB memory or CD-ROM.

[0039] The control unit 210, in cooperation with the information processing device 10, provides various functions necessary to determine, for example, whether an identification code such as a barcode obtained from a user is legitimate. For example, the control unit 210 provides functions such as obtaining various information (image data, text data, etc.) from the information processing device 10 for drawing on the screen of the terminal device 20. The control unit 210 includes, for example, a communication unit 211, a UI (User Interface) unit 212, and an imaging unit 213.

[0040] The memory unit 200 stores various programs and data necessary for the control unit 210 to perform this information processing.

[0041] The communication unit 211 has the function of performing various types of communication with the information processing device 10 using the communication IF 13.

[0042] The UI unit 212 has a function to receive various inputs from the user and to display a predetermined screen on the display when reading, for example, an identification code 32. The UI unit 212 also displays an identification code guide frame 40, etc., on the output device 15 (display) of the terminal device 20 according to the instructions of the information processing device 10.

[0043] The imaging unit 213 has the function of performing various types of imaging, such as reading identification codes 32, including one-dimensional codes and two-dimensional codes. One-dimensional codes include, for example, barcodes, and two-dimensional codes include, for example, QR codes (registered trademarks). The imaging unit 213 can also capture still images and videos. The imaging unit 213 can capture an image 33 of the area surrounding the identification code 32, within a predetermined range that includes the identification code 32. The imaging unit 213 can read barcodes and QR codes from the image of the identification code 32 included in the image 33 of the area surrounding the identification code.

[0044] Furthermore, the imaging unit 213 can be operated by the user to capture an image of the entire product 30 or an image showing a part of the product 30. The terminal device 20 may, for example, transmit the captured image of the entire product 30 and the user ID to the information processing device 10.

[0045] In this embodiment, an example is described in which the identification code 32 read by the terminal device 20 and the image 33 surrounding the identification code that has been captured are transmitted to the information processing device 10 each time, but the invention is not limited to this. For example, the terminal device 20 may, once it has finished capturing an image of the entire product 30, send the identification information obtained from the identification code 32, the image 33 surrounding the identification code, and the image of the entire product 30 together to the information processing device 10.

[0046] Users of terminal device 20 may, for example, enable the determination processing of information processing device 10 by installing a predetermined application (hereinafter also referred to as "app") on terminal device 20. By installing the app and registering as a user, a user ID is assigned to the user. The user ID may be the terminal ID of terminal device 20, a mobile phone number, an email address, etc.

[0047] A user who has installed the app can, for example, when purchasing product 30, operate the terminal device 20 to read the identification code 32 displayed on product 30 and take an image 33 of the area around the identification code on product 30. The terminal device 20 may also transmit the read identification code 32, the captured image 33 of the area around the identification code, and the user ID to the information processing device 10 at any time, for example, when the identification code 32 is read or when the image 33 of the area around the identification code is taken.

[0048] The information processing device 10 and / or terminal device 20 may consist of one or more information processing devices or computers, or they may be configured using a virtual server (such as a cloud server). Regarding the functional block configuration described above, the various processes according to this embodiment may be executed by the information processing device 10, by the terminal device 20, or by the information processing device 10 and the terminal device 20 working together.

[0049] <Product 30> Figure 6 shows an example of product 30 and identification code 32 in this embodiment. As shown in Figure 6, for example, an identification code 32 such as a barcode is displayed on the front of product 30. The location where the identification code 32 is displayed on product 30 is not limited to the front; the identification code 32 may be displayed anywhere on product 30, such as the side or back of product 30.

[0050] The identification code surrounding image 33 is an image that includes at least a portion of the area surrounding the identification code 32. As shown in Figure 6, the identification code surrounding image 33 may be, for example, an image that includes region A. Region A is an area that includes the entire identification code 32 and the surrounding area of ​​the identification code 32. Alternatively, the identification code surrounding image 33 may be, for example, an image that surrounds the identification code 32 by x centimeters (where x is any number). The identification code surrounding image 33 does not have to surround the entire area surrounding the identification code 32; it may be an image that includes a portion of the area surrounding the identification code 32. Furthermore, the identification code surrounding image 33 may be an image that includes characters present around the identification code 32 (for example, the product name A61 of product 30 or the material name A62 of the product 30's packaging).

[0051] Figure 7 is a schematic diagram showing an example of the processing flow of the information processing device 10 in this embodiment. In this embodiment, the identification code 32 is described as a barcode. Note that the identification code 32 is not limited to a barcode displayed on an item, but may also be an identification code 32 installed in a predetermined environment, for example, a QR code for payment installed in front of a cash register in a store.

[0052] For example, the user operates the terminal device 20 and takes a picture of the area around the barcode displayed on the product 30, thereby reading the identification information and taking a picture of the area around the identification code 33. The terminal device 20 transmits the read identification information and the captured picture of the area around the identification code 33 to the information processing device 10, and the receiving unit 111 of the information processing device 10 receives the identification information and the picture of the area around the identification code 33.

[0053] Alternatively, instead of reading identification information from the barcode, the terminal device 20 may take an image 33 of the area surrounding the identification code and transmit it to the information processing device 10. The information processing device 10 may then read the identification information from the barcode contained in the image taken by the terminal device 20.

[0054] Furthermore, the receiving unit 111 may receive, for example, a user ID, an overall image of the product 30, location information, etc., from the terminal device 20. If location information or an overall image of the product 30, etc., is received, this information may be associated with the received identification information, etc., and stored in the image information DB 101.

[0055] The acquisition unit 112 may acquire one or more reference images from the reference image information DB 102 that correspond to the identification information received by the receiving unit 111. If the acquisition unit 112 acquires multiple reference images, it may acquire any one reference image or multiple reference images. The reference image information DB 102 stores, for example, surrounding reference images containing the identification code, collected from the product on which the identification code is actually displayed or the environment in which it is installed, in association with reference image identification information based on the identification code contained in the reference image.

[0056] The determination unit 113 determines whether the identification information is valid based on the received identification information and the image 33 surrounding the identification code. For example, the determination unit 113 determines that the identification information is valid if the image 33 surrounding the identification code satisfies predetermined conditions.

[0057] Furthermore, the determination unit 113 may compare, for example, the identification code surrounding image 33 with one or more reference images acquired by the acquisition unit 112. The determination unit 113 may determine that the received identification information is valid if the identification code surrounding image 33 has a predetermined degree of similarity with at least one reference image.

[0058] Furthermore, the determination unit 113 may, for example, extract features from the received identification code surrounding image 33. Any known method can be arbitrarily adopted as the feature extraction method. For example, features may be extracted using a feature extractor that retains spatial information, such as a CNN (convolutional neural network).

[0059] Furthermore, the determination unit 113 may extract feature quantities of one or more reference images from the reference image information DB 102 based on the reference image identification information corresponding to the identification information received by the receiving unit 111. Any known method can be arbitrarily adopted as the feature quantity extraction method. In addition, the determination unit 113 may extract feature quantities of the identification code surrounding image 33 from the image information DB 101 in addition to, or instead of, the reference images.

[0060] Furthermore, the determination unit 113 may calculate similarity based on the features of the received identification code surrounding image 33 and the features of the reference image. Similarity may be calculated using a statistical similarity method, or it may be calculated using an approximate method such as clustering. For example, the determination unit 113 may calculate the similarity between the identification code surrounding image 33 and the reference image using statistical similarity methods such as KL distance, PE distance, optimal transport, and independence test.

[0061] Furthermore, the determination unit 113 may calculate the similarity in a range of, for example, 0 to 100%, and a higher similarity indicates that the received identification code surrounding image 33 and the reference image are similar.

[0062] Furthermore, if the calculated similarity is greater than a predetermined threshold, the determination unit 113 may determine that the image surrounding the identification code 33 and the reference image are similar and that the identification information is valid. Alternatively, if the calculated similarity is less than a predetermined threshold, the determination unit 113 may determine that the image surrounding the identification code 33 and the reference image are dissimilar and that the identification information is invalid.

[0063] Furthermore, the determination unit 113 compares the identification code surrounding image 33 with one or more reference images and can determine that the identification information is valid if the identification code surrounding image 33 has a predetermined degree of similarity with at least one reference image. The determination unit 113 may also store the result of determining whether the received identification code 32 is an image that has been illegally altered in the image information DB 101.

[0064] The output unit 114 outputs the result determined by the determination unit 113. The output unit 114 may, for example, output the similarity calculated by the determination unit 113 in a range of, for example, 0 to 100%. Alternatively, the output unit 114 may output the determination result to the terminal device 20.

[0065] The output unit 114 may, for example, output a message to the terminal device 20 such as "This image may have been illegally altered" if the determination unit 113 determines that the identification information is fraudulent.

[0066] Based on the above processing, the determination unit 113 can determine whether the identification code 32 received by the receiving unit 111 is legitimate or not. That is, using the read identification code 32 and the captured image 33 of the area surrounding the identification code, it can determine whether the barcode or other identification code obtained from the user is an image that has been fraudulently generated or altered.

[0067] Figure 8 is a schematic diagram showing another example of the processing flow of the information processing device 10 in this embodiment. In this embodiment, the learning model G may use training data stored in the image information DB 101, which includes previously received identification information and images 33 surrounding the identification code, as well as determination results indicating whether or not the previously received identification information is valid.

[0068] The determination unit 113 may input the identification information received by the receiving unit 111 and the image 33 surrounding the identification code to the learning model G, and determine whether the identification information is valid based on the output from the learning model G.

[0069] As an example of training data, for instance, in the image 33 surrounding the identification code, the area around the identification code 32 may contain not only color information such as white and black, but also text information, pattern information, etc. Furthermore, the identification code 32 for the said image 33 may contain a determination result that it has not been illegally tampered with.

[0070] The learning model G is input with an image 33 surrounding the identification code 32 that contains no text information, pattern information, etc., but only color information such as white or black, and the corresponding identification information. The learning model G is trained to output a result indicating that the identification information has been fraudulently altered, and for example, the learning model G can be generated as a mathematical model.

[0071] The generated learning model G can determine whether the identified information is legitimate or not. The learning model G may also be trained to determine whether a portion of an image has been fraudulently altered.

[0072] Furthermore, the learning model G may calculate a score in the range of 0 to 100%, and a higher score may mean that the image 33 surrounding the identification code is less likely to be an image that has been fraudulently altered.

[0073] The determination unit 113 may input the identification information received by the receiving unit 111 and the image 33 surrounding the identification code to the learning model G, and determine whether the identification information is valid based on the output from the learning model G.

[0074] The determination unit 113 may, for example, determine whether the identification information is valid based on the score output by the learning model G. The determination unit 113 may determine that the identification information is invalid if the score is below a predetermined threshold. The determination unit 113 may also determine that the identification information is valid if the score is greater than a predetermined threshold. The determination unit 113 may store the result of determining whether the received identification code 32 corresponding to the identification information is an image that has been illegally altered in the image information DB 101.

[0075] The output unit 114 outputs the result determined by the judgment unit 113. The output unit 114 may, for example, output a score that the judgment unit 113 has output, for example, in the range of 0 to 100%. The output unit 114 may also output the judgment result to the terminal device 20.

[0076] Through the above process, the read identification information and the captured image of the identification code surrounding the image 33 are input into the learning model G, which has learned the relationship between previously received identification information, the image of the identification code surrounding the image 33, and the judgment result. The judgment unit 113 can use the learning model G to determine whether or not the identification information is valid. As a result, it becomes possible to acquire information including the image of the product 30 with greater accuracy.

[0077] Furthermore, the learning model G may use training data that includes, instead of image information DB101, reference images containing the identification code and reference image identification information based on the identification code contained in the reference image, which are collected from products on which the identification code is actually displayed or from the environment on which the identification code is actually installed, and which are stored in reference image information DB102.

[0078] Furthermore, the determination unit 113 is not limited to similarity or determination methods using the learning model G, but may also determine whether the identification information is valid or not using a rule-based method.

[0079] For example, suppose that in the image 33 surrounding the identification code, there is no text information, pattern information, etc., around the identification code 32, and more than 95% of the entire image 33 consists only of color information such as white or black. Then, a rule may be created to determine that the identification information corresponding to the image 33 surrounding the identification code is fraudulent. The determination unit 113 may determine whether or not the identification information is legitimate based on the created rule.

[0080] For example, if data showing distortion caused by illegally altering an image is detected, a rule may be created to determine that the image 33 surrounding the identification code is an image that has been illegally altered. If the determination unit 113 detects data showing distortion caused by illegally altering an image, it may determine that the identification information is invalid based on the created rule.

[0081] Furthermore, the determination unit 113 may determine whether the identification information is valid based, for example, on both the similarity determination method and the learning model G determination method. The determination unit 113 may also determine that the identification information is invalid if it is determined that the identification information is invalid based on at least one of the determination results.

[0082] Alternatively, if both the similarity and the learning model G determination methods determine that the identification information is invalid, then the system may determine that the identification information is invalid. The determination methods can be combined in any way, including similarity, the learning model G, and rule-based methods. Combining these methods can improve the accuracy of determining whether the identification information is valid or not.

[0083] <3. Screen example> Next, Figure 9 is a diagram showing an example of a screen displayed on the terminal device 20 in this embodiment. The guide frame display unit 115 may, for example, display an identification code guide frame 40 that defines a predetermined range on the terminal device 20 when the terminal device 20 reads the identification code 32. The identification code guide frame 40 is a guide frame that supports the placement of the identification code 32 within the readable area of ​​the UI unit 212, which is readable by the user operating the terminal device 20.

[0084] By displaying the identification code guide frame 40 on the terminal device 20, the user can easily operate the terminal device 20 to place the identification code 32 within the identification code guide frame 40. The user can then use the terminal device 20 to assist in reading the identification code 32.

[0085] Furthermore, the guide frame display unit 115 may cause the terminal device 20 to display an identification code surrounding image guide frame 41 that defines a predetermined range when the terminal device 20 captures an identification code surrounding image 33. The identification code surrounding image guide frame 41 is a guide frame for positioning the identification code surrounding image 33 within the captureable area of ​​the UI unit 212, which is the range that the terminal device 20 can capture.

[0086] By displaying the identification code surrounding image guide frame 41 on the terminal device 20, the user can operate the terminal device 20 so that the identification code surrounding image 33 is placed within the identification code surrounding image guide frame 41. The user can easily take the identification code surrounding image 33 using the terminal device 20.

[0087] Through the above processing, the guide frame display unit 115 can support the user in accurately reading the identification code guide frame 40 and / or the identification code surrounding image guide frame 41 on the screen of the terminal device 20. The information processing device 10 can improve the judgment accuracy of the judgment unit 113 by receiving the identification code 32 and / or the identification code surrounding image 33 more accurately from the terminal device 20.

[0088] <System Operation> Next, the operation of the information processing device 10 according to this embodiment will be described. Figure 10 is a flowchart of the processing procedure of the information processing device 10 in this embodiment. Note that the information processing device 10 may perform the following processing before the purchase of product 30 or after the purchase of product 30.

[0089] In step S11, the receiving unit 111 receives identification information read from the identification code 32 displayed on the product 30 by the user's terminal device 20. Similarly, the receiving unit 111 receives an identification code surrounding image 33, which includes the area around the identification code 32, taken by the terminal device 20, from the terminal device 20.

[0090] In step S12, the determination unit 113 determines whether the identification information is valid based on the identification information and the surrounding image 33 of the identification code. The determination unit 113 determines that the identification information is valid if the surrounding image satisfies predetermined conditions.

[0091] For example, the acquisition unit 112 may use a storage unit 100 that stores reference images containing the identification code 32, collected from products in which the identification code 32 is actually displayed, and reference identification information based on the identification code 32 contained in the reference images, in association with each other. The acquisition unit 112 may acquire one or more reference images from the storage unit 100 based on the reference identification information corresponding to the identification information received by the receiving unit 111.

[0092] The determination unit 113 may compare the identification code surrounding image 33 with one or more reference images and determine that the identification information is valid if the identification code surrounding image 33 has a predetermined degree of similarity with at least one reference image.

[0093] Furthermore, the determination unit 113 may input the identification information received by the receiving unit 111 and the image 33 surrounding the identification code to the learning model G. The determination unit 113 may determine whether or not the identification information is valid based on the output from the learning model G.

[0094] The determination unit 113 may store the determination result corresponding to the received identification information and the image 33 surrounding the identification code in the image information DB 101.

[0095] In step S13, the output unit 114 outputs the result determined by the determination unit 113. Note that step S13 is not necessarily required.

[0096] <Variation> The above embodiments or examples are provided to facilitate understanding of the present invention and are not intended to limit it. The present invention can be modified or improved without departing from its spirit, and equivalents thereof are also included. Furthermore, the present invention can form various disclosures by appropriately combining the multiple components disclosed in the above embodiments or examples. For example, some components may be deleted from all the components shown in the embodiments. Furthermore, components may be appropriately combined in different embodiments. Moreover, the present invention can be modified in various ways without departing from its spirit.

[0097] For example, the determination unit 113 may determine whether the identification information is legitimate based on the frequency with which a user transmits an image 33 surrounding the identification code. For example, if the same user transmits an image 33 surrounding the identification code 32 a predetermined number of times in a day (e.g., 100 times), the determination unit 113 may determine that the identification information based on the read identification code is fraudulent. The determination unit 113 may also store in the storage unit the transmission rate of fraudulently altered images for each user.

[0098] The receiving unit 111 may restrict reception from the user terminal device if the transmission rate of images that have been illegally altered by the user exceeds a predetermined threshold.

[0099] The program of the present invention can be installed or loaded onto a computer by downloading it via various recording media such as CD-ROMs, magnetic disks, or semiconductor memory, or via a communication network.

[0100] Furthermore, in this specification, "part" does not merely mean a physical configuration, but also includes cases where the functions of that configuration are realized by software. Also, even if the functions of one configuration are realized by two or more physical configurations, the functions of two or more configurations may be realized by one physical configuration. [Explanation of Symbols]

[0101] 1...Information processing system, 10...Information processing device, 11...Processor, 12...Storage device, 13...Communication interface, 14...Input device, 15...Output device, 20...Terminal device, 30...Product, 32...Identification code, 33...Image surrounding identification code, 40...Identification code guide frame, 41...Identification code surrounding image guide frame, 100...Storage unit, 101...Image information DB, 102...Reference image information DB, 110...Control unit, 111...Receiving unit, 112...Acquisition unit, 113...Determination unit, 114...Output unit, 115...Guide frame display unit, 200...Storage unit, 210...Control unit, 211...Communication unit, 212...UI unit, 213...Imaging unit, G...Learning model

Claims

1. A receiving unit that receives identification information based on an identification code read by the user's terminal device and an image of the area surrounding the identification code, including the area around the identification code, taken by the terminal device, from the terminal device. A determination unit that determines whether the identification information is valid based on the identification information and the image surrounding the identification code, the determination unit that determines the identification information is valid if the image surrounding the identification code satisfies predetermined conditions, An information processing device equipped with the following features.

2. The system further includes an acquisition unit that uses a storage device that stores, in association with a surrounding reference image containing the identification code collected from a product or environment in which the identification code is actually displayed, and reference image identification information based on the identification code contained in the reference image, to acquire one or more reference images corresponding to the identification information received by the receiving unit from the storage device. The determination unit compares the image surrounding the identification code with the one or more reference images and determines that the identification information is valid if the image surrounding the identification code has a predetermined degree of similarity with at least one of the reference images. The information processing apparatus according to claim 1.

3. The determination unit inputs the identification information and the image surrounding the identification code received by the receiving unit into a learning model that has learned the relationship between the previously received identification information and the image surrounding the identification code and the judgment result, using training data that includes previously received identification information and the image surrounding the identification code, as well as a judgment result indicating whether or not the previously received identification information is valid. Based on the output from the learning model, the determination unit determines whether or not the identification information is valid. The information processing apparatus according to claim 1.

4. The system further includes an output unit that outputs the result determined by the determination unit. The information processing apparatus according to claim 1.

5. A terminal device that transmits identification information and an image surrounding an identification code to an information processing device according to claim 1, The aforementioned terminal device is A UI section that displays a guide frame on the screen used when reading the aforementioned identification code, A terminal device equipped with the following features.

6. The steps include receiving identification information based on an identification code read by the user's terminal device, and an image of the area surrounding the identification code, including the area around the identification code, taken by the terminal device, from the terminal device; A step of determining whether the identification information is valid based on the identification information and the image surrounding the identification code, the step of determining that the identification information is valid if the image surrounding the identification code satisfies predetermined conditions, An information processing method performed by an information processing device.

7. The steps include receiving identification information based on an identification code read by the user's terminal device, and an image of the area surrounding the identification code, including the area around the identification code, taken by the terminal device, from the terminal device; A step of determining whether the identification information is valid based on the identification information and the image surrounding the identification code, wherein the step of determining that the identification information is valid if the image surrounding the identification code satisfies predetermined conditions, A program that causes a computer to execute something.

Images