PROGRAM, AUTHENTICITY DETERMINATION METHOD, AND AUTHENTICITY DETERMINATION DEVICE

A program using a terminal device with a camera and machine learning to analyze light reflection patterns on cards determines authenticity effectively, addressing the limitations of existing methods by avoiding additional hardware and environmental dependencies.

JP7594140B1Active Publication Date: 2024-12-03CYGAMES INC

Patent Information

Application Number
JP2024026182
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2024-02-26
Publication Date
2024-12-03
Estimated Expiration
2044-02-26

AI Technical Summary

Technical Problem

Existing methods for determining the authenticity of trading cards, such as using NFC chips, wide-angle microscopes, or holograms, are costly, require additional hardware, or are not suitable for movable cards due to environmental dependencies.

Method used

A program that determines authenticity by processing changes in reflected light patterns from a card's surface using a terminal device with a camera, a guide frame, and machine learning to analyze time-series light reflection data without additional hardware.

Benefits of technology

Enables authenticity determination of cards without additional hardware, preventing increased costs and aesthetic impairment, while being robust to environmental factors and cheating methods.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007594140000001_ABST
    Figure 0007594140000001_ABST
Patent Text Reader

Abstract

To determine the authenticity of a card medium without using an authenticity determining device. [Solution] A program according to one embodiment of the present invention causes a computer to execute the steps of: displaying a guide on the screen of terminal device 2 for varying the shooting angle of a video image of a card captured by camera 26, and changing the display position of the guide at least once; and determining the authenticity of the card based on time-series information of reflected light from a specific surface of the card medium, which is contained in the video image data of the card captured by camera 26 while the guide is displayed.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a program, an authenticity determining method, and an authenticity determining device. [Background technology]

[0002] Conventionally, various techniques have been devised for determining the authenticity of cards used in trading card games. For example, a technique is known in which a Near Field Communication (NFC) chip that stores information for determining authenticity is embedded in a card and the information is read by an NFC tag reader to determine the authenticity of the card. By using this technique, the authenticity of the card can be determined with high accuracy. However, since the manufacturing cost of NFC chips is high, mounting an NFC chip on a card increases the manufacturing cost of the card.

[0003] Also, a technology is known that uses hardware such as a microscope capable of wide-angle photography to determine the authenticity of products that are not marked with markers, etc. Non-Patent Document 1 discloses a technology that uses a machine learning algorithm to determine the authenticity of a product based on an image of the physical product captured by a microscope camera capable of wide-angle photography that is compatible with mobile devices. However, the technology described in Non-Patent Document 1 requires a wide-angle microscope to be connected to a mobile device as a peripheral device, making it difficult for general users to use this technology to determine authenticity at home.

[0004] Also known is a method of judging the authenticity of a product by attaching a marker such as a hologram or a special barcode to the product to be judged and reading the hologram with software. For example, a technology has been put into practical use for theme park tickets, etc., where a hologram is printed on the ticket and the hologram is read with software to authenticate the ticket (see Non-Patent Document 2). This technology is effective when a new product is produced or when the addition of a marker does not detract from the appeal of the product. However, when an aesthetically pleasing card is required and an existing card is to be judged for authenticity, the aesthetics of the card may be marred by the addition of a hologram to the card. [Prior art documents] [Patent documents]

[0005] [Non-Patent Document 1] Ashlesh Sharma, Vidyuth Srinivasan, Vishal Kanchan, and Lakshminarayanan Subramanian. 2017. The Fake vs Real Goods Problem: Microscopy and Machine Learning to the Rescue. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD '17). Association for Computing Machinery, New York, NY, USA, 2011-2019. https: / / doi.org / 10.1145 / 3097983.3098186 [Non-Patent Document 2] Oumayma Kada, Camille Kurtz, Cuong van Kieu, and Nicole Vincent. 2022. Hologram Detection for Identity Document Authentication. In Pattern Recognition and Artificial Intelligence: Third International Conference, ICPRAI 2022, Paris, France, June 1-3, 2022, Proceedings, Part I. Springer-Verlag, Berlin, Heidelberg, 346-357. https: / / doi.org / 10.1007 / 978-3-031-09037-0_29 Summary of the Invention [Problem to be solved by the invention]

[0006] Also, as a method of determining authenticity using software, a technology called "visual check-in" invented by the applicant is known. The visual check-in technology allows users to obtain rewards in a game by taking pictures of TV programs, posters, the exterior of products, etc. with a smartphone camera. When an authenticity determination device using the visual check-in technology captures a template image to be recognized at a predetermined position in a camera image, it recognizes feature points that indicate local features of the image, and uses information on the positional relationship of the feature points to determine the match between the image and the template. This matching method using feature points is less susceptible to environmental factors such as the position of the light source and the color of the lighting, and is not dependent on the model of the camera or the lens characteristics.

[0007] However, current visual check-in technology is designed to capture images of fixed objects such as posters that are installed in specific locations, and is therefore not suitable for authenticating movable cards such as those used in trading card games.

[0008] The present invention has been made in consideration of the above circumstances, and has an object to make it possible to determine the authenticity of a card medium without using an authenticity determining device or the like. [Means for solving the problem]

[0009] The program according to the present invention is a program for determining the authenticity of a card medium having a predetermined surface that has been subjected to processing capable of changing the reflection state of light reflected from the predetermined surface. The program according to the present invention causes a computer to execute a display control procedure for displaying, on a screen of a terminal device, a guide for varying the shooting angle of a moving image of the card medium captured by a camera and changing the display position of the guide at least once, and a determination procedure for determining the authenticity of the card medium based on time-series information on the reflected light from the predetermined surface of the card medium that is included in the moving image data of the card medium captured by the camera with the guide displayed. Effect of the Invention

[0010] According to the present invention, it becomes possible to determine the authenticity of a card medium without using an authenticity determining device or the like. Problems, configurations and effects other than those described above will become apparent from the following description of the embodiments. [Brief description of the drawings]

[0011] [Figure 1] 1 is an overall configuration diagram showing an overview of an authenticity determination system according to an embodiment of the present invention; [Diagram 2] 2 is a block diagram showing an example of the hardware configuration of a server and a terminal device constituting the authenticity determination system according to one embodiment of the present invention. FIG. [Diagram 3] 1 is a block diagram showing a functional configuration of an authenticity determination system according to an embodiment of the present invention. [Figure 4] 11A to 11C are diagrams illustrating an example of a display of a guide frame based on a guide frame display process performed by a guide frame display processing unit according to an embodiment of the present invention. [Diagram 5] 11A to 11C are diagrams illustrating an example of changes in the angle at which a card is photographed by a camera according to an embodiment of the present invention. [Figure 6] 4 is a flowchart showing an example of a procedure for authenticity determination processing by an authenticity determination system according to an embodiment of the present invention. [Figure 7] 10 is a flowchart illustrating an example of a procedure for a guide frame display process by a guide frame display processing unit according to an embodiment of the present invention. [Figure 8] 10 is a flowchart illustrating an example of a procedure of a first determination process performed by a first determination unit according to an embodiment of the present invention. [Figure 9] 10 is a flowchart illustrating an example of a procedure of a second determination process by a second determination unit according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings. In this specification and the drawings, components having substantially the same functions or configurations are designated by the same reference numerals, and redundant description will be omitted.

[0013] [One embodiment] <Outline of the authenticity determination system> FIG. 1 is a diagram showing an overall configuration of an authenticity determination system 100 according to an embodiment of the present invention. The authenticity determination system 100 is a system that determines whether cards (an example of card media) used in a trading card game are genuine (authentic) or counterfeit (non-authentic). The authenticity determination system 100 includes a server 1, a smartphone 2A, and a PC (Personal Computer) 2B. The smartphone 2A and the PC 2B are connectable to the server 1 via a network N such as the Internet. In the following description, the smartphone 2A and the PC 2B are collectively referred to as a terminal device 2. Note that a dedicated gaming terminal may be used as the terminal device 2.

[0014] A card Cd (see FIG. 4) that is the subject of authentication determination by the authenticity determination system 100 is a card with a hologram and embossing (an example of a textured surface) on its surface (an example of a predetermined surface). The terminal device 2 of the authenticity determination system 100 performs authenticity determination of the card based on information obtained by photographing the card with a camera 26 (see FIG. 2) provided in the terminal device 2.

[0015] The authenticity determination system 100 according to this embodiment uses information such as the change in reflected light from the card according to the shooting angle of the camera 26 of the terminal device 2 as information obtained by the camera 26 photographing the card. Furthermore, the terminal device 2 according to this embodiment changes the reflection pattern of the reflected light from the card by displaying on the screen of the display device 30 a guide frame that prompts the user to move the terminal device in each direction such as up, down, left, right, etc. while photographing the card, i.e., a guide frame for changing the shooting angle of the card by the camera 26.

[0016] Then, terminal device 2 performs a first judgment process and a second judgment process as a process for judging the authenticity of the card. In the first judgment, terminal device 2 judges whether camera 26 of terminal device 2 is photographing a physical card or not, using as a key whether a movement vector of 6-axis sensor 27 (see FIG. 3) provided in terminal device 2 and a movement vector of a feature point extracted from an image captured by camera 26 change simultaneously and in the same direction.

[0017] In the second determination, the terminal device 2 determines the authenticity of the card based on information on the change in the reflection pattern of the light reflected from the card over time, which is included in the video recorded when the first determination is performed, that is, based on time-series information on the light reflected from the card. The terminal device 2 performs the second determination using a trained machine learning model 102a (see FIG. 3, hereinafter also referred to simply as the "trained model").

[0018] The server 1 is a device that manages a master image used in the first judgment by the terminal device 2 and a learning model 102a used in the second judgment. The server 1 synchronizes the master image stored in the terminal device 2 with the master image managed by the server 1 by transmitting the master image to the terminal device 2 at a predetermined cycle. In addition, the server 1 synchronizes the learning model stored in the terminal device 2 with the master image managed by the server 1 by transmitting the learning model to the terminal device 2 at a predetermined cycle.

[0019] The terminal device 2 is an example of an authenticity determination device. A non-volatile storage 31 (see FIG. 2) of the terminal device 2 stores an application (not shown) including a program for performing the authenticity determination process according to this embodiment. The control unit 20 (see FIG. 2) of the terminal device 2 starts the application based on a user's operation, and performs a predetermined process by reading the program recorded in the non-volatile storage 31.

[0020] The predetermined process includes the above-mentioned first judgment, second judgment, etc. The predetermined process also includes a process in which the control unit 20 of the terminal device 2 displays, on the screen of the display device 30, an image of a guide frame (hereinafter also simply referred to as a "guide frame") that specifies the display position of the card on the screen of the display device 30. Furthermore, the predetermined process includes a process in which information captured by the camera 26 while the user is performing an action to align the display position of the card with the guide frame is recorded as a moving image.

[0021] The input device 29 generates an operation signal according to an operation input by the user, and supplies the operation signal to the control unit 20 . The display device 30, under the control of the control unit 20, displays a guide frame and the like on the screen.

[0022] <Example of hardware configuration for authenticity determination system> Next, an example of a hardware configuration of an authenticity determination system 100 according to an embodiment of the present invention will be described with reference to Fig. 2. Fig. 2 is a block diagram showing an example of the hardware configuration of the server 1 and the terminal device 2 that constitute the authenticity determination system 100.

[0023] (Server configuration example) The server 1 is an example of a calculator that operates as a computer. The server 1 includes a control unit 10, a non-volatile storage 15, and a network interface 16, which are all connected to a bus B1.

[0024] The control unit 10 includes a central processing unit (CPU) 11, a graphics processing unit (GPU) 12, a read only memory (ROM) 13, and a random access memory (RAM) 14. The control unit 10 may be configured with a field programmable gate array (FPGA).

[0025] The CPU 11 reads out program code of software for realizing each function according to the present embodiment from the ROM 13, loads it into the RAM 14, and executes it. Variables, parameters, etc. generated during the arithmetic processing of the CPU 11 are temporarily written into the RAM 14. These variables, parameters, etc. written into the RAM 14 are read out by the CPU 11 as appropriate. The GPU 12 performs the calculations required to draw an image.

[0026] The non-volatile storage 15 may be, for example, a hard disk drive (HDD), a solid state drive (SSD), a Blu-ray (registered trademark) disk, a flexible disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a magnetic tape, or a non-volatile memory. In addition to an operating system (OS) and various parameters, a program for making the server 1 function is recorded in the non-volatile storage 15. The program for making the server 1 function may be stored in the ROM 12. In other words, the ROM 12 and the non-volatile storage 15 are used as an example of a computer-readable non-transient recording medium that stores a program executed by the server 1.

[0027] Moreover, a master image DB (Database) 101 (see FIG. 3) in which a master image is stored is formed in the non-volatile storage 15. Moreover, a learning model DB 102 (see FIG. 3) in which a learning model 102a is stored is formed in the non-volatile storage 15.

[0028] For example, a network interface card (NIC) is used as the network interface 15. The network interface 15 transmits and receives various data to and from the terminal device 2 via a dedicated line or the like connected to a terminal of the NIC and via the network N.

[0029] (Example of terminal device configuration) The terminal device 2 is an example of a calculator that operates as a computer capable of executing various programs. The terminal device 2 includes a control unit 20, a network interface 25, a camera 26, a six-axis sensor 27, an input / output interface 27, an input device 29, a display device 30, and a non-volatile storage 31, which are all connected to a bus B2.

[0030] The control unit 20 includes a CPU 21, a ROM 23, and a RAM 24. The control unit 20 may include a GPU, or may be configured as an FPGA.

[0031] The CPU 21 reads out program code of software for realizing each function according to the present embodiment from the ROM 23, loads it into the RAM 24, and executes it. Variables, parameters, etc. generated during the arithmetic processing of the CPU 21 are temporarily written into the RAM 24. These variables, parameters, etc. written into the RAM 24 are read out as appropriate by the CPU 21. The CPU 21 performs processes such as processing of the OS in the terminal device 2 and management of input and output of data performed by each part in the terminal device 2. The GPU 22 performs calculations required for drawing an image.

[0032] For example, a NIC or the like is used as the network interface 25. The network interface 25 can obtain various data from the server 1 via a LAN, a dedicated line, or the like connected to a terminal of the NIC, and can communicate with other terminal devices 2.

[0033] The camera 26 includes an image sensor such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor), and converts light of a subject imaged through a lens (not shown) into image data. The camera 26 according to this embodiment is capable of capturing not only still images but also moving images.

[0034] The six-axis sensor 27 is an inertial sensor capable of detecting a total of six-axis inertial forces, including acceleration in three axes, namely forward / backward, left / right, and up / down, and angular velocity in the three axes, acting on the terminal device 2. Information on the detection values ​​by the six-axis sensor 27 is referred to when the above-mentioned first determination is performed. Note that the inertial sensor possessed by the terminal device 2 is not limited to the six-axis sensor 27, and may be any other sensor as long as it is an inertial sensor mounted on the terminal device 2.

[0035] The input / output interface 28 converts an operation signal received from the input device 29 into data of a predetermined format and passes it to the CPU 21. The input / output interface 28 also passes an image of a guide frame or the like output from the control unit 20 to the display device 30. The display device 30 may include a speaker (not shown), in which case the input / output interface 28 can also output an audio signal to the speaker.

[0036] The input device 29 is configured by a touch sensor in a touch panel in the smartphone 2A (see FIG. 1), and is configured by a mouse, keyboard, and the like in the PC 2B.

[0037] The display device 30 is configured with an LCD (Liquid Crystal Display), an organic EL (Electro Luminescence) display, or the like.

[0038] The non-volatile storage 31 may be, for example, an HDD, an SSD, a Blu-ray disc, a flexible disc, an optical disc, a magneto-optical disc, a CD-ROM, a CD-R, or a non-volatile memory. In addition to the OS and various parameters, the non-volatile storage 31 records a program for operating the terminal device 2, a master image DB 101 and a learning model 102a (see FIG. 3) transmitted from the server 1, and the like.

[0039] The program for causing the server 1 to function may be stored in the ROM 23. In other words, the ROM 23 and the non-volatile storage 31 are used as an example of a computer-readable non-transitory recording medium storing a program executed by the terminal device 2.

[0040] When the non-volatile storage 31 is an external recording medium configured with an optical disk or the like, an optical drive (not shown) reads the non-volatile storage 31 and extracts data. The data extracted by the optical drive is used for predetermined arithmetic processing by the CPU 21.

[0041] <Example of functional configuration of authenticity determination system> Next, the functional configuration of the authenticity determining system 100 will be described with reference to FIG. Fig. 3 is a block diagram showing the functional configuration of the authenticity determination system 100. In Fig. 3, the communication network N is omitted.

[0042] (Server functional configuration example) 3, the server 1 includes a master image DB 101 and a learning model DB 102. The master image DB 101 is an image that is referred to when the first determination unit 206 of the terminal device 2 performs the first determination, and stores photographed images of regular (authentic) cards of all types.

[0043] The learning model DB 102 stores a trained learning model 102a constructed by a neural network having time axis information. The learning model 102a is composed of weight data whose weights have been adjusted by learning. For example, 3D-CNN (3-Dimensional Convolutional Neural Network), SlowFast, TimeSformer (Time-Space Transformer), etc. can be applied to the neural network having time axis information.

[0044] For example, when the learning model 102a is constructed using a deep learning framework called PyTorch, the learning model 102a is implemented in a PTH file format. The file format of the learning model 102a may be other file formats such as a GGML file format or an ONNX (Open Neural Network eXchange) file format.

[0045] In addition, the server 1 periodically transmits the master images stored in the master image DB 101 and the learning models 102a stored in the learning model DB 102 to the terminal device 2.

[0046] (Example of terminal device functional configuration) The terminal device 2 includes a local image DB 201, a learning model DB 202, a guide frame display processing unit 203, a moving image shooting unit 204, a card area detection unit 205, a six-axis sensor 27, a first judgment unit 206, a normalization unit 207, and a second judgment unit.

[0047] The local image DB 201 is the master image DB 101 transmitted from the server 1 in a form that can be accessed (referenced) by an application of the terminal device 2. The learning model DB 202 is a database that stores the learning model 102 a transmitted from the server 1 .

[0048] The learning data of learning model 102a is time series information of reflected light from the surface of a card, contained in video data of a genuine (authentic) card photographed by camera 26 based on guidance from a guide frame. Learning model 102a receives as input video data of a card to be judged for authenticity, photographed based on guidance from a guide frame, and outputs a result of authenticity judgment of the card. Learning model 102a judges the authenticity of the card by verifying whether the change in the reflection pattern of reflected light from the card to be judged for authenticity matches the reflection pattern of reflected light from a genuine card.

[0049] The guide frame display processing unit 203 (an example of a display control unit) is a processing unit that displays a guide frame on the screen of the display device 30 (see FIG. 2). The guide frame is indicated, for example, by a thick red frame. When the guide frame display processing unit 203 detects that the frame of the card fits within the guide frame, it displays a sign such as "OK" on the screen. Thereafter, the guide frame display processing unit 203 moves the guide frame to another position within the screen, and prompts the user to perform an operation of fitting the frame of the card to that position. An example of the display of the guide frame by the guide frame display processing unit 203 will be described in detail with reference to FIGS. 4 and 5 described later.

[0050] The video shooting unit 204 encodes image data obtained from the camera 26 as a video at a predetermined frame rate. The video shooting unit 204 stores the shot video in the non-volatile storage 31 in a general-purpose format such as H.264 / MP4. The video shooting unit 204 continues shooting video while the guide frame display processing unit 203 is changing the display position of the guide frame.

[0051] The card area detection unit 205 uses feature (keypoint) matching technology to detect a card area corresponding to a card from image data obtained from the camera 26. The card area detection unit 205 also compares the digital images of all types of cards in the master image DB 101 with the image of the detected card area to determine the type of card being photographed.

[0052] The feature matching technology used by the card area detection unit 205 may be ORB (Oriented FAST and Rotated BRIEF) keypoint matching or other technology. For example, by implementing ORB keypoint matching in parallel using multi-threading in the terminal device 2, it is possible to maintain a sufficient processing speed even when the terminal device 2 is a smartphone 2A.

[0053] In addition, the feature matching technique used by the card area detection unit 205 is not limited to ORB keypoint matching, and may be other techniques such as SIFT (Scale Invariant Feature Transform), SURF (Speed-Up Robust Features), KAZE, and AKAZE (Accelerated-KAZE).

[0054] First determination unit 206 determines whether camera 26 of terminal device 2 is capturing an image of a physical card to be subjected to authenticity determination, based on the movement of six-axis sensor 27 (see FIG. 2) and the movement of key points (one example of a feature point) detected by card area detection unit 205. Specifically, first determination unit 206 determines whether the movement vector of the key points extracted from the video data transmitted from camera 26 and the movement vector acquired from the sensor are moving simultaneously in the same direction.

[0055] By first judgment unit 206 making the first judgment, even if a cheating method (unauthorized operation) regarding the method of photographing a card is executed, the cheating method can be detected. Examples of cheating methods regarding the method of photographing a card include a method of photographing a moving image obtained by photographing the card in advance, playing the moving image in front of camera 26 of terminal device 2, and a method of photographing a high-definition copy image of the card with camera 26.

[0056] Furthermore, the first determination unit 206 generates coordinate information of the guide frame for changing the guide frame to the next display position during execution of the first determination, and outputs the coordinate information to the guide frame display processing unit 203.

[0057] The normalization unit 207 extracts the image of the card area from the moving image captured by the moving image capturing unit 204. Then, the normalization unit 207 normalizes to a fixed length by cropping the number of vertical and horizontal pixels of the image of the extracted area to the size assumed as the input by the learning model 102a. What the normalization unit 207 extracts by cropping is the image area of the card. By performing the normalization process by the normalization unit 207, an image composed only of the image portion of the card can be input to the learning model 102a in an appropriate size for the learning model 102a.

[0058] Also, for the purpose of preventing the top and bottom of the moving image input to the learning model 102a from being reversed, the normalization unit 207 rotates and resizes the image of the extracted area. The normalization unit 207 performs the rotation and resizing of the image while maintaining the aspect ratio of the original moving image as much as possible. The image size after normalization by the normalization unit 207 can be, for example, an image size of about width m pixels × height n pixels (m < n).

[0059] The second determination unit 208 inputs the moving image including the time-series information of the reflected light from the card, which has been normalized by the normalization unit 207, to the learning model 102a, and obtains the authenticity determination result of the card classified into two values of normal (genuine) or non-normal (fake) from the learning model 102a.

[0060] <Display processing of the guide frame> Next, with reference to FIGS. 4 and 5, the display processing of the guide frame by the guide frame display processing unit 203 will be described. FIG. 4 is a diagram showing an example of the display of the guide frame based on the display processing of the guide frame by the guide frame display processing unit 203.

[0061] On the left side of FIG. 4, a state is shown in which the frame of the card Cd is contained within the guide frame Fm displayed on the screen of the display device 30 by the guide frame display processing unit 203. When such a state occurs, that is, when it is detected that the frame of the card Cd is contained within the guide frame Fm, the guide frame display processing unit 203 causes a sign such as "OK" to be displayed on the screen.

[0062] Next, the guide frame display processing unit 203 performs guidance to change the position of the guide frame displayed on the screen to a different position. As shown on the right side of FIG. 4, the guide frame display processing unit 203 shifts the display position of the guide frame Fm on the screen diagonally upward and to the right. The user who sees this display moves the terminal device 2 held by the user to fit the frame of the card Cd into the card frame Fm. By performing such processing by the guide frame display processing unit 203, the position of the terminal device 2 held by the user changes up, down, left, right, etc. in accordance with the change in the display position of the guide frame. In other words, the shooting angle of the card by the camera 26 of the terminal device 2 changes. Note that the number of times the display position of the guide frame is changed by the guide frame display processing unit 203 may be any number of times as long as it is at least once or more.

[0063] Fig. 5 is a diagram showing an example of changes in the shooting angle of a card taken by camera 26. By changing the display position of the guide frame by guide frame display processing unit 203, the shooting angle of the card taken by camera 26 changes to various angles as shown in Fig. 5. Then, video shooting unit 204 records, as a video, images taken by camera 26 in such a state where the shooting angle has changed. By recording, as a video, images taken by camera 26 in a state where the shooting angle has changed, it becomes possible to include in the video shot by video shooting unit 204 a feature that essentially has time axis information, that is, a change in the reflection pattern of reflected light according to the shooting angle.

[0064] The card to be subjected to the authenticity determination by the authenticity determination system 100 according to the present embodiment is a card that has been subjected to fine hologram processing and embossing. The embossing is performed, for example, by printing a transparent varnish on the surface of the card. Therefore, when such a card is photographed, the reflection pattern of the reflected light from the card changes in a fine and diverse manner as the photographing angle of the camera 26 changes. In other words, the learning model 102a according to the present embodiment has learned the fine and diverse changes in the reflection pattern of the reflected light from such a card. Note that the processing on the card surface is not limited to both fine hologram processing and embossing, and may be either one of the processing. Also, the processing on the card surface may be processing other than hologram processing and embossing, such as gold foil stamping, silver foil stamping, and mother-of-pearl processing, as long as it is processing that can change the reflection state of the reflected light from the card surface.

[0065] <Authenticity determination process using the authenticity determination system> Next, an authenticity determination process performed by the authenticity determination system 100 according to this embodiment will be described with reference to Fig. 6. Fig. 6 is a flowchart showing an example of the procedure of the authenticity determination process performed by the authenticity determination system 100.

[0066] First, the guide frame display processing unit 203 (see FIG. 3) executes the guide frame display process (step S1). The guide frame display process in step S1 is started when the user starts an application incorporating a program that performs the authenticity determination process according to this embodiment and selects a menu for determining the authenticity of the card. The contents of the guide frame display process performed in step S1 will be described in detail with reference to FIG. 7 described later.

[0067] Next, shooting by camera 26 and recording of the image data shot by camera 26 as a moving image are started (step S2). The process of step S2 is performed continuously while a guide frame is displayed on the screen of display device 30 in step S1 and the user moves terminal device 2 up, down, left, right, etc. to fit the frame of the card into the guide frame.

[0068] Next, the card area detection unit 205 extracts the card area from the image data captured by the camera 26 in step S2 using a keypoint matching technique (step S3). Next, the first judgment unit 206 performs a first judgment process (step S4). The first judgment process will be described in detail with reference to FIG. 8 described later. Next, the normalization unit 207 normalizes the size of the video recorded in step S2 to a size suitable for input to the learning model 102a (step S5). Next, the second judgment unit 208 performs a second judgment process (step S6). The second judgment process performed in step S6 will be described in detail with reference to FIG. 9 described later.

[0069] (Guide frame display process) Next, the guide frame display processing performed in step S2 in Fig. 6 will be described with reference to Fig. 7. Fig. 7 is a flowchart showing an example of the procedure of the guide frame display processing by the guide frame display processing unit 203.

[0070] First, the guide frame display processing unit 203 displays a guide frame on the screen of the display device 30 (step S11). Next, the guide frame display processing unit 203 determines whether or not the frame of the card included in the image data captured by the camera 26 in step S2 is within the guide frame (step S12). If it is determined that the card does not fit within the guide frame (NO in step S12), the guide frame display processing unit 203 continues to perform the determination in step S12.

[0071] On the other hand, if it is determined that the card has fit within the guide frame (step S12: YES), the guide frame display processing unit 203 displays a sign such as "OK" on the screen to indicate that the card has fit within the guide frame (step S13). Note that the sign indicating that the card has fit within the guide frame is not limited to the word "OK" and may be other characters. Also, the sign may not be text, but may be notified in other forms such as by changing the color or shape of the guide frame, or by emitting a sound.

[0072] Next, the guide frame display processing unit 203 judges whether or not the end condition of the first judgment process by the first judgment unit 206 (see FIG. 3) is satisfied (step S14). The end condition of the first judgment process is that both the judgment result of "success" or "failure" by the first judgment process and the image (moving image) captured by the camera 26 in the case where it is judged as "success" are obtained. The moving image in the case where it is judged as "success" is obtained when the first judgment process for all the guide frames is completed. If it is judged in step S14 that the end condition of the first judgment process is not satisfied (step S14 is NO), the guide frame display processing unit 203 changes the display position of the guide frame on the screen (step S15). After the process of step S15, the guide frame display processing unit 203 returns to step S11 and performs the process.

[0073] On the other hand, if it is determined in step S14 that the end condition of the first determination process is satisfied (YES in step S14), the guide frame display process by the guide frame display processing unit 203 ends. Note that the guide frame display process by the guide frame display processing unit 203 may end at a predetermined timing after the output of the determination result by the first determination process.

[0074] (First judgment process) Next, the first determination process performed in step S4 in Fig. 6 will be described with reference to Fig. 8. Fig. 8 is a flowchart showing an example of the procedure of the first determination process performed by first determination unit 206.

[0075] First, the first determination unit 206 extracts key points from the card area detected in the captured image in step S3 of Fig. 6 (step S41). Next, the first determination unit 206 determines whether or not the movement vector of the key point extracted in step S41 and the movement vector of the six-axis sensor 27 of the terminal device 2 move simultaneously in the same direction (step S42).

[0076] If it is determined that the movement vector of the key point and the movement vector of the six-axis sensor 27 move simultaneously in the same direction (step S42 is YES), the first determination unit 206 determines that the camera 26 is photographing a physical card (step S43). On the other hand, if it is determined that the movement vector of the key point and the movement vector of the six-axis sensor 27 do not move simultaneously in the same direction (step S42 is NO), the first determination unit 206 determines that the camera 26 is photographing an object other than a physical card (step S44). After the process of step S44 or step S45, the first determination process by the first determination unit 206 ends. Note that the determination in step S42 can be made by analyzing the movement directions of both movement vectors, or the like, but can also be made based on the magnitude of the correlation coefficient between both movement vectors, or the like.

[0077] (Second judgment process) Next, the second determination process performed in step S6 in Fig. 6 will be described with reference to Fig. 9. Fig. 9 is a flowchart showing an example of the procedure of the second determination process performed by second determination unit 208.

[0078] First, the second judgment unit 208 inputs the video image of the card area normalized in step S5 of Fig. 6 to the learning model 102a (step S51). Next, the learning model 102a verifies whether or not the change in the reflection pattern of the reflected light obtained from the image of the card area photographed by the camera 26 and normalized matches the change in the reflection pattern of the reflection from a genuine card that has been learned in advance (step S52). Next, the second judgment unit 208 outputs the authenticity judgment result of the card as the result of the verification by the learning model 102a (step S53).

[0079] If the learning model 102a is able to confirm that the change in the reflection pattern of the reflected light obtained from the image of the card area matches the change in the reflection pattern of the reflection from a pre-learned genuine card, it outputs an authenticity determination result that the card photographed by the camera 26 is a genuine (authentic) card. On the other hand, if the learning model 102a is unable to confirm that the change in the reflection pattern of the reflected light obtained from the image of the card area matches the change in the reflection pattern of the reflection from a pre-learned genuine card, it outputs an authenticity determination result that the card photographed by the camera 26 is an unauthorized (counterfeit) card. After the process of step S63, the second determination process by the second determination unit 208 ends.

[0080] In the embodiment described above, the second determination unit 208 of the terminal device 2 determines the authenticity of the card based on time series information of the reflected light from the card, that is, information on changes in the reflection pattern of the reflected light, obtained by the camera 26. Therefore, according to this embodiment, it is possible to determine the authenticity of the card without adding or connecting an authenticity determination device, such as a wide-angle microscope, to the terminal device 2.

[0081] Furthermore, in this embodiment, the terminal device 2 determines the authenticity of the card based on the time series information of the reflected light from the card obtained by the camera 26, so there is no need to embed an NFC tag or the like for authenticity determination in the card, or to apply a watermark or special processing for authenticity determination. This not only prevents an increase in the manufacturing cost of the card, but also prevents the aesthetic appeal of the card from being impaired.

[0082] Furthermore, in this embodiment, the first determination unit 206 performs the first determination of whether or not the camera 26 is photographing a physical card, using a detection value by the 6-axis sensor 27 provided in the terminal device 2. Therefore, in this embodiment, it is possible to perform the authenticity determination (first determination) of the card without introducing new hardware into the terminal device 2.

[0083] Furthermore, for example, in authenticating a personal identification number (My Number) card, information on an image of the card is used, so a guide frame or the like cannot be displayed on the screen, but in this embodiment, the authenticity of the card is determined by inputting a moving image captured by camera 26. Therefore, a guide frame can be displayed in real time on the screen of display device 30.

[0084] The guide frame display processing unit 203 can then move the position of the guide frame displayed on the screen of the display device 30, thereby allowing the user to perform the action required for the terminal device 2 to acquire information necessary for determining the authenticity of the card. Furthermore, since this action is a simple action of moving the terminal device 2 so that the frame portion of the card fits within the guide frame, even a user who is unfamiliar with operating the terminal device 2 or a user who is a beginner at trading card games can properly photograph the card.

[0085] In this embodiment, since the learning model 102a constructed by a neural network is used to determine whether the card is genuine or counterfeit, there is a risk that a deceptive signal that exploits the vulnerability of the neural network model using a method not anticipated by the designer may be input when the second determination is performed. In fact, there are known methods of deceiving the neural network model using a noisy image, and an attack method called "jailbreak" for avoiding the constraints of large language models (LLMs). Furthermore, a method for safely using a neural network model has not yet been established.

[0086] In contrast, the authenticity determination system 100 according to this embodiment introduces keypoint matching, an algorithm that is less flexible but behaves deterministically and therefore less vulnerable, into the first determination. On the other hand, the neural network model used in the second determination is highly flexible but behaves non-deterministically. By configuring the authenticity determination system 100 in this way, it becomes possible to determine the authenticity of a card by having the neural network model (learning model 102a) learn video images of the card, without having the neural network model learn cheating techniques on a large scale.

[0087] The present invention is not limited to the above-described embodiment, and various other applications and modifications are possible without departing from the gist of the present invention as set forth in the claims. For example, the above-described embodiment describes the configuration of the system and device in detail and specifically in order to easily explain the present invention, and is not necessarily limited to having all of the configurations described. In addition, it is also possible to add, delete, or replace part of the configuration of the present embodiment with other configurations. In addition, the control lines and information lines shown are those that are considered necessary for the explanation, and not all control lines and information lines in the product are necessarily shown. In reality, it can be considered that almost all components are connected to each other. [Explanation of symbols]

[0088] 1...server, 2...terminal device, 25...camera, 26...6-axis sensor, 29...display device, 100...authenticity determination system, 101...master image DB, 102, 202...learning model, 203...guide frame display processing unit, 204...moving image capturing unit, 205...card area detection unit, 206...first determination unit, 207...normalization unit, 208...second determination unit

Claims

1. A program for determining authenticity of a card medium having a predetermined surface that has been modified to change the reflection state of light reflected from the predetermined surface, comprising: a display control procedure for displaying a guide for changing the shooting angle of a moving image of the card medium captured by a camera on a screen of a terminal device and changing a display position of the guide at least once; a determination procedure for determining the authenticity of the card medium based on time series information of reflected light from a predetermined surface of the card medium, the time series information being included in video image data of the card medium captured by the camera while the guide is displayed; A program for a computer to execute.

2. The determination step includes a step of verifying the authenticity of the card medium based on time series information of the reflected light included in the video image data and time series information of the reflected light from the predetermined surface included in the video image data obtained by photographing an authentic card medium. The program according to claim 1.

3. The determination step includes a step of having a machine learning model perform the matching using the video data as input and outputting a determination result of the authenticity of the card medium. The program according to claim 2.

4. The learning data of the machine learning model is learning data including time series information of reflected light from the predetermined surface, which is included in video image data of the authentic card medium photographed based on the guide. The program according to claim 3.

5. The machine learning model is constructed by a neural network that has time axis information in addition to spatial information. The program according to claim 4.

6. The determination step includes a step of performing a first determination step of determining whether or not the camera is photographing the physical card medium by comparing a movement vector of a feature point extracted from the video image data transmitted from the camera with each movement vector in a plurality of coordinate axis directions acquired from an inertial sensor included in the terminal device, prior to a second determination step of determining the authenticity of the card medium. The program according to claim 5.

7. The determination step performs the first determination by determining whether or not a movement vector of a feature point extracted from the video image data transmitted from the camera and a movement vector acquired from the inertial sensor are moving simultaneously in the same direction. The program according to claim 6.

8. The present invention further includes a normalization unit that extracts a card area corresponding to the card medium from within the video captured by the camera, and normalizes the number of pixels in the length and width of the extracted card area in the video to a fixed length pixel number suitable for input to the machine learning model. The program according to claim 7.

9. The display control step includes a step of displaying, as the guide, an image designating a display position of the card medium on a screen of the terminal device on the screen of the terminal device. The program according to claim 5.

10. The display control step includes a step of changing a display position of an image designating the display position of the card medium on the screen to a position different from the position where the image was displayed when it is determined that a frame portion of the card medium fits into the display position of the card medium indicated by the image designating the display position of the card medium. The program according to claim 9.

11. Due to the processing applied to the predetermined surface of the card medium, the reflection patterns of the reflected light from the predetermined surface corresponding to the incident light from a plurality of different incident angles are different from each other. The program according to any one of claims 1 to 10.

12. The processing of the predetermined surface includes processing to form a concave-convex portion by transparent varnish printed on the first surface of the card medium. The program according to claim 11.

13. An authenticity determination method for determining the authenticity of a card medium having a predetermined surface to which a process capable of changing a reflection state of reflected light from the predetermined surface has been applied, comprising: a display control procedure for displaying a guide for changing the shooting angle of a moving image of the card medium captured by a camera on a screen of a terminal device and changing a display position of the guide at least once; and a determination step of determining the authenticity of the card medium based on time-series information of reflected light from a predetermined surface of the card medium, the time-series information being included in video image data of the card medium captured by the camera with the guide displayed. Authenticity determination method.

14. An authenticity determination device for determining authenticity of a card medium having a predetermined surface that has been modified to change a reflection state of reflected light from the predetermined surface, comprising: a display control unit that displays a guide for changing the shooting angle of the moving image of the card medium captured by a camera on a screen of a terminal device and changes the display position of the guide at least once; a determination unit that determines the authenticity of the card medium based on time-series information of reflected light from a predetermined surface of the card medium, the time-series information being included in video image data of the card medium captured by the camera with the guide displayed. Authenticity determination device.

Citation Information

Patent Citations

  • Recognizing device for optical reflecting element and recognizing device for storage medium

    JP2001092916A

  • Recognition device by reelecting optical element

    JP2002008089A

  • Authenticating objects using machine learning from microscopic differences

    JP2017520864A

  • How to authenticate magnetic induction marks with a portable device

    JP2022522726A

  • Computer program, method for processing information, and information processor

    JP2023166847A

Cited By

  • Authenticity determination program, authenticity determination method, and authenticity determination device

    JP7766835B1

  • Learning program, learning method, learning device, authenticity determination program, authenticity determination method, and authenticity determination device

    JP7810849B1

  • Information processing device, information processing method, and program

    JP7888817B1