Program, authenticity determination method and authenticity determination device

Abstract

To determine authenticity of a card medium, without using a device for authenticity determination.SOLUTION: A program makes a computer execute: a procedure which displays a guide for changing a shooting angle of a moving image of a card captured by a camera 26 on a screen of a terminal device 2, and changes a display position of the guide at least once or more; and a procedure which determines authenticity of the card on the basis of time sequence information of reflected light from a predetermined surface of a card medium, included in moving image data of the card captured by the camera 26 in a state in which the guide is displayed.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a program, an authenticity determining method, and an authenticity determining device. [Background technology]

[0002] Various techniques have been devised to determine the authenticity of cards used in trading card games. For example, a technique is known in which a near-field communication (NFC) chip storing information for authenticity determination is embedded in a card and the information is read by an NFC tag reader to determine the authenticity of the card. This technique makes it possible to determine the authenticity of a card with high accuracy. However, because the manufacturing cost of NFC chips is high, incorporating an NFC chip into a card increases the manufacturing cost of the card.

[0003] There is also known a technology for determining the authenticity of products that do not have markers or the like attached, using hardware such as a microscope capable of wide-angle photography. Non-Patent Document 1 discloses a technology for determining the authenticity of a product using a machine learning algorithm based on an image of the physical product captured by a microscope camera capable of wide-angle photography that is compatible with mobile devices. However, the technology described in Non-Patent Document 1 requires a wide-angle microscope to be connected to a mobile device as a peripheral device, making it difficult for an average user to use this technology to determine authenticity at home.

[0004] Another known method is to authenticate a product by attaching a marker, such as a hologram or a special barcode, to the product to be authenticated and reading it with software. For example, a technology has been put into practical use for theme park tickets, where a hologram is printed on the ticket and the hologram is read by software to authenticate the ticket (see Non-Patent Document 2). This technology is effective when creating a new product or when attaching a marker does not detract from the product's appeal. However, if the card requires aesthetic appeal and the existing card is to be authenticated, adding a hologram to the card may detract from its aesthetic appeal. [Prior art documents] [Patent documents]

[0005] [Non-Patent Document 1] Ashlesh Sharma, Vidyuth Srinivasan, Vishal Kanchan, and Lakshminarayanan Subramanian. 2017. The Fake vs Real Goods Problem: Microscopy and Machine Learning to the Rescue. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD '17). Association for Computing Machinery, New York, NY, USA, 2011-2019. https: / / doi.org / 10.1145 / 3097983.3098186 [Non-patent document 2] Oumayma Kada, Camille Kurtz, Cuong van Kieu, and Nicole Vincent. 2022. Hologram Detection for Identity Document Authentication. In Pattern Recognition and Artificial Intelligence: Third International Conference, ICPRAI 2022, Paris, France, June 1-3, 2022, Proceedings, Part I. Springer-Verlag, Berlin, Heidelberg, 346-357. https: / / doi.org / 10.1007 / 978-3-031-09037-0_29 Summary of the Invention [Problem to be solved by the invention]

[0006] Another known software-based authentication method is a technology called "visual check-in," invented by the applicant. Visual check-in technology allows users to obtain in-game rewards by taking photos of television programs, posters, product exteriors, and the like with a smartphone camera. When a template image to be recognized is captured at a predetermined position within a camera image, an authentication device using visual check-in technology recognizes feature points, which are points that indicate local features of the image, and determines the match between the image and the template using information on the positional relationship of the feature points. This feature-based matching method is less susceptible to environmental factors such as the position of the light source and the color of the lighting, and is also independent of the camera model, lens characteristics, and the like.

[0007] However, current visual check-in technology is designed to photograph fixed objects such as posters that are installed in specific locations, and therefore is not suitable for authenticating movable cards such as those used in trading card games.

[0008] The present invention has been made in view of the above circumstances, and aims to make it possible to determine the authenticity of a card medium without using an authenticity determination device or the like. [Means for solving the problem]

[0009] A program according to the present invention is a program for determining the authenticity of a card medium whose predetermined surface has been subjected to processing capable of changing the reflection state of light reflected from the predetermined surface. The program according to the present invention causes a computer to execute a display control procedure for displaying, on the screen of a terminal device, a guide for changing the shooting angle of a moving image of the card medium captured by a camera and changing the display position of the guide at least once, and a determination procedure for determining the authenticity of the card medium based on time-series information of the light reflected from the predetermined surface of the card medium, which is included in the moving image data of the card medium captured by the camera while the guide is displayed. [Effects of the Invention]

[0010] According to the present invention, it becomes possible to determine the authenticity of a card medium without using an authenticity determining device or the like. Problems, configurations, and effects other than those described above will become apparent from the following description of the embodiments. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is an overall configuration diagram showing an overview of an authenticity determination system according to an embodiment of the present invention; [Figure 2] 1 is a block diagram showing an example of the hardware configuration of a server and a terminal device that constitute an authenticity determination system according to an embodiment of the present invention. [Figure 3] 1 is a block diagram showing the functional configuration of an authenticity determination system according to an embodiment of the present invention. [Figure 4] 10A and 10B are diagrams illustrating an example of a guide frame display based on a guide frame display process performed by a guide frame display processing unit according to an embodiment of the present invention. [Figure 5] 10A and 10B are diagrams illustrating an example of changes in the angle at which a card is photographed by a camera according to an embodiment of the present invention. [Figure 6] 1 is a flowchart showing an example of the procedure of an authenticity determination process performed by an authenticity determination system according to an embodiment of the present invention. [Figure 7] 10 is a flowchart illustrating an example of a procedure for a guide frame display process performed by a guide frame display processing unit according to an embodiment of the present invention. [Figure 8] 10 is a flowchart illustrating an example of a procedure of a first determination process performed by a first determination unit according to an embodiment of the present invention. [Figure 9] 10 is a flowchart illustrating an example of a procedure for a second determination process performed by a second determination unit according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functions or configurations are designated by the same reference numerals, and redundant description will be omitted.

[0013] [One embodiment] <Outline of the authenticity determination system> FIG. 1 is a diagram showing the overall configuration of an authentication system 100 according to one embodiment of the present invention. The authenticity determination system 100 is a system that determines whether cards (an example of card media) used in a trading card game are genuine (authentic) or counterfeit (non-genuine). The authenticity determination system 100 includes a server 1, a smartphone 2A, and a PC (Personal Computer) 2B. The smartphone 2A and PC 2B can be connected to the server 1 via a network N such as the Internet. In the following description, the smartphone 2A and PC 2B are collectively referred to as terminal devices 2. Note that a dedicated gaming terminal may also be used as the terminal device 2.

[0014] The card Cd (see FIG. 4) that is the subject of authentication determination by the authentication determination system 100 is a card with a hologram and embossing (an example of textured processing) on ​​its surface (an example of a predetermined surface). The terminal device 2 of the authentication determination system 100 determines the authenticity of the card based on information obtained by photographing the card with a camera 26 (see FIG. 2) provided in the terminal device 2.

[0015] The authenticity determination system 100 according to this embodiment uses information such as changes in reflected light from the card according to the shooting angle of the camera 26 of the terminal device 2 as information obtained by the camera 26 photographing the card. Furthermore, the terminal device 2 according to this embodiment changes the reflection pattern of the light reflected from the card by displaying on the screen of the display device 30 a guide frame that prompts the user to move the terminal device in various directions such as up, down, left, and right while photographing the card, i.e., a guide frame for changing the shooting angle of the card by the camera 26.

[0016] Then, terminal device 2 performs a first determination process and a second determination process as a process for determining the authenticity of the card. In the first determination, terminal device 2 determines whether camera 26 of terminal device 2 is capturing an image of a physical card, using as a key whether the movement vector of 6-axis sensor 27 (see FIG. 3) provided in terminal device 2 and the movement vector of feature points extracted from the image captured by camera 26 change simultaneously and in the same direction.

[0017] In the second determination, the terminal device 2 determines the authenticity of the card based on information on the change in the reflection pattern of the light reflected from the card over time, which is included in the video recorded when the first determination is performed, i.e., time-series information on the light reflected from the card. The terminal device 2 performs the second determination using a trained machine learning model 102a (see FIG. 3; hereinafter, also simply referred to as the "trained model").

[0018] The server 1 is a device that manages a master image used in the first determination by the terminal device 2 and a learning model 102a used in the second determination. The server 1 synchronizes the master image stored in the terminal device 2 with the master image managed by the server 1 by transmitting the master image to the terminal device 2 at predetermined intervals. The server 1 also synchronizes the learning model stored in the terminal device 2 with the master image managed by the server 1 by transmitting the learning model to the terminal device 2 at predetermined intervals.

[0019] The terminal device 2 is an example of an authenticity determination device. An application (not shown) including a program for performing the authenticity determination process according to this embodiment is stored in the nonvolatile storage 31 (see FIG. 2) of the terminal device 2. The control unit 20 (see FIG. 2) of the terminal device 2 starts the application based on a user operation and performs a predetermined process by reading the program recorded in the nonvolatile storage 31.

[0020] The predetermined processing includes the above-mentioned first determination, second determination, etc. The predetermined processing also includes a process in which the control unit 20 of the terminal device 2 displays, on the screen of the display device 30, an image of a guide frame (hereinafter also simply referred to as a "guide frame") that specifies the display position of the card on the screen of the display device 30. Furthermore, the predetermined processing includes a process in which the information captured by the camera 26 while the user is performing the action of aligning the display position of the card with the guide frame is recorded as a moving image.

[0021] The input device 29 generates an operation signal according to an operation input by the user, and supplies the operation signal to the control unit 20 . The display device 30 displays a guide frame or the like on the screen based on the control of the control unit 20.

[0022] <Example of hardware configuration for authenticity determination system> Next, an example of the hardware configuration of an authenticity determination system 100 according to one embodiment of the present invention will be described with reference to Fig. 2. Fig. 2 is a block diagram showing an example of the hardware configuration of each of the server 1 and the terminal device 2 that constitute the authenticity determination system 100.

[0023] (Server configuration example) The server 1 is an example of a calculator that operates as a computer. The server 1 includes a control unit 10, a nonvolatile storage 15, and a network interface 16, which are all connected to a bus B1.

[0024] The control unit 10 includes a CPU (Central Processing Unit) 11, a GPU (Graphics Processing Unit) 12, a ROM (Read Only Memory) 13, and a RAM (Random Access Memory) 14. The control unit 10 may be configured as an FPGA (Field Programmable Gate Array).

[0025] The CPU 11 reads out the program code of the software that realizes each function according to the present embodiment from the ROM 13, loads it into the RAM 14, and executes it. Variables, parameters, etc. that are generated during the calculation processing of the CPU 11 are temporarily written to the RAM 14. These variables, parameters, etc. written to the RAM 14 are read out by the CPU 11 as appropriate. The GPU 12 performs the calculations required to draw an image.

[0026] The nonvolatile storage 15 may be, for example, a hard disk drive (HDD), a solid state drive (SSD), a Blu-ray (registered trademark) disc, a flexible disk, an optical disc, a magneto-optical disc, a CD-ROM, a CD-R, a magnetic tape, or a nonvolatile memory. In addition to an operating system (OS) and various parameters, a program for causing the server 1 to function is recorded in the nonvolatile storage 15. The program for causing the server 1 to function may be stored in the ROM 12. In other words, the ROM 12 and the nonvolatile storage 15 are used as an example of a computer-readable non-transitory recording medium that stores a program executed by the server 1.

[0027] Furthermore, a master image DB (Database) 101 (see FIG. 3) in which master images are stored is formed in the nonvolatile storage 15. Furthermore, a learning model DB 102 (see FIG. 3) in which learning models 102a are stored is formed in the nonvolatile storage 15.

[0028] For example, a network interface card (NIC) or the like is used as the network interface 15. The network interface 15 transmits and receives various data to and from the terminal device 2 via a dedicated line or the like connected to a terminal of the NIC and via the network N.

[0029] (Example of terminal device configuration) The terminal device 2 is an example of a calculator that operates as a computer capable of executing various programs. The terminal device 2 includes a control unit 20, a network interface 25, a camera 26, a six-axis sensor 27, an input / output interface 27, an input device 29, a display device 30, and a non-volatile storage 31, all of which are connected to a bus B2.

[0030] The control unit 20 includes a CPU 21, a ROM 23, and a RAM 24. The control unit 20 may include a GPU, or may be configured as an FPGA.

[0031] The CPU 21 reads out the program code of the software that realizes each function according to the present embodiment from the ROM 23, loads it into the RAM 24, and executes it. Variables, parameters, etc. that are generated during the calculation processing of the CPU 21 are temporarily written to the RAM 24. These variables, parameters, etc. written to the RAM 24 are read out as appropriate by the CPU 21. The CPU 21 performs processes such as processing the OS in the terminal device 2 and managing the input and output of data performed by each part in the terminal device 2. The GPU 22 performs calculations required for drawing images.

[0032] For example, a NIC or the like is used as the network interface 25. The network interface 25 can acquire various data from the server 1 via a LAN, a dedicated line, or the like connected to a terminal of the NIC, and can communicate with other terminal devices 2.

[0033] The camera 26 includes an image sensor such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor), and converts light from a subject focused on a lens (not shown) into image data. The camera 26 according to this embodiment is capable of capturing not only still images but also moving images.

[0034] The six-axis sensor 27 is an inertial sensor that can detect six-axis inertial forces, including accelerations in three axes (front-back, left-right, and up-down) and angular velocities in the three axes, acting on the terminal device 2. Information on the detected values ​​by the six-axis sensor 27 is referenced when the first determination described above is performed. Note that the inertial sensor included in the terminal device 2 is not limited to the six-axis sensor 27, and may be any other inertial sensor mounted on the terminal device 2.

[0035] The input / output interface 28 converts operation signals received from the input device 29 into data in a predetermined format and passes the data to the CPU 21. The input / output interface 28 also passes images of guide frames and the like output from the control unit 20 to the display device 30. The display device 30 may be equipped with a speaker (not shown), in which case the input / output interface 28 can also output audio signals to the speaker.

[0036] The input device 29 is configured by a touch sensor on a touch panel in the smartphone 2A (see FIG. 1), and is configured by a mouse, keyboard, and the like in the PC 2B.

[0037] The display device 30 is configured by an LCD (Liquid Crystal Display), an organic EL (Electro Luminescence) display, or the like.

[0038] The nonvolatile storage 31 may be, for example, an HDD, SSD, Blu-ray disc, flexible disk, optical disc, magneto-optical disc, CD-ROM, CD-R, or nonvolatile memory. In addition to the OS and various parameters, the nonvolatile storage 31 also stores programs for operating the terminal device 2, the master image DB 101 and learning model 102a (see FIG. 3) transmitted from the server 1, and the like.

[0039] The program for causing the server 1 to function may be stored in the ROM 23. In other words, the ROM 23 and the non-volatile storage 31 are used as an example of a computer-readable non-transitory recording medium that stores the program executed by the terminal device 2.

[0040] If the nonvolatile storage 31 is an external recording medium such as an optical disc, an optical drive (not shown) reads the nonvolatile storage 31 and extracts data from it. The data extracted by the optical drive is used for predetermined arithmetic processing by the CPU 21.

[0041] <Example of functional configuration of authenticity determination system> Next, the functional configuration of the authenticity determination system 100 will be described with reference to FIG. Fig. 3 is a block diagram showing the functional configuration of the authenticity determination system 100. In Fig. 3, the communication network N is omitted.

[0042] (Server functional configuration example) 3, the server 1 includes a master image DB 101 and a learning model DB 102. The master image DB 101 is an image that is referenced when the first determination unit 206 of the terminal device 2 makes the first determination, and stores photographed images of all types of regular (authentic) cards.

[0043] The learning model DB 102 stores a trained learning model 102a constructed by a neural network with time axis information. The learning model 102a is composed of weight data whose weights have been adjusted by learning. For example, a 3D-CNN (3-Dimensional Convolutional Neural Network), SlowFast, or a TimeSformer (Time-Space Transformer) can be applied to the neural network with time axis information.

[0044] For example, if the learning model 102a is constructed using a deep learning framework called PyTorch, the learning model 102a is implemented in a PTH file format. The file format of the learning model 102a may be other file formats such as a GGML file format or an ONNX (Open Neural Network eXchange) file format.

[0045] Furthermore, the server 1 periodically transmits the master images stored in the master image DB 101 and the learning models 102a stored in the learning model DB 102 to the terminal device 2.

[0046] (Example of terminal device functional configuration) The terminal device 2 includes a local image DB 201, a learning model DB 202, a guide frame display processing unit 203, a moving image shooting unit 204, a card area detection unit 205, a six-axis sensor 27, a first judgment unit 206, a normalization unit 207, and a second judgment unit.

[0047] The local image DB 201 is the master image DB 101 transmitted from the server 1 in a format that can be accessed (referenced) by an application on the terminal device 2. The learning model DB 202 is a database that stores the learning model 102 a transmitted from the server 1 .

[0048] The learning data of learning model 102a is time-series information of reflected light from the surface of a card, contained in video data of a genuine (authentic) card photographed by camera 26 based on guidance from a guide frame. Learning model 102a then inputs video data of a card to be authenticated, photographed based on guidance from a guide frame, and outputs an authenticity determination result for the card. Learning model 102a determines the authenticity of a card by verifying whether changes in the reflection pattern of reflected light from the card to be authenticated match the reflection pattern of reflected light from a genuine card.

[0049] The guide frame display processing unit 203 (an example of a display control unit) is a processing unit that displays a guide frame on the screen of the display device 30 (see FIG. 2). The guide frame is indicated, for example, by a thick red frame. When the guide frame display processing unit 203 detects that the frame of the card has fit within the guide frame, it displays a sign such as "OK" on the screen. Thereafter, the guide frame display processing unit 203 moves the guide frame to another position on the screen and prompts the user to align the frame of the card with that position. Examples of guide frame display by the guide frame display processing unit 203 will be described in detail with reference to FIGS. 4 and 5 below.

[0050] The video shooting unit 204 encodes image data obtained from the camera 26 as a video at a predetermined frame rate. The video shooting unit 204 saves the captured video in a general-purpose format such as H.264 / MP4 in the non-volatile storage 31. The video shooting unit 204 continues to shoot video while the guide frame display processing unit 203 is changing the display position of the guide frame.

[0051] The card area detection unit 205 uses feature (keypoint) matching technology to detect a card area corresponding to a card from image data obtained from the camera 26. The card area detection unit 205 also compares the image of the detected card area with the digital images of all types of cards in the master image DB 101 to determine the type of card being photographed.

[0052] The feature matching technology used by the card area detection unit 205 may be ORB (Oriented FAST and Rotated BRIEF) keypoint matching or other technology. For example, by parallelizing ORB keypoint matching using multiple threads and implementing it in the terminal device 2, sufficient processing speed can be maintained even when the terminal device 2 is a smartphone 2A.

[0053] Note that the feature matching technique used by the card area detection unit 205 is not limited to ORB keypoint matching, but may be other techniques such as SIFT (Scale Invariant Feature Transform), SURF (Speeded-Up Robust Features), KAZE, and AKAZE (Accelerated-KAZE).

[0054] The first determination unit 206 determines whether the camera 26 of the terminal device 2 is capturing an image of a physical card whose authenticity is to be determined, based on the movement of the six-axis sensor 27 (see FIG. 2) and the movement of the key points (an example of feature points) detected by the card area detection unit 205. Specifically, the first determination unit 206 determines whether the movement vector of the key points extracted from the video data transmitted from the camera 26 and the movement vector acquired from the sensor are moving simultaneously in the same direction.

[0055] By making the first determination by the first determination unit 206, even if a cheating method (unauthorized operation) related to the method of photographing a card is performed, the cheating method can be detected. Examples of cheating methods related to the method of photographing a card include a method of photographing a moving image of the card in advance, playing the moving image in front of the camera 26 of the terminal device 2, and a method of photographing a high-resolution copy image of the card with the camera 26.

[0056] Furthermore, the first determination unit 206 generates coordinate information of the guide frame for changing the guide frame to the next display position during the execution of the first determination, and outputs the information to the guide frame display processing unit 203.

[0057] The normalization unit 207 extracts the image of the card area from the moving image captured by the moving image capturing unit 204. Then, the normalization unit 207 normalizes it to a fixed length by cropping the number of vertical and horizontal pixels of the image of the extracted area to the size assumed as the input by the learning model 102a. What the normalization unit 207 extracts by cropping is the image area of the card. By performing the normalization process by the normalization unit 207, an image composed only of the image part of the card can be input to the learning model 102a in an appropriate size for the learning model 102a.

[0058] Also, the normalization unit 207 rotates and resizes the image of the extracted area for the purpose of preventing the top and bottom of the moving image input to the learning model 102a from being reversed. The normalization unit 207 performs the rotation and resizing of the image while maintaining the aspect ratio of the original moving image as much as possible. The image size after normalization by the normalization unit 207 can be, for example, an image size of about width m pixels × height n pixels (m < n).

[0059] The second determination unit 208 inputs the moving image including the time-series information of the reflected light from the card, which has been normalized by the normalization unit 207, to the learning model 102a, and obtains the authenticity determination result of the card classified into two values of normal (genuine) or non-normal (fake) from the learning model 102a.

[0060] <Display processing of the guide frame> Next, referring to FIGS. 4 and 5, the display processing of the guide frame by the guide frame display processing unit 203 will be described. FIG. 4 is a diagram showing an example of the display of the guide frame based on the display processing of the guide frame by the guide frame display processing unit 203.

[0061] On the left side of FIG. 4, a state where the frame of the card Cd is contained within the guide frame Fm displayed on the screen of the display device 30 by the guide frame display processing unit 203 is shown. When such a state occurs, that is, when it is detected that the frame of the card Cd is contained within the guide frame Fm, the guide frame display processing unit 203 causes a sign such as "OK" to be displayed on the screen.

[0062] Next, the guide frame display processing unit 203 provides guidance to change the position of the guide frame displayed on the screen to a different position. As shown on the right side of FIG. 4, the guide frame display processing unit 203 shifts the display position of the guide frame Fm on the screen diagonally upward and to the right. Seeing this display, the user moves the terminal device 2 being held in order to fit the frame of the card Cd into the card frame Fm. As a result of this processing being performed by the guide frame display processing unit 203, the position of the terminal device 2 being held by the user changes up, down, left, right, etc. in accordance with the change in the display position of the guide frame. In other words, the angle at which the camera 26 of the terminal device 2 captures the card changes. Note that the number of times the display position of the guide frame is changed by the guide frame display processing unit 203 may be any number of times as long as it is at least one or more.

[0063] FIG. 5 is a diagram showing an example of changes in the shooting angle of a card taken by camera 26. As the display position of the guide frame is changed by guide frame display processing unit 203, the shooting angle of the card taken by camera 26 changes to various angles, as shown in FIG. 5. Then, video shooting unit 204 records, as a video, images taken by camera 26 in states where the shooting angle has changed in this way. By recording, as a video, images taken by camera 26 in states where the shooting angle has changed, it becomes possible to include in the video taken by video shooting unit 204 a feature that essentially has time axis information, namely, a change in the reflection pattern of reflected light according to the shooting angle.

[0064] The authenticity determination system 100 according to the present embodiment determines the authenticity of cards that have undergone fine hologram and embossing processes. Embossing is achieved, for example, by printing a transparent varnish on the surface of the card. Therefore, when such a card is photographed, the reflection pattern of light reflected from the card changes in a subtle and diverse manner as the shooting angle of the camera 26 changes. In other words, the learning model 102a according to the present embodiment learns the subtle and diverse changes in the reflection pattern of light reflected from such a card. The surface treatment of the card is not limited to both fine hologram and embossing, and may be either one of them. Furthermore, the surface treatment of the card may be other than hologram or embossing, such as gold foil stamping, silver foil stamping, or mother-of-pearl inlay, as long as it can change the reflection state of light reflected from the card surface.

[0065] <Authenticity determination process using an authenticity determination system> Next, the authenticity determination process performed by the authenticity determination system 100 according to this embodiment will be described with reference to Fig. 6. Fig. 6 is a flowchart showing an example of the procedure for the authenticity determination process performed by the authenticity determination system 100.

[0066] First, the guide frame display processing unit 203 (see FIG. 3) executes a guide frame display process (step S1). The guide frame display process of step S1 starts when a user starts an application incorporating a program that executes the authenticity determination process according to this embodiment and selects a menu for determining the authenticity of a card. The contents of the guide frame display process performed in step S1 will be described in detail with reference to FIG. 7 below.

[0067] Next, the camera 26 starts taking pictures and recording the image data taken by the camera 26 as a moving image (step S2). The process of step S2 continues while the guide frame is displayed on the screen of the display device 30 in step S1 and the user moves the terminal device 2 up, down, left, right, etc. to align the frame of the card with the guide frame.

[0068] Next, the card area detection unit 205 extracts a card area from the image data captured by the camera 26 in step S2 using keypoint matching technology (step S3). Next, the first determination unit 206 performs a first determination process (step S4). The first determination process will be described in detail with reference to FIG. 8, which will be described later. Next, the normalization unit 207 normalizes the size of the video recorded in step S2 to a size suitable for input to the learning model 102a (step S5). Next, the second determination unit 208 performs a second determination process (step S6). The second determination process performed in step S6 will be described in detail with reference to FIG. 9, which will be described later.

[0069] (Guide frame display processing) Next, the guide frame display processing performed in step S2 of Fig. 6 will be described with reference to Fig. 7. Fig. 7 is a flowchart showing an example of the procedure for the guide frame display processing performed by the guide frame display processing unit 203.

[0070] First, the guide frame display processing unit 203 displays a guide frame on the screen of the display device 30 (step S11). Next, the guide frame display processing unit 203 determines whether the frame of the card included in the image data captured by the camera 26 in step S2 fits within the guide frame (step S12). If it is determined that the card does not fit within the guide frame (NO in step S12), the guide frame display processing unit 203 continues to make the determination in step S12.

[0071] On the other hand, if it is determined that the card has fit into the guide frame (YES in step S12), the guide frame display processing unit 203 displays a sign such as "OK" on the screen to indicate that the card has fit into the guide frame (step S13). Note that the sign indicating that the card has fit into the guide frame is not limited to the letters "OK" and may be other letters. Furthermore, the sign may not be letters, but may be other forms of notification, such as a change in the color or shape of the guide frame, or the emission of a sound.

[0072] Next, the guide frame display processing unit 203 determines whether or not the termination condition of the first determination process by the first determination unit 206 (see FIG. 3) has been satisfied (step S14). The termination condition of the first determination process is that both the determination result of "success" or "failure" by the first determination process and an image (moving image) captured by the camera 26 in the case where the determination is "success" are obtained. The moving image in the case where the determination is "success" is obtained when the first determination process for all guide frames has been completed. If it is determined in step S14 that the termination condition of the first determination process is not satisfied (NO in step S14), the guide frame display processing unit 203 changes the display position of the guide frame on the screen (step S15). After the processing of step S15, the guide frame display processing unit 203 returns to step S11 and performs the processing.

[0073] On the other hand, if it is determined in step S14 that the termination condition for the first determination process is satisfied (YES in step S14), the guide frame display process is terminated by the guide frame display processing unit 203. Note that the guide frame display process by the guide frame display processing unit 203 may be terminated at a predetermined timing after the determination result from the first determination process is output.

[0074] (First determination process) Next, the first determination process performed in step S4 of Fig. 6 will be described with reference to Fig. 8. Fig. 8 is a flowchart showing an example of the procedure of the first determination process performed by first determination unit 206.

[0075] First, the first determination unit 206 extracts key points from the card area detected in the captured image in step S3 of Fig. 6 (step S41). Next, the first determination unit 206 determines whether the movement vector of the key point extracted in step S41 and the movement vector of the six-axis sensor 27 of the terminal device 2 are moving simultaneously in the same direction (step S42).

[0076] If it is determined that the movement vector of the key point and the movement vector of the six-axis sensor 27 are moving simultaneously in the same direction (YES in step S42), the first determination unit 206 determines that the camera 26 is photographing a physical card (step S43). On the other hand, if it is determined that the movement vector of the key point and the movement vector of the six-axis sensor 27 are not moving simultaneously in the same direction (NO in step S42), the first determination unit 206 determines that the camera 26 is photographing an object other than a physical card (step S44). After the processing of step S44 or step S45, the first determination process by the first determination unit 206 ends. Note that the determination in step S42 can be made by analyzing the movement directions of both movement vectors, or can also be made based on the magnitude of the correlation coefficient between both movement vectors, or the like.

[0077] (Second determination process) Next, the second determination process performed in step S6 of Fig. 6 will be described with reference to Fig. 9. Fig. 9 is a flowchart showing an example of the procedure of the second determination process performed by second determination unit 208.

[0078] First, the second determination unit 208 inputs the video image of the card area normalized in step S5 of Fig. 6 to the learning model 102a (step S51). Next, the learning model 102a verifies whether the change in the reflection pattern of reflected light obtained from the image of the card area photographed by the camera 26 and normalized matches the change in the reflection pattern of reflection from a genuine card that has been learned in advance (step S52). Next, the second determination unit 208 outputs the authenticity determination result of the card as a result of the verification by the learning model 102a (step S53).

[0079] If the learning model 102a can confirm that the change in the reflection pattern of the reflected light obtained from the image of the card area matches the change in the reflection pattern of the reflection from a pre-learned genuine card, it outputs an authenticity determination result that the card photographed by the camera 26 is a genuine (genuine) card. On the other hand, if the learning model 102a cannot confirm that the change in the reflection pattern of the reflected light obtained from the image of the card area matches the change in the reflection pattern of the reflection from a pre-learned genuine card, it outputs an authenticity determination result that the card photographed by the camera 26 is an illegal (counterfeit) card. After the processing of step S63, the second determination processing by the second determination unit 208 ends.

[0080] In the above-described embodiment, the second determination unit 208 of the terminal device 2 determines the authenticity of the card based on time-series information of the light reflected from the card, i.e., information on changes in the reflection pattern of the reflected light, obtained by the camera 26. Therefore, according to this embodiment, it is possible to determine the authenticity of the card without adding or connecting to the terminal device 2 any equipment for authenticity determination, such as a wide-angle microscope.

[0081] Furthermore, in this embodiment, the terminal device 2 determines the authenticity of a card based on time-series information of the light reflected from the card obtained by the camera 26, so there is no need to embed an NFC tag or the like for authenticity determination in the card, or to apply a watermark or special processing for authenticity determination. This not only prevents an increase in the manufacturing cost of the card, but also prevents the aesthetic appeal of the card from being impaired.

[0082] Furthermore, in this embodiment, the first determination unit 206 performs the first determination of whether or not the camera 26 is capturing an image of a physical card, using a detection value from the six-axis sensor 27 provided in the terminal device 2. Therefore, in this embodiment, it is possible to perform the authenticity determination (first determination) of the card without introducing new hardware into the terminal device 2.

[0083] Furthermore, for example, in authenticating a personal identification number (My Number) card, information from a photographed image of the card is used, so a guide frame or the like cannot be displayed on the screen, but in this embodiment, the authenticity of the card is determined using a moving image taken by camera 26 as input. Therefore, a guide frame can be displayed in real time on the screen of display device 30.

[0084] Then, the guide frame display processing unit 203 moves the position of the guide frame displayed on the screen of the display device 30, thereby allowing the user to perform the operation required for the terminal device 2 to acquire the information necessary for determining the authenticity of the card. Furthermore, since this operation is a simple operation of moving the terminal device 2 so that the frame portion of the card fits into the guide frame, even users who are unfamiliar with operating the terminal device 2 or users who are beginners to trading card games can properly photograph the card.

[0085] In this embodiment, since the learning model 102a constructed by a neural network is used to determine whether a card is authentic or counterfeit, there is a risk that a deceptive signal that exploits the vulnerability of the neural network model using a method not anticipated by the designer may be input when the second determination is performed. In fact, there are known methods of deceiving a neural network model using a noisy image and an attack method called "jailbreak" that circumvents the constraints of large language models (LLMs). However, a method for safely using a neural network model has not yet been established.

[0086] In contrast, the authentication determination system 100 according to this embodiment employs keypoint matching, an algorithm that is less flexible but behaves deterministically and therefore less vulnerable, in the first determination. On the other hand, the neural network model used in the second determination is more flexible but behaves non-deterministically. By configuring the authentication determination system 100 in this way, it becomes possible to determine the authenticity of a card by having the neural network model (learning model 102a) learn video images of the card, without having the neural network model learn cheating techniques on a large scale.

[0087] The present invention is not limited to the above-described embodiment, and it goes without saying that various other applications and modifications are possible without departing from the gist of the present invention as set forth in the claims. For example, the above-described embodiment has described in detail and specifically the configuration of the system and device in order to clearly explain the present invention, and is not necessarily limited to having all of the described configurations. Furthermore, it is also possible to add, delete, or replace part of the configuration of the present embodiment with other configurations. In addition, the control lines and information lines shown are those that are considered necessary for the explanation, and do not necessarily show all the control lines and information lines in the product. In reality, it can be assumed that almost all components are interconnected. [Explanation of symbols]

[0088] 1...server, 2...terminal device, 25...camera, 26...6-axis sensor, 29...display device, 100...authenticity determination system, 101...master image DB, 102, 202...learning model, 203...guide frame display processing unit, 204...moving image capturing unit, 205...card area detection unit, 206...first determination unit, 207...normalization unit, 208...second determination unit

Claims

1. A program for determining the authenticity of a card medium having a predetermined surface that has been processed to change the reflection state of light reflected from the predetermined surface, a display control procedure for displaying a guide for changing the shooting angle of the moving image of the card medium captured by a camera on a screen of a terminal device and changing the display position of the guide at least once; a determination procedure for determining the authenticity of the card medium based on time-series information of reflected light from a predetermined surface of the card medium, which is included in video image data of the card medium captured by the camera while the guide is displayed; A program for a computer to run.

2. The determination step includes a step of verifying the authenticity of the card medium based on time series information of the reflected light included in the video image data and time series information of the reflected light from the predetermined surface included in video image data obtained by photographing the authentic card medium. The program according to claim 1.

3. The determination step includes a step of having a machine learning model perform the matching, which receives the video data as input and outputs a determination result of the authenticity of the card medium. The program according to claim 2.

4. The learning data of the machine learning model is learning data obtained by learning time-series information of reflected light from the predetermined surface contained in video image data of the authentic card medium photographed based on the guide. The program according to claim 3.

5. The learning model is constructed by a neural network that has time axis information in addition to spatial information. The program according to claim 4.

6. The determination procedure includes a step of performing a first determination, prior to a second determination process for determining whether the card is authentic, of determining whether the camera is capturing an image of the physical card medium by comparing a movement vector of a feature point extracted from the video image data transmitted from the camera with each movement vector in a plurality of coordinate axis directions acquired from an inertial sensor provided in the terminal device. The program according to claim 5.

7. The determination step performs the first determination by determining whether or not a movement vector of a feature point extracted from the video image data transmitted from the camera and a movement vector acquired from the inertial sensor are moving simultaneously in the same direction. The program according to claim 6.

8. The apparatus further includes a normalization unit that extracts a card area corresponding to the card from the video captured by the camera and normalizes the number of pixels in the length and width of the extracted card area in the video to a fixed length of pixel number suitable for input to the learning model. The program according to claim 7.

9. The display control step includes a step of displaying, as the guide, an image specifying a display position of the card medium on the screen of the terminal device on the screen of the terminal device. The program according to claim 5.

10. The display control procedure includes a procedure for changing a display position of an image specifying the display position of the card medium on the screen to a position different from the displayed position when it is determined that a frame portion of the card medium fits into the display position of the card medium indicated by the image specifying the display position of the card medium. The program according to claim 9.

11. Due to the processing applied to the predetermined surface of the card medium, the reflection patterns of the reflected light from the predetermined surface corresponding to the incident light from a plurality of different incident angles are different from each other. The program according to any one of claims 1 to 10.

12. The processing on the predetermined surface includes processing to form a concave-convex portion by using a transparent varnish printed on the first surface of the card medium. The program according to claim 11.

13. An authenticity determination method for determining the authenticity of a card medium having a predetermined surface to which processing capable of changing the reflection state of reflected light from the predetermined surface has been applied, a display control procedure for displaying a guide for changing the shooting angle of the moving image of the card medium captured by a camera on a screen of a terminal device and changing the display position of the guide at least once; and a determination procedure for determining the authenticity of the card medium based on time-series information of reflected light from a predetermined surface of the card medium, the time-series information being included in video image data of the card medium captured by the camera while the guide is displayed. Authenticity determination method.

14. An authenticity determination device for determining the authenticity of a card medium having a predetermined surface that has been processed to change the reflection state of light reflected from the predetermined surface, a display control unit that displays a guide for changing the shooting angle of the moving image of the card medium captured by the camera on a screen of the terminal device and changes the display position of the guide at least once; a determination unit that determines the authenticity of the card medium based on time-series information of reflected light from a predetermined surface of the card medium, the time-series information being included in video image data of the card medium captured by the camera while the guide is displayed. Authenticity determination device.

Images