Program, authenticity determination method, and authenticity determination apparatus

Abstract

The present invention determines the authenticity of a card medium without using an authenticity determination device. A program according to one embodiment of the present invention causes a computer to execute: a procedure for causing a screen of a terminal apparatus to display a guide for causing an imaging angle of a moving image of a card being imaged by a camera to vary, and for causing the display position of the guide to change at least once; and a procedure for determining the authenticity of the card on the basis of time series information relating to reflected light from a prescribed surface of a card medium, included in the moving image data of the card imaged by the camera in a state in which the guide is being displayed.

Description

Program, authenticity determination method, and authenticity determination device

[0001] The present invention relates to a program, an authenticity determining method, and an authenticity determining device.

[0002] Various techniques have been devised to determine the authenticity of cards used in trading card games. For example, a technique is known in which a near-field communication (NFC) chip storing information for authenticity determination is embedded in a card and the information is read by an NFC tag reader to determine the authenticity of the card. This technique allows the authenticity of the card to be determined with high accuracy. However, because the manufacturing cost of NFC chips is high, incorporating an NFC chip into a card increases the manufacturing cost of the card.

[0003] There is also a known technology for determining the authenticity of products that do not have markers or the like attached, using hardware such as a microscope capable of wide-angle photography. Non-Patent Document 1 discloses a technology for determining the authenticity of a product using a machine learning algorithm based on an image of the physical product captured by a microscope camera capable of wide-angle photography that is compatible with mobile devices. However, the technology described in Non-Patent Document 1 requires a wide-angle microscope to be connected to the mobile device as a peripheral device, making it difficult for an average user to use this technology to determine the authenticity of a product at home.

[0004] Another known method is to authenticate a product by attaching a marker, such as a hologram or a special barcode, to the product to be authenticated and reading it with software. For example, a technology has been put into practical use for theme park tickets, where a hologram is printed on the ticket and the hologram is read by software to authenticate the ticket (see Non-Patent Document 2). This technology is effective when creating a new product or when attaching a marker does not detract from the product's appeal. However, if the card requires aesthetic appeal and the existing card is to be authenticated, adding a hologram to the card may detract from the card's aesthetic appearance.

[0005] Ashlesh Sharma, Vidyuth Srinivasan, Vishal Kanchan, and Lakshminarayanan Subramanian. 2017. The Fake vs Real Goods Problem: Microscopy and Machine Learning to the Rescue. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD '17). Association for Computing Machinery, New York, NY, USA, 2011-2019. https: / / doi.org / 10.1145 / 3097983.3098186Oumayma Kada, Camille Kurtz, Cuong van Kieu, and Nicole Vincent. 2022. Hologram Detection for Identity Document Authentication. In Pattern Recognition and Artificial Intelligence: Third International Conference, ICPRAI 2022, Paris, France, June 1-3, 2022, Proceedings, Part I. Springer-Verlag, Berlin, Heidelberg, 346-357. https: / / doi.org / 10.1007 / 978-3-031-09037-0_29

[0006] Another known software-based authentication method is a technology called "visual check-in," invented by the applicant. Visual check-in technology allows users to obtain in-game rewards by taking photos of television programs, posters, product exteriors, and the like with a smartphone camera. When a template image to be recognized is captured at a predetermined position within a camera image, an authentication device using visual check-in technology recognizes feature points, which are points that indicate local features of the image, and determines the match between the image and the template using information on the positional relationship of the feature points. This feature-based matching method is less susceptible to environmental factors such as the position of the light source and the color of the lighting, and is also independent of the camera model, lens characteristics, and the like.

[0007] However, current visual check-in technology is designed to photograph fixed objects such as posters that are installed in specific locations, and therefore is not suitable for authenticating movable cards such as those used in trading card games.

[0008] The present invention has been made in view of the above circumstances, and aims to make it possible to determine the authenticity of a card medium without using an authenticity determination device or the like.

[0009] A program according to the present invention is a program for determining the authenticity of a card medium whose predetermined surface has been subjected to processing capable of changing the reflection state of light reflected from the predetermined surface. The program causes a computer to execute the following steps: a display control procedure for displaying, on a screen of a terminal device, a guide for changing the shooting angle of a moving image of the card medium captured by a camera and changing the display position of the guide at least once; and a determination procedure for determining the authenticity of the card medium based on time-series information of the light reflected from the predetermined surface of the card medium, which is included in the moving image data of the card medium captured by the camera while the guide is displayed.

[0010] According to the present invention, it becomes possible to determine the authenticity of a card medium without using an authenticity determining device, etc. Problems, configurations, and effects other than those described above will become clear from the following description of the embodiments.

[0011] FIG. 1 is an overall configuration diagram showing an overview of an authenticity determination system according to one embodiment of the present invention. FIG. 2 is a block diagram showing an example of the hardware configuration of a server and a terminal device that constitute the authenticity determination system according to one embodiment of the present invention. FIG. 3 is a block diagram showing the functional configuration of an authenticity determination system according to one embodiment of the present invention. FIG. 4 is a diagram showing an example of a guide frame display based on guide frame display processing by a guide frame display processing unit according to one embodiment of the present invention. FIG. 5 is a diagram showing an example of a change in the angle at which a card is photographed by a camera according to one embodiment of the present invention. FIG. 6 is a flowchart showing an example of the procedure of authenticity determination processing by an authenticity determination system according to one embodiment of the present invention. FIG. 7 is a flowchart showing an example of the procedure of guide frame display processing by a guide frame display processing unit according to one embodiment of the present invention. FIG. 8 is a flowchart showing an example of the procedure of first determination processing by a first determination unit according to one embodiment of the present invention. FIG. 9 is a flowchart showing an example of the procedure of second determination processing by a second determination unit according to one embodiment of the present invention.

[0012] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functions or configurations are designated by the same reference numerals, and redundant description will be omitted.

[0013] [One Embodiment] <Outline of Authenticity Determination System> FIG. 1 is an overall configuration diagram showing an overview of an authenticity determination system 100 according to one embodiment of the present invention. The authenticity determination system 100 is a system that determines whether cards (an example of card media) used in a trading card game are genuine (authentic) or counterfeit (non-authentic). The authenticity determination system 100 includes a server 1, a smartphone 2A, and a PC (Personal Computer) 2B. The smartphone 2A and the PC 2B are connectable to the server 1 via a network N such as the Internet. In the following description, the smartphone 2A and the PC 2B are collectively referred to as terminal devices 2. Note that a dedicated gaming terminal may also be used as the terminal device 2.

[0014] The card Cd (see FIG. 4 ) that is the subject of authentication determination by the authenticity determination system 100 is a card that has a hologram and embossing (an example of textured processing) applied to its surface (an example of a predetermined surface). The terminal device 2 of the authenticity determination system 100 determines the authenticity of the card based on information obtained by photographing the card with a camera 26 (see FIG. 2 ) provided in the terminal device 2.

[0015] The authenticity determination system 100 according to this embodiment uses information such as changes in reflected light from the card according to the shooting angle of the camera 26 of the terminal device 2 as information obtained by the camera 26 photographing the card. Furthermore, the terminal device 2 according to this embodiment changes the reflection pattern of the light reflected from the card by displaying on the screen of the display device 30 a guide frame that prompts the user to move the terminal device in various directions such as up, down, left, and right while photographing the card, i.e., a guide frame for changing the shooting angle of the card by the camera 26.

[0016] Then, the terminal device 2 performs a first determination process and a second determination process as a process for determining the authenticity of the card. In the first determination, the terminal device 2 determines whether the camera 26 of the terminal device 2 is capturing an image of a physical card, using as a key whether the movement vector of the six-axis sensor 27 (see FIG. 3 ) provided in the terminal device 2 and the movement vector of the feature point extracted from the image captured by the camera 26 change simultaneously and in the same direction.

[0017] In the second determination, the terminal device 2 determines the authenticity of the card based on information on the change in the reflection pattern of the light reflected from the card over time, which is included in the video recorded when the first determination is performed, i.e., based on the time-series information of the light reflected from the card. The terminal device 2 performs the second determination using a trained machine learning model 102a (see FIG. 3 ; hereinafter, also simply referred to as the “training model”).

[0018] The server 1 is a device that manages a master image used in the first determination by the terminal device 2 and a learning model 102a used in the second determination. The server 1 transmits the master image to the terminal device 2 at predetermined intervals, thereby synchronizing the master image stored in the terminal device 2 with the master image managed by the server 1. The server 1 also transmits the learning model to the terminal device 2 at predetermined intervals, thereby synchronizing the learning model stored in the terminal device 2 with the master image managed by the server 1.

[0019] The terminal device 2 is an example of an authenticity determination device. An application (not shown) including a program for performing the authenticity determination process according to this embodiment is stored in the non-volatile storage 31 (see FIG. 2) of the terminal device 2. The control unit 20 (see FIG. 2) of the terminal device 2 starts the application based on a user operation and performs a predetermined process by reading the program recorded in the non-volatile storage 31.

[0020] The predetermined processing includes the above-mentioned first determination, second determination, etc. The predetermined processing also includes a process in which the control unit 20 of the terminal device 2 displays, on the screen of the display device 30, an image of a guide frame (hereinafter also simply referred to as a "guide frame") that specifies the display position of the card on the screen of the display device 30. The predetermined processing also includes a process in which the information captured by the camera 26 while the user is aligning the display position of the card with the guide frame is recorded as a moving image.

[0021] The input device 29 generates an operation signal according to an operation input by the user, and supplies the operation signal to the control unit 20. The display device 30 displays a guide frame or the like on the screen based on the control of the control unit 20.

[0022] <Example of Hardware Configuration of Authenticity Determination System> Next, an example of the hardware configuration of an authenticity determination system 100 according to one embodiment of the present invention will be described with reference to Fig. 2. Fig. 2 is a block diagram showing an example of the hardware configuration of each of the server 1 and the terminal device 2 that constitute the authenticity determination system 100.

[0023] (Example of Server Configuration) The server 1 is an example of a calculator that operates as a computer. The server 1 includes a control unit 10, a nonvolatile storage 15, and a network interface 16, which are all connected to a bus B1.

[0024] The control unit 10 includes a central processing unit (CPU) 11, a graphics processing unit (GPU) 12, a read-only memory (ROM) 13, and a random access memory (RAM) 14. The control unit 10 may be configured as a field programmable gate array (FPGA).

[0025] The CPU 11 reads out program code of software that realizes each function according to this embodiment from the ROM 13, loads it into the RAM 14, and executes it. Variables, parameters, etc. that are generated during the calculation processing of the CPU 11 are temporarily written to the RAM 14. These variables, parameters, etc. written to the RAM 14 are read out by the CPU 11 as appropriate. The GPU 12 performs calculation processing required for drawing an image.

[0026] The nonvolatile storage 15 may be, for example, a hard disk drive (HDD), a solid state drive (SSD), a Blu-ray (registered trademark) disk, a flexible disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a magnetic tape, or a nonvolatile memory. In addition to an operating system (OS) and various parameters, the nonvolatile storage 15 also stores programs for causing the server 1 to function. The programs for causing the server 1 to function may be stored in the ROM 12. In other words, the ROM 12 and the nonvolatile storage 15 are used as an example of a computer-readable, non-transitory recording medium that stores programs executed by the server 1.

[0027] A master image database (DB) 101 (see FIG. 3) for storing master images is also formed in the nonvolatile storage 15. A learning model DB 102 (see FIG. 3) for storing learning models 102a is also formed in the nonvolatile storage 15.

[0028] For example, a network interface card (NIC) or the like is used as the network interface 15. The network interface 15 transmits and receives various data to and from the terminal device 2 via a dedicated line or the like connected to a terminal of the NIC and via the network N.

[0029] (Configuration Example of Terminal Device) The terminal device 2 is an example of a calculator that operates as a computer capable of executing various programs. The terminal device 2 includes a control unit 20, a network interface 25, a camera 26, a six-axis sensor 27, an input / output interface 27, an input device 29, a display device 30, and a non-volatile storage 31, all of which are connected to a bus B2.

[0030] The control unit 20 includes a CPU 21, a ROM 23, and a RAM 24. The control unit 20 may include a GPU or may be configured as an FPGA.

[0031] The CPU 21 reads out program code of software that realizes each function according to this embodiment from the ROM 23, loads it into the RAM 24, and executes it. Variables, parameters, etc. generated during the calculation processing of the CPU 21 are temporarily written to the RAM 24. These variables, parameters, etc. written to the RAM 24 are read out as appropriate by the CPU 21. The CPU 21 performs processes such as OS processing in the terminal device 2 and management of input and output of data performed by each section within the terminal device 2. The GPU 22 performs calculation processes required for drawing images.

[0032] For example, a NIC or the like is used as the network interface 25. The network interface 25 can acquire various data from the server 1 via a LAN, a dedicated line, or the like connected to a terminal of the NIC, and can communicate with other terminal devices 2.

[0033] The camera 26 includes an image sensor such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS), and converts light from a subject imaged through a lens (not shown) into image data. The camera 26 according to this embodiment is capable of capturing not only still images but also moving images.

[0034] The six-axis sensor 27 is an inertial sensor that can detect six-axis inertial forces, including accelerations in the three axes (front-back, left-right, and up-down) and angular velocities in the three axes applied to the terminal device 2. Information on the detected values ​​by the six-axis sensor 27 is referenced when the first determination described above is performed. Note that the inertial sensor included in the terminal device 2 is not limited to the six-axis sensor 27, and may be any other inertial sensor implemented in the terminal device 2.

[0035] The input / output interface 28 converts operation signals received from the input device 29 into data in a predetermined format and passes the data to the CPU 21. The input / output interface 28 also passes images of guide frames and the like output from the control unit 20 to the display device 30. The display device 30 may be equipped with a speaker (not shown), in which case the input / output interface 28 can also output audio signals to the speaker.

[0036] The input device 29 is configured by a touch sensor on a touch panel in the smartphone 2A (see FIG. 1), and is configured by a mouse, keyboard, and the like in the PC 2B.

[0037] The display device 30 is configured by an LCD (Liquid Crystal Display), an organic EL (Electro Luminescence) display, or the like.

[0038] The nonvolatile storage 31 may be, for example, an HDD, SSD, Blu-ray disc, flexible disk, optical disc, magneto-optical disc, CD-ROM, CD-R, or nonvolatile memory. In addition to the OS and various parameters, the nonvolatile storage 31 also stores programs for operating the terminal device 2, the master image DB 101 and the learning model 102a (see FIG. 3) transmitted from the server 1, and the like.

[0039] The program for causing the server 1 to function may be stored in the ROM 23. In other words, the ROM 23 and the non-volatile storage 31 are used as an example of a computer-readable non-transitory recording medium that stores the program executed by the terminal device 2.

[0040] If the nonvolatile storage 31 is an external recording medium such as an optical disk, an optical drive (not shown) reads the nonvolatile storage 31 and extracts data from it. The data extracted by the optical drive is used for predetermined arithmetic processing by the CPU 21.

[0041] <Example of Functional Configuration of Authenticity Determination System> Next, the functional configuration of the authenticity determination system 100 will be described with reference to Fig. 3. Fig. 3 is a block diagram showing the functional configuration of the authenticity determination system 100. In Fig. 3, the communication network N is omitted.

[0042] 3, the server 1 includes a master image DB 101 and a learning model DB 102. The master image DB 101 is an image that is referenced by the first determination unit 206 of the terminal device 2 when making the first determination, and stores photographed images of all types of regular (genuine) cards.

[0043] The learning model DB 102 stores a trained learning model 102a constructed by a neural network with time axis information. The learning model 102a is composed of weight data whose weights have been adjusted through learning. For example, a 3D-CNN (3-Dimensional Convolutional Neural Network), SlowFast, or a TimeSformer (Time-Space Transformer) can be applied to the neural network with time axis information.

[0044] For example, when the learning model 102a is constructed using a deep learning framework called PyTorch, the learning model 102a is implemented in a PTH file format. Note that the file format of the learning model 102a may be other file formats such as a GGML file format or an ONNX (Open Neural Network eXchange) file format.

[0045] In addition, the server 1 periodically transmits the master images stored in the master image DB 101 and the learning models 102 a stored in the learning model DB 102 to the terminal device 2 .

[0046] (Example of functional configuration of terminal device) The terminal device 2 includes a local image DB 201, a learning model DB 202, a guide frame display processing unit 203, a moving image capturing unit 204, a card area detection unit 205, a six-axis sensor 27, a first determination unit 206, a normalization unit 207, and a second determination unit.

[0047] The local image DB 201 is a version of the master image DB 101 transmitted from the server 1 that can be accessed (referenced) by an application on the terminal device 2. The learning model DB 202 is a database that stores the learning model 102a transmitted from the server 1.

[0048] The learning data of the learning model 102a is time-series information of reflected light from the surface of a card, which is included in video data of a genuine (authentic) card photographed by the camera 26 based on guidance from the guide frame. The learning model 102a then inputs video data of the card to be authenticated, photographed based on guidance from the guide frame, and outputs the authenticity determination result for the card. The learning model 102a determines the authenticity of the card by verifying whether the change in the reflection pattern of the reflected light from the card to be authenticated matches the reflection pattern of the reflected light from a genuine card.

[0049] The guide frame display processing unit 203 (an example of a display control unit) is a processing unit that displays a guide frame on the screen of the display device 30 (see FIG. 2). The guide frame is indicated, for example, by a thick red frame. When the guide frame display processing unit 203 detects that the frame of the card has fit within the guide frame, it displays a sign such as "OK" on the screen. Thereafter, the guide frame display processing unit 203 moves the guide frame to another position on the screen and prompts the player to align the frame of the card with that position. Examples of guide frame display by the guide frame display processing unit 203 will be described in detail below with reference to FIGS. 4 and 5.

[0050] The video shooting unit 204 encodes image data obtained from the camera 26 as a video at a predetermined frame rate. The video shooting unit 204 stores the captured video in the non-volatile storage 31 in a general-purpose format such as H.264 / MP4. The video shooting unit 204 continues to shoot video while the guide frame display processing unit 203 is changing the display position of the guide frame.

[0051] The card area detection unit 205 uses feature (keypoint) matching technology to detect a card area corresponding to a card from image data obtained from the camera 26. The card area detection unit 205 also compares the image of the detected card area with digital images of all types of cards in the master image DB 101 to determine the type of card being photographed.

[0052] The feature matching technology used by the card area detection unit 205 may be ORB (Oriented FAST and Rotated BRIEF) keypoint matching or other technology. For example, by parallelizing ORB keypoint matching using multiple threads and implementing it in the terminal device 2, sufficient processing speed can be maintained even when the terminal device 2 is a smartphone 2A.

[0053] Note that the feature matching technique used by the card area detection unit 205 is not limited to ORB keypoint matching, but may be other techniques such as SIFT (Scale Invariant Feature Transform), SURF (Speeded-Up Robust Features), KAZE, and AKAZE (Accelerated-KAZE).

[0054] The first determination unit 206 determines whether the camera 26 of the terminal device 2 is capturing an image of a physical card whose authenticity is to be determined, based on the movement of the six-axis sensor 27 (see FIG. 2 ) and the movement of the key points (an example of feature points) detected by the card area detection unit 205. Specifically, the first determination unit 206 determines whether the movement vector of the key points extracted from the video data transmitted from the camera 26 and the movement vector acquired from the sensor are moving simultaneously in the same direction.

[0055] By making the first determination by the first determination unit 206, even if a cheating method (unauthorized operation) related to the method of photographing a card is performed, the cheating method can be detected. Examples of cheating methods related to the method of photographing a card include a method of photographing a moving image of a card in advance, playing the moving image in front of the camera 26 of the terminal device 2, and a method of photographing a high-resolution copy image of the card with the camera 26.

[0056] Furthermore, the first determination unit 206 generates coordinate information of the guide frame for changing the guide frame to the next display position during the execution of the first determination, and outputs the information to the guide frame display processing unit 203 .

[0057] The normalization unit 207 extracts an image of a card area from the video captured by the video capture unit 204. The normalization unit 207 then normalizes the image of the extracted area to a fixed length by cropping the number of pixels in both the vertical and horizontal directions to a size assumed as input by the learning model 102a. The image area of ​​the card is extracted by cropping. As a result of the normalization process performed by the normalization unit 207, an image consisting only of the image portion of the card can be input to the learning model 102a at an appropriate size.

[0058] Furthermore, the normalization unit 207 rotates and resizes the image of the extracted region to prevent the video image input to the learning model 102a from being upside down. The normalization unit 207 rotates and resizes the image while maintaining the aspect ratio of the original video image as much as possible. The image size after normalization by the normalization unit 207 can be, for example, approximately m pixels wide by n pixels high (m<n).

[0059] The second judgment unit 208 inputs the video image containing the time series information of the reflected light from the card, which has been normalized by the normalization unit 207, into the learning model 102a, and obtains an authenticity judgment result for the card, classified into two values: regular (genuine) or non-genuine (counterfeit), from the learning model 102a.

[0060] 4 and 5, a description will be given of the display processing of the guide frame by the guide frame display processing unit 203. Fig. 4 is a diagram showing an example of the display of the guide frame based on the display processing of the guide frame by the guide frame display processing unit 203.

[0061] 4 shows a state in which the frame of the card Cd is contained within the guide frame Fm displayed on the screen of the display device 30 by the guide frame display processing unit 203. When this state occurs, that is, when it is detected that the frame of the card Cd is contained within the guide frame Fm, the guide frame display processing unit 203 displays a sign such as "OK" on the screen.

[0062] Next, the guide frame display processing unit 203 provides guidance to change the position of the guide frame displayed on the screen to a different position. As shown on the right side of FIG. 4 , the guide frame display processing unit 203 shifts the display position of the guide frame Fm on the screen diagonally upward and to the right. Seeing this display, the user moves the terminal device 2 being held in order to fit the frame of the card Cd into the card frame Fm. As a result of this processing being performed by the guide frame display processing unit 203, the position of the terminal device 2 being held by the user changes up, down, left, right, etc. in accordance with the change in the display position of the guide frame. In other words, the angle at which the camera 26 of the terminal device 2 captures the card changes. Note that the number of times the display position of the guide frame is changed by the guide frame display processing unit 203 may be any number of times as long as it is at least once.

[0063] Figure 5 is a diagram showing an example of changes in the shooting angle of a card taken by the camera 26. As the display position of the guide frame is changed by the guide frame display processing unit 203, the shooting angle of the card taken by the camera 26 changes to various angles, as shown in Figure 5. The video shooting unit 204 then records, as a video, images taken by the camera 26 in states where the shooting angle has changed in this way. By recording, as a video, images taken by the camera 26 in states where the shooting angle has changed, it becomes possible to include in the video shot by the video shooting unit 204 a feature that essentially has time axis information, namely, a change in the reflection pattern of reflected light according to the shooting angle.

[0064] The authenticity determination system 100 according to the present embodiment determines authenticity of cards that have undergone fine hologram and embossing processes. Embossing is achieved, for example, by printing a transparent varnish on the surface of the card. Therefore, when such a card is photographed, the reflection pattern of light reflected from the card changes in a subtle and diverse manner as the shooting angle of the camera 26 changes. In other words, the learning model 102a according to the present embodiment learns the subtle and diverse changes in the reflection pattern of light reflected from such a card. The surface treatment of the card is not limited to both fine hologram and embossing, and may be either one of the two. Furthermore, the surface treatment of the card may be other than hologram or embossing, such as gold foil stamping, silver foil stamping, or mother-of-pearl inlay, as long as it can change the reflection state of light reflected from the card surface.

[0065] <Authenticity Determination Process by Authentication Determination System> Next, the authenticity determination process by the authenticity determination system 100 according to this embodiment will be described with reference to Fig. 6. Fig. 6 is a flowchart showing an example of the procedure of the authenticity determination process by the authenticity determination system 100.

[0066] First, the guide frame display processing unit 203 (see FIG. 3) executes a guide frame display process (step S1). The guide frame display process of step S1 starts when a user starts an application incorporating a program that executes the authenticity determination process according to this embodiment and selects a card authenticity determination menu. The guide frame display process executed in step S1 will be described in detail later with reference to FIG. 7.

[0067] Next, the camera 26 starts taking pictures and recording the image data taken by the camera 26 as a moving image (step S2). The process of step S2 continues while the guide frame is displayed on the screen of the display device 30 in step S1 and the user moves the terminal device 2 up, down, left, right, etc. to align the frame of the card with the guide frame.

[0068] Next, the card area detection unit 205 extracts a card area from the image data captured by the camera 26 in step S2 using keypoint matching technology (step S3). Next, the first determination unit 206 performs a first determination process (step S4). The first determination process will be described in detail with reference to FIG. 8, which will be described later. Next, the normalization unit 207 normalizes the size of the video recorded in step S2 to a size suitable for input to the learning model 102a (step S5). Next, the second determination unit 208 performs a second determination process (step S6). The second determination process performed in step S6 will be described in detail with reference to FIG. 9, which will be described later.

[0069] (Guide Frame Display Processing) Next, the guide frame display processing performed in step S2 of Fig. 6 will be described with reference to Fig. 7. Fig. 7 is a flowchart showing an example of the procedure of the guide frame display processing performed by the guide frame display processing unit 203.

[0070] First, the guide frame display processing unit 203 displays a guide frame on the screen of the display device 30 (step S11). Next, the guide frame display processing unit 203 determines whether the frame of the card included in the image data captured by the camera 26 in step S2 fits within the guide frame (step S12). If it is determined that the card does not fit within the guide frame (NO in step S12), the guide frame display processing unit 203 continues to perform the determination in step S12.

[0071] On the other hand, if it is determined that the card has fit into the guide frame (YES in step S12), the guide frame display processing unit 203 displays a sign such as "OK" on the screen to indicate that the card has fit into the guide frame (step S13). Note that the sign indicating that the card has fit into the guide frame is not limited to the letters "OK" and may be other letters. Furthermore, the sign may not be letters, but may be other forms of notification, such as a change in the color or shape of the guide frame, or the emission of a sound.

[0072] Next, the guide frame display processing unit 203 determines whether the termination condition of the first determination process by the first determination unit 206 (see FIG. 3) has been satisfied (step S14). The termination condition of the first determination process is that both a determination result of "success" or "failure" by the first determination process and an image (moving image) captured by the camera 26 in the case where the determination is "success" are obtained. The moving image in the case where the determination is "success" is obtained when the first determination process for all guide frames has been completed. If it is determined in step S14 that the termination condition of the first determination process has not been satisfied (step S14 is NO), the guide frame display processing unit 203 changes the display position of the guide frame on the screen (step S15). After processing of step S15, the guide frame display processing unit 203 returns to step S11 and performs processing.

[0073] On the other hand, if it is determined in step S14 that the termination condition for the first determination process is satisfied (YES in step S14), the guide frame display process is terminated by the guide frame display processing unit 203. Note that the guide frame display process by the guide frame display processing unit 203 may be terminated at a predetermined timing after the determination result of the first determination process is output.

[0074] (First Determination Process) Next, the first determination process performed in step S4 of Fig. 6 will be described with reference to Fig. 8. Fig. 8 is a flowchart showing an example of the procedure of the first determination process performed by the first determination unit 206.

[0075] First, the first determination unit 206 extracts key points from the card area detected in the captured image in step S3 of Fig. 6 (step S41). Next, the first determination unit 206 determines whether the movement vector of the key point extracted in step S41 and the movement vector of the six-axis sensor 27 of the terminal device 2 are moving simultaneously in the same direction (step S42).

[0076] If it is determined that the movement vector of the key point and the movement vector of the six-axis sensor 27 are moving simultaneously in the same direction (YES in step S42), the first determination unit 206 determines that the camera 26 is photographing a physical card (step S43). On the other hand, if it is determined that the movement vector of the key point and the movement vector of the six-axis sensor 27 are not moving simultaneously in the same direction (NO in step S42), the first determination unit 206 determines that the camera 26 is photographing an object other than a physical card (step S44). After processing step S44 or step S45, the first determination process by the first determination unit 206 ends. Note that the determination in step S42 can be made by, for example, analyzing the movement directions of both movement vectors, but can also be made based on, for example, the magnitude of the correlation coefficient between both movement vectors.

[0077] (Second Determination Process) Next, the second determination process performed in step S6 of Fig. 6 will be described with reference to Fig. 9. Fig. 9 is a flowchart showing an example of the procedure of the second determination process performed by the second determination unit 208.

[0078] First, the second determination unit 208 inputs the video image of the card area normalized in step S5 of Fig. 6 into the learning model 102a (step S51). Next, the learning model 102a verifies whether the change in the reflection pattern of reflected light obtained from the normalized image of the card area photographed by the camera 26 matches the change in the reflection pattern of reflection from a genuine card that has been previously learned (step S52). Next, the second determination unit 208 outputs the authenticity determination result of the card as a result of verification by the learning model 102a (step S53).

[0079] If the learning model 102a can confirm that the change in the reflection pattern of reflected light obtained from the image of the card area matches the change in the reflection pattern of reflection from a pre-learned genuine card, it outputs an authenticity determination result that the card photographed by the camera 26 is a genuine (genuine) card. On the other hand, if the learning model 102a cannot confirm that the change in the reflection pattern of reflected light obtained from the image of the card area matches the change in the reflection pattern of reflection from a pre-learned genuine card, it outputs an authenticity determination result that the card photographed by the camera 26 is an illegal (counterfeit) card. After the processing of step S63, the second determination processing by the second determination unit 208 ends.

[0080] In the embodiment described above, the second determination unit 208 of the terminal device 2 determines the authenticity of a card based on time-series information of the light reflected from the card, i.e., information on changes in the reflection pattern of the reflected light, obtained by the camera 26. Therefore, according to this embodiment, it is possible to determine the authenticity of a card without adding or connecting an authenticity determination device such as a wide-angle microscope to the terminal device 2.

[0081] Furthermore, in this embodiment, the terminal device 2 determines the authenticity of the card based on the time-series information of the light reflected from the card obtained by the camera 26, so there is no need to embed an NFC tag or the like for authenticity determination in the card, or to apply a watermark or special processing for authenticity determination. This not only prevents an increase in the manufacturing cost of the card, but also prevents the aesthetic appeal of the card from being impaired.

[0082] Furthermore, in this embodiment, the first determination unit 206 makes the first determination of whether or not the camera 26 is capturing an image of a physical card, using a detection value from the six-axis sensor 27 provided in the terminal device 2. Therefore, in this embodiment, it is possible to make a determination of the authenticity of a card (first determination) without introducing new hardware into the terminal device 2.

[0083] Furthermore, for example, in authenticating a personal identification number (My Number) card, information from a photographed image of the card is used, so a guide frame or the like cannot be displayed on the screen, but in this embodiment, the authenticity of the card is determined using a moving image taken by the camera 26 as input. Therefore, a guide frame can be displayed in real time on the screen of the display device 30.

[0084] The guide frame display processing unit 203 then moves the position of the guide frame displayed on the screen of the display device 30, thereby allowing the user to perform the operation required for the terminal device 2 to acquire the information necessary for determining the authenticity of the card. Furthermore, since this operation is a simple operation of moving the terminal device 2 so that the frame portion of the card fits into the guide frame, even users who are unfamiliar with operating the terminal device 2 or users who are beginners to trading card games can properly photograph the card.

[0085] In this embodiment, since the learning model 102a constructed by a neural network is used to determine whether a card is authentic or counterfeit, there is a risk that a deceptive signal that exploits the vulnerability of the neural network model using a method not anticipated by the designer may be input when the second determination is performed. In fact, known attack methods include a method of deceiving a neural network model using a noisy image and a "jailbreak" attack method for circumventing the constraints of large language models (LLMs). However, a method for safely using neural network models has not yet been established.

[0086] In contrast, the authenticity determination system 100 according to the present embodiment employs keypoint matching, an algorithm that is less flexible but behaves deterministically and therefore less vulnerable, in the first determination. Meanwhile, the neural network model used in the second determination is more flexible but behaves non-deterministically. By configuring the authenticity determination system 100 in this manner, it becomes possible to determine the authenticity of a card by having the neural network model (learning model 102a) learn video images of the card, without requiring the neural network model to learn cheating techniques on a large scale.

[0087] The present invention is not limited to the above-described embodiments, and various other applications and modifications are possible without departing from the spirit of the present invention as defined in the claims. For example, the above-described embodiments provide detailed and specific descriptions of the system and device configurations to clearly explain the present invention, and are not necessarily limited to those including all of the described configurations. Furthermore, it is also possible to add, delete, or replace some of the configurations of the present embodiments with other configurations. Furthermore, the control lines and information lines shown are those considered necessary for explanation, and do not necessarily represent all control lines and information lines in the product. In reality, it can be assumed that almost all of the configurations are interconnected.

[0088] REFERENCE SIGNS LIST 1...server, 2...terminal device, 25...camera, 26...6-axis sensor, 29...display device, 100...authenticity determination system, 101...master image DB, 102, 202...learning model, 203...guide frame display processing unit, 204...moving image capturing unit, 205...card area detection unit, 206...first determination unit, 207...normalization unit, 208...second determination unit

Claims

1. A program for determining the authenticity of a card medium whose specified surface has been modified to change the reflection state of light reflected from the specified surface, comprising: a display control procedure for displaying a guide for changing the shooting angle of a video image of the card medium captured by a camera on the screen of a terminal device and changing the display position of the guide at least once; and a determination procedure for determining the authenticity of the card medium based on time series information of the light reflected from the specified surface of the card medium contained in the video image data of the card medium captured by the camera while the guide is displayed.

2. The program according to claim 1, wherein the determination procedure includes a procedure for verifying the authenticity of the card medium based on time series information of the reflected light contained in the video image data and time series information of the reflected light from the specified surface contained in video image data obtained by photographing an authentic card medium.

3. The program according to claim 2, wherein the determination procedure includes a procedure for having a machine learning model perform the matching using the video data as input and outputting a determination result of the authenticity of the card medium.

4. The program described in claim 3, wherein the learning data for the machine learning model is learning data obtained by learning the time series information of reflected light from the specified surface contained in the video image data of the authentic card medium photographed based on the guide.

5. The program according to claim 4, wherein the learning model is constructed by a neural network having time axis information in addition to spatial information.

6. The program described in claim 5, wherein the determination procedure includes a step of making a first determination as to whether the camera is photographing the physical card medium by comparing the movement vectors of feature points extracted from the video image data transmitted from the camera with each movement vector in multiple coordinate axis directions obtained from an inertial sensor provided in the terminal device, prior to a second determination process of determining the authenticity of the card.

7. The program described in claim 6, wherein the determination procedure performs the first determination by determining whether the movement vector of the feature point extracted from the video image data transmitted from the camera and the movement vector obtained from the inertial sensor are moving simultaneously in the same direction.

8. The program according to claim 7, further comprising a normalization unit that extracts a card area corresponding to the card from the video captured by the camera and normalizes the number of pixels in the width and height of the extracted card area in the video to a fixed length of pixel number suitable for input to the learning model.

9. The program according to claim 5, wherein the display control procedure includes a procedure for displaying, as the guide, an image on the screen of the terminal device that specifies the display position of the card medium on the screen of the terminal.

10. The program according to claim 9, wherein the display control procedure includes a procedure for changing the display position of the image specifying the display position of the card medium on the screen to a position different from the previous display position when it is determined that the frame portion of the card medium has fit into the display position of the card medium indicated by the image specifying the display position of the card medium.

11. A program described in any one of claims 1 to 10, wherein the reflection patterns of the reflected light from the specified surface corresponding to the incident light from a plurality of different incident angles are different due to processing applied to the specified surface of the card medium.

12. The program according to claim 11, wherein the processing of the predetermined surface includes processing to form an uneven portion using a transparent varnish printed on the first surface of the card medium.

13. A method for determining the authenticity of a card medium whose specified surface has been subjected to processing capable of changing the reflection state of light reflected from the specified surface, comprising: a display control procedure for displaying a guide for changing the shooting angle of a video image of the card medium captured by a camera on the screen of a terminal device and changing the display position of the guide at least once; and a determination procedure for determining the authenticity of the card medium based on time series information of the light reflected from the specified surface of the card medium, which is included in the video image data of the card medium captured by the camera while the guide is displayed.

14. An authenticity determination device that determines the authenticity of a card medium whose specified surface has been processed to change the reflection state of light reflected from the specified surface, comprising: a display control unit that displays a guide on the screen of a terminal device to change the shooting angle of a video image of the card medium captured by a camera and changes the display position of the guide at least once; and a determination unit that determines the authenticity of the card medium based on time series information of the light reflected from the specified surface of the card medium contained in the video image data of the card medium captured by the camera while the guide is displayed.

Images