Authenticity determination program, authenticity determination method, and authenticity determination device

Abstract

To provide an authenticity determination program for determining the authenticity of a card medium by obtaining an image that can be used to determine the authenticity of the card medium by instructing re-photography when a defective photographing part such as whiteout or the like is detected in the image. [Solution] The authenticity determination program causes a computer to execute a poorly photographed area detection procedure, which detects whether or not there are any poorly photographed areas in the image of the card medium when it is not certain that a known image has been detected, and if it detects that there are any poorly photographed areas, instructs the computer to take a video image of the card medium again and executes a display control procedure; and a determination procedure, which determines the authenticity of the card medium based on time series information of reflected light from a specific surface of the card medium contained in video image data of the card medium photographed by the camera with a guide displayed when it is certain that a known image has been detected, or if it detects that there are no poorly photographed areas or that there are significantly fewer poorly photographed areas.

Description

[Technical Field]

[0001] The present invention relates to an authenticity determination program, an authenticity determination method, and an authenticity determination device. [Background technology]

[0002] Various techniques have been devised to determine the authenticity of cards used in trading card games. For example, a technique is known in which a near-field communication (NFC) chip storing information for authenticity determination is embedded in a card and the information is read by an NFC tag reader to determine the authenticity of the card. This technique makes it possible to determine the authenticity of a card with high accuracy. However, because the manufacturing cost of NFC chips is high, incorporating an NFC chip into a card increases the manufacturing cost of the card.

[0003] For this reason, attempts have been made to photograph the card with a camera built into a mobile device such as a smartphone and use the photographed image to verify the authenticity of the card. In addition to verifying the authenticity of the card, attempts have also been made to perform authentication processing (also called check-in) using the photographed image in an access-managed authentication system.

[0004] For example, Patent Document 1 discloses a technology in which a guide for changing the shooting angle of a moving image of a card medium captured by a camera is displayed on the screen of a terminal device, the display position of the guide is changed at least once, and the authenticity of a card medium is determined based on time series information of reflected light from a specific surface of the card medium contained in the moving image data of the card medium captured by the camera while the guide is displayed. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent No. 7594140 Summary of the Invention [Problem to be solved by the invention]

[0006] A well-known software-based authentication method is a technology called "visual check-in," which was invented by the applicant. Visual check-in technology allows users to obtain in-game rewards by taking photos of TV programs, posters, product exteriors, etc. with their smartphone cameras.

[0007] Conventional cards often feature special processing such as holograms and embossing, which can create a glossy surface. For this reason, in photography environments with light sources, excessive overexposure (hereafter referred to as "blown-out highlights") can easily occur, affecting authentication using artificial intelligence (AI). Blown-out highlights are a phenomenon in which excessive light during photography causes the loss of color and gradation information, resulting in a partially or entirely pure white image. When blown-out highlights occur, the shape and texture information contained in the blown-out highlights is saturated and lost, significantly reducing the accuracy of authenticity determination and check-in processing. Therefore, there is a need for a system that can automatically detect blown-out highlights and warn users.

[0008] Although excessive light incident on a camera's image sensor or the like can cause photon saturation and result in overexposure, processing can be performed to reduce the effect of photon saturation on the captured image by changing the dynamic range, etc. On the other hand, even if excessive light is not incident on a camera's image sensor or the like, overexposure can occur in the captured image due to light reflection from the surface of the subject, etc.

[0009] However, the software built into mobile devices alone cannot perform simple processes such as determining areas of a captured image that have been blown out due to the effects of auto exposure (AE). Furthermore, mobile devices are equipped with software that processes captured images, and automatically corrected captured images do not contain data before development, such as RAW data. For this reason, authenticity of card media must be determined based on captured images that have been developed by the software or hardware built into the mobile device.

[0010] The technology disclosed in the above-mentioned Patent Document 1 does not take into consideration reducing the effects of overexposure. Therefore, if the captured image contains poorly captured areas such as overexposure, check-in cannot be completed normally.

[0011] The present invention has been made in consideration of such circumstances, and aims to enable the authenticity of the card medium to be determined by instructing the user to take a photo again when a defective photographed area is detected in the image of the card medium. [Means for solving the problem]

[0012] The authenticity determination program of the present invention determines the authenticity of a card medium whose predetermined surface has been subjected to processing that changes the reflection state of light reflected from the predetermined surface. This authenticity determination program includes: a display control procedure that displays a guide on the screen of a terminal device to change the shooting angle of a moving image of the card medium captured by a camera and changes the display position of the guide during the shooting period; a known image detection procedure that detects a known image formed on the card medium from moving image data of the card medium captured in accordance with the guide; a poorly photographed portion detection procedure that, when it is not certain that the known image has been detected, detects whether or not there are any poorly photographed areas in the image of the card medium and, when it is detected that there are any poorly photographed areas, instructs the camera to capture a moving image of the card medium again and causes the display control procedure to be performed; and a determination procedure that, when it is certain that the known image has been detected or when it is detected that there are no poorly photographed areas or that there are significantly fewer poorly photographed areas, determines the authenticity of the card medium based on time-series information of reflected light from the predetermined surface of the card medium contained in the moving image data of the card medium captured by the camera with the guide displayed. Let the computer run it. The above-described authenticity determination program is one aspect of the present invention, and an authenticity determination method and an authenticity determination device that reflect one aspect of the present invention are configured in the same manner as the above-described authenticity determination program. [Effects of the Invention]

[0013] According to the present invention, when a poorly photographed area is detected in an image of a card medium, a command to take a photograph again is given, thereby obtaining an image that can be used to determine the authenticity of the card medium, thereby making it possible to determine the authenticity of the card medium. Problems, configurations, and effects other than those described above will become apparent from the following description of the embodiments. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is an overall configuration diagram showing an overview of an authenticity determination system according to a first embodiment of the present invention. [Figure 2]1 is a block diagram showing an example of the hardware configuration of each of a server and a terminal device that constitute an authenticity determination system according to a first embodiment of the present invention. [Figure 3] 1 is a block diagram showing the functional configuration of an authenticity determination system according to a first embodiment of the present invention. [Figure 4] 1 is a block diagram showing an example of the internal configuration of a learning device according to a first embodiment of the present invention. [Figure 5] 3 is a flowchart showing an example of a known item learning process according to the first embodiment of the present invention. [Figure 6] 4 is a flowchart showing an example of an authenticity determination learning process according to the first embodiment of the present invention. [Figure 7] 5A to 5C are diagrams illustrating an example of a guide frame display based on a guide frame display process performed by a display processing unit according to the first embodiment of the present invention. [Figure 8] 5A to 5C are diagrams showing examples of changes in the angle at which a card is photographed by a camera according to the first embodiment of the present invention. [Figure 9] 1A and 1B are diagrams showing examples of images in which whiteout appears according to the first embodiment of the present invention. [Figure 10] 3 is a flowchart showing an example of the procedure of an authenticity determination process performed by the authenticity determination system according to the first embodiment of the present invention. [Figure 11] 10 is a flowchart showing an example of a procedure for a guide frame display process by a display processing unit according to the first embodiment of the present invention. [Figure 12] 5 is a flowchart showing an example of a procedure for shooting determination processing by a shooting determination unit according to the first embodiment of the present invention. [Figure 13] 5 is a flowchart showing an example of a procedure for a known item detection process performed by a known item detection unit according to the first embodiment of the present invention. [Figure 14] 10 is a flowchart showing an example of the procedure of two-stage blown-out highlight detection processing by a two-stage blown-out highlight detection unit according to the first embodiment of the present invention. [Figure 15] 5 is a flowchart showing an example of the procedure of an authenticity determination process performed by an authenticity determination unit according to the first embodiment of the present invention. [Figure 16]FIG. 10 is a block diagram showing an example of the hardware configuration of each of a server and a terminal device that constitute an authenticity determination system according to a second embodiment of the present invention. [Figure 17] FIG. 10 is a block diagram showing the functional configuration of an authenticity determination system according to a second embodiment of the present invention. [Figure 18] FIG. 10 is a block diagram showing the functional configuration of an authenticity determination system according to a third embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0015] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functions or configurations are designated by the same reference numerals, and redundant description will be omitted.

[0016] [First embodiment] <Outline of the authenticity determination system> FIG. 1 is a diagram showing the overall configuration of an authentication system 100 according to a first embodiment of the present invention. The authenticity determination system 100 is a system that determines whether, for example, cards (an example of card media) used in a trading card game are genuine (authentic) or counterfeit (non-genuine). The authenticity determination system 100 includes a learning device 3, a smartphone 21, and a PC (Personal Computer) 22. The smartphone 21 and PC 22 can be connected to the learning device 3 via a network N such as the Internet. In the following description, the smartphone 21 and PC 22 are collectively referred to as the information processing terminal 2. Note that a dedicated gaming terminal may also be used as the information processing terminal 2.

[0017] The card Cd (see FIG. 7) that is the subject of authentication determination by the authentication determination system 100 is a card that has a hologram and embossing (an example of textured surface) on its surface (an example of a predetermined surface). The information processing terminal 2 determines the authenticity of the card based on information obtained by photographing the card with the camera 116 (see FIG. 2).

[0018] The authenticity determination system 100 according to this embodiment uses information such as information about changes in reflected light from the card according to the shooting angle of the camera 116 of the information processing terminal 2 as information obtained by the camera 116 photographing the card. Furthermore, the information processing terminal 2 according to this embodiment changes the pattern of changes over time in the reflected light from the card by displaying on the screen of the display device 119 a guide frame that prompts the user to move the terminal device in various directions such as up, down, left, and right while photographing the card, i.e., a guide frame for changing the shooting angle of the card by the camera 116.

[0019] However, light reflected from the card can cause defective photographs, such as overexposed highlights, in the captured image. As mentioned above, overexposed highlights prevent accurate authentication of the captured image. Therefore, a warning message is displayed instructing the user to retake a photo of the card to prevent overexposed highlights.

[0020] Then, information processing terminal 2 performs photography determination processing, known item detection processing, two-stage blown-out highlight detection processing, normalization processing, and authenticity determination processing. In the photography determination processing, information processing terminal 2 determines whether camera 116 of information processing terminal 2 is photographing a physical card medium, using as a key whether the movement vector of 6-axis sensor 117 (see FIG. 2) provided in information processing terminal 2 and the movement vector of feature points extracted from the image photographed by camera 116 change simultaneously and in the same direction.

[0021] Furthermore, in a known item detection process, the information processing terminal 2 determines whether or not the subject is a known item (for example, a card) based on the captured image. Furthermore, in a two-stage blown-out highlight detection process, the information processing terminal 2 determines whether or not the captured image has blown-out highlights that would prevent accurate authenticity determination. Furthermore, in a normalization process, the information processing terminal 2 normalizes a captured image that is determined not to have blown-out highlights.

[0022] In the authentication determination process, the information processing terminal 2 determines the authenticity of the card based on information on the change in the time direction of the pattern of change in the light reflected from the card, which is included in the video recorded when the photographing and determination is performed, i.e., based on the time-series information of the light reflected from the card. The information processing terminal 2 performs the authenticity determination process using the trained authenticity determination model 202b.

[0023] The learning device 3 is a server that manages a master image used for photographic determination by the information processing terminal 2, a known item detection model 45a used for known item detection, and an authenticity determination model 45b used for authenticity determination (see FIG. 3 for all of these). The learning device 3 synchronizes the master image stored in the information processing terminal 2 with the master image managed by the learning device 3 by transmitting the master image to the information processing terminal 2 via the network N at predetermined intervals. The learning device 3 also synchronizes the known item detection model 45a and the authenticity determination model 45b stored in the information processing terminal 2 with the master image managed by the learning device 3 by transmitting the known item detection model 45a and the authenticity determination model 45b to the information processing terminal 2 via the network N at predetermined intervals.

[0024] The information processing terminal 2 is an example of an authenticity determination device. An application (not shown) including a program for executing each process related to authenticity determination according to this embodiment is stored in the nonvolatile storage 121 (see FIG. 2) of the information processing terminal 2. The control unit 110 (see FIG. 2) of the information processing terminal 2 starts the application based on a user operation, and performs a predetermined process by reading the program recorded in the nonvolatile storage 121.

[0025] The predetermined processing includes the above-mentioned processing of photograph determination, authenticity determination, etc. The predetermined processing also includes processing in which control unit 110 of information processing terminal 2 displays, as a guide, an image of a guide frame (hereinafter also simply referred to as "guide frame") that specifies the display position of the card on the screen of display device 119 on the screen of display device 119. Furthermore, the predetermined processing includes processing in which information captured by camera 116 while the user is performing the action of aligning the display position of the card with the guide frame is recorded as a moving image.

[0026] The input device 118 generates an operation signal according to an operation input by the user, and supplies the operation signal to the control unit 110 . The display device 119 displays a guide frame and the like on the screen based on the control of the control unit 110.

[0027] <Example of hardware configuration for authenticity determination system> Next, an example of the hardware configuration of the authenticity determination system 100 according to the first embodiment of the present invention will be described with reference to Fig. 2. Fig. 2 is a block diagram showing an example of the hardware configuration of the learning device 3 and the information processing terminal 2 that constitute the authenticity determination system 100.

[0028] (Example of learning device configuration) The learning device 3 is an example of a calculator that operates as a computer. The learning device 3 includes a control unit 130, a nonvolatile storage 135, and a network interface 136, which are all connected to a bus B1.

[0029] The control unit 130 includes a central processing unit (CPU) 131, a graphics processing unit (GPU) 132, a read only memory (ROM) 133, and a random access memory (RAM) 134. The control unit 130 may be configured using a field programmable gate array (FPGA).

[0030] The CPU 131 reads out program code of software that realizes each function according to this embodiment from the ROM 133, loads it into the RAM 134, and executes it. Variables, parameters, etc. that are generated during the calculation processing of the CPU 131 are temporarily written to the RAM 134. These variables, parameters, etc. written to the RAM 134 are read out by the CPU 131 as appropriate. The GPU 132 performs calculations required when performing various learning processes according to this embodiment.

[0031] Examples of nonvolatile storage 135 include a hard disk drive (HDD), a solid state drive (SSD), an optical disk, a magneto-optical disk, and nonvolatile memory. In addition to an operating system (OS) and various parameters, this nonvolatile storage 135 also stores programs for running the learning device 3. The programs for running the learning device 3 may be stored in ROM 133. In other words, ROM 133 and nonvolatile storage 135 are used as an example of a computer-readable, non-transitory recording medium that stores a program executed by the learning device 3.

[0032] The nonvolatile storage 135 also includes a learning data database (DB) 40 that stores learning data used in machine learning, and a master image DB 50 (see FIG. 3) that stores master images. The nonvolatile storage 135 also includes a server-side learning model DB 45 (see FIG. 3) that stores a known item detection model 202a and an authenticity determination model 202b.

[0033] For example, a network interface card (NIC) or the like is used as the network interface 136. The network interface 136 transmits and receives various data to and from the information processing terminal 2 via a dedicated line or the like connected to a terminal of the NIC and via the network N.

[0034] (Example of terminal device configuration) The information processing terminal 2 is an example of a calculator that operates as a computer capable of executing various programs. The information processing terminal 2 includes a control unit 110, a network interface 115, a camera 116, a six-axis sensor 117, an input device 118, a display device 119, an input / output interface 120, and a non-volatile storage 121, which are all connected to a bus B2.

[0035] The control unit 110 includes a CPU 111, a GPU 112, a ROM 113, and a RAM 114. The control unit 110 may be configured as an FPGA.

[0036] The CPU 111 reads out program code of software that realizes each function according to the present embodiment from the ROM 113, loads it into the RAM 114, and executes it. Variables, parameters, etc. that are generated during the arithmetic processing of the CPU 111 are temporarily written to the RAM 114. These variables, parameters, etc. written to the RAM 114 are read out as appropriate by the CPU 111. The CPU 111 performs processes such as processing of the OS in the information processing terminal 2 and management of input and output of data performed by each unit in the information processing terminal 2. The GPU 112 performs calculations required for performing various inference processes according to this embodiment in addition to rendering images.

[0037] For example, a NIC or the like is used as the network interface 115. The network interface 115 can acquire various data from the learning device 3 via a LAN, a dedicated line, or the like connected to a terminal of the NIC, and can communicate with other information processing terminals 2.

[0038] The camera 116 includes an image sensor such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor), and converts light from a subject focused on a lens (not shown) into image data. The camera 116 according to this embodiment is capable of capturing not only still images but also moving images.

[0039] The six-axis sensor 117 is an inertial sensor that can detect six-axis inertial forces, including accelerations in the three axes (front-back, left-right, and up-down) and angular velocities in the three axes that are applied to the information processing terminal 2. Information on the values ​​detected by the six-axis sensor 117 is referenced when performing the above-described photographing determination. Note that the inertial sensor included in the information processing terminal 2 is not limited to the six-axis sensor 117, and may be any other inertial sensor implemented in the information processing terminal 2.

[0040] The input device 118 is configured by a touch sensor on a touch panel in the smartphone 21 (see FIG. 1), and is configured by a mouse, keyboard, and the like in the PC 22. The display device 119 is configured by an LCD (Liquid Crystal Display), an organic EL (Electro Luminescence) display, or the like.

[0041] The input / output interface 120 converts operation signals received from the input device 118 into data in a predetermined format and passes the data to the CPU 111. The input / output interface 120 also passes images of guide frames and the like output from the control unit 110 to the display device 119. The display device 119 may be equipped with a speaker (not shown), in which case the input / output interface 120 can also output audio signals to the speaker.

[0042] The nonvolatile storage 121 may be, for example, an HDD, SSD, optical disk, magneto-optical disk, or nonvolatile memory. In addition to the OS and various parameters, the nonvolatile storage 121 stores programs for operating the information processing terminal 2, a local image DB 201 based on the master image DB 50 transmitted from the learning device 3, a known item detection model 202a, and an authenticity determination model 202b (see FIG. 3 ).

[0043] A program for causing the learning device 3 to function may be stored in the ROM 113. In other words, the ROM 113 and the nonvolatile storage 121 are used as an example of a computer-readable non-transitory recording medium that stores a program executed by the information processing terminal 2.

[0044] If the nonvolatile storage 121 is an external recording medium such as an optical disc, an optical drive (not shown) reads the nonvolatile storage 121 and extracts data from it. The data extracted by the optical drive is used for predetermined arithmetic processing by the CPU 111.

[0045] <Example of functional configuration of authenticity determination system> Next, the functional configuration of the authenticity determination system 100 will be described with reference to FIG. Fig. 3 is a block diagram showing the functional configuration of the authenticity determination system 100. In Fig. 3, the communication network N is omitted.

[0046] (Server functional configuration example) As shown in Fig. 3, the learning device 3 includes a learning data DB 40, a master image DB 50, and a server-side learning model DB 45, which are configured in the non-volatile storage 121 shown in Fig. 2. The learning data DB 40 is an example of a learning data recording unit that records learning data for each learning model stored in the server-side learning model DB 45. The master image DB 50 is a DB that serves as the basis for the local image DB 201 that stores images referenced by the photography determination unit 206 of the information processing terminal 2 when performing photography determination, and stores photographed images of all types of legitimate (authentic) cards.

[0047] The server-side learning model DB 45 is an example of a learning model recording unit that records a known item detection model 45a and a trained authenticity determination model 45b constructed by a neural network that performs learning taking time axis information into consideration. The known item detection model 45a and the authenticity determination model 45b are composed of weight data whose weights have been adjusted by learning.

[0048] The known item detection model 45a is an example of an AI that is based on, for example, a YOLO-based high-speed multitasking object detection network and can simultaneously estimate the identifier of a card for a card game and the written language (Japanese, English, Simplified Chinese, etc.). The known item detection model 45a can be automatically learned and updated by the learning device 3 even when a new type of card is added.

[0049] The authenticity determination model 45b is an image recognition model, such as SlowFast, that analyzes the temporal change patterns of reflected light from diffractive optical structures, such as holograms and embossments, on the card surface and classifies these temporal change patterns based on learning to determine the authenticity of the card medium. Depending on the processing applied to a specific surface of the card medium, the temporal change patterns of reflected light from each specific surface corresponding to incident light from multiple different angles of incidence are different. Therefore, the input to the authenticity determination model 45b is a short video captured by a user while slowly tilting their smartphone up, down, left, and right. The authenticity determination model 45b continuously acquires brightness changes that vary for each shooting angle due to the movement during shooting.

[0050] For neural networks that perform learning taking time axis information into account, spatiotemporal architectures such as 3D-CNN (3-Dimensional Convolutional Neural Network), SlowFast, and TimeSformer (Time-Space Transformer) can be applied. The authenticity determination model 45b comprehensively maps the spatial features of each frame of the input video image and the temporal changes between frames into a feature space. During inference for authenticity determination, the authenticity determination model 45b inputs this spatiotemporal feature sequence into a transfer learning model that performs binary classification of authenticity. The authenticity determination model 45b determines the card as "authentic" only if the confidence score exceeds a predetermined threshold. Therefore, the authenticity determination model 45b can identify the authenticity of gaming cards with extremely high accuracy using only optical features, without embedding semiconductor elements such as NFC in the card.

[0051] For example, when the known item detection model 45a and the authenticity determination model 45b are constructed using a deep learning framework called PyTorch, the known item detection model 45a and the authenticity determination model 45b are implemented in the PTH file format. Note that the file formats of the known item detection model 45a and the authenticity determination model 45b may be other file formats such as the GGML file format or the ONNX (Open Neural Network eXchange) file format.

[0052] The learning device 3 periodically transmits the master image stored in the master image DB 50, and the known item detection model 45a and the authenticity determination model 45b stored in the server-side learning model DB 45 to the information processing terminal 2. The known item detection model 45a and the authenticity determination model 45b received by the information processing terminal 2 are stored in the local-side inference model DB 202 as the known item detection model 202a and the authenticity determination model 202b, respectively.

[0053] (Example of terminal device functional configuration) The information processing terminal 2 includes a local image DB 201, a local inference model DB 202, a display processing unit 203, a video image capturing unit 204, a card area detection unit 205, a six-axis sensor 117, a capturing judgment unit 206, a known item detection unit 207, a two-stage whiteout detection unit 208, a normalization unit 209, and an authenticity judgment unit 210.

[0054] The local image DB 201 is the master image DB 50 transmitted from the learning device 3 in a format that can be accessed (referenced) by an application of the information processing terminal 2.

[0055] The local-side inference model DB 202 is a database that stores the known item detection model 202a and the authenticity determination model 202b transmitted from the learning device 3. As described above, the known item detection model 202a and the authenticity determination model 202b have the same contents as the known item detection model 45a and the authenticity determination model 45b stored in the server-side learning model DB 45, but have different symbols.

[0056] The learning data for the authenticity determination model 202b is time-series information on the light reflected from the surface of a card, which is included in video data of a genuine (authentic) card photographed by the camera 116 based on guidance from the guide frame. The authenticity determination model 202b then inputs video data of the card to be authenticated, photographed based on guidance from the guide frame, and outputs an authenticity determination result for the card. The authenticity determination model 202b analyzes the pattern of change over time in the light reflected from the card to be authenticated using, for example, a classifier that uses SlowFast, and determines the authenticity of the card by learning-based classification based on whether the pattern of change over time matches the characteristic movement seen in genuine cards.

[0057] The display processing unit 203 (an example of a display control unit) is a processing unit that displays a guide frame on the screen of the display device 119 (see FIG. 2). The guide frame is indicated, for example, by a thick red frame. When the display processing unit 203 detects that the frame of the card has fit within the guide frame, it displays a sign such as "OK" on the screen. Thereafter, the display processing unit 203 moves the guide frame to another position on the screen and prompts the user to perform an operation to fit the frame of the card into that position. Examples of display of the guide frame by the display processing unit 203 will be described in detail with reference to FIGS. 7 and 8 below.

[0058] The video capture unit 204 encodes image data obtained from the camera 116 as a video at a predetermined frame rate. The video capture unit 204 saves the captured video in the non-volatile storage 121 in a general-purpose format such as H.264 / MP4. The video capture unit 204 continues capturing video while the display processing unit 203 is changing the display position of the guide frame.

[0059] The card area detection unit 205 detects the position and type of the card from image data obtained from the camera 116, for example, using YOLO (You Only Look Once). YOLO is a real-time object detection model that processes an image all at once and simultaneously estimates the position and category of an object at high speed. After determining the type of card using YOLO, the card area detection unit 205 detects the card area using, for example, KAZE. However, the card area detection unit 205 may also use YOLO to perform both the detection of the card type and the card area. Alternatively, the card area detection unit 205 may also use AKAZE (Accelerated-KAZE) to perform both the detection of the card type and the card area. By using such a technique, sufficient processing speed can be maintained even when the information processing terminal 2 is a smartphone 21.

[0060] The photography determination unit 206 determines whether the camera 116 of the information processing terminal 2 is photographing a physical card medium whose authenticity is to be determined, based on the movement of the six-axis sensor 117 (see FIG. 2) and the movement of the card position detected by the card area detection unit 205. Specifically, the photography determination unit 206 executes a photography determination procedure to determine whether the camera 116 is photographing a physical card medium by comparing the movement vector of the card position extracted from the video data transmitted from the camera 116 with each of the movement vectors in the multiple coordinate axis directions acquired from the six-axis sensor 117. Through this photography determination procedure, the photography determination unit 206 can determine whether the movement vector of the card position extracted from the video data and the movement vector acquired from the six-axis sensor 117 are moving simultaneously in the same direction.

[0061] By performing the photographing judgment by the photographing judgment unit 206, even if a cheating method (unauthorized operation) related to the method of photographing a card is executed, the cheating method can be detected. Examples of cheating methods related to the method of photographing a card include a method of photographing a moving image obtained by photographing the card in advance, playing it in front of the camera 116 of the information processing terminal 2, and a method of photographing a high-resolution copy image of the card with the camera 116.

[0062] Furthermore, the shooting determination unit 206 generates coordinate information of the guide frame for changing the guide frame to the next display position during execution of the shooting determination, and outputs the information to the display processing unit 203 .

[0063] The known item detection unit 207 uses the known item detection model 202a to execute a known image detection procedure that detects a known image formed on a card medium that has been processed on a predetermined surface from video image data of the card medium that has been photographed according to the guide. In this embodiment, a known item is an example of a known image (genuine image), and the known item detection unit 207 is an example of a known image detection unit. The known item detection unit 207 estimates information unique to the card medium that is included in the image of the card medium based on the known image, and outputs a certainty score that indicates the certainty of the estimated unique information.

[0064] The two-stage overexposure detection unit 208 is activated only when the certainty score of the subject area output by the known item detection model 202a of the known item detection unit 207 is neither too low nor too high, but is in the middle range. This is because the design takes into consideration that overexposure can occur frequently even in well-intentioned usage environments, but has characteristics that differ from intentional concealment.

[0065] In the procedure for detecting poorly photographed areas executed by the two-stage blown-out highlight detection unit 208, when it is not certain that a known image has been detected, the two-stage blown-out highlight detection unit 208 detects, for example, the presence or absence of blown-out highlights in the image as a poorly photographed area in the image on the card medium. Note that when the certainty score is less than a predetermined certainty threshold, the two-stage blown-out highlight detection unit 208 determines that it is not certain that the photographed subject is a known image. Here, the two-stage blown-out highlight detection unit 208 detects the presence or absence of blown-out highlights in two stages: global analysis and local analysis. Details of the global analysis and local analysis will be described later with reference to FIG. 14.

[0066] This two-stage blown-out highlight detection unit 208 is an example of a poorly captured part detection unit that, when it detects that an image has blown-out highlights, issues an instruction to shoot a moving image on the card medium again and operates the display processing unit 203. The two-stage blown-out highlight detection unit 208 issues an instruction to shoot a moving image on the card medium again by outputting, to the display processing unit 203, information instructing the display processing unit 203 to correct the shooting conditions.

[0067] The display processing unit 203 displays a message for re-shooting, as shown in Fig. 9, described below, based on the shooting condition correction instruction information. In this way, the display processing unit 203 displays correction instruction information that prompts the user to immediately correct the lighting conditions and the angle of the camera 116, and performs the display control procedure again. For this reason, it is expected that the re-shot image will not have whiteout, and the shooting determination procedure and known item detection procedure are performed using moving images that are less affected by whiteout, thereby reducing erroneous determination of known images.

[0068] The normalization unit 209 is executed when the known item detection unit 207 is certain to have detected a known image, or when the two-stage highlight detection unit 208 detects that the image has no highlights or has significantly less highlights. The normalization unit 209 extracts an image of a card area from the video captured by the video capture unit 204. The normalization unit 209 then normalizes the number of pixels in both the vertical and horizontal directions of the extracted image area to a size expected as input by the authenticity determination model 202b, thereby normalizing the image to a fixed length. The image area extracted by the normalization unit 209 through cropping is the image area of ​​the card. As a result of the normalization process performed by the normalization unit 209, an image consisting only of the image portion of the card can be input to the authenticity determination model 202b at an appropriate size.

[0069] Further, the normalization unit 209 rotates and resizes the image of the extracted region for the purpose of preventing the top and bottom of the moving image input to the authenticity determination model 202b from being reversed. The normalization unit 209 performs the rotation and resizing of the image while maintaining the aspect ratio of the original moving image as much as possible. The size of the image after normalization by the normalization unit 209 can be, for example, an image size of about width m pixels × height n pixels (m < n).

[0070] The authenticity determination unit 210 is executed when the known item detection unit 207 can be confident that a known image has been detected, or when the two-stage white skip detection unit 208 detects that there is no white skip or significantly less white skip in the image. That is, the authenticity determination unit 210 inputs a moving image including time-series information of reflected light from the card, which has been normalized by the normalization unit 209, into the authenticity determination model 202b, and obtains an authenticity determination result of the card classified into two binary values of normal (genuine) or abnormal (fake) from the authenticity determination model 202b. The authenticity determination model 202b executes a determination procedure for classifying the temporal change pattern based on a learning basis, using the temporal change pattern of the reflected light from a predetermined surface of the card medium included in the moving image data of the card medium photographed by the camera 116 in a state where the guide is displayed, and the temporal change pattern of the reflected light from the predetermined surface included in the moving image data obtained by photographing a genuine card medium. This determination procedure includes a procedure for causing the authenticity determination model 202b, which takes the moving image data as an input and outputs an authenticity determination result of the card medium, to classify the temporal change pattern based on a learning basis. Then, by this determination procedure, the authenticity determination model 202b outputs a numerical value for determining the authenticity of the card medium, and this numerical value is classified into two binary values indicating the authenticity determination result of the card.

[0071] The authenticity determination model 202b is characterized by its efficient collection of time-series data from multiple viewpoints or illumination levels through a photography interface that naturally guides the user's actions when photographing a card, and its ability to make maximum use of the temporal change patterns of reflected light that are dependent on the direction of a hologram for both learning and inference. It is also important to note that the data used for authenticity determination is photographed manually. Because the authenticity determination model 202b learns counterfeiting methods generically, without relying on a specific subject, it is a "cold model" that is retrained only when a new counterfeiting method is discovered.

[0072] 4 is a block diagram showing an example of the internal configuration of the learning device 3. In FIG. 4, the master image DB 50 of the learning device 3 is omitted.

[0073] The learning data DB 40 of the learning device 3 stores video data 40a, known item learning data 40b, and authenticity determination learning data 40c. The learning device 3 also includes a data expansion unit 41, a known item learning unit 42, and an authenticity determination learning unit 43. The server-side learning model DB 45 also stores a known item detection model 45a and an authenticity determination model 45b.

[0074] The moving image data 40a is, for example, data of moving images captured by a developer on a card medium under various conditions. The moving image data 40a may also include moving images captured by automatically changing the light source, pan, tilt, and focus based on data of known images formed on the card medium. The moving image data 40a includes data of hundreds to tens of thousands of moving images for each known image formed on the card medium.

[0075] The data extension unit 41 reads the video data 40a from the data extension unit 41 and extends the video data 40a through various image processing. The data extension unit 41 stores the extended video data 40a in the learning data DB 40 as known item learning data 40b and authenticity judgment learning data 40c.

[0076] The known item learning data 40b is data that is read by the known item learning unit 42 to learn known items into the known item detection model 45a. The known item learning unit 42 reads the known item detection model 45a from the server-side learning model DB 45, inputs the known item learning data 40b, and causes the known item detection model 45a to learn the known items. The known item learning unit 42 saves the learned known item detection model 45a in the server-side learning model DB 45.

[0077] The authenticity determination learning data 40c is data that is read by the authenticity determination learning unit 43 to train the authenticity determination model 45b to perform authenticity determination. The authenticity determination learning unit 43 reads the authenticity determination model 45b from the server-side learning model DB 45, inputs the authenticity determination learning data 40c, and trains the authenticity determination model 45b to perform authenticity determination. The authenticity determination learning unit 43 then stores the trained authenticity determination model 45b in the server-side learning model DB 45.

[0078] <Learning process> Next, the learning process will be described with reference to FIGS. FIG. 5 is a flowchart showing an example of a known item learning process.

[0079] First, the data extension unit 41 acquires video data 40a as an image (original data) to be detected from the learning data DB 40 (S1). The image (original data) to be detected includes a video of a known image formed on a card medium.

[0080] Next, the data expansion unit 41 expands the number of data by expanding the image (original data) of the detection target acquired from the video data 40a using a data expansion algorithm (S2), and stores the expanded data as known item learning data 40b in the learning data DB 40. The data expansion algorithm is used by the data expansion unit 41 to perform image processing such as adding blur to the known image, cutting out a rectangular area from the known image, enlarging, reducing, rotating, and translating the known image, changing the brightness of a part of the known image, and cutting out the edges of the known image.

[0081] Next, the data extension unit 41 inputs the known item learning data 40b into the known item detection model 45a read from the server-side learning model DB 45, thereby learning known items based on the original data with the expanded number of data (S3).The data extension unit 41 then saves the learned known item detection model 45a in the server-side learning model DB 45 (S4), and ends this process.

[0082] FIG. 6 is a flowchart showing an example of the authenticity determination learning process.

[0083] First, the authenticity determination learning unit 43 acquires authenticity determination learning data 40c from the learning data DB 40 (S11). Next, the data expansion unit 41 expands the number of pieces of data by expanding the image (original data) of the detection target acquired from the video data 40a using a data expansion algorithm (S12), and stores the expanded data in the learning data DB 40 as authenticity determination learning data 40c.

[0084] Next, the authenticity determination learning unit 43 inputs the authenticity determination learning data 40c into the authenticity determination model 45b read from the server-side learning model DB 45, thereby causing the model to learn how to determine the authenticity of a known image (S13).

[0085] Then, the authenticity determination learning unit 43 stores the learned authenticity determination model 45b in the server-side learning model DB 45 (S14), and ends this process.

[0086] <Guide frame display processing> 7 and 8, a description will be given of a guide frame display process performed by the display processing unit 203 of the information processing terminal 2. FIG. 7 is a diagram showing an example of a guide frame display based on the guide frame display process performed by the display processing unit 203.

[0087] The left side of Fig. 7 shows a state in which the frame of the card Cd is displayed within a guide frame Fm displayed on the screen of the display device 119 by the display processing unit 203. The guide frame Fm is an example of an image that specifies the display position of the card medium. When this state occurs, that is, when it is detected that the frame of the card Cd has fit within the guide frame Fm, the display processing unit 203 displays a sign such as "OK" on the screen.

[0088] Next, the display processing unit 203 provides guidance to change the position of the guide frame displayed on the screen to a position different from the previous position. As shown on the right side of Fig. 7, the display processing unit 203 shifts the display position of the guide frame Fm on the screen diagonally upward to the right. Upon seeing the guide displayed with the display position shifted, the user moves the information processing terminal 2 that is being held in order to fit the frame of the card Cd into the card frame Fm.

[0089] By performing such processing by the display processing unit 203, the position of the information processing terminal 2 held by the user changes up, down, left, right, etc. in accordance with the change in the display position of the guide frame. In other words, the angle at which the card is photographed by the camera 116 of the information processing terminal 2 changes. Note that the number of times the display position of the guide frame is changed by the display processing unit 203 may be any number of times as long as it is at least once or more.

[0090] Fig. 8 is a diagram showing an example of changes in the photographing angle of a card by camera 116. When the display processing unit 203 changes the display position of the guide frame, the photographing angle of the card by camera 116 changes to various angles, as shown in Fig. 8.

[0091] Then, video shooting unit 204 records, as a video, the images captured by camera 116 with the imaging angle changed. By recording, as a video, the images captured by camera 116 with the imaging angle changed, it becomes possible to include, in the video shot by video shooting unit 204, a feature that essentially has time axis information, that is, a temporal change pattern of reflected light according to the imaging angle.

[0092] The authenticity determination system 100 according to the present embodiment determines the authenticity of cards that have undergone fine hologram and embossing processes. Embossing is, for example, a process of printing a transparent varnish on the surface (an example of the first surface) of a card to form a concave and convex portion. Therefore, when such a card is photographed, the temporal change pattern of the reflected light from the card changes in a subtle and diverse manner as the photographing angle of the camera 116 changes. In other words, the authenticity determination model 202b according to the present embodiment learns the subtle and diverse changes in the temporal change pattern of the reflected light from such a card. The surface treatment of the card is not limited to both fine hologram and embossing, and may be either one of them. Furthermore, the surface treatment of the card may be other than hologram or embossing, such as gold foil stamping, silver foil stamping, or mother-of-pearl inlay, as long as it can change the reflection state of the reflected light from the card surface.

[0093] <Example of an image with overexposed highlights> Figure 9 shows an example of an image that contains overexposed highlights. When a card that has been treated to make it more reflective is photographed under bright lighting, overexposed highlights may appear in the image.

[0094] For example, a blown-out highlight Wh occurs in the area indicated by the dashed line in FIG. 9. Because the blown-out highlight Wh covers the photographed card Cd, part of the frame of the card Cd and part of the character image formed on the card Cd appear white. Note that the card frame Fm is displayed by the display processing unit 203 and is therefore not affected by the blown-out highlight Wh. When blown-out highlights Wh cover the card Cd in this way, information necessary for determining the authenticity of the card Cd may be missing. For this reason, the display processing unit 203 displays a message saying, "The screen is too bright. Please darken the surroundings and take the photograph again," and instructs the user to take the photograph again. When the user sees this message, it is expected that they will photograph the card Cd again.

[0095] <Authenticity determination process using an authenticity determination system> Next, the authentication determination process by the authentication determination system 100 according to this embodiment will be described with reference to Fig. 10. Fig. 10 is a flowchart showing an example of the procedure of the authentication determination process by the authentication determination system 100.

[0096] First, the display processing unit 203 (see FIG. 3) of the information processing terminal 2 executes a guide frame display process (S21). The guide frame display process of step S21 starts when the user starts an application incorporating a program that executes the authenticity determination process according to this embodiment and selects a menu for determining the authenticity of a card. The details of the guide frame display process performed in step S21 will be described in detail with reference to FIG. 11, which will be described later.

[0097] Next, the camera 116 starts taking pictures and recording the image data taken by the camera 116 as a moving image (S22). The process of step S22 continues while the guide frame is displayed on the screen of the display device 119 in step S21 and the user moves the information processing terminal 2 up, down, left, right, etc. to fit the frame of the card into the guide frame.

[0098] Next, the card area detection unit 205 extracts a card area from the image data captured by the camera 116 in step S22 (S23). Next, the photography determination unit 206 performs photography determination processing (S24). The photography determination processing will be described in detail later with reference to FIG. 12.

[0099] Next, the known item detection unit 207 performs a known item detection process (S25). The known item detection process will be described in detail with reference to FIG. 13 below. Next, the known item detection unit 207 determines whether the certainty score s obtained by the known item detection process falls within a range of predetermined thresholds t1 to t2 (S26). If the certainty score s exceeds the threshold t2 (NO in S26), it is sufficiently certain that a known image has been detected as a known item, and the process proceeds to step S28.

[0100] On the other hand, if the certainty score s falls within the range of the predetermined thresholds t1 to t2 (YES in S26), it is not certain that a known image has been detected as a known item, so the process proceeds to step S27. Then, the two-stage blown out highlight detection unit 208 performs two-stage blown out highlight detection processing (S27). The two-stage blown out highlight detection processing will be described in detail later with reference to FIG. 14.

[0101] If the certainty score s is less than the threshold t1, it is highly likely that the subject is outside the guide frame Fm or is obscured. If the certainty score s is equal to or greater than the threshold t2, the authenticity of the image can be determined with sufficient confidence, and the two-stage blown-out highlight detection process in step S27 is unnecessary.

[0102] Furthermore, in this embodiment, thresholds t1 to t2 are set, but t1 may be set to 0. In this case, if the threshold value t2 is less than the threshold value t2, it is not certain that a known image has been detected, and the two-stage blown-out highlight detection process of step S27 may be performed.

[0103] If it is sufficiently certain that a known image has been detected in step S26, or after the two-stage blown-out highlight detection process in step S27, the normalization unit 209 normalizes the size of the video recorded in step S22 to a size suitable for input to the authenticity determination model 202b (S28). Next, the authenticity determination unit 210 performs an authenticity determination process (S29). The authenticity determination process performed in step S29 will be described in detail with reference to FIG. 15 below.

[0104] (Guide frame display processing) Next, the guide frame display process performed in step S21 of Fig. 10 will be described with reference to Fig. 11. Fig. 11 is a flowchart showing an example of the procedure for the guide frame display process performed by the display processing unit 203.

[0105] First, the display processing unit 203 displays a guide frame on the screen of the display device 119 (S31). Next, the display processing unit 203 determines whether the frame of the card included in the image data captured by the camera 116 in step S22 fits within the guide frame (S32). If it is determined that the card does not fit within the guide frame (NO in S32), the display processing unit 203 continues to make the determination in step S32.

[0106] On the other hand, if it is determined that the card has fit into the guide frame (YES in S32), the display processing unit 203 displays a sign such as "OK" on the screen to indicate that the card has fit into the guide frame (S33). Note that the sign indicating that the card has fit into the guide frame is not limited to the letters "OK" and may be other letters. Furthermore, the sign may not be letters, but may be other forms of notification to the user, such as a change in the color or shape of the guide frame, or the emission of a sound.

[0107] Next, the display processing unit 203 determines whether or not the termination condition for the photography determination process by the photography determination unit 206 (see FIG. 3) has been met (S34). The termination condition for the photography determination process is that both the determination result of "success" or "failure" by the photography determination process and the captured image (video) by the camera 116 in the case of a "success" determination are obtained.

[0108] A moving image determined to be "successful" is obtained when the shooting determination process has been completed for all guide frames. If it is determined in step S34 that the conditions for ending the shooting determination process have not been met (NO in S34), the display processing unit 203 changes the display position of the guide frames on the screen (S35). After processing step S35, the display processing unit 203 returns to step S31 and performs processing.

[0109] On the other hand, if it is determined in step S34 that the termination condition for the shooting determination process is satisfied (YES in S34), the guide frame display process is terminated by the display processing unit 203. Note that the guide frame display process by the display processing unit 203 may be terminated at a predetermined timing after the determination result of the shooting determination process is output.

[0110] (Shooting judgment processing) Next, the shooting determination process performed in step S24 of Fig. 10 will be described with reference to Fig. 12. Fig. 12 is a flowchart showing an example of the procedure for the shooting determination process performed by the shooting determination unit 206.

[0111] First, the photography determination unit 206 extracts the position of the card from the card area detected in the photographed image in step S23 of Fig. 10 (S41). Next, the photography determination unit 206 determines whether the movement vector of the card position extracted in step S41 and the movement vector of the six-axis sensor 117 of the information processing terminal 2 are moving simultaneously in the same direction (S42).

[0112] If it is determined that the movement vector of the card position and the movement vector of the six-axis sensor 117 are moving simultaneously in the same direction (YES in S42), the photography determination unit 206 determines that the camera 116 is photographing a physical card medium (S43). On the other hand, if it is determined that the movement vector of the card position and the movement vector of the six-axis sensor 117 are not moving simultaneously in the same direction (NO in S42), the photography determination unit 206 determines that the camera 116 is photographing a subject other than a physical card medium (S44). After processing step S44 or step S45, the photography determination process by the photography determination unit 206 ends.

[0113] The determination in step S42 can be made by analyzing the movement directions of both movement vectors, but can also be made based on the magnitude of the correlation coefficient between both movement vectors, etc.

[0114] (Known item detection process) Next, the known item detection process performed in step S25 of Fig. 10 will be described with reference to Fig. 13. Fig. 13 is a flowchart showing an example of the procedure of the known item detection process performed by the known item detection unit 207.

[0115] First, the known item detection unit 207 reads an image captured by the video image capturing unit 204, and inputs the image to the known item detection model 202a read from the local-side inference model DB 202 (S51).

[0116] The known item detection model 202a estimates the presence or absence of an identifier of the card Cd and a written language (Japanese, Chinese, English, etc.) based on the input image (S52). Then, the known item detection model 202a outputs a certainty score s (S53). After the processing of step S53, the known item detection processing by the known item detection unit 207 ends.

[0117] (Two-stage overexposure detection processing) Next, the two-stage blown-out highlight detection process performed in step S27 of Fig. 10 will be described with reference to Fig. 14. Fig. 14 is a flowchart showing an example of the procedure for the two-stage blown-out highlight detection process performed by the two-stage blown-out highlight detection unit 208.

[0118] First, the two-stage blown-out highlight detection unit 208 performs a global analysis in steps S61 to S63. In the global analysis, the two-stage blown-out highlight detection unit 208 converts an image captured by the video capture unit 204 and input (referred to as an input image) into a luminance image (grayscale image) (S61). The input image is, for example, an 8-bit, 3-channel video image. After formal validation of the input image (changing the image size, etc.), the two-stage blown-out highlight detection unit 208 converts the input image into a grayscale image, i.e., a 1-channel luminance image, if the input image is normal. This is because blown-out highlight determination uses only luminance values.

[0119] Next, in a first analysis step, the two-stage blown-out highlight detection unit 208 analyzes the proportion of pixels in the image of the card medium whose brightness values ​​exceed a predetermined brightness threshold. The first analysis step involves converting the entire image of the card medium into a brightness image and comparing the brightness value of each pixel in the brightness image with the brightness threshold. Specifically, the two-stage blown-out highlight detection unit 208 calculates the proportion of pixels in the entire converted brightness image whose brightness values ​​exceed a certain brightness threshold (e.g., a brightness value of 250) (S62). Here, the two-stage blown-out highlight detection unit 208 calculates the frequency of brightness values ​​for each pixel as a histogram. The brightness values ​​have 256 bins ranging from 0 to 255. Each bin represents a specific brightness value. The two-stage blown-out highlight detection unit 208 then calculates the number of pixels in the entire image and calculates the total number of bins in the histogram whose brightness values ​​exceed the brightness threshold, i.e., whose brightness values ​​are greater than or equal to 250 and less than or equal to 255.

[0120] Next, the two-stage blown-out highlight detection unit 208 calculates the ratio of the total number of bins exceeding the brightness threshold to the total number of pixels in the entire image. This calculation is performed to determine the ratio of high-brightness pixels that can be considered "almost white" to the entire image. The two-stage blown-out highlight detection unit 208 then determines whether the calculated ratio exceeds a preset ratio threshold (S63). This ratio threshold is preferably less than 10%, for example.

[0121] If it is determined that the proportion of pixels exceeding the brightness threshold exceeds a predetermined proportion threshold (YES in S63), it is considered that blown-out highlights have occurred over a wide area. Therefore, the two-stage blown-out highlight detection unit 208 outputs shooting condition correction instruction information to the display processing unit 203 (S69). Furthermore, if the proportion of pixels exceeds a predetermined pixel proportion threshold, the two-stage blown-out highlight detection unit 208 performs a first warning procedure to warn of the possibility of a poorly shot area.

[0122] If it is determined that the proportion of pixels exceeding the brightness value threshold does not exceed the preset proportion threshold (is equal to or less than the proportion threshold) (NO in S63), the two-stage blown-out highlight detection unit 208 performs local analysis shown in steps S64 to S68.

[0123] In the local analysis, if the two-stage blown-out highlight detection unit 208 determines in the first analysis step that the pixel ratio does not exceed the pixel ratio threshold, it divides the image of the card medium into blocks of a predetermined size and analyzes the relationship between the high luminance density and the predetermined high luminance density threshold for each block. Specifically, the two-stage blown-out highlight detection unit 208 first divides the luminance image into small blocks (a 4x4 grid) (S64). Here, the luminance image is divided into 16 blocks by dividing the original image size into 4 vertically and 4 horizontally, and the two-stage blown-out highlight detection unit 208 calculates the height and width of each block in terms of the number of pixels. Note that the number of blocks into which the two-stage blown-out highlight detection unit 208 divides the luminance image (16) is just an example; the original image size may be divided into 2 vertically and 2 horizontally, resulting in 4 blocks, or the original image size may be divided into 5 vertically and 5 horizontally, resulting in 25 blocks.

[0124] Next, the two-stage blown-out highlight detection unit 208 performs a loop process for each divided block (S65). In this loop process, the two-stage blown-out highlight detection unit 208 first scans one of the blocks and counts the number of pixels bC whose luminance value exceeds 250, for example (S66).

[0125] Next, the two-stage blown-out highlight detection unit 208 calculates the high luminance density bR of the block (S67). The high luminance density bR is calculated in the second analysis procedure as the ratio of the number of pixels whose luminance values ​​are equal to or greater than the luminance threshold value to the total number of pixels in the block. Specifically, it is calculated using the following formula (1): High luminance density bR = number of pixels bC / total number of pixels in the block × 100 …(1)

[0126] Next, the two-stage blown-out highlight detection unit 208 determines whether the high luminance density bR is equal to or greater than a preset density threshold (S68). The density threshold may be set to, for example, 10% or less. By calculating the high luminance density bR, the two-stage blown-out highlight detection unit 208 detects local overexposure that causes blown-out highlights in the luminance image.

[0127] On the other hand, if the two-stage blown-out highlight detection unit 208 determines that the high luminance density bR is not equal to or greater than the preset density threshold (NO in S68), it performs local analysis on the next block. For example, after processing the block in the upper left of the original image, it processes the blocks to the right, and after processing the rightmost block, it processes the block one block below and to the left. This loop process is then performed on all blocks.

[0128] If the two-stage blown-out highlight detection unit 208 performs local analysis on all blocks and determines NO in step S68, the possibility of blown-out highlights is low, and the process ends.

[0129] On the other hand, if the two-stage blown-out highlight detection unit 208 determines that the high luminance density bR is equal to or greater than the preset density threshold (YES in S68), it exits the loop even if local analysis has not been performed on all blocks. In other words, the two-stage blown-out highlight detection unit 208 exits the loop if it determines that the high luminance density bR of even one block is equal to or greater than the preset density threshold.

[0130] If the two-stage blown-out highlight detection unit 208 determines through global analysis that the pixel ratio exceeds the ratio threshold (YES in S63), or if the local analysis determines that the high luminance density bR is equal to or greater than a preset density threshold (YES in S68), it outputs shooting condition correction instruction information to the display processing unit 203 (S69). In this way, if there is a block where the high luminance density is equal to or greater than the high luminance density threshold, the two-stage blown-out highlight detection unit 208 performs a second warning procedure that warns of the possibility of a poorly shot area.

[0131] When the display processing unit 203 receives the correction instruction information for the shooting conditions, it displays a warning of the possibility of overexposure as shown in Fig. 9 (S70), and returns the process to step S23 (detecting the card area from the captured image) in Fig. 10. In this way, even in a state affected by the automatic exposure control of the camera 116, the display processing unit 203 can detect only the light that impairs the information of the subject and display a warning of the possibility of overexposure. Thereafter, the process from step S23 onwards is performed again.

[0132] In this way, the two-stage blown-out highlight detection unit 208 determines the final result as true, i.e., blown-out highlights detected, if either the global analysis or the local analysis is true. If both the global analysis and the local analysis are false, blown-out highlights are not detected.

[0133] (Authenticity determination process) Next, the authenticity determination process performed in step S29 of Fig. 10 will be described with reference to Fig. 15. Fig. 15 is a flowchart showing an example of the procedure for the authenticity determination process performed by the authenticity determination unit 210.

[0134] First, the authenticity determination unit 210 inputs the video image of the card area normalized in step S28 of Fig. 10 to the authenticity determination model 202b (S71). Next, the authenticity determination model 202b analyzes the pattern of change over time of the reflected light obtained from the image of the card area that was captured by the camera 116 and normalized, and classifies the pattern of change over time on a learning basis (S72).

[0135] Next, the authenticity determining unit 210 outputs the result of the authentication of the card as a result of the verification by the authenticity determining model 202b (S73).

[0136] The authenticity determination model 202b analyzes the temporal change pattern of the reflected light obtained from the image of the card area, and based on the results of learning-based classification of the temporal change pattern, obtains an authenticity determination result that the card photographed by the camera 116 is a regular (genuine) card, and outputs this authenticity determination result.

[0137] On the other hand, the authenticity determination model 202b analyzes the pattern of change over time of the reflected light obtained from the image of the card area, and based on the result of classifying the pattern of change over time based on learning, obtains an authenticity determination result that the card photographed by the camera 116 is an illegal (counterfeit) card, and outputs this authenticity determination result. After the processing of step S73, the authenticity determination processing by the authenticity determination unit 210 ends.

[0138] In the first embodiment described above, when a known subject is present within the frame and the certainty score of the subject detected by the known item detection unit 207 is less than the certainty threshold, a two-stage blown-out highlight detection process, consisting of a global analysis and a local analysis, is performed on the captured image. The two-stage blown-out highlight detection process analyzes the brightness characteristics of the entire image and each block to detect loss of information about the known subject due to blown-out highlights. Furthermore, the authenticity determination unit 210 of the information processing terminal 2 analyzes the time-series information of the reflected light from the card obtained by the camera 116, i.e., the temporal change pattern of the reflected light, and classifies the temporal change pattern based on learning to determine the authenticity of the card. Therefore, according to this embodiment, the authenticity of the card can be determined without adding or connecting an authenticity determination device, such as a wide-angle microscope, to the information processing terminal 2.

[0139] Furthermore, in the first embodiment, a simple threshold-based image processing method is used to detect blown-out highlights across the entire image and for each block, while maintaining high computational efficiency that enables real-time processing on a smartphone. To ensure versatility that is not dependent on specific device-specific functions, the two-stage blown-out highlight detection unit 208 detects blown-out highlights only through image analysis of image data after AE (Auto Exposure) or AI correction, rather than raw data specific to the camera device.

[0140] Furthermore, unlike intentional obscuration (illegal manipulation) caused by covering the subject with a finger or a card other than the subject, blown-out highlights can frequently occur when a well-intentioned user takes a photo due to lighting conditions or shooting angle. Therefore, when the two-stage blown-out highlight detection unit 208 detects blown-out highlights, the display processing unit 203 immediately prompts the user to correct the shooting conditions, such as by dimming the lighting or adjusting the camera angle. This reduces unnecessary reshoots and false detections of blown-out highlights, and enables blown-out highlight detection to be achieved with the minimum number of activations of each AI in the information processing terminal 2.

[0141] The two-stage blown-out highlight detection unit 208 according to the first embodiment detects blown-out highlights in an image in two stages: global analysis and local analysis. Therefore, the two-stage blown-out highlight detection unit 208 can detect blown-out highlights that occur in a captured image even if excessive light is not incident on the image sensor or the like.

[0142] Furthermore, the two-stage blown-out highlight detection unit 208 according to the first embodiment can perform more accurate determinations based on the "information loss rate of the subject" than conventional cameras and image processing software, which perform blown-out highlight detection based on the entire image. For example, the two-stage blown-out highlight detection unit 208 can detect localized blown-out highlights (e.g., a condition in which light reflects off the surface of a vinyl-coated card, making part of the card information unreadable) that are difficult to detect using existing automatic exposure control algorithms, based on global and local analysis. When the two-stage blown-out highlight detection unit 208 detects blown-out highlights, it outputs information to the display processing unit 203 instructing the user to modify the shooting conditions. The display processing unit 203 then displays information instructing the user to change the shooting position, etc. This ensures stable capture of images without blown-out highlights. In particular, the authenticity determination unit 210, which uses the results of the two-stage blown-out highlight detection unit 208 to determine the authenticity of card media, provides greater stability and accuracy in authenticity determination for subject authentication and check-in applications than conventional methods.

[0143] In addition, camera devices and control software for general smartphones are equipped with an automatic exposure control function as standard, making it difficult to directly acquire raw sensor data. On the other hand, the first embodiment specifically detects only overexposure, which impairs subject information even when using corrected image data, and prompts the user to change the shooting angle appropriately, leading to successful shooting, thereby improving the user experience (UX).

[0144] Furthermore, in this embodiment, the information processing terminal 2 determines the authenticity of the card based on time-series information of the light reflected from the card obtained by the camera 116. This eliminates the need to embed an NFC tag or the like for authenticity determination in the card, or to apply a watermark or special processing for authenticity determination. This not only prevents an increase in the manufacturing cost of the card, but also prevents the aesthetic appeal of the card from being impaired.

[0145] Furthermore, in this embodiment, the photography determination unit 206 performs photography determination to determine whether or not the camera 116 is photographing a physical card medium, using a detection value from the six-axis sensor 117 provided in the information processing terminal 2. Therefore, in this embodiment, it is possible to perform card authenticity determination (photography determination) without introducing new hardware into the information processing terminal 2.

[0146] Furthermore, for example, in authenticating a personal identification number (My Number) card, information on an image of the card is used, so a guide frame or the like cannot be displayed on the screen, but in this embodiment, the authenticity of the card can be determined by inputting a moving image captured by camera 116. Therefore, a guide frame can be displayed in real time on the screen of display device 119.

[0147] Then, the display processing unit 203 moves the position of the guide frame displayed on the screen of the display device 119, thereby allowing the user to perform the operation required for the information processing terminal 2 to acquire the information necessary for determining the authenticity of the card. Furthermore, this operation is a simple operation of moving the information processing terminal 2 so that the frame portion of the card fits into the guide frame, so even users who are unfamiliar with operating the information processing terminal 2 or users who are beginners at trading card games can properly photograph the card.

[0148] In this embodiment, since the authentication determination model 202b constructed by a neural network is used to determine the authenticity of a card, there is a risk that a deceptive signal that exploits the vulnerability of the neural network model using a method not anticipated by the designer may be input during the authentication process. In fact, known attack methods include a method of deceiving a neural network model using a noisy image and a "jailbreak" attack method for circumventing the limitations of large language models (LLMs). However, a method for safely using neural network models has not yet been established.

[0149] In contrast, the authentication system 100 according to this embodiment uses YOLO to detect the position and type of the card, and KAZE to detect the card area. The neural network model used for authentication is highly flexible but behaves non-deterministically. By configuring the authentication system 100 in this way, it becomes possible to determine the authenticity of a card by having the neural network model (authentication model 202b) learn video images of the card, without having the neural network model learn cheating techniques on a large scale.

[0150] As an additional function, the two-stage blown-out highlight detection unit 208 may also perform an RGB saturation check. The RGB saturation check is a method in which the two-stage blown-out highlight detection unit 208 separates the R (Red), G (Green), and B (Blue) channels of the input image, counts the number of pixels whose luminance values ​​in all three channels are equal to or greater than a luminance threshold (e.g., 250), and calculates the ratio of the luminance value of each channel to the overall image. For example, if the two-stage blown-out highlight detection unit 208 cannot correctly detect blown-out highlights in an image captured under lighting of a specific color, performing an RGB saturation check is expected to enable correct detection of blown-out highlights in the image.

[0151] [Second embodiment] <Outline of the authenticity determination system> Next, an authenticity determination system according to a second embodiment of the present invention will be described. In the authenticity determination system according to the second embodiment, the processes related to authenticity determination that were previously performed by the information processing terminal 2 are performed by an inference device on the server side.

[0152] 16 is a block diagram showing an overall configuration of an authenticity determination system 100A according to a second embodiment of the present invention, which is a block diagram showing an example of the hardware configuration of a learning device 3, an inference device 6, and an information processing terminal 2A that constitute the authenticity determination system 100A.

[0153] The authenticity determination system 100A includes an information processing terminal 2A and a learning device 3 according to the second embodiment, as well as an inference device 6 that is capable of communicating via a network N. Here, an example configuration of the inference device 6 will be described.

[0154] (Example of inference device configuration) The inference device 6 is an example of a calculator that operates as a computer. The inference device 6 includes a control unit 160, a nonvolatile storage 165, and a network interface 166, all of which are connected to a bus B3.

[0155] The control unit 160 includes a CPU 161, a GPU 162, a ROM 163, and a RAM 164. The control unit 160 may be configured using an FPGA.

[0156] The CPU 161 reads out program code of software that realizes each function according to this embodiment from the ROM 163, loads it into the RAM 164, and executes it. Variables, parameters, etc. that are generated during the calculation processing of the CPU 161 are temporarily written to the RAM 164. These variables, parameters, etc. written to the RAM 164 are read out by the CPU 161 as appropriate. The GPU 162 performs calculations required when performing various inference processes according to this embodiment.

[0157] The nonvolatile storage 165 may be, for example, an HDD, SSD, optical disk, magneto-optical disk, or nonvolatile memory. In addition to an OS and various parameters, this nonvolatile storage 165 stores a program for causing the inference device 6 to function. The program for causing the inference device 6 to function may be stored in ROM 163. In other words, ROM 163 and nonvolatile storage 165 are used as an example of a computer-readable non-transitory recording medium that stores a program executed by the inference device 6. In addition, a server-side inference model DB 60 (see FIG. 6) that stores the authenticity determination model 60b is formed in the nonvolatile storage 165.

[0158] For example, an NIC or the like is used as the network interface 166. The network interface 166 transmits and receives various data to and from the learning device 3 and the information processing terminal 2A via a dedicated line or the like connected to a terminal of the NIC and via the network N.

[0159] <Example of functional configuration of authenticity determination system> Next, the functional configuration of the authenticity determining system 100A according to the second embodiment will be described with reference to FIG. 17 is a block diagram showing the functional configuration of an authenticity determination system 100A according to the second embodiment. In FIG. 17, the communication network N and the learning device 3 are omitted.

[0160] (Example of terminal device functional configuration) The information processing terminal 2A includes a local image DB 201, a local-side inference model DB 202, a display processing unit 203, a video image capturing unit 204, a card area detection unit 205, a six-axis sensor 117, a capturing determination unit 206, a known item detection unit 207, a two-stage blown-out highlight detection unit 208, and a normalization unit 209. The functions of each unit included in the information processing terminal 2A are the same as those of each unit shown in FIG. 3 according to the first embodiment, and therefore detailed description thereof will be omitted.

[0161] Note that only the known item detection model 202a is stored in the local-side inference model DB 202. When the normalization unit 209 receives an image in which the known item detection unit 207 has detected a known item with a high degree of certainty, or an image in which no blown-out highlights have been detected by the two-stage blown-out highlight detection unit 208, the normalization unit 209 normalizes the image and transmits the image data of the normalized image to the inference device 6. At this time, the image data is transmitted from the network interface 115 of the information processing terminal 2A via the communication network N, and the control unit 160 captures the image data received by the network interface 166 of the inference device 6.

[0162] (Example of functional configuration of inference device) The inference device 6 includes a server-side inference model DB 60 and an authenticity determination unit 61. The server-side inference model DB 60 stores the authenticity determination model 45b imported from the server-side learning model DB 45 of the learning device 3 as the authenticity determination model 60b.

[0163] The authenticity determination unit 61 inputs the image data of the normalized image acquired from the information processing terminal 2A into the authenticity determination model 60b read from the server-side inference model DB 60, and obtains an authenticity determination result. The operation of the authenticity determination model 60b is the same as the authenticity determination process described with reference to Fig. 15. Then, the authenticity determination result is transmitted to the information processing terminal 2A, and the display processing unit 203 displays the authenticity determination result.

[0164] In the authenticity determination system 100A according to the second embodiment described above, the authenticity determination process is performed by the inference device 6 on the server side. This makes it possible to prevent reverse engineering of the authenticity determination model 60b. Furthermore, even if an image that has been fraudulently edited by an unauthorized user is detected as a known item by the processing of the information processing terminal 2A and no blown-out highlights are detected in the image, the authenticity determination unit 61 can correctly determine its authenticity. Because the inference device 6 is managed by the developer, it becomes easy to identify unauthorized users who repeatedly send image data determined to be counterfeit and take measures such as suspending their accounts.

[0165] [Third embodiment] <Outline of the authenticity determination system> Next, an authenticity determination system according to a third embodiment of the present invention will be described. In the authenticity determination system according to the third embodiment, the processes related to known item detection, two-stage overexposure detection, and authenticity determination, which were previously performed by the information processing terminal 2, are performed by an inference device on the server side. Since the hardware configuration of the authenticity determination system according to the third embodiment is the same as that of the authenticity determination system according to the second embodiment, only an example of the functional configuration of the authenticity determination system will be described.

[0166] <Example of functional configuration of authenticity determination system> Next, with reference to FIG. 18, the functional configuration of an authenticity determination system 100B according to the third embodiment will be described. 18 is a block diagram showing the functional configuration of an authenticity determination system 100B according to the third embodiment. In FIG. 18, the communication network N and the learning device 3 are omitted.

[0167] (Example of terminal device functional configuration) The information processing terminal 2B includes a local image DB 201, a display processing unit 203, a moving image shooting unit 204, a card area detection unit 205, a six-axis sensor 117, and a shooting determination unit 206. The functions of the units included in the information processing terminal 2B are the same as the functions of the units shown in Fig. 3 according to the first embodiment, and therefore detailed description thereof will be omitted.

[0168] It should be noted that the information processing terminal 2B itself does not perform AI determination. Image data of the moving image for which the shooting determination unit 206 has performed shooting determination is transmitted to the inference device 6. At this time, the image data is transmitted from the network interface 115 of the information processing terminal 2B via the communication network N, and the control unit 160 captures the image data received by the network interface 166 of the inference device 6A.

[0169] (Example of functional configuration of inference device) The inference device 6A includes a server-side inference model DB 60, a known item detection unit 62, a two-stage overexposure detection unit 63, a normalization unit 64, and an authenticity determination unit 61. The server-side inference model DB 60 stores the known item detection model 45a and the authenticity determination model 45b imported from the server-side learning model DB 45 of the learning device 3 as the known item detection model 60a and the authenticity determination model 60b.

[0170] The known item detection unit 62 inputs the image data obtained from the information processing terminal 2B and after the photographing determination into the known item detection model 60a read from the server-side inference model DB 60, and detects known items from the subjects appearing in the image. The known item detection unit 62 then outputs a certainty score. The processing of the known item detection unit 62 is the same as the processing shown in FIG. 13.

[0171] The two-stage blown-out highlight detection unit 63 performs two-stage blown-out highlight detection processing when the certainty score falls within a range of predetermined thresholds t1 to t2. The processing by the two-stage blown-out highlight detection unit 63 is the same as the processing shown in Fig. 14. When the two-stage blown-out highlight detection unit 63 detects blown-out highlights in the image through global analysis or local analysis, it transmits correction instruction information for the shooting determination conditions to the display processing unit 203 of the information processing terminal 2.

[0172] The normalization unit 64 normalizes an image whose certainty score exceeds t2 or an image in which no blown-out highlights have been detected by the two-stage blown-out highlights detection unit 63. The processing by the normalization unit 64 is the same as the processing in step S28 in FIG.

[0173] As described in the second embodiment, the authenticity determination unit 61 obtains an authenticity determination result based on the image data of the normalized image. The operation of the authenticity determination model 60b is the same as the authenticity determination process described with reference to Fig. 15. Then, the authenticity determination result is transmitted to the information processing terminal 2A, and the display processing unit 203 displays the authenticity determination result.

[0174] In the authenticity determination system 100B according to the third embodiment described above, the known item detection process, the two-stage whiteout detection process, the normalization process, and the authenticity determination process are performed by the inference device 6 on the server side. This prevents reverse engineering of the known item detection model 60a and the authenticity determination model 60b. Furthermore, the image captured by the information processing terminal 2B is determined in real time by the image capture determination unit 206, and the image data after the image capture determination is acquired in real time by the inference device 6A. The inference device 6A performs the known item detection process, the two-stage whiteout detection process, the normalization process, and the authenticity determination process in real time. Even if an unauthorized user manipulates an image, the developer managing the inference device 6A can understand the manipulation process from the video. This allows unauthorized operations by unauthorized users to be detected and dealt with promptly.

[0175] It should be noted that the present invention is not limited to the above-described embodiments, and various other applications and modifications are possible without departing from the gist of the present invention as set forth in the claims. For example, the above-described embodiments have described the configuration of the device and system in detail and specifically in order to clearly explain the present invention, and are not necessarily limited to those including all of the described configurations. Furthermore, it is possible to replace part of the configuration of the embodiments described here with the configuration of other embodiments, and it is also possible to add the configuration of one embodiment to the configuration of another embodiment. Furthermore, it is also possible to add, delete, or replace part of the configuration of each embodiment with other configurations. In addition, the control lines and information lines shown are those that are considered necessary for the explanation, and do not necessarily show all the control lines and information lines in the product. In reality, it can be assumed that almost all components are interconnected. [Explanation of symbols]

[0176] 2...information processing terminal, 3...learning device, 6...inference device, 40...learning data DB, 45...server-side learning model DB, 50...master image DB, 60...server-side inference model DB, 100...authenticity determination system, 202a...known item detection model, 202b...authenticity determination model, 203...display processing unit, 204...video image capturing unit, 205...card area detection unit, 206...photography determination unit, 207...known item detection unit, 208...two-stage overexposure detection unit, 209...normalization unit, 210...authenticity determination unit, 201...local image DB, 202...local-side inference model DB

Claims

1. An authenticity determination program for determining the authenticity of a card medium having a predetermined surface to which processing has been applied to change the reflection state of light reflected from the predetermined surface, a display control procedure for displaying a guide for changing the shooting angle of the moving image of the card medium captured by a camera on a screen of a terminal device and changing the display position of the guide during the shooting period; a known image detection step of detecting a known image formed on the card medium from the moving image data of the card medium captured according to the guide; a poorly photographed portion detection procedure for detecting the presence or absence of a poorly photographed portion in the image on the card medium when it is not certain that the known image has been detected, and instructing the device to photograph a moving image on the card medium again and performing the display control procedure when it is detected that the poorly photographed portion exists; a determination procedure for determining the authenticity of the card medium based on time series information of reflected light from a predetermined surface of the card medium, which is included in video image data of the card medium captured by the camera with the guide displayed, when it is certain that the known image has been detected, or when it is detected that there are no imperfectly photographed areas or that the number of imperfectly photographed areas is significantly small; An authenticity determination program to be run by a computer.

2. The known image detection step estimates information specific to the card medium contained in the image of the card medium based on the known image, and outputs a certainty score representing a certainty of the estimation of the specific information; The imaging defect detection step determines that it is not certain that the image is a known image when the certainty score is less than a predetermined certainty threshold. The authenticity determination program according to claim 1.

3. The step of detecting a defective photographed portion includes a first analysis step of analyzing a ratio of pixels in the image of the card medium whose brightness values ​​exceed a predetermined brightness threshold value; a first warning step of warning of the possibility of the defective imaging location when the proportion of the pixels exceeds a predetermined pixel proportion threshold. The authenticity determination program according to claim 2.

4. The first analysis step converts the entire image of the card medium into a luminance image and compares the luminance value for each pixel of the luminance image with the luminance threshold. The authenticity determination program according to claim 3.

5. The imaging defect detection procedure includes a second analysis procedure, in which, when the first analysis procedure determines that the pixel ratio does not exceed the pixel ratio threshold, the image of the card medium is divided into blocks of a predetermined size, and the relationship between high luminance density and a predetermined high luminance density threshold is analyzed for each block. a second warning step of warning of a possibility of the poorly photographed portion when the block in which the high luminance density is equal to or greater than the high luminance density threshold exists. The authenticity determination program according to claim 4.

6. In the imaging defect detection step, the second analysis step calculates, as the high luminance density, a ratio of the number of pixels whose luminance value is equal to or greater than a luminance value threshold to the total number of pixels in the block. The authenticity determination program according to claim 5.

7. The determination step includes a step of analyzing time-series information of the reflected light included in the video image data and a time-varying pattern of the reflected light from the predetermined surface included in the video image data obtained by photographing the authentic card medium, and classifying the time-varying pattern on a learning basis. The authenticity determination program according to claim 2.

8. The determination step includes a step of causing an authenticity determination model that receives the video data as input and outputs a determination result of the authenticity of the card medium to perform the classification. The authenticity determination program according to claim 7.

9. The determination step includes a photographing determination step of determining whether the camera is photographing the physical card medium by comparing a movement vector of a feature point extracted from the video image data transmitted from the camera with each movement vector in a plurality of coordinate axis directions acquired from an inertial sensor provided in the terminal device; an authenticity determination procedure in which the authenticity determination model determines the authenticity of the card medium. The authenticity determination program according to claim 8.

10. The photographing determination step determines whether or not a movement vector of a feature point extracted from the moving image data transmitted from the camera and a movement vector acquired from the inertial sensor are moving simultaneously in the same direction. The authenticity determination program according to claim 9.

11. a normalization procedure for extracting a card area corresponding to the card medium from the video image data of the card medium photographed by the camera, and normalizing the number of pixels in the length and width of the extracted video image of the card area to a fixed length of pixel number suitable for input to the authentication model; The authenticity determination procedure inputs the normalized card area into the authenticity determination model to determine the authenticity of the card medium. The authenticity determination program according to claim 10.

12. The display control step includes a step of displaying, as the guide, an image that specifies a display position of the card medium on the screen of the terminal device on the screen of the terminal device. The authenticity determination program according to claim 8.

13. The display control procedure includes a procedure for changing a display position of an image specifying the display position of the card medium on the screen to a position different from the displayed position when it is determined that a frame portion of the card medium fits into the display position of the card medium indicated by the image specifying the display position of the card medium. The authenticity determination program according to claim 12.

14. Due to the processing applied to the predetermined surface of the card medium, the temporal change patterns of the reflected light from the predetermined surface corresponding to the incident light from a plurality of different incident angles are different from each other. The authenticity determination program according to any one of claims 1 to 13.

15. The processing on the predetermined surface includes processing to form a concave-convex portion by using a transparent varnish printed on the first surface of the card medium. The authenticity determination program according to claim 14.

16. A method for determining the authenticity of a card medium having a predetermined surface to which a processing treatment for changing a reflection state of reflected light from the predetermined surface has been applied, comprising: a display control procedure for displaying a guide for changing the shooting angle of the moving image of the card medium captured by a camera on a screen of a terminal device and changing the display position of the guide during the shooting period; a known image detection step of detecting a known image formed on the card medium from the moving image data of the card medium captured according to the guide; a poorly photographed portion detection procedure for detecting the presence or absence of a poorly photographed portion in the image on the card medium when it is not certain that the known image has been detected, and instructing the device to photograph a moving image on the card medium again and performing the display control procedure when it is detected that the poorly photographed portion exists; and a determination procedure for determining the authenticity of the card medium based on time-series information of reflected light from a predetermined surface of the card medium, which is included in video image data of the card medium captured by the camera with the guide displayed, when it is certain that the known image has been detected, or when it is detected that there are no defectively captured areas or that the number of defectively captured areas is significantly small. Authenticity determination method.

17. An authenticity determination device for determining the authenticity of a card medium having a predetermined surface that has been subjected to a processing treatment that changes the reflection state of light reflected from the predetermined surface, a display control unit that displays a guide for changing the shooting angle of the moving image of the card medium captured by the camera on a screen of the terminal device and changes the display position of the guide during the shooting period; a known image detection unit that detects a known image formed on the card medium from the moving image data of the card medium captured according to the guide; a poorly photographed portion detecting unit that detects the presence or absence of a poorly photographed portion in the image on the card medium when it is not certain that the known image has been detected, and that, when it detects the presence of the poorly photographed portion, instructs the display control unit to again photograph a moving image on the card medium; and a determination unit that, when it is certain that the known image has been detected, or when it is detected that there are no defective photographing areas or that the number of defective photographing areas is significantly small, determines the authenticity of the card medium based on time-series information of reflected light from a predetermined surface of the card medium, which is included in moving image data of the card medium captured by the camera with the guide displayed. Authenticity determination device.

Images