Information processing method, program, and information processing device

Abstract

An information processing method and the like that makes it possible to detect impersonation is provided. [Solution] Before the online exam date, a computer acquires first photographic data of the examinee's official photo ID and second photographic data of the examinee. The computer then allows the examinee to take the online exam if the examinee's identity is verified based on the first and second photographic data. The computer also acquires third photographic data of the examinee who has been permitted to take the online exam on the day of the online exam.

Description

[Technical Field]

[0001] The present disclosure relates to an information processing method, a program, and an information processing device. [Background technology]

[0002] In recent years, online tests have become available, allowing test takers to take exams such as qualification exams or certification exams from their homes via communication networks such as the Internet. Patent Document 1 discloses a technique for monitoring test takers during online tests. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 7458595 Summary of the Invention [Problem to be solved by the invention]

[0004] In online exams, measures must be taken to prevent not only cheating during the exam but also "impersonation," where someone other than the examinee takes the exam. In Patent Document 1, a photo taken for identity verification is printed on the certificate, so the examinee can be identified by the photo on the certificate, preventing impersonation. However, the technology disclosed in Patent Document 1 cannot detect impersonation when applying for or taking the exam.

[0005] An object of the present disclosure is to provide an information processing method and the like that makes it possible to detect spoofing. [Means for solving the problem]

[0006] In one embodiment of the information processing method of the present disclosure, a computer executes a process to acquire first photographed data of an official document with a photograph of the examinee and second photographed data of the examinee before the online exam date, and if the examinee's identity is authenticated based on the first photographed data and the second photographed data, allows the examinee to take the online exam, and acquires third photographed data of the examinee who has been allowed to take the online exam on the day of the online exam. [Effects of the Invention]

[0007] According to the present disclosure, it is possible to detect spoofing. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is an explanatory diagram illustrating an example of the configuration of an online examination system. [Figure 2] FIG. 2 is a block diagram showing an example of the configuration of a server and an examinee terminal. [Figure 3] FIG. 2 is an explanatory diagram showing an example of a record layout of an examinee DB stored in a server. [Figure 4] FIG. 1 is an explanatory diagram illustrating an example of the configuration of a learning model. [Figure 5] 10 is a flowchart showing an example of a processing procedure for advance authentication of an examinee. [Figure 6] FIG. 10 is an explanatory diagram showing an example of a screen on an examinee terminal. [Figure 7] 10 is a flowchart illustrating an example of a processing procedure for an online test. [Figure 8] 10 is a flowchart illustrating an example of a processing procedure for an online test. [Figure 9] FIG. 10 is an explanatory diagram showing an example of a screen on an examinee terminal. [Figure 10] FIG. 10 is an explanatory diagram showing an example of a screen on an examinee terminal. [Figure 11] 10 is a flowchart illustrating an example of a procedure for determining fraudulent activity. [Figure 12] FIG. 10 is an explanatory diagram showing an example of a screen on an operator terminal. [Figure 13] 10 is a flowchart showing an example of a processing procedure for an online test according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] The information processing method, program, and information processing device of the present disclosure will be described in detail below with reference to the drawings showing an online examination system that is an embodiment thereof.

[0010] (Embodiment 1) This embodiment describes an online examination system that provides online examinations in which examinees take proficiency tests, certification tests, etc. from home or the like via a network. The online examination in this embodiment is an Internet-Based Testing (IBT) examination that examinees take at home or at work, etc., using a terminal owned by the examinee or the company, etc. However, the online examination system of this embodiment can also be applied to a system that provides Computer-Based Testing (CBT) examinations that examinees take at a designated examination venue using a terminal provided by the operator. Furthermore, the online examination in this embodiment is an examination in which a specified examination period is specified and the examinee can take the examination any time within the examination period, but it may also be an examination in which a specified examination date is specified and the examinee can take the examination only on the specified examination date.

[0011] 1 is an explanatory diagram showing an example of the configuration of an online examination system. The online examination system of this embodiment includes a server 10, an examinee terminal 20, an authentication server 30, an operator terminal 40, etc., and each device is communicatively connected via a network N. There may be multiple examinee terminals 20 and multiple operator terminals 40. The network N may be the Internet or a public telephone line network, or may be a LAN (Local Area Network) constructed within the facility where the online examination system is installed.

[0012] The server 10 and the authentication server 30 are information processing devices capable of various information processing and information transmission / reception, such as a server computer, a personal computer, or a quantum computer. The server 10 may be managed by the company that administers the online exam, or by a company entrusted with administering the online exam. The authentication server 30 is a server that provides user authentication (examinee authentication) using a public photo ID, such as an eKYC (electronic Know Your Customer) system. The authentication server 30 may be managed by the company that administers the online exam, or by another company entrusted with examinee authentication. The examinee terminal 20 and the operator terminal 40 are information processing devices capable of various information processing and information transmission / reception, such as a personal computer, a tablet terminal, or a smartphone. The examinee terminal 20 is a terminal device of an examinee who takes the online exam provided by the server 10, and may be a laptop or desktop. The operator terminal 40 is, for example, a terminal device of a person in charge (operator) of the company that administers the online exam.

[0013] In the online examination system of this embodiment, the server 10 has web server functionality and provides an online examination site 12S (see FIG. 2) for conducting online examinations to the examinee terminal 20 via the network N. Before taking the online examination, the examinee terminal 20 transmits to the server 10, as advance information, photographed images of the examinee's official certificate and photographed images of the examinee's face. The server 10 then transmits the photographed images of the official certificate and photographed images of the examinee to the authentication server 30, which then authenticates the examinee (verifies the examinee's identity). Only examinees authenticated by the authentication server 30 are permitted to take the online examination. Just before taking the online examination, the examinee terminal 20 transmits to the server 10, as advance information, photographed images of the examinee's face and video footage of the area above and below the desk where the examination will be conducted and the interior of the room. The server 10 authenticates the examinee based on the photographed images of the examinee acquired here and the photographed images of the official certificate or the examinee acquired as advance information, thereby confirming the examinee's authenticity. During the test, the examinee terminal 20 also captures video of the examinee's face (upper body), collects indoor audio, and tracks the cursor position, and transmits the examinee's captured video, indoor audio, and cursor position to the server 10 as in-test information. The in-test information may be transmitted to the server 10 in real time each time the examinee terminal 20 acquires it, or may be transmitted to the server 10 all at once after the test is completed. The examinee terminal 20 is not limited to a configuration in which the examinee's captured video, indoor audio, and cursor position are acquired separately during the test. For example, the examinee's captured video, including indoor audio, may be displayed on part of the test screen, and screen data may be acquired that includes the test screen on which the captured video is displayed and the cursor moving on the test screen. In this case, the examinee's captured video, indoor audio, and cursor position can be acquired together as a single screen data. For example, in a configuration in which an online exam is taken while screens are shared between the server 10 and the examinee terminal 20 using a system that exchanges filmed video and spoken audio via the network N, the server 10 may obtain various information from the examinee terminal 20 (the examinee's filmed video, room audio, cursor position) through screen sharing. The server 10 determines whether the examinee is engaging in cheating based on the information during the exam.In this way, in this embodiment, examinee authentication is performed in advance using an official certificate, and examinee authentication is performed based on a photographed image immediately before the exam, thereby confirming the legitimacy of the examinee and preventing impersonation by others. In addition, video, audio, and cursor position data collected during the exam can be used to determine whether cheating or other fraudulent behavior has occurred during the exam.

[0014] FIG. 2 is a block diagram showing an example configuration of the server 10 and the examinee terminal 20. The server 10 includes a control unit 11, a memory unit 12, a communication unit 13, a reading unit 14, etc., and each unit is connected to each other via a bus. The control unit 11 includes one or more processors (arithmetic processing devices), such as a CPU (Central Processing Unit), an MPU (Micro-Processing Unit), or a GPU (Graphics Processing Unit). The control unit 11 executes the processes to be performed by the server 10 by appropriately executing a program 12P stored in the memory unit 12. Note that when the control unit 11 includes multiple processors, each process may be executed by the same processor, or each process may be executed by a different processor.

[0015] The memory unit 12 includes RAM (Random Access Memory), flash memory, a hard disk, an SSD (Solid State Drive), etc. The memory unit 12 stores a program 12P (program product, computer program) executed by the control unit 11 and various data. The memory unit 12 also temporarily stores data generated when the control unit 11 executes the program 12P. The memory unit 12 also stores an online examination site 12S and an examinee DB 12a. The memory unit 12 further stores a learning model 12M that has learned training data through machine learning. The learning model 12M is expected to be used as a program module constituting artificial intelligence software. The learning model 12M performs a predetermined calculation on input data and outputs the calculation result. The memory unit 12 stores data such as coefficients and thresholds of functions that define this calculation as the learning model 12M. In addition to being configured to be stored in the memory unit 12, the learning model 12M may be read by the server 10 by accessing another server that stores the learning model 12M. The memory unit 12 may be composed of multiple storage devices, and part of the memory unit 12 may be another storage device connected to the server 10, or another storage device with which the server 10 can communicate.

[0016] The communication unit 13 is a communication module for performing processes related to wired or wireless communication, and transmits and receives information to and from other devices via the network N. The reading unit 14 reads information recorded on a recording medium 10a such as a memory card or an optical disk. The program 12P and various data stored in the storage unit 12 may be read by the control unit 11 from the recording medium 10a via the reading unit 14 and stored in the storage unit 12. Furthermore, the program 12P and various data stored in the storage unit 12 may be written to the storage unit 12 during the manufacturing stage of the server 10, or may be downloaded by the control unit 11 from another device via the communication unit 13 and stored in the storage unit 12.

[0017] In this embodiment, the server 10 is not limited to a single computer, but may be a multi-computer consisting of multiple computers, or may be a virtual machine virtually constructed by software within a single device. The server 10 may also be a local server installed in a facility where the server 10 (online testing system) is installed, or a cloud server connected to the facility via a network N. The program 12P may be deployed and executed on a single computer or at a single site, or may be distributed across multiple sites and deployed to be executed on multiple computers interconnected via the network N. Furthermore, the server 10 may be configured to include an input unit for accepting user inputs, a display unit for displaying various information, and the like.

[0018] The authentication server 30 has a similar configuration to the server 10, and therefore a description of the configuration will be omitted. The storage unit of the authentication server 30 stores an authentication program for authenticating examinees based on a photographed image of the examinee's official photo ID and a photographed image of the examinee's face. In this embodiment, the authentication server 30, which performs examinee authentication based on the photographed image of the official photo ID, is provided separately from the server 10. However, the server 10 may also have the functions of the authentication server 30. Alternatively, the online test administration company may manage the server 10 and the authentication server 30. Furthermore, the server 10 may perform examinee authentication based on the photographed image of the official photo ID. For example, the server 10 may perform examinee authentication using a method such as pattern matching. Specifically, the server 10 may extract features from the photographed image of the examinee's official photo ID and the photographed image of the examinee's face, calculate the similarity between the features, and determine authentication success if the calculated similarity is equal to or greater than a threshold, or failure if the calculated similarity is less than the threshold. The similarity may be determined using a correlation coefficient, cosine similarity, or the like.

[0019] The examinee terminal 20 includes a control unit 21, a memory unit 22, a communication unit 23, an input unit 24, a display unit 25, a camera 26, a microphone 27, etc., and each unit is interconnected via a bus. The control unit 21, memory unit 22, and communication unit 23 of the examinee terminal 20 have the same configuration as the control unit 11, memory unit 12, and communication unit 13 of the server 10, so a description of their configuration will be omitted. In addition to the program 22P executed by the control unit 21, the memory unit 22 of the examinee terminal 20 also stores a web browser (hereinafter referred to as browser 22B) for accessing the web server.

[0020] The display unit 25 is a liquid crystal display or organic EL display, etc., and displays various information in accordance with instructions from the control unit 21. The input unit 24 includes, for example, a mouse and keyboard, etc., and accepts operation inputs by the examinee and sends control signals corresponding to the operation content to the control unit 21. The input unit 24 of the examinee terminal 20 of this embodiment has a pointing device 24a that can operate a cursor displayed on the screen of the display unit 25, and the pointing device 24a can be, for example, a mouse, touchpad, trackball, joystick, etc.

[0021] Camera 26 is an imaging device having a lens, an imaging element, etc., and takes pictures in accordance with instructions from control unit 21, acquiring one frame of image data (still image data) or 15 or 30 frames of image data (video data) per second, and stores the acquired image data in memory unit 22. Camera 26 is provided in a position where it can capture an image of the face (for example, the area above the chest) of an examinee taking the test using examinee terminal 20. Microphone 27 collects sound in accordance with instructions from control unit 21 to acquire audio data, and stores the acquired audio data in memory unit 22. Camera 26 and microphone 27 may be built into examinee terminal 20, or may be configured to be externally attached to examinee terminal 20. Furthermore, multiple cameras 26 and microphones 27 may be provided.

[0022] The operator terminal 40 has the same configuration as the examinee terminal 20, and therefore a description thereof will be omitted. The operator terminal 40 may be configured without a camera or microphone.

[0023] Fig. 3 is an explanatory diagram showing an example of the record layout of the examinee DB 12a stored in the server 10. The examinee DB 12a is a database that stores information about examinees who have registered to take online exams provided by the server 10. The examinee DB 12a is prepared, for example, for each online exam or for each exam period specified for an online exam, and in the example of Fig. 3, the examinee DB 12a is stored in the storage unit 12 in association with the exam ID assigned to each online exam and the exam period for that online exam.

[0024] The examinee DB 12a shown in FIG. 3 includes an examinee ID sequence, an examinee information sequence, a prior information sequence, a recent information sequence, an in-test information sequence, and a test information sequence, and stores various information in association with identification information (examinee ID) assigned to each examinee. The examinee information sequence stores information about the examinee, such as the examinee's name, date of birth, address or location, contact information, and authentication information used for logging in when taking an online test. The prior information sequence stores information registered in advance before the online test date (test period). The prior information is information related to examinee authentication using an official certificate, which is performed in advance, and includes photographed images of the official certificate and photographed images of the examinee's face, as well as the results of the examinee authentication. The official certificate used for examinee authentication is an official certificate with a face photo, and can be, for example, a driver's license, My Number card, residence card, or special permanent resident certificate. The recent information sequence stores information registered before the start of the online test on the test date (test day). The latest information includes an image of the examinee's face and a video of the examination environment (room) taken by the examinee terminal 20 before (just before) the start of the exam, as well as the results of examinee authentication performed based on the image of the examinee's face. The video of the examination environment can be, for example, a video of the top and bottom of the desk used for the exam and the interior of the room used for the exam taken over a predetermined period of time (e.g., 30 seconds). The test information sequence stores information collected by the examinee terminal 20 during the exam. The test information includes a video of the examinee's face (e.g., the area above the chest) taken during the exam, audio data collected in the room during the exam, and cursor position data detected by the pointing device 24a during the exam. The cursor position data is data indicating the results of tracking the cursor position on the exam screen. Note that the cursor position data may also be, for example, the time period when the cursor position left the exam screen (such as the time when it moved off the exam screen and the time when it returned from off the exam screen to the exam screen). The test information sequence stores information related to the exam. The test information includes the test date, answer data, test results, whether or not there was any cheating, and the operator's confirmation results. The test date may be the year, month, and day, or the start and end dates and times of the test.The answer data is the answer data to the test questions entered by the examinee via the examinee terminal 20, and the test result is the pass / fail judgment result (pass or fail) based on the answer data. The presence or absence of cheating is the result of the server 10's judgment of the presence or absence of cheating using the in-test information, and the operator's confirmation result is the result of the operator's confirmation of the presence or absence of cheating based on the server 10's judgment of cheating. Note that the operator performs confirmation when the server 10 judges that cheating has occurred, so the operator's confirmation result is stored only when the server 10 judges that cheating has occurred. The contents stored in the examinee DB 12a are not limited to the example shown in FIG. 3, and various information necessary for online testing may be stored. For example, if the examinee terminal 20 acquires screen data including video footage of the examinee, room audio, and cursor position as in-test information, the screen data may be stored as in-test information. Furthermore, the examinee DB 12a may be divided into multiple DBs and each piece of information may be stored, or some information may be registered in a DB on another server. For example, the in-test information and / or test information may be stored in another server in association with the examinee ID.

[0025] Fig. 4 is an explanatory diagram showing an example of the configuration of learning model 12M. Learning model 12M shown in Fig. 4A is trained to receive inputs of video data of the examinee, audio data in the room, and cursor position data obtained by tracking the cursor position, all collected by examinee terminal 20 while the examinee is taking an online exam, perform calculations to estimate whether or not the examinee has engaged in cheating based on the input data, and output the calculation results. Learning model 12M is configured using algorithms such as CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), LSTM (Long Short-Term Memory), Transformer, decision tree, random forest, and SVM (Support Vector Machine), and may also be configured by combining multiple algorithms.

[0026] The learning model 12M has an input layer, an intermediate layer, and an output layer. The input layer has multiple input nodes, through which the examinee's video data, in-room audio data, and cursor position data are input. The intermediate layer uses various functions, thresholds, etc. to calculate output values ​​from each piece of data input through the input layer, and outputs the calculated output values ​​to the output layer. The output layer has multiple output nodes. Each output node is associated with a state of cheating due to gaze direction, cheating due to in-room audio, cheating due to cursor position, and normal (no cheating), and each output node outputs the probability (certainty) of determining that the associated state is true. The output value of each output node is, for example, a value between 0 and 1, and the sum of the probabilities output from each output node is 1 (100%).

[0027] The server 10 identifies the output node in the learning model 12M that outputs the largest output value (certainty) among the output values ​​from each output node, and identifies the state associated with the identified output node as the state of the examinee to be estimated. Note that the learning model 12M may be configured to have a single output node that outputs the state with the highest certainty, instead of having multiple output nodes that output certainty for each state.

[0028] The learning model 12M is generated by machine learning using training data that associates training video data, audio data, and cursor position data with correct labels indicating the examinee's state (presence and type of cheating). The training data is generated by acquiring, for example, data on an examinee taking an online exam, including a video of the examinee's face (e.g., the area above the chest), audio from the room where the examinee is taking the exam, and the position of the cursor operated by the examinee, and assigning the examinee's state to each acquired data. The training data may also be generated by assigning the examinee's state to video data, audio from the room, and cursor position data acquired from an examinee taking a simulated online exam or from an examinee in a situation similar to that in the online exam. The examinee's state may be represented by a label determined by an evaluator who can determine the presence or absence of cheating based on the video data, audio data, and cursor position data of each examinee. The training data generated in this manner is stored, for example, in a training database (not shown) provided in the memory unit 12 and used during the learning process of the learning model 12M.

[0029] When video data, audio data, and cursor position data included in the training data are input, the learning model 12M learns so that the output value from the output node corresponding to the state indicated by the correct label approaches 1 and the output values ​​from other output nodes approach 0. In the learning process, the learning model 12M performs calculations based on the input video data, audio data, and cursor position data to calculate output values ​​from each output node. The learning model 12M then compares the calculated output value of each output node with a value corresponding to the correct label (1 for the output node corresponding to the state indicated by the correct label, and 0 for other output nodes), and optimizes parameters used in the calculation process so that each output value approximates the value corresponding to the correct label. The parameters to be optimized include weights (coupling coefficients) between nodes in the learning model 12M, and the optimization method can be an error backpropagation method, a steepest descent method, or the like. This results in a learning model 12M that, when video data, audio data, and cursor position data collected by the examinee terminal 20 during an online exam are input, estimates whether or not an examinee has engaged in any fraudulent activity and the type of such activity, and outputs the estimated results.

[0030] The learning model 12M may be trained not only by the server 10 but also by other learning devices. The trained learning model 12M, generated by training on another learning device, is downloaded from the learning device to the server 10 via the network N or the recording medium 10a, for example, and stored in the memory unit 12. The learning model 12M is not limited to the configuration shown in FIG. 4. For example, the learning model 12M may be configured to receive video data of the examinee and in-room audio data collected during an online exam and output confidence levels for three states: presence of misconduct due to gaze direction, presence of misconduct due to in-room audio, and normal. The states associated with each output node are not limited to the example shown in FIG. 4 and may include other types of misconduct that can be determined from the examinee's video, in-room audio, and cursor position. The states associated with each output node may also include various states related to the examinee's behavior and the exam environment, such as whether or not the examinee cheated, whether or not the examinee left their seat, whether or not there were people other than the examinee, and whether or not there was noise. The process of determining whether or not there was misconduct due to cursor position based on cursor position data may be rule-based. For example, by setting a time (threshold) for determining that cheating has occurred for the time the cursor is positioned outside the test screen, the time that the cursor is positioned outside the test screen during the test can be measured, and if the measured time exceeds the threshold, it can be determined that cheating has occurred (or is suspected).

[0031] FIG. 4B is an explanatory diagram showing another example of the configuration of the learning model. When the server 10 acquires the examinee's video recording, room audio, and cursor position as a single screen data from the examinee terminal 20, the learning model 12Ma in FIG. 4B may be used to estimate whether or not the examinee has engaged in cheating. The learning model 12Ma shown in FIG. 4B is trained to input screen data (including the examinee's video data, room audio data, and cursor position data) collected by the examinee terminal 20 during an online exam, estimate whether or not the examinee has engaged in cheating based on the input screen data, and output the estimation result. Although the input data of the learning model 12Ma is different from that of the learning model 12M, the output values ​​from each output node are the same as those of the learning model 12M. Even when the learning model 12Ma in FIG. 4B is used, the presence and type of cheating by the examinee can be estimated. In addition, the learning models may be prepared separately for a model that estimates whether or not an examinee has engaged in cheating based on a video of the examinee, a model that estimates whether or not an examinee has engaged in cheating based on the audio in the room, and a model that estimates whether or not an examinee has engaged in cheating based on the cursor position.

[0032] The following describes the processing related to the online examination performed by the online examination system of this embodiment. In this system, the examinee undergoes examinee authentication in advance using a public certificate. The examinee also undergoes examinee authentication using a photographed image of their face immediately before taking the exam, and can begin the exam if authenticated. FIG. 5 is a flowchart showing an example of the processing procedure for examinee authentication in advance, and FIG. 6 is an explanatory diagram showing an example screen on the examinee terminal 20. In FIG. 5, the left side shows the processing performed by the examinee terminal 20, the center shows the processing performed by the server 10, and the right side shows the processing performed by the authentication server 30. Note that the examinee terminal 20 used for examinee authentication using a public certificate in advance and the examinee terminal 20 used for examinee authentication using a photographed image of the examinee immediately before taking the exam and for taking the exam may be the same terminal or different terminals. For example, the examinee may perform examinee authentication in advance using their own smartphone, and perform examinee authentication immediately before taking the exam and take the exam using their own personal computer.

[0033] In the system of this embodiment, the control unit 21 of the examinee terminal 20 accesses the online examination site 12S in accordance with operational input from the examinee via the input unit 24 (S11). After applying for the online examination, the examinee obtains a Uniform Resource Locator (URL) for examinee authentication in advance from the examination administration company via an examination information email or the like. The control unit 21 then starts the browser 22B and accesses the online examination site 12S based on the obtained URL.

[0034] The control unit 11 of the server 10 transmits the web page accessed by the examinee terminal 20, in this case the pre-authentication screen of the online examination site 12S, to the examinee terminal 20 (S12). The control unit 21 of the examinee terminal 20 receives the pre-authentication screen from the server 10 and displays it on the display unit 25 (S13). FIG. 6 shows an example of the pre-authentication screen, which displays the exam type, exam period, and the examinee's name and date of birth. This information is, for example, information registered in the examinee DB 12a of the server 10 when the examinee applies for the online examination. If there is information not registered in the examinee DB 12a, the screen of FIG. 6 may have an input field for that information, which can be entered at this point and registered in the examinee DB 12a. The screen of FIG. 6 also has an input field R1 for an image of a photographed official certificate taken from the front in a planar view, an input field R2 for an image of the official certificate taken from an oblique angle, and an input field R3 for an image of the examinee's face.

[0035] For example, the examinee operates input field R1 on the pre-authentication screen to activate camera 26, capture a front image of their official certificate, and operates input field R2 to activate camera 26 and capture an oblique image of their official certificate. The examinee also operates input field R3 on the pre-authentication screen to activate camera 26 and capture an image of their face from the front. Control unit 21 of examinee terminal 20 captures images using camera 26 in accordance with the examinee's operations as described above, and acquires the front and oblique images of the official certificate (first captured data) and the captured image of the examinee (second captured data) as examinee authentication data (S14). Control unit 21 determines whether the authentication button on the pre-authentication screen has been operated (S15). If it determines that the button has not been operated (S15: NO), the process returns to step S14 and continues acquiring examinee authentication data. If it determines that the authentication button has been operated (S15; YES), control unit 21 transmits the acquired examinee authentication data to server 10 (S16). At this time, the control unit 21 transmits the test type, the name of the testee, and other information for testee authentication.

[0036] The control unit 11 of the server 10 acquires the examinee authentication data from the examinee terminal 20 and stores it as advance information in the examinee DB 12a (S17). Specifically, the control unit 11 stores the photographed image of the official certificate and the photographed image of the examinee in the advance information sequence corresponding to the examinee ID in the examinee DB 12a corresponding to the test ID and test period according to the test type. The control unit 11 transmits the photographed image of the official certificate and the photographed image of the examinee (examinee authentication data) to the authentication server 30 and requests that examinee authentication be performed (S18).

[0037] The control unit of the authentication server 30 acquires the examinee authentication data from the server 10, and performs examinee authentication (identity verification) based on the photographed image of the examinee's official certificate and the photographed image of the examinee in accordance with the authentication program (S19), and sends the authentication result to the server 10 (S20). The control unit 11 of the server 10 acquires the authentication result from the authentication server 30 and stores it in the examinee DB 12a as the authentication result of the preliminary information (S21). For example, if the authentication result is successful, "Authentication" is stored, and if the authentication result is not successful, "Authentication Failed" is stored. The control unit 11 stores "Authentication" as the authentication result, thereby allowing the examinee to take the exam. The control unit 11 then transmits the authentication result to the examinee terminal 20 (S22), and the control unit 21 of the examinee terminal 20 acquires the authentication result from the server 10 and displays it on the display unit 25 (S23). The control unit 11 notifies the examinee of the authentication result by generating a screen according to the authentication result and sending it to the examinee terminal 20. For example, the control unit 11 generates a screen for authenticated examinees to notify them that examinee authentication has been completed and they are permitted to take the exam, and generates a screen for unauthenticated examinees to notify them that authentication using a public certificate has failed. Examinees who are notified that authentication has failed will retry examinee authentication using a public certificate, and will be permitted to take the exam if authentication is successful.

[0038] The above-described process allows examinees to be authenticated using a public certificate before taking an online exam, thereby confirming that the examinee who has applied for the exam is the person listed in the public certificate. By using photographs of examinees whose identities have been verified (authenticated) in this way for examinee authentication before the exam begins, it becomes possible to verify the examinee's identity at the start of the exam, and to detect impersonation at the start of the exam. Furthermore, photographs of examinees whose identities have been verified may also be used for examinee authentication based on photographs of the examinee taken during the exam, in which case it becomes possible to verify the examinee's identity during the exam.

[0039] FIGS. 7 and 8 are flowcharts showing an example of the processing procedure for an online exam, and FIGS. 9 and 10 are explanatory diagrams showing example screens on the examinee terminal 20. In FIGS. 7 and 8, the left side shows the processing performed by the examinee terminal 20, and the right side shows the processing performed by the server 10. In the system of this embodiment, after applying for an online exam or being authenticated through prior examinee authentication, the examinee obtains an exam URL from the exam administration company via an exam invitation email or the like. The examinee takes the online exam by accessing the online exam site 12S using the exam URL on any exam date (online exam date) within the designated exam period. Before taking the online exam (before the exam starts on the day of the online exam), the examinee takes a photograph of their face and sends it to the server 10. The server 10 then confirms, based on the photographed image of the examinee, that the examinee is the examinee authenticated through prior examinee authentication. If the examinee's identity is confirmed, the examinee is allowed to take the exam.

[0040] The control unit 21 of the examinee terminal 20 accesses the online examination site 12S in accordance with operational input from the examinee via the input unit 24 (S31). The control unit 21 then launches the browser 22B and accesses the online examination site 12S based on the URL for the examination. The control unit 11 of the server 10 then transmits the web page accessed by the examinee terminal 20—in this case, the last-minute authentication screen of the online examination site 12S—to the examinee terminal 20 (S32). The control unit 21 of the examinee terminal 20 then receives the last-minute authentication screen from the server 10 and displays it on the display unit 25 (S33). FIG. 9 shows an example of the last-minute authentication screen, which displays the exam type, exam period, exam date, and the examinee's name and date of birth. The examinee's name and date of birth may be displayed on the last-minute authentication screen, or they may be entered by the examinee in input fields. The screen of FIG. 9 also includes an input field R4 for an image of the examinee's face and an input field R5 for an image of the exam environment.

[0041] For example, the examinee activates the camera 26 by operating input field R4 on the last-minute authentication screen and captures an image of their face from the front. The examinee also activates the camera 26 by operating input field R5 on the last-minute authentication screen and captures video of the area above and below the desk used for the exam, as well as the room. If the examinee's examinee terminal 20 is a portable terminal such as a laptop computer or tablet, the examinee holds the examinee terminal 20 and captures the exam environment while changing the shooting direction of the camera 26 built into the examinee terminal 20. If the examinee terminal 20 is a desktop computer and the camera 26 is an external camera attached to the examinee terminal 20, the examinee captures the exam environment while changing the shooting direction of the camera 26. If the shooting direction of the camera 26 cannot be changed, the examinee may capture the exam environment using a portable examinee terminal 20 different from the examinee terminal 20 used for the exam and send the captured image to the server 10. If the examinee uses a different portable examinee terminal 20 to capture the exam environment, the capture may be performed before the start of the online exam or after the online exam is completed.

[0042] The control unit 21 of the examinee terminal 20 captures images using the camera 26 in accordance with the examinee's operation as described above, and acquires the captured image of the examinee (third captured data) and the captured video of the examination environment (fourth captured data) as immediate-precedent authentication data (S34). The control unit 21 determines whether the authentication button on the immediate-precedent authentication screen has been operated (S35). If it determines that it has not been operated (S35: NO), the control unit 21 returns to step S34 and continues acquiring immediate-precedent authentication data. If it determines that the authentication button has been operated (S35; YES), the control unit 21 transmits the acquired immediate-precedent authentication data to the server 10 (S36). At this time, the control unit 21 transmits the immediate-precedent authentication data together with the test type, the examinee's name, etc. If the immediate-precedent authentication screen has input fields for the examinee's name and date of birth, the control unit 21 accepts input of the examinee's name and date of birth via the input fields, and transmits the input name and date of birth, the test type, and immediate-precedent authentication data to the server 10. In this case, the server 10 can verify the identity of the examinee using the examinee's name and date of birth entered by the examinee and the latest authentication data (a photographed image of the examinee).

[0043] The control unit 11 of the server 10 acquires the last-minute authentication data from the examinee terminal 20 and stores it in the examinee DB 12a as last-minute information (S37). Specifically, the control unit 11 stores the photographed images of the examinee and the photographed video of the examination environment in the last-minute information sequence corresponding to the examinee ID in the examinee DB 12a corresponding to the test ID and examination period according to the exam type. Next, the control unit 11 performs examinee authentication (identity verification) based on the photographed images of the examinee stored in the examinee DB 12a (S38). For example, the control unit 11 verifies whether the person about to take the exam is an examinee who has been previously authenticated as an examinee, based on the photographed images of the examinee in the last-minute information and the photographed images of the official certificate or the examinee in the prior information. Specifically, the control unit 11 extracts facial features of the subject examinee in the photographed images of the examinee in the last-minute information and the photographed images of the official certificate or the examinee in the prior information, calculates the similarity between the extracted features, and certifies that the examinee is a previously authenticated examinee if the similarity is equal to or greater than a predetermined value. The similarity can be calculated using, for example, a correlation coefficient or cosine similarity, or the like. Alternatively, the similarity between two captured images can be estimated using a learning model constructed by machine learning. For example, a learning model configured by CNN that is trained to output the similarity between the two captured images or a determination result as to whether or not the image is the same as the person in question when two captured images are input can be used.

[0044] The control unit 11 stores the authentication result in the examinee DB 12a as the authentication result of the most recent information (S39). The control unit 11 then determines whether or not authentication was successful (S40), and if it determines that authentication was not successful (S40: NO), it sends an authentication failure screen to the examinee terminal 20 (S41), and the control unit 21 of the examinee terminal 20 obtains the authentication failure screen from the server 10 and displays it on the display unit 25 (S42). Figure 10A shows an example of the authentication failure screen, and the screen in Figure 10A notifies the examinee that examinee authentication has failed and has a "Retry authentication" button to instruct the examinee to try again, and a "Cancel" button to cancel the exam.

[0045] The control unit 21 of the examinee terminal 20 determines whether the "Retry authentication" button has been operated (S43), and if it determines that it has been operated (S43: YES), it returns to step S33. The control unit 21 then re-displays the last-minute authentication screen as shown in Figure 9 (S33) and re-executes the processes from step S34 onwards to authenticate the examinee just before taking the exam. If the control unit 21 determines that the "Retry authentication" button has not been operated (S43: NO), that is, if the "Cancel" button has been operated, it displays a screen notifying the examinee that the exam has been cancelled, and then terminates the process.

[0046] If the control unit 11 of the server 10 determines that the examinee authentication immediately prior to the test was successful (S40: YES), it sends a test start screen to the examinee terminal 20 (S44), and the control unit 21 of the examinee terminal 20 obtains the test start screen from the server 10 and displays it on the display unit 25 (S45). FIG. 10B shows an example of the test start screen, which notifies the examinee that the examinee authentication was successful and displays important points to note during the test. During the test, the examinee's face (the area above the chest) will be photographed, so the examinee should position the camera 26 of the examinee terminal 20 in a position where it can capture their own face. The control unit 21 may also activate the camera 26 and display the image captured by the camera 26 on the test start screen of FIG. 10B, allowing the examinee to confirm that the camera 26 is positioned appropriately by looking at the image displayed on the test start screen.

[0047] The control unit 21 of the examinee terminal 20 determines whether the "Start Test" button has been pressed (S46). If it determines that the button has not been pressed (S46: NO), it waits. If the control unit 21 determines that the "Start Test" button has been pressed (S46: YES), it starts displaying the test screen (S47) and starts accepting answer input via the test screen (S48). FIG. 10C shows an example test screen. The screen in FIG. 10C displays multiple questions and answer options for each question, with a check box for each option. In addition to multiple-choice questions like those in FIG. 10C, the test screen may also include questions that require the examinee to enter text or numbers. The screen in FIG. 10C also displays images captured by the camera 26. Information about the test screen may be transmitted (downloaded) from the server 10 to the examinee terminal 20 along with the test start screen. In this case, the control unit 21 stores the test screen information acquired from the server 10 in the memory unit 22 and sequentially displays the information on the display unit 25 according to the examinee's answer input. In addition, the information on the test screen may be sent from the server 10 to the examinee terminal 20 in accordance with the examinee's input of answers. In this case, the control unit 21 obtains the information for the next page from the server 10 and displays it on the display unit 25, for example, each time the examinee inputs answers for one page.

[0048] When the online test starts (when the test screen starts to be displayed), the control unit 21 starts counting down the remaining test time using a counter (S49). The control unit 21 also starts capturing video of the test taker using the camera 26 and acquiring room audio using the microphone 27 (S50). The control unit 21 also starts acquiring the position of the cursor operated by the pointing device 24a on the test screen (S51). The video of the test taker captured here may be video data of, for example, one to several frames per second, and room audio may be acquired at a predetermined sampling period (for example, the same sampling period as the captured video). The cursor position may also be acquired at a predetermined sampling period. The control unit 21 stores the captured video of the test taker (video data), room audio data, and cursor position data in the storage unit 22. Through the above-mentioned processing, the control unit 21 can acquire test-in-progress data (test-in-progress captured data) that captures the test taker's situation during the test. The control unit 21 may also acquire screen data including the captured video data of the test taker, room audio data, and cursor position data. For example, as shown in FIG. 10C, the control unit 21 may acquire screen data of a screen that displays a captured video including the image of the examinee and the room sound, and the cursor position on the test screen.

[0049] When the control unit 21 acquires the cursor position at a predetermined sampling period, it determines whether the cursor is within the test screen based on the acquired cursor position and the test screen being displayed on the display unit 25 (S52). If the control unit 21 determines that the cursor is not within the test screen (S52: NO), that is, if the cursor has moved outside the test screen, it measures the time that the cursor continues to be outside the test screen (S53). In other words, the control unit 21 measures the time that the cursor is outside the test screen. If the control unit 21 determines that the cursor is within the test screen (S52: YES), it skips step S53.

[0050] The control unit 21 determines whether to end the online exam (S54), and if it determines not to end it (S54: NO), it returns to step S52. The control unit 21 continues the process of determining whether the cursor is within the exam screen based on the cursor position that is sequentially acquired, and the process of measuring the time the cursor is outside the exam screen if it determines that the cursor is outside the exam screen. The control unit 21 determines to end the online exam when the remaining exam time that is being counted down reaches 0 or when the "End exam" button on the exam screen is operated. The control unit 21 may also determine that cheating by the examinee has occurred and end the online exam if the time the cursor has been outside the exam screen exceeds a predetermined time.

[0051] If the control unit 21 determines that the test should be ended (S54: YES), it displays a screen notifying the user that the test has ended, terminates the test, and transmits the answer data entered via the test screen, along with the examinee's video data, room audio data, and cursor position data (during-test data) acquired during the test to the server 10 (S55). The during-test data may include the time measured when the cursor moves off the test screen. The control unit 21 transmits the answer data and during-test data along with the test type, the examinee's name, the test date, etc. The control unit 11 of the server 10 acquires the answer data and during-test data from the examinee terminal 20 and stores them in the examinee DB 12a as during-test information or test information (S56). Specifically, the control unit 11 stores the examinee's video data, room audio data, and cursor position data in the during-test information sequence corresponding to the examinee ID in the examinee DB 12a. The control unit 11 also stores the test date and answer data in the test information sequence corresponding to the examinee ID.

[0052] The above-described process captures the examinee's face again immediately before taking the online exam and authenticates the examinee based on the captured image, thereby confirming that the examinee about to take the exam is the previously authenticated examinee. Therefore, impersonation by another person can be detected at the start of the exam. In addition to authenticating the examinee based on the examinee's captured image obtained from the examinee terminal 20, the server 10 may also perform a process to determine whether the exam environment complies with the rules based on captured video of the exam environment. For example, the control unit 11 may detect objects on the desk based on video of the desk, determine whether the detected objects are intended for use during the exam, and notify the examinee that the exam environment is not suitable for use during the exam if they are not intended for use during the exam. The control unit 11 may also detect the presence of objects that could lead to cheating or other misconduct based on video of the desk and the room, and if an object that could lead to cheating is detected, notify the examinee that the exam environment is not suitable. Upon receiving such a notification, the examinee may repeatedly capture images of the exam environment and have the server 10 perform the assessment process until the server 10 determines that the exam environment is appropriate. If the online exam is available after it has been determined that the exam environment is appropriate, you may take the online exam in the appropriate exam environment.

[0053] Furthermore, by acquiring photographic images of the examinee, audio data in the room, and cursor position data while the examinee is taking the online test, the acquired data can be used to determine whether the examinee has engaged in dishonest behavior during the test. This makes it possible to detect the occurrence of dishonest behavior, such as cheating, by the examinee. The control unit 11 of the server 10 scores the answer data acquired from the examinee terminal 20, determines whether the examinee has passed or failed, and stores the pass / fail result in the examinee DB 12a.

[0054] The following describes the process by which the server 10 determines whether or not an examinee has engaged in fraudulent behavior based on the in-test information of the examinee who has completed the test. Fig. 11 is a flowchart showing an example of the fraudulent behavior determination process, and Fig. 12 is an explanatory diagram showing an example screen on the operator terminal 40. In Fig. 11, the process performed by the server 10 is shown on the left, and the process performed by the operator terminal 40 is shown on the right.

[0055] The control unit 11 of the server 10 performs a process to determine whether or not an examinee who has completed an online exam and whose in-exam data (the examinee's video data, in-room audio data, and cursor position data) is stored in the examinee DB 12a has engaged in misconduct, such as cheating, during the exam. The control unit 11 reads the in-exam data of one examinee from the examinee DB 12a (S61). Based on the read in-exam data, the control unit 11 determines whether or not the examinee engaged in misconduct (S62). Here, the control unit 11 inputs the examinee's video data, in-room audio data, and cursor position data contained in the in-exam data into the learning model 12M and obtains the presence and type of misconduct as output values ​​from the learning model 12M. Note that when the control unit 11 obtains screen data including the examinee's video data, in-room audio data, and cursor position data, it may input the screen data into the learning model 12Ma and obtain the presence and type of misconduct as output values ​​from the learning model 12Ma.

[0056] The process for determining whether or not cheating has occurred is not limited to processing using the learning model 12M. For example, rule-based processing may be used to determine whether or not cheating has occurred due to cursor position based on cursor position data. For example, cheating due to cursor position may be determined if the cursor has been off the test screen for a predetermined period of time or more. Cheating due to cursor position may also be determined if the cursor has been off the test screen a predetermined number of times or more. The examinee's gaze direction may be tracked based on a video of the examinee, and the presence or absence of cheating may be determined based on the trajectory of the gaze direction. Furthermore, the presence or absence of speech other than that of the examinee may be detected based on audio data in the room, and cheating may be determined based on the presence or absence of speech other than that of the examinee.

[0057] The control unit 11 stores the result of the determination of whether or not there has been misconduct in the examinee DB 12a in association with the examinee ID of the examinee (S63). The control unit 11 determines whether or not the determination process has been completed for all examinees to be determined (S64). If it determines that the process has not been completed (S64: NO), the control unit 11 returns to step S61 and performs the processes of steps S61 to S63 for the unprocessed examinees. As a result, the presence or absence of misconduct is determined for each examinee to be determined based on the in-test data, and the determination results are stored in the examinee DB 12a. It is also possible to determine whether or not there has been misconduct for only those examinees who passed. In this case, the control unit 11 reads the in-test data of the examinees who passed in step S61, performs the processes of steps S62 to S63, and performs the process of step S64 for the passed examinees as the subjects of determination. If the control unit 11 determines that the above-mentioned process has been completed for all examinees to be determined (S64: YES), it generates a list of examinees who have been determined to have engaged in misconduct (a misconduct list) (S65).

[0058] FIG. 12 shows an example of a cheating list. For both successful and unsuccessful test takers, the list in FIG. 12 displays information about test takers who were determined to have cheated, including the test date, test taker ID, details of the cheating (the cheating), and confidence level. The confidence level can be, for example, the confidence level output from the learning model 12M. The list in FIG. 12 also displays thumbnail images of the test taker's video included in the in-test data, and includes a play button for instructing playback of the in-test data. The thumbnail image may be, for example, the first image from the test taker's video during the time period in which cheating was determined to have occurred, or an image taken a predetermined time before that time period. The control unit 11 determines whether cheating occurred in step S62 and identifies the time period during which cheating was determined to have occurred, and can generate thumbnail images based on images from the identified time period. The play button contains links to the test taker's video, in-room audio data, and cursor position data included in the in-test data. Operating the play button allows the test taker to view the test taker's video, in-room audio, and cursor position. The test data may be configured to allow playback of all video data, indoor audio data, and cursor position data collected during the test, or may be configured to allow playback of video data, indoor audio data, and cursor position data from a time period in which cheating was determined to have occurred. Furthermore, the list in FIG. 12 includes a "Cheating" button and a "No Cheating" button for the operator to input the results of their confirmation of whether or not cheating occurred. In this embodiment, as shown in FIG. 12, the operator is notified of information about not only successful but also unsuccessful test takers suspected of cheating. However, the list may be configured to only notify successful test takers of information about suspected cheating. In this case, in step S65, the control unit 11 generates a list of successful test takers who were determined to have cheated based on the test results stored in the test taker DB 12a, and transmits the list to the operator terminal 40. Whether the information to be sent to the operator terminal 40 is limited to successful test takers who were determined to have cheated or all test takers who were determined to have cheated may be configurable in the server 10.

[0059] The control unit 11 sends the generated misconduct list (detection results) to the operator terminal 40 (S66), and the control unit of the operator terminal 40 obtains the misconduct list from the server 10 and displays it on the display unit (S67). The operator checks the test data of each examinee in the misconduct list to determine whether or not misconduct has occurred, and inputs the confirmation result by operating the "Misconduct Present" button or the "No Misconduct" button. The control unit of the operator terminal 40 accepts the input of the confirmation result of whether or not misconduct has occurred through the operator's operation via the input unit (S68), and after accepting the confirmation result for each examinee, transmits the confirmation result of whether or not misconduct has occurred for each examinee to the server 10 (S69). When the control unit 11 of the server 10 obtains the confirmation result of whether or not misconduct has occurred for each examinee from the operator terminal 40, it stores the operator's confirmation result in the examinee DB 12a in association with each examinee's examinee ID (S70). This allows the operator to confirm the suitability of the determination result of whether or not there is fraudulent activity determined by the server 10 using the learning model 12M, and associate the confirmation result determined by the operator with the determination result.

[0060] The operator terminal 40 may transmit to the server 10 the operator's confirmation results only for examinees for whom the server 10's determination of whether or not they have engaged in misconduct differs from the operator's confirmation results. Since the server 10 notifies the operator terminal 40 of information about examinees who have been determined to have engaged in misconduct, the operator terminal 40 may transmit confirmation results only for examinees for whom the operator has determined that they have not engaged in misconduct. In this case, the control unit 11 of the server 10 obtains from the operator terminal 40 the examinee IDs of examinees who the server 10 has notified of as having engaged in misconduct and who the operator has determined have not engaged in misconduct, as well as the confirmation results that no misconduct has been engaged in for those examinees. The server 10 then stores "no misconduct" as the operator confirmation result corresponding to the obtained examinee ID.

[0061] Through the above-described processing, in this embodiment, it is possible to determine whether or not an examinee has engaged in misconduct using each piece of data acquired during the online exam. Furthermore, a list of examinees suspected of misconduct can be presented to the operator terminal 40, thereby notifying the online exam management company. At the online exam management company, an operator can review the exam data of each examinee, confirm whether or not they have engaged in misconduct, and register the results of the review in the server 10 (examinee DB 12a). The online exam management company can take subsequent action based on the operator's review results. For example, the pass of an examinee who is confirmed to have engaged in misconduct can be revoked, and the pass of an examinee who is confirmed not to have engaged in misconduct can be confirmed.

[0062] In this embodiment, examinee identity is confirmed in advance by authenticating the examinee using an official certificate, and then confirmed again by authenticating the examinee using an image taken just before the start of the exam, thereby preventing impersonation by others. Furthermore, cheating can be detected using data collected during the exam, preventing cheating from being used to pass the exam.

[0063] In this embodiment, the learning model 12M is used to determine whether or not there is misconduct due to gaze direction, indoor audio, or cursor position based on the video of the examinee, in-room audio data, and cursor position data acquired during the online exam. Alternatively, for example, the server 10 may perform gaze direction tracking processing based on the video of the examinee and determine whether or not there is misconduct due to gaze direction based on the gaze movement. Furthermore, by recording the examinee's speech in advance or before the start of the online exam, the presence or absence of misconduct due to indoor audio may be determined based on whether or not the in-room audio data acquired during the online exam contains audio other than that of the examinee. Various methods and systems may be used to determine whether or not there is misconduct due to gaze direction, indoor audio, or cursor position.

[0064] (Embodiment 2) An online examination system that determines whether or not an examinee has engaged in fraudulent conduct during an online examination will be described. The online examination system of this embodiment can be realized using the same device as the online examination system of embodiment 1, so a description of the configuration will be omitted.

[0065] Figure 13 is a flowchart showing an example of the processing procedure for an online examination in embodiment 2. In Figure 13, the left side shows the processing performed by the examinee terminal 20, the center shows the processing performed by the server 10, and the right side shows the processing performed by the operator terminal 40. The processing shown in Figure 13 is the processing shown in Figures 7 and 8 with steps S81 to S94 added instead of steps S52 to S56. Explanations of the same steps as in Figures 7 and 8 will be omitted. Furthermore, steps S31 to S46 in Figures 7 and 8 are not shown in Figure 13.

[0066] In this embodiment, after processing step S51, the control unit 21 of the examinee terminal 20 transmits to the server 10 the video data of the examinee, the audio data in the room, and the cursor position data (during-test data) acquired after the start of the online test (S81). The control unit 21 transmits the during-test data along with the test type, the examinee's name, etc. The control unit 21 is configured to transmit the during-test data acquired sequentially during the examinee's test to the server 10 at predetermined time intervals. The control unit 21 determines whether a predetermined time has elapsed since transmitting the during-test data to the server 10 (S82). If it determines that the predetermined time has elapsed (S82: YES), the control unit 11 returns to step S81 and transmits the during-test data accumulated since the previous transmission process to the server 10 (S81). If it determines that the predetermined time has not elapsed (S82: NO), the control unit 11 determines whether to end the online test (S83). If it determines that the online test should not be ended (S83: NO), the control unit 11 returns to step S82. Therefore, the control unit 11 repeats the process of transmitting the test data accumulated since the previous transmission process to the server 10 every time a predetermined time has elapsed.

[0067] The control unit 11 of the server 10 acquires the test-in-progress data from the examinee terminal 20 and stores it in the examinee DB 12a as test-in-progress information (S84). Specifically, the control unit 11 stores the test-in-progress video of the examinee, the audio data in the room, and the cursor position data in the test-in-progress information sequence corresponding to the examinee ID in the examinee DB 12a. The control unit 11 determines whether or not the examinee has engaged in cheating based on the test-in-progress data stored in the examinee DB 12a (S85). The process of step S85 can be the same as step S62 in FIG. 11. The control unit 11 stores the determination result of whether or not cheating has occurred in the examinee DB 12a in association with the examinee ID of the examinee (S86). The control unit 11 may store the determination result (that cheating has occurred) only if it determines that cheating has occurred, or may store the determination result in association with the date and time when the determination process was performed.

[0068] The control unit 11 determines whether or not cheating has been determined in the determination process of step S85 (S87). If it determines that cheating has been determined (S87: YES), it transmits the examinee's examinee ID, the details of the cheating (the details of the cheating) and the certainty level, the examinee's test-in-progress data, etc. to the operator terminal 40, and notifies the operator of the occurrence of cheating (S88). When the control unit of the operator terminal 40 acquires information about an examinee who has been determined to have engaged in cheating from the server 10, it adds the acquired information about the examinee (information about the cheater) to a cheating list, such as that shown in FIG. 12, and displays it on the display unit (S89). This allows the information about the examinee suspected of cheating to be notified to the operator via the operator terminal 40. If the operator terminal 40 is equipped with a warning lamp or a speaker that outputs a warning sound, the control unit may notify the operator by lighting or flashing the lamp, or by outputting a warning message from the speaker, when notified of the occurrence of cheating by the server 10.

[0069] The operator checks the notified examinee's in-test data, determines whether or not there is any misconduct, and inputs the confirmation result. The control unit of the operator terminal 40 accepts the input of the confirmation result regarding whether or not there is any misconduct (S90) and transmits the accepted confirmation result to the server 10 (S91). The control unit 11 of the server 10 acquires the confirmation result from the operator terminal 40 for the examinee notified in step S88, and stores the operator's confirmation result in the examinee DB 12a in association with the examinee ID of the examinee (S92). This allows the operator's confirmation result to be associated with the determination result regarding whether or not there is any misconduct determined using the learning model 12M. If the control unit 11 of the server 10 determines that there is no misconduct (S87: NO), it skips steps S88 and S92. If the control unit 11 acquires the confirmation result indicating the presence or absence of misconduct from the operator terminal 40, it may be configured to end (cancel) the examinee's exam at this point, or to send a warning message to the examinee terminal 20 of the examinee.

[0070] If the control unit 21 of the examinee terminal 20 determines that the test should be ended (S83: YES), it ends the test and transmits the answer data entered via the test screen to the server 10 (S93). The control unit 21 transmits the answer data together with the test type, the examinee's name, the test date, etc. The control unit 11 of the server 10 acquires the answer data from the examinee terminal 20 and stores it in the examinee DB 12a as test information (S94). Here, the control unit 11 stores the test date and answer data in the test information string corresponding to the examinee ID in the examinee DB 12a. The control unit 11 may then score the answer data of each examinee to determine whether they pass or fail, and store the pass / fail results in the examinee DB 12a.

[0071] In this embodiment, the above-described process automatically determines whether or not a test taker is engaging in misconduct while taking an online test, allowing an operator to identify test takers who should be checked for misconduct in real time. If cheating or other misconduct is suspected, the operator is notified, so the operator only needs to check the test data of the test taker suspected of misconduct. Furthermore, if the operator confirms that a test taker has engaged in misconduct, the operator notifies the server 10, allowing the online test operator to take prompt action against the test taker who engaged in misconduct.

[0072] The embodiments disclosed herein are to be considered in all respects as illustrative and not restrictive. The scope of the present disclosure is defined by the claims, not by the above meaning, and is intended to include all modifications within the meaning and scope of the claims.

[0073] The matters described in each embodiment can be combined with each other. In addition, the independent claims and dependent claims described in the claims can be combined with each other in any combination, regardless of the reference format. Furthermore, although the claims use a format in which a claim references two or more other claims (multiple claim format), this is not limited to this format. A multiple claim (multi-multi claim) that references at least one other multiple claim may also be used. [Explanation of symbols]

[0074] 10 Servers 11 Control section 12 Storage section 13 Communications Department 20 Examinee's terminal 21 Control section 22 Memory section 23 Communications Department 24 Input section 24a Pointing Device 25 Display section 26 Camera 27. Mike 30 Authentication Server 40 Operator Terminal

Claims

1. Before the online test date, first photographed data of an official photo ID of the examinee and second photographed data of the examinee are obtained; permitting the examinee to take the online test if the examinee's identity is authenticated based on the first photographed data and the second photographed data; During the online test, photographed data of the test taker taken by a camera attached to the test taker's terminal device, audio data of the room in which the test taker is taking the online test, and cursor position data of a pointing device operated by the test taker are acquired; a learning model that has been trained to output information indicating, when photographed data, audio data, and cursor position data are input, whether there is no misconduct by the examinee in the photographed data that was input, whether misconduct is suspected due to the examinee's line of sight, whether misconduct is suspected due to the position indicated by the pointing device operated by the examinee, or whether misconduct is suspected due to the sound in the room; and by inputting the acquired photographed data during the test, audio data, and cursor position data into the learning model, information indicating whether there is no misconduct by the examinee in the photographed data during the test, whether misconduct is suspected due to the examinee's line of sight, whether misconduct is suspected due to the position indicated by the pointing device operated by the examinee, or whether misconduct is suspected due to the sound in the room is obtained; Based on the acquired information, the presence or absence and type of cheating by the examinee is detected. An information processing method in which processing is performed by a computer.

2. Acquire third photographic data of the examinee who is permitted to take the online exam on the day of the exam, authenticating the examinee based on the third photographed data and the first photographed data or the second photographed data; The authentication result is stored in the storage unit in association with the identification information that identifies the examinee. The information processing method according to claim 1 , wherein the processing is executed by the computer.

3. Before the start of the online exam on the day of the exam, fourth photographic data is obtained by photographing the area above and below the examinee's desk and the interior of the room.

3. The information processing method according to claim 1, wherein the processing is executed by the computer.

4. The detection results are output to the operator's terminal device.

3. The information processing method according to claim 1, wherein the processing is executed by the computer.

5. If fraudulent activity by the examinee is detected, information regarding the fraudulent activity, the test-taken image data in which the fraudulent activity was detected, and identification information for identifying the examinee are associated with each other and output to the operator's terminal device.

5. The information processing method according to claim 4, wherein the processing is executed by the computer.

6. The test-taking photograph data of each of the examinees and the information regarding the fraudulent acts are output to the terminal device of the operator in association with identification information that identifies each of the examinees.

5. The information processing method according to claim 4, wherein the processing is executed by the computer.

7. Before the online test date, first photographed data of an official photo ID of the examinee and second photographed data of the examinee are obtained; permitting the examinee to take the online test if the examinee's identity is authenticated based on the first photographed data and the second photographed data; During the online test, photographed data of the test taker taken by a camera attached to the test taker's terminal device, audio data of the room in which the test taker is taking the online test, and cursor position data of a pointing device operated by the test taker are acquired; a learning model that has been trained to output information indicating, when photographed data, audio data, and cursor position data are input, whether there is no misconduct by the examinee in the photographed data that was input, whether misconduct is suspected due to the examinee's line of sight, whether misconduct is suspected due to the position indicated by the pointing device operated by the examinee, or whether misconduct is suspected due to the sound in the room; and by inputting the acquired photographed data during the test, audio data, and cursor position data into the learning model, information indicating whether there is no misconduct by the examinee in the photographed data during the test, whether misconduct is suspected due to the examinee's line of sight, whether misconduct is suspected due to the position indicated by the pointing device operated by the examinee, or whether misconduct is suspected due to the sound in the room is obtained; Based on the acquired information, the presence or absence and type of cheating by the examinee is detected. A program that causes a computer to perform a process.

8. In an information processing device having a control unit, The control unit Before the online test date, first photographed data of an official photo ID of the examinee and second photographed data of the examinee are obtained; permitting the examinee to take the online test if the examinee's identity is authenticated based on the first photographed data and the second photographed data; During the online test, photographed data of the test taker taken by a camera attached to the test taker's terminal device, audio data of the room in which the test taker is taking the online test, and cursor position data of a pointing device operated by the test taker are acquired; a learning model that has been trained to output information indicating, when photographed data, audio data, and cursor position data are input, whether there is no misconduct by the examinee in the photographed data that was input, whether misconduct is suspected due to the examinee's line of sight, whether misconduct is suspected due to the position indicated by the pointing device operated by the examinee, or whether misconduct is suspected due to the sound in the room; and by inputting the acquired photographed data during the test, audio data, and cursor position data into the learning model, information indicating whether there is no misconduct by the examinee in the photographed data during the test, whether misconduct is suspected due to the examinee's line of sight, whether misconduct is suspected due to the position indicated by the pointing device operated by the examinee, or whether misconduct is suspected due to the sound in the room is obtained; Based on the acquired information, the presence or absence and type of cheating by the examinee is detected. Information processing device.

Images