Programs, information processing methods, information processing devices

Abstract

In an exam where test-takers answer by typing, efficiently determining whether or not each test-taker is cheating during the exam. [Solution] A program that causes an information processing device to perform the following actions in an examination in which each user answers by inputting a key into each processing device: acquiring first log information relating to the log of key input by each user to each processing device; extracting predetermined actions of each user during the examination based on the first log information; identifying a score related to each user's cheating in the examination based on the predetermined actions; and outputting information based on the score.

Description

Technical Field

[0001] The present invention relates to a program, an information processing method, and an information processing apparatus.

Background Art

[0002] Conventionally, a technique for evaluating the skills of examinees online is known. For example, Patent Document 1 discloses that a test for measuring the programming skills of engineers is carried out online.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, according to the prior art, in an online test where examinees input keys to answer questions, when determining whether an examinee is cheating, the evaluator has to visually check the answers of each examinee and the process leading to the answers. Therefore, in the conventional method for confirming cheating, the burden on the evaluator was large.

[0005] Therefore, the disclosed technology aims to provide a new mechanism that enables efficient grasping of whether each examinee is cheating during an online test where examinees input keys to answer questions.

Means for Solving the Problems

[0006] A program, which is one aspect of the disclosed technology, causes an information processing device to perform the following actions in an examination in which each user answers by typing on each processing device: to acquire first log information relating to the log of each user's key input to each processing device; to extract predetermined actions of each user during the examination based on the first log information; to identify a score relating to each user's cheating in the examination based on the predetermined actions; and to output information based on the score. [Effects of the Invention]

[0007] According to the present invention, in an online examination in which examinees answer by typing, evaluators can efficiently determine whether or not each examinee is cheating during the examination. [Brief explanation of the drawing]

[0008] [Figure 1] Diagram showing an example of the configuration of an information processing system relating to one embodiment of the present invention. [Figure 2] A diagram showing an example of a test-taker's answer screen. [Figure 3] Block diagram showing an example of a server relating to one embodiment of the present invention. [Figure 4] A diagram showing an example of a test results confirmation screen. [Figure 5] A diagram showing an example of a test results confirmation screen. [Figure 6] Sequence diagram showing an example of processing by the information processing device and server according to this embodiment. [Modes for carrying out the invention]

[0009] Embodiments of the present invention will be described below with reference to the attached drawings. The following embodiments are illustrative for explaining the present invention and are not intended to limit the present invention to these embodiments only. Furthermore, the present invention can be modified in various ways without departing from its essence. In addition, the same reference numerals are used for the same components in each drawing whenever possible, and redundant explanations are omitted whenever possible.

[0010] <System Overview> Figure 1 shows an example of the configuration of an information processing system 1 according to one embodiment of the present invention. In the example shown in Figure 1, the information processing system 1 is exemplary configured to include an information processing device 10, an information processing device 20, a server 30, and a network N. The number of information processing devices 10, 20, and 30 may be one or more.

[0011] Information processing system 1 is used, for example, by a user who detects cheating in a skills measurement test (hereinafter also referred to as the "operator"), a user who takes the test (hereinafter also referred to as the "test taker"), and a user who administers the test and / or evaluates the test results (hereinafter also referred to as the "evaluator" or "company"). The operator and the evaluator may be the same user. The test questions may also be created by the operator or the evaluator. When the user who administers the test and / or evaluates the test results is referred to as the "company," the "company" may include at least one company that wants to recruit personnel and a company that wants to measure the skills of its employees, etc.

[0012] Information processing system 1 generates, for example, an examination containing predetermined questions, which the examinee answers by typing, upon request from an operator or evaluator. For example, information processing system 1 generates a coding examination in a predetermined programming language upon request from an operator or evaluator. The predetermined programming language includes at least one of the following: JavaScript®, Python, C language (a scripting language), Go language, Swift, Kotlin®, and other well-known programming languages. The examination for measuring the examinee's skills is not limited to coding tests; it may also include questions testing knowledge of coding, or questions testing knowledge of system design, development, maintenance, requirements definition, etc. Furthermore, the examination for measuring the examinee's skills may include tests in which the examinee writes text by typing. For example, the examination may be a test in which the examinee writes text in the language of a predetermined country.

[0013] Information processing system 1 provides the test to the test taker or evaluator. For example, information processing system 1 provides an environment for the test taker to take the test online and records the contents of key input during the test as a log. Information processing system 1 outputs information to the evaluator or company regarding cheating in each test taker's test based on the recorded log. Information processing system 1 may also reproduce the answers entered by the test taker based on the recorded log for the evaluator or company. This allows the evaluator or company to view or evaluate the content of the test taker's answers. Information processing system 1 may also reproduce the usage history of at least one of the test taker's search tools or generating AI for the evaluator or company based on the recorded log. This allows the evaluator or company to view or evaluate the test taker's use of at least one of the search tools or generating AI. Note that the search tool or generating AI may be an external tool that has functions to support the test regarding the measurement of the test taker's skills.

[0014] The information processing device 10 is an information processing device used by examinees when taking a skills measurement test. The information processing device 10 is, for example, a personal computer, a smartphone, or a tablet device. The information processing device 10 acquires the test questions via the network N and presents them to the examinee. The information processing device 10 transmits the answers that the examinee has given to the test questions to the server 30. The information processing device 10 may also transmit to the server 30 the content that the examinee has entered into at least one of the search tool or the generation AI. The information processing device 10 may also transmit to the server 30 the content that at least one of the search tool or the generation AI has output.

[0015] Candidates may use at least one of the search tool or the generative AI displayed on the answer screen. Candidates may answer questions based, for example, on the output of at least one of the search tool or the generative AI displayed on the answer screen. The search tool is, but is not limited to, the search engine Google®. The generative AI is, but is not limited to, the ChatGPT provided by OpenAI OpCo, LLC. Candidates using at least one of the search tool or the generative AI during the exam is achieved, for example, by the communication between the web browser or application providing the exam and the software providing at least one of the search tool or the generative AI via an API (Application Programming Interface). By communicating via API, the software can communicate information with each other, for example, the search tool or the generative AI can return a response in response to a request from the software providing the exam.

[0016] Figure 2 shows an example of a test-taker's answer screen. The test-taker answers the skills test using an answer screen A10 displayed on an information processing device 10, such as a personal computer, smartphone, or tablet device.

[0017] In the answer screen A10, screen area B10 displays the exam questions. In the example shown in Figure 2, screen area B10 displays the question, "Create code to sort 7, 11, 4, 2, 9, 5, 14 in ascending order using bubble sort." If multiple exam questions are presented, the question displayed in screen area B10 may switch to the next question, for example, once the examinee has finished answering the current question. Alternatively, the question displayed in screen area B10 may switch to the next question, for example, after a predetermined amount of time has elapsed since the current question was displayed.

[0018] On the screen area B20 in the answer screen A10, the answers input by the examinee are displayed. The examinee may input the answers, for example, by using an input device such as a keyboard provided in the information processing device 10. The examinee may also input the answers, for example, by pressing the keys displayed on the screen provided in the information processing device 10 with a finger.

[0019] On the screen area B30 in the answer screen A10, a button (hereinafter also referred to as the "search button") for enabling the examinee to use the search tool is displayed. By selecting the search button during the test, the examinee can use the search tool. The examinee may select the search button, for example, by moving the cursor to the screen area B30 and clicking the mouse. Also, the examinee may select the search button, for example, by pressing the screen area B30 with a finger. If the operator, evaluator or company does not permit the examinee to use the search tool during the test, the search button is not active and may not be displayed on the answer screen A10. Also, the operator, evaluator or company can preset whether the examinee can use the search tool for one or more questions before the start of the test.

[0020] On the screen area B40 in the answer screen A10, a button (hereinafter also referred to as the "AI button") for enabling the examinee to use the generative AI is displayed. By selecting the AI button during the test, the examinee can use the generative AI. The examinee may select the AI button, for example, by moving the cursor to the screen area B40 and clicking the mouse. Also, the examinee may select the AI button, for example, by pressing the screen area B40 with a finger. If the operator, evaluator or company does not permit the examinee to use the generative AI during the test, the AI button is not active and may not be displayed on the answer screen A10. Also, the operator, evaluator or company can preset whether the examinee can use the generative AI for one or more questions before the start of the test.

[0021] When a test-taker selects the AI ​​button, the request input field B50 (hereinafter also referred to as "input field B50") on answer screen A10 becomes available. That is, the test-taker can enter a request (e.g., a prompt) for the generated AI in input field B50. The rightmost area of ​​answer screen A10, which includes input field B50, may be displayed when either the search button B30 or the AI ​​button B40 is selected.

[0022] When a request is entered in input field B50, the response (e.g., answer) output by the generating AI is displayed in response output field B60 (hereinafter also referred to as "output field B60"). This allows test takers to answer the questions based on the generating AI's response displayed in output field B60.

[0023] When a test-taker selects the search button, the request input field (not shown in the diagram; hereinafter also referred to as the "input field") on answer screen A10 becomes available. In other words, the test-taker can enter a request for the search tool (e.g., a search query) into the input field.

[0024] When a request is entered into the input field, the response (e.g., search results) output by the search tool is displayed in the response output field (not shown in the diagram; hereinafter also referred to as the "output field"). This allows test-takers to answer the questions based on the search tool's response displayed in the output field.

[0025] Returning to Figure 1, the information processing device 10 may have an application program (app) installed for using the examinee skills measurement test service (hereinafter also referred to as "the Service") provided by the server 30. This app causes the information processing device 10 to execute at least a part of the processing related to the examinee skills measurement test disclosed in the embodiments shown below in the Service provided by the server 30. When this app is executed, the information processing device 10 may access the server 30 to send and receive information used to execute the app.

[0026] The information processing device 20 is an information processing device used by evaluators or companies. The information processing device 20 is, for example, a personal computer, a smartphone, or a tablet terminal. The information processing device 20 receives requests regarding examination questions from evaluators or companies based on user input and sends these requests to the server 30.

[0027] The information processing device 20 obtains the test question corresponding to the request or a list of selectable test questions from the server 30 via the network N. The information processing device 20 may also transmit the finally determined test question to the information processing device 10 via the network N. The information processing device 20 may also obtain the answers that the examinee has given to the test questions from the information processing device 10 or the server 30 via the network N.

[0028] The information processing device 20 may, for each exam question, set whether the examinee can use external tools, including a search tool or a generation AI, based on user interaction. For example, the information processing device 20 sends information on whether external tools can be used to the server 30. The server 30 sets whether external tools can be used for the exam question based on the information on whether external tools can be used. If the external tools are available, the server 30 may configure the screen of the exam question to display a button for API integration with the external tools mentioned above.

[0029] Server 30 is an information processing device used by operators, etc., and may be a general-purpose computer or the like. Server 30 provides the services disclosed in this embodiment. Server 30 generates test questions related to measuring the skills of test takers. Server 30 transmits the generated test questions to at least one of the information processing device 10 or information processing device 20 via the network N.

[0030] The server 30 obtains, via the network N, the content of the examinee's answers to the examination questions, the requests the examinee inputs to at least one of the search tool or the generation AI, and the responses output by at least one of the search tool or the generation AI from the information processing device 10.

[0031] Server 30 transmits, for example, the content of the examinee's answers to the examination questions or information on predetermined processing performed on said content to the information processing device 20 via the network N. Server 30 may also transmit, for example, log information regarding key inputs to the information processing device 10 for answering to the information processing device 20.

[0032] The server 30 may, for example, transmit to the information processing device 20, via the network N, a request entered by the examinee into at least one of the search tool or the generating AI, or information on which predetermined processing has been performed on said request.

[0033] The server 30 may, for example, transmit to the information processing device 20 via the network N the response output by at least one of the search tool or the generating AI, or information on which predetermined processing has been performed on the response.

[0034] Server 30 may evaluate the answers, for example, by scoring or correcting them, and then transmit the evaluated answers to at least one of the information processing devices 10 or 20. The detailed processing by Server 30 will be described later.

[0035] Network N can be implemented by networks such as the Internet, mobile phone networks, LANs (Local Area Networks), or a combination of these.

[0036] <Server Configuration> Figure 3 is a block diagram showing an example of a server 30 according to one embodiment of the disclosure. The server 30 includes one or more processors (e.g., CPUs) 310, one or more network communication interfaces 320, a storage device (storage unit) 330, and one or more communication buses 350 for interconnecting these components.

[0037] The server 30 may optionally include a user interface 340. The user interface 340 includes a display and / or input devices (such as a keyboard and / or mouse, or any other pointing device).

[0038] The storage device 330 is, for example, a high-speed random-access memory (main memory) such as DRAM, SRAM, or other random-access solid-state memory. Alternatively, the storage device 330 may be one or more non-volatile memories (auxiliary storage) such as magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. The storage device 330 may also be a non-temporary recording medium that is readable by a computer and stores programs, etc. Furthermore, the storage device 330 may be either main memory (memory) or auxiliary storage (storage), or it may include both.

[0039] The storage device 330 is a storage unit that stores data, programs, etc., used by the information processing system 1. The storage device 330 stores information that identifies the examinee (hereinafter also referred to as "examinee ID") in association with each log information (for example, the first log information, second log information, and third log information described later). The storage device 330 may also store the questions asked to the examinee, the correct answers to the questions, evaluation criteria, and examination results.

[0040] Another example of the storage device 330 may be one or more storage devices located remotely from the processor 310. In one embodiment, the storage device 330 stores programs, modules, and data structures, or subsets thereof, executed by the processor 310.

[0041] The processor 310 controls, for example, the processing performed by the information processing unit 312 by executing a program stored in the memory device 330.

[0042] The information processing unit 311 includes, for example, an acquisition unit 312, an extraction unit 313, a specification unit 314, and an output unit 315.

[0043] The acquisition unit 312 acquires first log information regarding the log of key inputs made by each examinee to each processing unit in an examination in which each examinee answers by making key inputs to each processing unit. The examinee's operation log can be acquired using known technology. For example, when answers are made using the web browser of the information processing unit 10, the acquisition unit 312 acquires key inputs during the answer as log information. The first log information is recorded in the storage device 330. In this case, the time at which the key input was made may be associated with the input.

[0044] The acquisition unit 312 may acquire the first log information from the information processing device 10 via the network N. The acquisition unit 312 may also acquire the first log information by reading the first log information stored in the storage device 330.

[0045] The extraction unit 313 extracts predetermined actions of each examinee during the exam based on the first log information acquired by the acquisition unit 312. For example, the extraction unit 313 extracts actions related to cheating by each examinee during the exam based on the first log information. For example, cheating (unfair behavior) that generally results in a negative evaluation may be defined, and key inputs indicating this cheating behavior may be set as conditions. In this case, the extraction unit 313 extracts examinee actions that satisfy the cheating behavior by comparing each key input with the key inputs set as conditions related to cheating behavior.

[0046] The identification unit 314 identifies a score for each test-taker's cheating in the exam based on predetermined actions extracted by the extraction unit 313. For example, when a predetermined action is extracted by the extraction unit 313, the identification unit 314 identifies a score corresponding to that predetermined action. In this case, the identification unit 314 may identify a score stored in the storage device 330 in association with the predetermined action. The identification unit 314 may also calculate a score corresponding to the number of times the predetermined action has been extracted. The identification unit 314 may input information about the extracted predetermined action to a learning model trained using learning data that associates predetermined actions and cheating behaviors of a large number of test-takers during the exam, and obtain a score output from the learning model. The score may be a numerical value representing the probability of cheating, or a rank indicating the possibility of cheating. The method of identification is not particularly limited as long as the test-taker's score for cheating can be identified.

[0047] The output unit 315 outputs information based on the score identified by the identification unit 314. The output unit 315 controls the network communication interface 320 to output the score-based information to the information processing device 20 used by the evaluator or company. As score-based information, the output unit 315 outputs at least one of the following: information showing the score itself, and information indicating whether or not the examinee based on the score cheated during the test.

[0048] Through the above process, in exams where test-takers answer by typing, evaluators or companies can efficiently determine whether or not each test-taker cheated during the exam by referring to information based on their scores.

[0049] Figure 4 shows an example of a test result confirmation screen displayed on an information processing device 20 used by an evaluator or company. The information processing device 20 displays, for example, screen A20, which includes the test results of each examinee shown in screen areas B70 to B100. The test results may include, but are not limited to, "information for identifying the examinee," "test score," "score for cheating," and "a mark indicating whether or not the examinee cheated during the test."

[0050] "Information for identifying the examinee" includes, but is not limited to, at least one of the following: the examinee's name, the examinee's nickname, the examinee ID, the email address, and the telephone number. In screen area B70, the examinee's name, "Taro Tanaka," is displayed as "Information for identifying the examinee."

[0051] "Exam score" refers to the score a test-taker achieved in an exam. "Exam score" is displayed on a scale, for example, between 0 and 100.

[0052] The "fraud score" is the fraud score identified by the identification unit 314. In the example shown in Figure 4, the score is displayed as a numerical value indicating the probability of fraud.

[0053] The "mark indicating whether or not a test-taker cheated during the exam" is a mark corresponding to the "score for cheating" and is used to identify whether or not a test-taker cheated during the exam. For example, in screen area B70, an "×" is displayed to indicate that test-taker (Taro Tanaka) cheated during the exam. In screen area B80, a "○" is displayed to indicate that test-taker (Hanako Suzuki) did not cheat during the exam. In screen area B90, a "△" is displayed to indicate that it is doubtful whether or not test-taker (Jiro Yamaguchi) cheated during the exam. In screen area B100, a "-" is displayed to indicate that it has not been evaluated or is impossible to evaluate whether or not test-taker (Hiroko Takahashi) cheated during the exam. Note that it is sufficient for at least one of the "score for cheating" and the "mark indicating whether or not a test-taker cheated during the exam" to be displayed.

[0054] The screen areas B70-100 shown in Figure 4 may be buttons that can be selected by the evaluator or company. For example, if screen area B70, which is a selectable button, is pressed by the evaluator or company, the screen may transition to screen A30 shown in Figure 5.

[0055] Figure 5 shows an example of a screen that plays back the examinee's answers (hereinafter also referred to as the "playback screen"). In the example shown in Figure 5, the timing of the cheating actions is identifiable by M10 and M20 on the timeline T10 at the bottom of the playback screen. That is, in the example shown in Figure 5, it is shown that the examinee performed cheating actions twice during the exam. The output unit 315 generates and outputs screen information that identifiablely displays the timing of the cheating actions, for example, based on the time of key input associated with the first log information.

[0056] Returning to Figure 3, the output unit 315 may determine the display order of the detection results (test results) regarding each examinee based on the score of each examinee identified by the identification unit 314. For example, the output unit 315 may determine the display order of the detection results in ascending or descending order of scores based on each examinee's score regarding fraud. Note that whether the display order of the detection results is ascending or descending may be set as appropriate by the evaluator or company.

[0057] The output unit 315 may generate display information that displays the detection results of each examinee based on the determined display order. For example, the output unit 315 generates screen information that shows the display information regarding the detection results of each examinee to be displayed on the information processing device 20 used by the evaluator or company, based on the determined display order.

[0058] The output unit 315 outputs the generated display information. The output unit 315 controls the network communication interface 320 to output the display information to the information processing device 20 used by the evaluator or company. For example, if the display order of the detection results is determined to be in descending order of score, in the example shown in Figure 4, the detection results for each examinee will be displayed in the order of "Takahashi Riko", "Tanaka Taro", "Yamaguchi Jiro", and "Suzuki Hanako". Note that examinees who were not evaluated for cheating during the exam or whose evaluation was not possible (e.g., "Takahashi Riko") may not be displayed. Also, examinees who were not evaluated for cheating during the exam or whose evaluation was not possible (e.g., "Takahashi Riko") may be displayed after examinees who were evaluated for cheating during the exam (e.g., "Tanaka Taro", "Yamaguchi Jiro", and "Suzuki Hanako").

[0059] Through the above processing, for example, by displaying each test-taker's test results in descending order of cheating scores on the results screen, evaluators or companies can efficiently identify test-takers who are highly likely to have cheated during the exam. Alternatively, by displaying each test-taker's test results in ascending order of cheating scores on the results screen, evaluators or companies can efficiently identify test-takers who are highly likely not to have cheated during the exam.

[0060] The extraction unit 313 may extract actions in which one or more examinees revise their answers based on the first log information acquired by the acquisition unit 312. For example, the extraction unit 313 extracts actions in which characters are deleted during the exam based on the first log information, which includes logs of key inputs for event keys that erase characters in an answer (e.g., Backspace or Delete). In other words, the extraction unit 313 may extract actions in which an examinee deletes an answer they typed during the exam and re-answers it, based on the first log information.

[0061] The identification unit 314 may identify a score related to the examinee's cheating during the exam based on the examinee's actions of correcting answers during the exam. For example, the identification unit 314 calculates the number of times the examinee corrected answers during the exam based on the first log information. The identification unit 314 may identify a score that is inversely proportional to the number of such corrections. That is, if the number of corrections is extremely low, it is considered that the examinee made an unusually small number of mistakes while answering, and there is a possibility that the examinee engaged in cheating, such as referring to a designated book or web page while answering, or having a third party tell them the answers. Therefore, the identification unit 314 may identify a score related to cheating based on the number of corrections.

[0062] This allows the information processing device 20 to output information regarding scores based on the examinee's actions of revising answers during the exam. As a result, evaluators or companies can, for example, identify examinees who have cheated based on the number of times they revised their answers during the exam.

[0063] The acquisition unit 312 may acquire second log information relating to the log of each examinee losing focus from the screen in which they input their answers. For example, an element implemented to detect an event in anticipation of an event occurring on that element is a known technique called an event target, and the acquisition unit 312 can use this known technique to acquire second log information relating to the log of the examinee losing focus from the screen in which they input their answers. The second log information may be stored in the storage device 330 in association with the time of focus loss.

[0064] The acquisition unit 312 may acquire third log information relating to the log of each examinee refocusing on the screen where they input their answers. For example, the acquisition unit 312 can acquire third log information relating to the log of the examinee refocusing on the screen where they input their answers using the above-mentioned known technology relating to event targets. The third log information may be stored in the storage device 330 in association with the time of focus-in.

[0065] The extraction unit 313 may extract copy-and-paste actions performed by each examinee after leaving and returning to the answer screen, based on the first log information, second log information, and third log information acquired by the acquisition unit 312. For example, the extraction unit 313 may extract copy-and-paste actions of content displayed on a screen different from the answer screen during the exam (for example, copying and pasting text created by others and published on other web pages), based on the first log information, second log information, and third log information, which include logs of key input for copy event keys (e.g., Ctrl+C) and paste event keys (e.g., Ctrl+V).

[0066] The identification unit 314 may identify a score related to the examinee's cheating in the exam based on the copy-and-paste behavior extracted by the extraction unit 313. For example, the identification unit 314 calculates the time from when the examinee leaves the answer screen until they return to the answer screen, based on the focus-out time associated with the second log information and the focus-in time associated with the third log information. The identification unit 314 may identify a score proportional to the calculated time. That is, if the time from when the examinee leaves the answer screen until they return to the answer screen is extremely short, it is possible that the examinee left the answer screen unintentionally, and if the time from when the examinee leaves the answer screen until they return to the answer screen is long, it is possible that the examinee cheated, so the identification unit 314 may use this time to identify a score related to cheating.

[0067] The identification unit 314 may calculate the number of times the examinee leaves the answer screen and then returns to it, based on the second log information and / or the third log information. The identification unit 314 may also identify a score proportional to the calculated number. That is, if the number of times the examinee leaves the answer screen and then returns to it is extremely low, it is possible that the examinee left the answer screen unintentionally. Furthermore, if the number of times the examinee leaves the answer screen and then returns to it is high, it is possible that the examinee cheated. Therefore, the identification unit 314 may use this number to identify a score related to cheating.

[0068] This allows the information processing device 20 to output information regarding the score based on the examinee's actions, such as leaving the answer screen and then copying and pasting after returning. As a result, evaluators or companies can identify examinees who have cheated, for example, by browsing other web pages during the exam.

[0069] Furthermore, the tests provided by this service may include tests related to coding.

[0070] The extraction unit 313 may extract actions in which each examinee changes the variable names contained in the code they copied and pasted, based on the first log information, second log information, and third log information acquired by the acquisition unit 312. For example, the extraction unit 313 extracts actions in which an examinee copies and pastes code displayed on a screen different from the answer screen to the answer screen, and changes the variable names contained in the code on that answer screen, based on the first log information, second log information, and third log information, which include logs of key inputs such as event keys that erase characters in the answer (e.g., Backspace or Delete).

[0071] The identification unit 314 may identify a score related to cheating by the examinee in the exam based on the action of changing the variable name contained in the copy-and-paste code extracted by the extraction unit 313. For example, the identification unit 314 calculates the time from when the examinee returns to the answer screen until the variable name is changed based on the focus-in time associated with the third log information and the time when the event key for deleting characters associated with the first log information is entered. The identification unit 314 may identify a score that is inversely proportional to the calculated time. That is, if the time from when the examinee returns to the answer screen until the variable name is changed is short, it is possible that the examinee copied and pasted code from a web page or the like and changed the variable name with the intention of cheating, so the identification unit 314 may use that time to identify a score related to cheating.

[0072] The identification unit 314 may calculate the number of times the examinee changed the variable names in the code that was copied and pasted, based on the first log information, the second log information, and / or the third log information. The identification unit 314 may identify a score corresponding to the calculated number of changes. For example, the identification unit 314 may identify a score that is proportional to the number of changes.

[0073] This allows the information processing device 20 to output information about the score based on the action of copying and pasting a code displayed on a different screen from the answer screen to the answer screen and changing the variable name contained in that code on the answer screen. Therefore, even if the examinee changes the variable name that they copied and pasted, the evaluator or company can identify the examinee who cheated, for example, by browsing other web pages during the exam.

[0074] The acquisition unit 312 may acquire permission information indicating that each examinee is permitted to use external services during the exam. For example, the acquisition unit 312 may acquire permission information indicating that each examinee is permitted to use a search tool or a generating AI during the exam. The acquisition unit 312 may acquire such permission information by, for example, reading permission information stored in the storage device 330.

[0075] The output unit 315 may output permission information acquired by the acquisition unit 312 in association with information based on the fraud score. The output unit 315 outputs screen information in which the permission information is associated with the information based on the fraud score to the information processing device 20 used by the evaluator or company. The permission information is displayed as "-" as shown in screen area B100 of Figure 4, for example.

[0076] This means that even if a test-taker's score regarding cheating is inflated due to the use of external services such as search tools or generation AI, evaluators or companies can confirm that they do not need to consider the test-taker to have cheated during the exam because the use of external services was permitted in advance.

[0077] <Operation> Next, the operation according to this embodiment will be described. Figure 6 is a sequence diagram showing an example of the processing of the information processing device 20 and server 30 according to this embodiment.

[0078] In step S11, the information processing device 20 used by the evaluator or company transmits a request for test results.

[0079] In step S12, the server 30 used by the operator acquires log information. For example, the server 30 acquires first log information, second log information, and third log information. However, the server 30 does not necessarily have to acquire second log information and third log information.

[0080] In step S13, the server 30 extracts a predetermined action taken by each examinee during the exam based on the log information.

[0081] In step S14, the server 30 identifies the score for each test-taker's cheating in the exam based on the extracted predetermined behavior.

[0082] In step S15, the server 30 outputs information based on the identified score to the information processing device 20.

[0083] <Variation> The above embodiments are provided to facilitate understanding of the present invention and are not intended to limit it. The present invention can be modified and improved without departing from its spirit, and equivalents thereof are also included. Furthermore, the present invention can form various disclosures by appropriately combining the multiple components disclosed in the above embodiments. For example, some components may be deleted from all the components shown in the embodiments. Moreover, components may be appropriately combined in different embodiments.

[0084] Server 30 may use information based on the test scores of other test takers to identify scores related to cheating. For example, the identification unit 314 of Server 30 may identify scores corresponding to the absolute difference between a test taker's score above the average score and the average score. That is, since a test taker who scores significantly above the average score may have cheated, the identification unit 314 may use the average score to identify scores related to cheating. Alternatively, the identification unit 314 may use the median score of test takers on the test questions to identify scores related to cheating.

[0085] Server 30 may use the percentage of copied and pasted characters in the answers submitted by the examinee to identify scores related to cheating. For example, the identification unit 314 of Server 30 may identify a score corresponding to that percentage. That is, the higher the percentage of copied and pasted characters in the submitted answers, the more likely the examinee is to have cheated, so the identification unit 314 may use that percentage to identify scores related to cheating.

[0086] Server 30 may use the time the examinee spent solving the exam questions to determine the score related to cheating. For example, the identification unit 314 of Server 30 may identify a score corresponding to the examinee's answer time. That is, if the answer time is extremely short, the examinee may have cheated, and therefore the identification unit 314 may use that answer time to identify the score related to cheating. [Explanation of Symbols]

[0087] 1...Information processing system, 10...Information processing device, 20...Information processing device, 30...Server, 310...Processor, 311...Information processing unit, 312...Acquisition unit, 313...Extraction unit, 314...Specification unit, 315...Output unit, 320...Network communication interface, 330...Storage device, 340...User interface, 350...Communication bus, A10...Screen, A20...Screen, A30...Screen, B10~40...Screen area, B50...Input field, B60...Output field, B10~100...Screen area, M10...Playback timing, M20...Playback timing, N...Network, T10...Timeline

Claims

1. In an information processing device, In an examination in which each user answers by typing on each processing unit, first log information is obtained regarding the log of each user's typing on each processing unit, Based on the first log information, extract a predetermined action of each user during the test, Based on the predetermined actions, identify the score for each user's misconduct in the test, To output information based on the aforementioned score, A program that executes the command.

2. The output mentioned above is, Based on the scores of each user, the display order of the detection results regarding fraud for each user is determined. Based on the aforementioned display order, display information is generated to display the detection results for each user, Outputting the aforementioned display information, The program according to claim 1, including the following:

3. The extraction described above means The program according to claim 1, which includes extracting actions by one or more users that have re-answered based on the first log information.

4. To obtain second log information regarding the log of each user who has removed focus from the screen where they enter their answer, To obtain third log information regarding the log of each user refocusing on the screen in which they enter their answer, Let's execute it further, The extraction described above means The program according to claim 1, which includes extracting the copy-and-paste actions performed by each user after leaving and returning from the screen, based on the first log information, the second log information, and the third log information.

5. The aforementioned test includes a test on coding, The extraction described above means The program according to claim 4, which includes extracting actions in which each user changes the variable names contained in the copied and pasted code, based on the first log information, the second log information, and the third log information.

6. Further, the system will obtain permission information indicating that each of the aforementioned users is permitted to use external services during the test, The output mentioned above is, The program according to claim 1, which includes outputting the permission information in association with the information based on the score.

7. An information processing method performed by an information processing device, In an examination in which each user answers by typing on each processing unit, first log information is obtained regarding the log of each user's typing on each processing unit, Based on the first log information, extract a predetermined action of each user during the test, Based on the predetermined actions, identify the score for each user's misconduct in the test, To output information based on the aforementioned score, An information processing method that performs the following.

8. An information processing device including a processor, The aforementioned processor, In an examination in which each user answers by typing on each processing unit, first log information is obtained regarding the log of each user's typing on each processing unit, Based on the first log information, extract a predetermined action of each user during the test, Based on the predetermined actions, identify the score for each user's misconduct in the test, To output information based on the aforementioned score, An information processing device that performs the following actions.

Images