Method, program, and apparatus for generating test

Abstract

Disclosed are a method, a program, and an apparatus for generating a test, according to an embodiment of the present disclosure. The method may comprise the steps of: acquiring input data regarding attributes of a test intended by a user; on the basis of a user input, determining detection conditions for monitoring behaviors of test takers; and opening the test on the basis of the determined detection conditions.

Description

Test creation method, program and device

[0001] The present disclosure relates to a test generation method, and more particularly, to a method and device for setting conditions necessary for test management and supervision in a computing environment and generating a test according to the set conditions.

[0002]

[0003] In educational environments where testing takes place, it's crucial to monitor test takers' behavior during the test. Unlike offline testing, it's difficult to effectively monitor test takers' behavior and surroundings in online testing. Therefore, in online testing environments, it's even more crucial for administrators to accurately analyze test takers' actions in real time to determine if there was any cheating.

[0004] Meanwhile, establishing standards for determining whether a test taker's actions constitute cheating is crucial. While there are generally accepted qualitative standards, these standards are not quantitatively defined for offline testing, making them difficult to directly apply to online testing. Therefore, in an online testing environment, appropriate detection standards for monitoring behaviors such as cheating must be established when designing the test, taking into account factors such as the test's characteristics, environment, and proctoring style. Furthermore, these detection standards for monitoring test taker behavior must be supplemented through feedback derived from data collected during the test administration.

[0005]

[0006] The present disclosure is conceived in response to the aforementioned background technology, and aims to provide a test construction environment capable of generating and recommending detection conditions necessary for test supervision in accordance with the properties of a test, and in which user customization of detection conditions is easy.

[0007] However, the problems to be solved in this disclosure are not limited to the problems mentioned above, and other problems not mentioned can be clearly understood based on the description below.

[0008]

[0009] According to one embodiment of the present disclosure, a test generation method, performed by a computing device, for achieving the aforementioned task is disclosed. The method may include: obtaining input data regarding attributes of a test intended by a user; determining detection conditions for monitoring the behavior of test takers based on the user input; and creating a test based on the determined detection conditions.

[0010] Alternatively, the attributes of the test may include at least one of the test type, whether it is open book, whether it is searchable, or the test period.

[0011] Alternatively, the step of determining detection conditions for monitoring the behavior of test takers based on the user input may include the step of determining the detection conditions based on user input via a first user interface through which a recommended ruleset is exposed.

[0012] Alternatively, the step of determining a detection condition based on a user input through a first user interface to which the recommended rule set is exposed may include the step of generating a recommended rule set from a rule database based on input data regarding the properties of the test; and the step of determining a detection condition based on the selected rule set when either the recommended rule set or a preset exposed through the first user interface is selected according to the user input.

[0013] Alternatively, the above rule database can be updated based on the results of analyzing big data accumulated as the test is opened and operated.

[0014] Alternatively, the big data may include at least one of data regarding detection conditions adjusted by the user's rule set customization during the process of opening a test or data regarding events detected based on rules during the process of running an opened test.

[0015] Alternatively, the update may be to create a rule to be included in the rule database, or to dynamically change the details of a rule that constitutes a ruleset included in the rule database.

[0016] Alternatively, when one of the recommended rulesets or presets exposed through the first user interface is selected based on the user input, the step of determining the detection condition based on the selected ruleset may include a step of adjusting the intensity of the detection condition based on a user input for controlling the level of the selected ruleset. In addition, the level of the ruleset may be distinguished based on the numerical values ​​of the detailed items of the rules constituting the ruleset.

[0017] Alternatively, the step of determining detection conditions for monitoring the behavior of test takers based on the user input may include the step of determining the detection conditions based on user input via a second user interface for customizing the ruleset.

[0018] Alternatively, the step of determining the detection condition based on user input via the second user interface for customizing the ruleset may include a step of determining the detection condition based on the ruleset including the set or adjusted rule when at least one rule included in the ruleset is set or adjusted based on the user input via the second user interface.

[0019] Alternatively, when setting or adjusting at least one rule included in the rule set is performed according to user input through the second user interface, the step of determining a detection condition based on the rule set including the set or adjusted rule may include: when setting or adjusting at least one rule is performed, a step of analyzing a detailed item determined to be suitable for the set or adjusted rule based on big data of a test to which rules corresponding to the set or adjusted rule are applied; and a step of recommending a detailed item determined to be suitable for the set or adjusted rule based on a result of the analysis.

[0020] Alternatively, when setting or adjusting at least one rule is performed, the step of analyzing a detailed item determined to be suitable for the set or adjusted rule based on big data of a test to which rules corresponding to the set or adjusted rule are applied may include a step of analyzing weights of the rules based on big data of a test to which rules corresponding to the set or adjusted rule are applied; and a step of determining a detailed item of a rule selected according to the analyzed weights as a detailed item suitable for the set or adjusted rule.

[0021] Alternatively, the big data that influences the analysis of the above weights may include at least one of the rate at which a result detected according to a specific rule was delivered to the test taker or the number of times a specific rule was activated by the test taker's behavior satisfying a subitem of the specific rule.

[0022] According to one embodiment of the present disclosure, a test generation method, performed by a computing device, for achieving the aforementioned task is disclosed. The method may include: displaying a first user interface in which a recommended ruleset generated from a rule database based on input data regarding attributes of a test intended by a user is displayed; and displaying a second user interface area for receiving user input for customizing the ruleset.

[0023] According to one embodiment of the present disclosure for achieving the aforementioned task, a computer program stored on a computer-readable storage medium is disclosed. When the computer program is executed on one or more processors, it causes the computer program to perform operations for generating a test. In this case, the operations may include: obtaining input data regarding properties of a test intended by a user; determining detection conditions for monitoring the behavior of test takers based on user input; and opening a test based on the determined detection conditions.

[0024] According to one embodiment of the present disclosure for achieving the aforementioned task, a computing device for generating a test is disclosed. The device may include a processor including at least one core; a memory including program codes executable by the processor; and a network unit for acquiring input data regarding the properties of a test intended by a user. In this case, the processor may determine detection conditions for monitoring the behavior of test takers based on user input, and may open a test based on the determined detection conditions.

[0025]

[0026] The present disclosure can provide a test construction environment that can generate and recommend detection conditions necessary for test supervision according to the properties of the test and facilitate user customization of detection conditions.

[0027]

[0028] FIG. 1 is a block diagram of a computing device according to one embodiment of the present disclosure.

[0029] FIG. 2 is a conceptual diagram illustrating a process for generating a recommended ruleset according to one embodiment of the present disclosure.

[0030] FIG. 3 is a block diagram illustrating a test opening process according to one embodiment of the present disclosure.

[0031] Figure 4 is a flowchart showing a test opening process according to one embodiment of the present disclosure.

[0032] FIG. 5 is a flowchart illustrating a test generation method according to one embodiment of the present disclosure.

[0033]

[0034] Below, embodiments of the present disclosure are described in detail with reference to the attached drawings so that those skilled in the art can easily implement the present disclosure. The embodiments presented in this disclosure are provided to enable those skilled in the art to utilize or implement the contents of the present disclosure. Accordingly, various modifications to the embodiments of the present disclosure will be apparent to those skilled in the art. That is, the present disclosure may be implemented in various different forms and is not limited to the embodiments described below.

[0035] Throughout the specification of this disclosure, identical or similar drawing numbers refer to identical or similar components. Furthermore, for clarity in the description of this disclosure, drawing numbers for parts unrelated to the description of this disclosure may be omitted in the drawings.

[0036] The term "or" as used herein is intended to mean an inclusive "or" rather than an exclusive "or." That is, unless otherwise specified herein or clear from context, "X employs A or B" should be understood to mean either of the natural inclusive permutations. For example, unless otherwise specified herein or clear from context, "X employs A or B" can be interpreted to mean either X employs A, X employs B, or X employs both A and B.

[0037] The term "and / or" as used herein should be understood to refer to and include all possible combinations of one or more of the related concepts listed.

[0038] The terms "comprises" and / or "comprising" as used herein should be understood to mean the presence of certain features and / or components. However, it should be understood that the terms "comprises" and / or "comprising" do not exclude the presence or addition of one or more other features, other components, and / or combinations thereof.

[0039] Unless otherwise specified in this disclosure or unless the context makes it clear that the singular form is intended to be referred to, the singular should generally be construed to include “one or more.”

[0040] The term "Nth (N is a natural number)" used in the present disclosure can be understood as an expression used to mutually distinguish components of the present disclosure based on a predetermined standard such as a functional perspective, a structural perspective, or convenience of explanation. For example, components performing different functional roles in the present disclosure can be distinguished as a first component or a second component. However, components that are substantially the same within the technical spirit of the present disclosure but must be distinguished for convenience of explanation may also be distinguished as a first component or a second component.

[0041] The term "acquisition" as used in this disclosure may be understood to refer to generating or receiving data in an on-device form, as well as receiving data via a wireless communication network with an external device or system.

[0042] Meanwhile, the term "module" or "unit" used in the present disclosure can be understood as a term referring to an independent functional unit that processes computing resources, such as a computer-related entity, firmware, software or a part thereof, hardware or a part thereof, or a combination of software and hardware. At this time, the "module" or "unit" may be a unit composed of a single element, or a unit expressed as a combination or set of multiple elements. For example, as a narrow concept, a "module" or "unit" may refer to a hardware element of a computing device or a set thereof, an application program that performs a specific function of software, a processing process implemented through software execution, or a set of instructions for program execution, etc. In addition, as a broad concept, a "module" or "unit" may refer to the computing device itself that constitutes the system, or an application running on the computing device, etc. However, since the above-described concept is only an example, the concept of “module” or “part” may be defined in various ways within a range understandable to those skilled in the art based on the contents of the present disclosure.

[0043] The term "model" as used herein may be understood as a system implemented using mathematical concepts and language to solve a specific problem, a set of software units to solve a specific problem, or an abstract model of a processing process to solve a specific problem. For example, a deep learning "model" may refer to an entire system implemented as a neural network that has problem-solving capabilities through learning. In this case, the neural network can have problem-solving capabilities by optimizing parameters connecting nodes or neurons through learning. A deep learning "model" may include a single neural network or a set of neural networks that are a combination of multiple neural networks.

[0044] The term "image" as used herein may refer to multidimensional data composed of discrete image elements. In other words, "image" may be understood as a term referring to a digital representation of an object visible to the human eye. For example, "image" may refer to multidimensional data composed of elements corresponding to pixels in a two-dimensional image. "Image" may refer to multidimensional data composed of elements corresponding to voxels in a three-dimensional image.

[0045] Meanwhile, the term "rule" used in this disclosure can be understood as an action corresponding to a detection condition for monitoring user behavior. For example, a rule may correspond to an abnormal behavior that is not an act of cheating or may be suspected of cheating, but is not an act of cheating, as defined by the user or the system. In this case, the detailed items of the rule may be composed of an action, such as "raising an arm" or "looking away from the monitor," and a numerical value, such as "0.5 seconds." In other words, a rule set as a cheating action may be composed of a combination of an action and a numerical value, such as "raising an arm for 0.5 seconds." In addition, the term "ruleset" can be understood as a set of rules, such as the examples described above.

[0046] The explanation of the above terms is intended to aid understanding of the present disclosure. Therefore, unless explicitly stated as limiting the contents of the present disclosure, it should be noted that the above terms are not intended to limit the technical ideas of the present disclosure.

[0047] FIG. 1 is a block diagram of a computing device according to one embodiment of the present disclosure.

[0048] A computing device (100) according to one embodiment of the present disclosure may be a hardware device or a part of a hardware device that performs comprehensive processing and calculation of data, or may be a software-based computing environment connected to a communication network. For example, the computing device (100) may be a server that performs intensive data processing functions and shares resources, or may be a client that shares resources through interaction with a server. In addition, the computing device (100) may be a cloud system that enables multiple servers and clients to interact with each other to comprehensively process data. Since the above description is only one example related to the type of computing device (100), the type of computing device (100) may be configured in various ways within a range understandable to those skilled in the art based on the contents of the present disclosure.

[0049] Referring to FIG. 1, a computing device (100) according to one embodiment of the present disclosure may include a processor (110), a memory (120), and a network unit (130). However, FIG. 1 is merely an example, and the computing device (100) may include other components for implementing a computing environment. In addition, only some of the disclosed components may be included in the computing device (100).

[0050] The processor (110) according to one embodiment of the present disclosure may be understood as a configuration unit including hardware and / or software for performing computing operations. For example, the processor (110) may read a computer program and perform operations for data processing. The processor (110) may process computational processes such as processing input data for machine learning, feature extraction for machine learning, and error calculation based on backpropagation. The processor (110) may generate a user interface for communication with an external terminal or device. The processor (110) for performing such data processing may include a central processing unit (CPU), a general purpose graphics processing unit (GPGPU), a tensor processing unit (TPU), an application specific integrated circuit (ASIC), or a field programmable gate array (FPGA). The type of processor (110) described above is only one example, and thus the type of processor (110) can be configured in various ways within a range understandable to those skilled in the art based on the contents of the present disclosure.

[0051] The processor (110) can receive information regarding the attributes of a test a user wishes to open, generate a ruleset used to determine detection conditions for monitoring the test taker's behavior, and recommend the ruleset to the user. Here, the detection conditions can be understood as judgment classifications and criteria for monitoring the test taker's behavior. For example, the detection conditions can be criteria for determining the test taker's behavior as cheating or criteria for determining the test taker's behavior as abnormal.

[0052] The processor (110) can obtain input data including information about the characteristics of the test, such as what type of test it is and how the test is conducted. In addition, the processor (110) can generate a recommended rule set from a rule database in which rules are stored based on input data regarding the properties of the test intended by the user. For example, the processor (110) can extract rules from a rule database tagged according to information regarding the properties of the test based on the input data to generate a recommended rule set in order to determine conditions for detecting cheating or abnormal behavior. The processor (110) can provide a user interface so that the user can use this recommended rule set to set conditions for determining cheating or abnormal behavior. The processor (110) can provide a rule set suitable for the user's intention through recommendations based on input data regarding the properties of the test, and can provide an environment in which the user can conveniently set detection conditions.

[0053] The processor (110) can update a rule database for generating a ruleset using big data collected during the test creation and operation. During the process of creating and operating a test in a computing environment, various pieces of information that may affect the behavior monitoring of test takers, such as information preset by users for test creation and information on results detected according to detection conditions during the test operation process, may be accumulated. The processor (110) can analyze data containing such information to update a rule database for generating a recommended ruleset. For example, if a detection condition commonly set by multiple users exists during the test creation process, the processor (110) can update a rule corresponding to the condition in the rule database. At this time, if the rule to be updated does not exist in the rule database, the processor (110) can add or store the corresponding rule in the rule database. If the detailed items of the rule to be updated in the rule database are different, the processor (110) can change the detailed items of the matching rule in the rule database. The processor (110) can increase the reliability and accuracy of rule set recommendations by dynamically updating the rule database that serves as the basis for recommendations based on big data collected during the test opening and operation process.

[0054] The processor (110) can dynamically respond to rule set customization based on user input and recommend rules or rule sets to the user. When the user customizes the rule set itself by creating, deleting, or modifying rules included in the rule set according to his / her own opinion, the processor (110) can recommend detailed items that are deemed appropriate for the rule being customized. At this time, the processor (110) can analyze big data of tests to which rules corresponding to the rule being customized are applied to determine how to appropriately set detailed items of the rule customized by the user. For example, when a rule called "raising the arm more than 10 degrees" is added to the rule set during the process of opening a test, the processor (110) can analyze data of tests to which rules highly correlated with the rule called "raising the arm more than 10 degrees" are applied. Specifically, the processor (110) can analyze the weights of the rules by analyzing how much the detection condition corresponding to the rule of 'raising the arm more than 10 degrees' was triggered, which condition among the detection conditions corresponding to the rule including the action of 'raising the arm' was triggered, etc. Based on the weights of the rules, the processor (110) can determine what value is more appropriate for the condition to determine that the rule of 'raising the arm more than 10 degrees' is a cheating or abnormal behavior than 10 degrees. If it is determined that the value of 10 degrees is too sensitive and the value of 15 degrees is appropriate, the processor (110) can recommend to the user that the rule be changed to 'raising the arm more than 15 degrees'. The processor (110) can provide the user with objective information based on statistical evidence through recommendations based on user customization. In addition, the processor (110) can increase the reliability and accuracy of behavior monitoring for test takers by guiding users to make rational choices and test settings based on objective information through recommendations based on user customization.

[0055] The processor (110) can determine detection conditions for monitoring the behavior of a test taker based on a ruleset established based on user input. Furthermore, the processor (110) can generate a test based on the determined detection conditions. For example, the processor (110) can determine conditions for determining cheating or abnormal behavior based on a ruleset established by the user. If a request is received to determine a ruleset including the rule "raising the arm more than 15 degrees" as cheating based on user input, the processor (110) can determine the rules included in the ruleset as monitoring targets for detecting cheating. The processor (110) can generate a test reflecting this determination. Furthermore, when the generated test is run, the processor (110) can detect whether behaviors matching the rules determined as monitoring targets occur.

[0056] The memory (120) according to one embodiment of the present disclosure may be understood as a configuration unit including hardware and / or software for storing and managing data processed in the computing device (100). That is, the memory (120) may store any type of data generated or determined by the processor (110) and any type of data received by the network unit (130). For example, the memory (120) may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory, a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, or an optical disk. In addition, the memory (120) may also include a database system that controls and manages data in a predetermined system. The type of memory (120) described above is only one example, and thus the type of memory (120) can be configured in various ways within a range understandable to those skilled in the art based on the contents of the present disclosure.

[0057] The memory (120) can structure and organize and manage data, combinations of data, and program codes executable by the processor (110) required for the processor (110) to perform operations. For example, the memory (120) can store input data acquired through the network unit (130) described below. The memory (120) can store data regarding user input and requests received through the user interface. The memory (120) can store program codes that cause the processor (110) to perform operations that generate or recommend detection conditions for behavior monitoring of a test taker based on the input data. In addition, the memory (120) can store program codes that cause the processor (110) to train a deep learning model, program codes that cause the processor (110) to perform inference operations on the trained deep learning model, and various data generated as the program codes are executed.

[0058] The network unit (130) according to one embodiment of the present disclosure may be understood as a component that transmits and receives data through any type of known wired or wireless communication system. For example, the network unit (130) may perform data transmission and reception using a wired or wireless communication system such as a local area network (LAN), wideband code division multiple access (WCDMA), long term evolution (LTE), wireless broadband internet (WiBro), fifth generation mobile communication (5G), ultra wide-band, ZigBee, radio frequency (RF) communication, wireless LAN, wireless fidelity, near field communication (NFC), or Bluetooth. Since the above-described communication systems are only examples, the wired and wireless communication system for data transmission and reception of the network unit (130) may be applied in various ways other than the above-described examples.

[0059] The network unit (130) can receive data necessary for the processor (110) to perform calculations through wired or wireless communication with any system, server, client, etc. In addition, the network unit (130) can transmit data generated through calculations of the processor (110) through wired or wireless communication with any system, server, client, etc. For example, the network unit (130) can receive user input and input data for test opening through wired or wireless communication with a client, etc. The network unit (130) can transmit various data generated through calculations of the processor (110) based on the input data through wired or wireless communication with a client, etc.

[0060] FIG. 2 is a conceptual diagram illustrating a process for generating a recommended ruleset according to one embodiment of the present disclosure.

[0061] Referring to FIG. 2, a computing device (100) according to one embodiment of the present disclosure can obtain input data (10) regarding the attributes of a test intended by a user. Here, the attributes of the test can be understood as indicators for determining the type of test the user intends to open and how it is administered. The computing device (100) can obtain the input data (10) used to determine the characteristics of the test the user intends to open by user input via a user interface. For example, the attributes of the test can include at least one of the test type, whether the test is open-book, whether the test is searchable, or the test cycle. The test type can indicate the type of test, such as a computer-based test (CBT), a written test, or a coding test. The presence of an open-book can indicate whether information designated by the test administrator or proctor is exposed to the test taker during the test. The presence of searchable can indicate whether the test taker can search for information related to the test during the test. The test cycle can indicate whether the test is conducted regularly or irregularly. The test attributes listed above are only examples, and various indicators may be included to determine the characteristics of the test in addition to the examples described above.

[0062] The computing device (100) can generate a recommended rule set (30) from the rule database (20) based on input data (10) regarding the properties of the test. The computing device (10) can generate a recommended rule set (30) by matching detailed information of the input data (10) regarding the properties of the test with rules classified according to the properties of the test in the rule database (20). For example, if the input data (10) includes detailed information such as the type of test, whether the test is open-book, and whether the test is searchable, the computing device (100) can extract rules by checking rules matching each detailed information of the input data (10) in the rule database (20). Specifically, the computing device (100) can analyze rules included in a group (A) related to the test type in the rule database (20) based on information about the test type of the input data (10), and include rules matching the information about the test type of the input data (10) in the recommended rule set (30). The computing device (100) can analyze rules included in a group (B) related to the presence or absence of an open book in the rule database (20) based on information about the presence or absence of an open book in the input data (10), and include rules matching the information about the presence or absence of an open book in the input data (10) in the recommended rule set (30). The computing device (100) can analyze rules included in a group (C) related to the presence or absence of search in the rule database (20) based on information about the searchability of the input data (10), and include rules matching the information about the searchability of the input data (10) in the recommended rule set (30). At this time, the analysis of rules included in the group within the rule database (20) may be performed according to logic defined by program code, or may be performed through a pre-trained neural network model. In this way, the computing device (100) can extract rules matching the detailed information of the input data (10) from the rule database (20) constructed by classifying rules according to the properties of the test, and generate a rule set (30) including the extracted rules.

[0063] Meanwhile, the computing device (100) can update the rule database (20) based on the results of analyzing the big data (40) accumulated in accordance with the opening and operation of the test. The computing device (100) can dynamically update the rule database (20) by continuously analyzing the big data (40) collected in the process of opening and operating the test. The big data (40) used to update the rule database (20) can include at least one of data regarding detection conditions adjusted by customizing the rule set in the process of opening the test by the user or data regarding events detected based on rules in the process of operating the opened test. In addition, dynamic updating can be understood as creating a rule to be newly included in the rule database (20) or changing the details of a rule already included in the rule database (20). For example, if the computing device (10) analyzes big data (40) and determines that a rule such as 'a body part is not visible in a designated area for more than 0.5 seconds' is frequently set by users through rule set customization, the computing device (100) can check whether the rule identified through the analysis of big data (40) is included in the rule database (20). If the rule identified through the analysis of big data (40) is not included in the rule database (20), the computing device (100) can classify the identified rule into Group A, Group B, or Group C according to the properties of the test and store it in the rule database (20). At this time, the classification may be performed according to logic defined by the program code or may be performed through a pre-learned neural network model. If a rule identified through analysis of big data (40) is included in the rule database (20) in a similar form, such as 'a body part is not visible in a designated area for more than 1 second', the computing device (100) can identify and change information that is inconsistent in the details between the rule already stored in the rule database (20) and the rule identified through analysis of big data (40).Specifically, the computing device (100) can modify the rule stored in the rule database (20) from 'no body part visible in the designated area for more than 1 second' to 'no body part visible in the designated area for more than 0.5 seconds.' In this way, the computing device (100) can improve the reliability and accuracy of user-tailored recommendations by utilizing big data (40) secured through test opening and operation for dynamic updates of the rule database (20).

[0064] FIG. 3 is a block diagram illustrating a test opening process according to one embodiment of the present disclosure.

[0065] Referring to FIG. 3, a computing device (100) according to one embodiment of the present disclosure may receive test information to open a test desired by a user (S110). At this time, the test information may be information regarding test properties necessary to understand the characteristics of the test desired by the user. At this time, the computing device (100) may provide a user interface to receive the test information. For example, the computing device (100) may generate a user interface to obtain test information input through a user terminal. The user interface for inputting test information may be implemented on a display. That is, the computing device (100) may receive information regarding test properties, such as test type, whether the test is open-book, whether the test is searchable, or the test cycle, through a user interface implemented on the display of the user terminal.

[0066] The computing device (100) may provide a user interface for determining detection conditions used to monitor the behavior of a test taker. In addition, the computing device (100) may obtain user input necessary to determine the detection conditions through the user interface and set the detection conditions (S120).

[0067] For example, the computing device (100) may provide a first user interface (10) that exposes relevant information so that a recommended rule set based on test information or a preset preset set in advance by the user can be selected. The user may perform an action of selecting one of the recommended rule sets or presets in an area (11) of the first user interface (10) implemented on the display of the user terminal. When one of the recommended rule sets or presets is finally selected according to the user's action, the computing device (100) may determine the finally selected rule set as a detection condition for determining cheating or abnormal behavior. Meanwhile, in the process of selecting the recommended rule set or preset according to the user's action, the computing device (100) may provide at least one of an area (12) that exposes detailed information of the rule set or an area (13) that exposes graphics for controlling the level of the rule set through the first user interface (10). The user can check which rules are included in the ruleset through the area (13) that exposes the details of the ruleset of the first user interface (10). The user can adjust the intensity of the detection condition in the area (13) that exposes the graphic for controlling the level of the ruleset of the first user interface (10). At this time, the level of the ruleset can be distinguished according to the numerical value of the detailed items of the rules that constitute the ruleset. Assuming that there is a rule that 'a body part is not visible in the designated area for more than 1 second', if the user lowers the level of the ruleset, the value of 1 second can be increased so that the detection sensitivity based on the rule is lowered. Conversely, if the user raises the level of the ruleset, the value of 1 second can be decreased so that the detection sensitivity based on the rule is increased. Through such level control of the ruleset, the user can easily and freely adjust the intensity of the detection condition under the designated ruleset.The first user interface (10) expressed in FIG. 3 is only an example, and thus the first user interface (10) may be implemented in various forms within a range understandable to those skilled in the art based on the present disclosure.

[0068] The computing device (100) may provide a second user interface (20) that provides a user customization environment so that the user can arbitrarily set or modify a rule set. The user may perform an operation to configure a rule set according to the user's own thoughts on the second user interface (20) implemented on the display of the user terminal. When rules for configuring a rule set are created or adjusted according to the user's operation, the computing device (100) may determine the customized rule set as a detection condition for determining cheating or abnormal behavior. Meanwhile, although not shown in FIG. 3, during the process of performing user customization, the computing device (100) may provide an area for recommending detailed items of a rule set or adjusted by the user through the second user interface (20). When a rule is set or adjusted by the user, the computing device (100) may analyze big data of a test to which rules corresponding to the set or adjusted rule are applied and recommend detailed items that are determined to be suitable for the set or adjusted rule. Here, recommending detailed items deemed appropriate can be understood as deriving an optimal value by comparing detailed items of rules through statistical analysis or neural network analysis based on big data. The user can select a recommended value displayed through the recommendation area of ​​the second user interface (20), or can set or adjust an arbitrary value without selecting one. In other words, the computing device (100) can assist the user's decision-making in setting detection conditions for test opening through recommendations through the second user interface (20). Since the second user interface (20) depicted in FIG. 3 is only an example, the second user interface (20) can be implemented in various forms within a range understandable to those skilled in the art based on the present disclosure.

[0069] When detection conditions are set through step S120, the computing device (100) can generate test questions based on information about test questions input by the user (S130). Furthermore, the computing device (100) can open a test based on the set detection conditions and the generated test questions (S140). For example, when a user inputs a test format, questions, and answers, the computing device (100) can organize the questions and answers according to the test format and generate test questions. Furthermore, the computing device (100) can open a test by generating an access link, such as a URL, and then provide the test to the user or test taker through a user terminal.

[0070] Meanwhile, the computing device (100) can continuously collect user input data acquired through step S120. Furthermore, the computing device (100) can analyze the collected user input data and manage it as big data. The computing device (100) can utilize this big data for updating a rule database for generating a recommended rule set, generating recommended rules during the user customization process, etc.

[0071] FIG. 4 is a flowchart illustrating a test establishment process according to one embodiment of the present disclosure. Below, the description of the remaining steps in FIG. 4, excluding step S260, will be replaced with the contents of FIGS. 2 and 3.

[0072] Referring to FIG. 4, a computing device (100) according to an embodiment of the present disclosure can analyze and recommend detailed items of a rule during a process of customizing a rule set by a user input (S260). When an input for setting or adjusting a rule is received (S240), the computing device (100) can analyze the weights of the rules based on big data of a test to which rules corresponding to the set or adjusted rule are applied. Then, the computing device (100) can select detailed items of the rule according to the analyzed weights and determine the selected detailed items of the rule as detailed items suitable for the set or adjusted rule. At this time, big data that influences the analysis of the weights can include at least one of the rate at which a result detected according to a specific rule is delivered to the test taker or the number of times a specific rule is activated by the test taker's behavior satisfying the detailed items of a specific rule.

[0073] Specifically, when a user-customized input is received, the computing device (100) can analyze the big data of a test to which rules corresponding to the user-customized input are applied. At this time, the rules corresponding to the user-customized input may be rules that have a high correlation with the rule that is the target of user customization. For example, if the rule of "raising the arm more than 10 degrees" is the target of customization, the computing device (100) can identify a rule that includes a body related to the arm, such as a hand, and an action related to the movement of the body as a detailed item as a rule with a high correlation, and can analyze the big data of the test to which the rule is applied.

[0074] The computing device (100) can analyze big data to determine the weights of rules. For example, a test supervisor may request an explanation from a test taker to accurately judge the detection results. The higher the rate of such explanation requests, the higher the probability that the detection results will be judged as cheating or abnormal behavior. Accordingly, the computing device (100) can assign a higher weight to a rule the higher the rate at which the detection results are transmitted to the test taker according to the rule. If the rate at which the detection results are transmitted to the test taker according to the rule "raising the arm more than 10 degrees" is 20%, and the rate at which the detection results are transmitted to the test taker according to the rule "raising the arm more than 20 degrees" is 30%, the computing device (100) can assign a higher weight to the rule for raising the arm more than 20 degrees than to the rule for raising the arm more than 10 degrees. In addition, the more times a rule is activated by satisfying the detection conditions according to the rule, the higher the probability that the detection result according to the rule corresponds to cheating or abnormal behavior. However, if the rule is activated too many times, there is a high probability that the rule corresponds to the general behavior of test takers rather than cheating or abnormal behavior. Therefore, the computing device (100) can assign a higher weight as the number of times the rule is activated increases below a predetermined threshold. In addition, the computing device (100) can assign a gradually lower weight as the number of times the rule is activated exceeds the predetermined threshold.

[0075] The computing device (100) can determine the detailed items of the rule to be recommended to the user based on the analyzed weights. The computing device (100) can determine the detailed items of the rule with the highest weight as the detailed items of the rule to be recommended to the user, or can determine the detailed items of the rule to be recommended to the user by combining the detailed items of rules with weights higher than a reference value. In addition, the computing device (100) can provide the detailed items determined as the recommendation target to the user through the user terminal. At this time, the detailed items of the rule to be recommended to the user can be distinguished into an action based on a specific body part of a person and a numerical value indicating the degree of the action. For example, if the rule "raising the arm more than 20 degrees" is analyzed to have the highest weight, the computing device (100) can provide the user through the user terminal with recommendation information suggesting that the rule customized as "raising the arm more than 10 degrees" be modified from 10 degrees to 20 degrees. Alternatively, if the rule of 'raising the hand more than 10 degrees' is analyzed to have the highest weight, the computing device (100) can provide the user with recommendation information suggesting to change the arm to a hand in the rule customized to 'raising the arm more than 10 degrees' through the user terminal.

[0076] FIG. 5 is a flowchart illustrating a test generation method according to one embodiment of the present disclosure.

[0077] Referring to FIG. 5, a computing device (100) according to one embodiment of the present disclosure can obtain input data regarding the properties of a test intended by a user (S310). For example, if the computing device (100) is a cloud server, the computing device (100) can input information regarding the properties of the test and receive the input data through a user interface implemented through an input / output unit of a user terminal. If the computing device (100) is a client directly equipped with an input / output unit, the computing device (100) can input information regarding the properties of the test through a user interface implemented through the input / output unit.

[0078] The computing device (100) can determine detection conditions for monitoring the behavior of test takers based on user input (S320). The computing device (100) can determine detection conditions based on user input via a first user interface where a recommended rule set is displayed. For example, the computing device (100) can generate a recommended rule set from a rule database based on input data regarding the properties of a test. In addition, if one of the recommended rule set or preset rule set displayed via the first user interface is selected based on user input, the computing device (100) can determine detection conditions based on the selected rule set. In addition, the computing device (100) can determine detection conditions based on user input via a second user interface for customizing the rule set. For example, if at least one rule included in the rule set is set or adjusted based on user input via the second user interface, the computing device (100) can determine detection conditions based on the rule set including the set or adjusted rule. Specifically, when setting or adjusting at least one rule is performed, the computing device (100) can analyze detailed items determined to be suitable for the set or adjusted rule based on big data of a test to which rules corresponding to the set or adjusted rule are applied. In addition, the computing device (100) can recommend detailed items determined to be suitable for the set or adjusted rule based on the analysis results.

[0079] The computing device (100) can open a test based on the determined detection conditions (S330). The computing device (100) can receive additional information, such as test questions, to open the test, and can open the test by reflecting the received additional information. In this case, the opened test can be provided to test takers through a platform implemented in the computing environment.

[0080] Meanwhile, according to one embodiment of the present disclosure, when the computing device (100) includes an input / output unit or is a client, it may display a first user interface area in which a recommended rule set generated from a rule database based on input data regarding the properties of a test intended by the user is exposed. In addition, the computing device (100) may display a second user interface area in which a user input for customizing the rule set is received. The user may perform operations such as selecting, modifying, or changing on the first user interface area or the second user interface area. The computing device (100) may process data based on the input according to the corresponding operation or transmit a data processing request through another server.

[0081] The various embodiments of the present disclosure described above can be combined with additional embodiments and modified within the scope understood by those skilled in the art in light of the detailed description above. It should be understood that the embodiments of the present disclosure are illustrative in all respects and not restrictive. For example, each component described as a single component may be implemented in a distributed manner, and likewise, components described as distributed may be implemented in a combined manner. Accordingly, all changes or modifications derived from the meaning, scope, and equivalent concepts of the claims of the present disclosure should be construed as being included within the scope of the present disclosure.

Claims

1. A test generation method performed by a computing device including at least one processor, A step of obtaining input data regarding the properties of a test intended by the user; A step of determining detection conditions for monitoring the behavior of test takers based on user input; and A step of opening a test based on the above-determined detection conditions; including, method.

2. In paragraph 1, The properties of the above test are: Including at least one of the following: exam type, open book, searchability, or exam cycle; method.

3. In paragraph 1, The step of determining detection conditions for monitoring the behavior of test takers based on the above user input is: A step of determining detection conditions based on user input through a first user interface where a recommended ruleset is exposed; including, method.

4. In paragraph 3, The step of determining detection conditions based on user input through the first user interface where the above recommended ruleset is exposed is: A step of generating a recommended rule set from a rule database based on input data regarding the properties of the above test; and A step of determining a detection condition based on the selected rule set when one of the recommended rule sets or presets exposed through the first user interface is selected based on user input; including, method.

5. In paragraph 4, The above rule database is, It is updated based on the results of analyzing big data accumulated through the opening and operation of the test. method.

6. In paragraph 5, The above big data is, Including at least one of data on detection conditions adjusted by customizing the user's ruleset during the process of opening a test or data on events detected based on rules during the process of running an opened test. method.

7. In paragraph 5, The above update is, Generating a rule to be included in the above rule database or dynamically changing the details of a rule that constitutes a rule set included in the above rule database. method.

8. In paragraph 3, When one of the recommended rulesets or presets exposed through the first user interface is selected according to the user input, the step of determining the detection condition based on the selected ruleset is as follows: A step of adjusting the intensity of a detection condition according to a user input for controlling the level of the above-mentioned selected ruleset; Including, The level of the above ruleset is Distinguished by the numerical value of the detailed items of the rules that constitute the above ruleset, method.

9. In paragraph 1, The step of determining detection conditions for monitoring the behavior of test takers based on the above user input is: A step of determining detection conditions based on user input via a second user interface for customizing the ruleset; including, method.

10. In paragraph 9, The step of determining detection conditions based on user input through a second user interface for customizing the above ruleset is: A step of determining a detection condition based on a ruleset including the set or adjusted rule when at least one rule included in the ruleset is set or adjusted according to user input through the second user interface; including, method.

11. In paragraph 10, When at least one rule included in the ruleset is set or adjusted based on user input through the second user interface, the step of determining the detection condition based on the ruleset including the set or adjusted rule is as follows: When setting or adjusting at least one rule is performed, a step of analyzing detailed items determined to be suitable for the set or adjusted rule based on big data of a test to which rules corresponding to the set or adjusted rule are applied; and Based on the results of the above analysis, a step of recommending detailed items deemed suitable in the above set or adjusted rules; including, method.

12. In paragraph 11, When setting or adjusting at least one rule is performed, the step of analyzing the detailed items that are determined to be suitable for the set or adjusted rule based on the big data of the test to which the rules corresponding to the set or adjusted rule are applied is as follows: A step of analyzing the weights of the rules based on the big data of the test to which the rules corresponding to the above-mentioned set or adjusted rules are applied; and A step of determining the detailed items of the rule selected based on the analyzed weight as detailed items suitable for the set or adjusted rule; including, method.

13. In paragraph 12, Big data that influences the analysis of the above weights are: At least one of the following: the rate at which a result detected according to a particular rule is delivered to the test taker, or the number of times the particular rule is activated by the test taker's action satisfying a subitem of the particular rule; method.

14. A test generation method performed by a computing device including at least one processor, A step of displaying a first user interface area in which a recommended ruleset generated from a rule database based on input data regarding the properties of a test intended by the user is exposed; and A step of displaying a second user interface area for receiving user input for customizing the ruleset; including, method.

15. A computer program stored on a computer-readable storage medium, wherein the computer program, when executed on one or more processors, causes the computer program to perform operations for generating a test. The above actions are, An action to obtain input data regarding the properties of a test intended by the user; An action to determine detection conditions for monitoring the behavior of test takers based on user input; and An action to open a test based on the detection conditions determined above; including, Computer program.

16. As a computing device for generating tests, A processor comprising at least one core; A memory containing program codes executable by the processor; and A network unit for obtaining input data regarding the properties of a test intended by the user; Including, The above processor, Determine detection conditions for monitoring the behavior of test takers based on user input, Opening a test based on the detection conditions determined above, device.

Images