Examination monitoring system and examination monitoring method

The test monitoring system uses camera-based image analysis to automate the detection of fraudulent behaviors in large-scale testing environments, reducing monitor workload and improving detection accuracy by analyzing upper limb and eye movements, heart rate, and prohibited items.

JP2024158466AActive Publication Date: 2024-11-08EARTH EYES CO LTD +1

Patent Information

Application Number
JP2023073687
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-04-27
Publication Date
2024-11-08
Estimated Expiration
2043-04-27

AI Technical Summary

Technical Problem

In large-scale test environments, detecting cheating among test takers is challenging due to the complexity of movements and the difficulty in accurately distinguishing fraudulent behaviors from normal actions, leading to a high workload for test monitors.

Method used

A test monitoring system that utilizes cameras and image analysis to detect fraudulent behaviors by analyzing upper limb and eye movements, heart rate, and the presence of prohibited items, with integrated image recognition and fraud detection units to automate the detection process.

Benefits of technology

The system effectively reduces the workload of test monitors by accurately detecting cheating and preventing prohibited items, while minimizing the need for multiple supervisors, enhancing detection accuracy through combined analysis of multiple factors.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024158466000001_ABST
    Figure 2024158466000001_ABST
Patent Text Reader

Abstract

To provide examination monitoring means capable of automatically detecting a dishonest act at high detection accuracy while mitigating a workload on an examination monitoring person at an examination site at which many examinees gather and take an examination.SOLUTION: An examination monitoring system 1 comprises: an imaging unit 11 for imaging an examinee 3; an operation recognition unit 121 for recognizing operation of an upper limb 31 of the examinee 3 from a monitoring picture 110 imaged by the imaging unit 11; a visual line detection unit 122 for detecting movement of a visual line 32 of the examinee 3 from the monitoring picture 110; a dishonest act detection unit 127 for detecting a dishonest act of the examinee 3; and a monitoring information output unit 13 for outputting monitoring information in a format that can be recognized by an examination monitoring person, when the dishonest act is detected by the dishonest act detection unit 127. The dishonest act detection unit 127 takes, as an input value, information consisting of a combination of the operation of the upper limb 31 and the movement of the visual line 32 and compares this input value with a threshold for dishonest act determination to detect the dishonest act of the examinee 3.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to an exam monitoring system and an exam monitoring method, and more particularly to a technology for detecting exam misconduct (cheating) by exam examinees in an exam center where many examinees gather together to take an exam, and for assisting the exam monitor in the exam monitoring work. [Background technology]

[0002] For example, in exam halls where many examinees gather to take exams such as various qualification exams and entrance exams, exam monitors visually monitor each examinee to see if they are engaging in dishonest behavior (cheating, etc.). Types of dishonest behavior generally anticipated in exam monitor work include bringing in prohibited items without being detected by the exam monitor, and writing information related to exam answers on the palm of the hand, arm, etc.

[0003] At examination centers where a certain number of test candidates gather together to take the exam, it becomes difficult for a single test supervisor to perform the test monitoring task of detecting and preventing such cheating, i.e., to completely visually monitor all the test candidates, and therefore, in order to reliably carry out the monitoring task, it becomes necessary to increase the number of test supervisors.

[0004] As an example of a technology to reduce the burden of test monitors at test venues, development is underway of a cheating detection means (see Patent Document 1) that automatically detects test takers' movements using wearable sensors worn by the test takers and detects cheating by test takers.

[0005] If it were possible to accurately detect cheating using sensing technology, as in the above-mentioned method, the workload of exam monitoring could be significantly reduced. However, the patterns of examinee behavior are complex, and it has not yet been easy to automatically and accurately extract only the movements related to cheating, such as those exemplified above. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2016-95817 A Summary of the Invention [Problem to be solved by the invention]

[0007] In view of the above-mentioned circumstances, the present invention has an objective of providing a test monitoring means that can automatically detect cheating with high detection accuracy while reducing the workload of test monitors at test centers where a large number of examinees gather to take a test, or while limiting the number of test monitors. [Means for solving the problem]

[0008] The inventors of the present invention came up with the idea that by focusing on the various behaviors of examinees, particularly the movements of the upper limbs and the movements of the eyes, and extracting and analyzing these from monitoring images, it is possible to automatically detect examinees' cheating behavior efficiently and with high detection accuracy, and thus completed the present invention. Specifically, the present invention provides the following test monitoring means.

[0009] (1) A test monitoring system for detecting cheating by test takers at a test center, comprising: a photographing unit that photographs the test taker; a movement recognition unit that recognizes the movement of the test taker's upper limbs from the monitoring image photographed by the photographing unit; a gaze detection unit that detects the movement of the test taker's gaze from the monitoring image; a cheating detection unit that detects the test taker's cheating; and a monitoring information output unit that outputs monitoring information related to the cheating in a format recognizable by a test monitor, wherein the cheating detection unit receives information consisting of a combination of the movement of the upper limbs and the movement of the gaze as an input value and detects the test taker's cheating by comparing this input value with a threshold value for determining cheating.

[0010] The test monitoring system of (1) can automatically detect cheating with a high degree of detection accuracy while reducing the workload of test monitors at test centers where a large number of test takers gather to take the test, or while limiting the number of test monitors.

[0011] (2) The exam monitoring system described in (1) further includes a heart rate detection unit that detects the examinee's heart rate from the monitoring image, and the misconduct detection unit uses information consisting of a combination of the upper limb movement, the eye movement, and the heart rate as an input value, and detects the examinee's misconduct by comparing this input value with the threshold value.

[0012] According to the test monitoring system of (2), the accuracy of detecting cheating in the test monitoring system of (1) can be further improved by using biometric information, which tends to show unique fluctuations in test-takers who engage in cheating, as information for detecting cheating.

[0013] (3) An exam monitoring system as described in (1) or (2), further comprising a prohibited object detection unit that detects prohibited objects present in the vicinity of the examinee from the surveillance image, and when the prohibited object is detected, the surveillance information output unit outputs a notification indicating the occurrence of cheating.

[0014] (3) The exam monitoring system can automatically and quickly detect the act of bringing in prohibited items without being detected by exam monitors, which is one of the most common types of cheating in exam centers where a large number of examinees gather together to take the exam.

[0015] (4) An exam monitoring system as described in (1) or (2), further comprising a seating / leaving action verification unit that detects the examinee's actions of leaving and sitting down from the monitoring image, wherein the seating / leaving action verification unit generates an image for verifying the seating / leaving action consisting of a combination of a specific seating / leaving image, which is an image of the examinee leaving his / her seat from his / her examinee seat, and a specific seating image, which is an image of the examinee's first sitting down action after the examinee leaves his / her seat, and the monitoring information output unit outputs the image for verifying the seating / leaving action.

[0016] (4) The exam monitoring system enables exam monitors to more accurately detect what is known as proxy exam cheating, which is a particularly serious type of cheating that occurs in exam halls where a large number of examinees gather together to take exams. This is the fraudulent substitution of examinees.

[0017] (5) The exam monitoring system described in (4) further includes a facial recognition information acquisition unit that acquires the examinee's unique facial recognition information from the surveillance image, and if the facial recognition information of the examinee who has left his / her seat and the examinee who is seated in the image for verifying leaving and sitting behavior does not match, the surveillance information output unit outputs a notification indicating the occurrence of fraudulent behavior.

[0018] According to the exam monitoring system of (5), in the exam monitoring system described in (4), the above-mentioned proxy taking of an exam can be automatically detected with higher detection accuracy.

[0019] (6) An exam monitoring system as described in (1) or (2), wherein the imaging unit is configured to include a table-mounted camera that is installed individually at each seat where each examinee answers questions and is capable of photographing the upper body of each examinee from the front, and a ceiling camera that is capable of photographing an overhead view of the entire monitored area of ​​the exam venue.

[0020] According to the exam monitoring system (6), in the exam monitoring system (1) or (2), the imaging unit that captures the surveillance images is configured by combining multiple cameras that capture the upper body movements of the upper limbs, face, etc. of each examinee from the front, and a camera that can capture images of all examinees from above. This reduces the risk of misconduct being missed due to blind spots in the surveillance images, and makes it possible to detect misconduct with even higher detection accuracy.

[0021] (7) An exam monitoring system as described in (6), wherein the table-mounted camera is a PC-built-in camera built into a personal computer for each examinee, which is installed individually at each seat where each examinee answers questions.

[0022] According to the exam monitoring system of (7), in an exam in which examinees answer questions using a personal computer dedicated to answering questions, it is possible to reliably capture the examinees' facial and upper limb movements while eliminating the expense of installing a separate dedicated monitoring camera, thereby achieving the above-mentioned effects of the exam monitoring system described in (6) with high accuracy.

[0023] (8) A test monitoring method for detecting misconduct by an examinee at a test center, comprising: a monitoring image capturing step in which a capturing unit captures an image of the examinee; a movement recognition step in which a movement recognition unit recognizes the movement of the examinee's upper limbs from the monitoring image captured in the monitoring image capturing step; a gaze detection step in which a gaze detection unit detects the gaze movement of the examinee from the monitoring image; a misconduct detection step in which a misconduct detection unit detects the examinee's misconduct; and a monitoring information output step in which a monitoring information output unit outputs monitoring information related to the misconduct in a format recognizable by a test monitor, wherein in the misconduct detection step, information consisting of a combination of the movement of the upper limbs and the gaze movement is used as an input value, and the misconduct is detected by comparing this input value with a threshold value for determining misconduct.

[0024] According to the test monitoring method (8), it is possible to automatically detect cheating with a high degree of detection accuracy while reducing the workload of test monitors at a test center where a large number of test takers gather to take the test.

[0025] (9) An exam monitoring method as described in (8), further comprising a heart rate detection step in which a heart rate detection unit detects the heart rate of the examinee from the monitoring image, and in the misconduct detection step, information consisting of a combination of the upper limb movement, the eye movement, and the heart rate is used as an input value, and the misconduct is detected by comparing this input value with a threshold value for determining misconduct.

[0026] According to the test monitoring method of (9), the accuracy of detecting cheating in the test monitoring method of (8) can be further improved by using biometric information, which tends to show unique fluctuations in test-takers who engage in cheating, as information for detecting cheating.

[0027] (10) An exam monitoring method as described in (8) or (9), further comprising a prohibited object detection step in which a prohibited object detection unit detects a prohibited object present in the vicinity of the examinee from the surveillance image, and if the prohibited object is detected, a notification indicating the occurrence of cheating is output in the surveillance information output step.

[0028] According to the examination monitoring method of (10), the bringing in of prohibited items, which is one of the most common types of cheating in examination centers where a large number of examinees gather together to take an examination, can be detected automatically and quickly.

[0029] (11) An exam monitoring method as described in (8) or (9), further comprising a seating / leaving verification step in which the seating / leaving verification unit detects the examinee's actions of leaving and sitting down from the surveillance image, in which an image for verifying seating / leaving behavior is generated that is a combination of a specific seating / leaving image, which is an image of the examinee's action of leaving their seat from their examinee's seat, and a specific seating image, which is an image of the examinee's first action of sitting down on their seat after the act of leaving their seat, and in which a surveillance information output step, the image for verifying seating / leaving behavior is output.

[0030] According to the test monitoring method of (11), the test monitor can more accurately detect the so-called proxy test-taking behavior, which is a particularly serious type of cheating that occurs in a test center where a large number of test takers gather together to take the test, in which a test taker illegally switches places with another test taker.

[0031] (12) The exam monitoring method described in (11) further includes a facial recognition information acquisition step in which a facial recognition information acquisition unit acquires the examinee's unique facial recognition information from the surveillance image, and if the facial recognition information of the person who has left their seat and the person who is seated in the image for verifying leaving and sitting behavior does not match, a notification indicating the occurrence of fraudulent behavior is output in the surveillance information output step.

[0032] According to the test monitoring method of (12), in the test monitoring system described in (11), the above-mentioned proxy taking of an exam can be automatically detected with higher detection accuracy.

[0033] (13) A test monitoring method as described in (8) or (9), wherein the photographing unit includes a table-mounted camera that is installed individually at each seat where each test taker answers questions and is capable of photographing the upper body of each test taker from the front, and a ceiling camera that is capable of photographing an overhead view of the entire monitored area of ​​the test venue, and in the monitoring image photographing step, the table-mounted camera photographs the face and upper body of each test taker.

[0034] According to the test monitoring method of (13), in the test monitoring method of (8) or (9), the imaging unit that captures the surveillance images is configured by combining a plurality of cameras that capture the movements of the upper body, such as the upper limbs and face, of each examinee from the front, and a camera that can capture images of all examinees from above. This reduces the risk of misconduct being missed due to the presence of blind spots in the surveillance images, and makes it possible to detect misconduct with even higher detection accuracy.

[0035] (14) An exam monitoring method as described in (13), wherein the table-mounted camera is a PC-built-in camera built into a personal computer for each examinee, which is installed individually at each seat where each examinee answers questions.

[0036] According to the test monitoring method of (14), in a test in which the examinee answers questions using a personal computer dedicated to answering questions, it is possible to reliably capture the examinee's face and upper limb movements while eliminating the expense of separately installing a dedicated monitoring camera, thereby achieving the above-mentioned effects of the test monitoring method described in (13) with high accuracy.

[0037] (15) In the exam monitoring method described in (8), a program for exam monitoring for detecting cheating by examinees at an exam venue, which causes an information processing device to execute the action recognition step, the gaze detection step, the cheating detection step, and the monitoring information output step.

[0038] The test monitoring program of (15) can automatically detect cheating with a high degree of accuracy while reducing the workload of test monitors at test centers where a large number of test takers gather to take the test. Effect of the Invention

[0039] According to the present invention, it is possible to provide a test monitoring means that can automatically detect cheating with high detection accuracy while reducing the workload of test monitors at a test center where a large number of test takers gather to take the test, or while limiting the number of test monitors. [Brief description of the drawings]

[0040] [Figure 1] 1 is a block diagram illustrating a basic configuration of a test monitoring system according to the present invention. [Diagram 2] 3 is a flowchart showing the operation flow of the test monitoring method of the present invention. [Diagram 3] 1 is a diagram showing a schematic view of an exam hall in which an exam monitoring system according to the present invention is installed; [Figure 4] 1 is a diagram showing a schematic example of a monitoring image in the test monitoring system of the present invention; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0041] Hereinafter, the best mode for carrying out the present invention will be described with reference to the drawings as appropriate.

[0042] <Test monitoring system> The "examination monitoring system" of the present invention is a system that can automatically detect dishonest behavior (cheating) by examinees at an examination venue from monitored images.

[0043] [Overall configuration] As shown in FIG. 1, the exam monitoring system 1 is composed of an imaging unit 11 that photographs the examinees 3 (3A, 3B, 3C, ... 3Z) and creates monitoring images 110, a calculation processing unit 12 that performs calculations to detect cheating by the examinees 3 (3A, 3B, 3C, ... 3Z) by analyzing the monitoring images 110, and a monitoring information output unit 13 that outputs monitoring information related to the cheating detected by the calculation processing unit 12 in a format recognizable by the exam monitor 5.

[0044] The above-mentioned arithmetic processing unit 12 constituting the test monitoring system 1 includes at least a motion recognition unit 121, a gaze detection unit 122, and a misconduct detection unit 127. More preferably, the arithmetic processing unit 12 further includes a heart rate detection unit 123, a prohibited item detection unit 124, a seating / leaving behavior verification unit 125, and a face authentication information acquisition unit 126 in addition to the above three components.

[0045] [Photography Department] The photographing unit 11 is a so-called surveillance camera that photographs the examinee 3 to be monitored. The photographing unit 11 can be configured with various types of conventionally known surveillance cameras that can continuously photograph the examinee 3 to be monitored. Note that the photographing unit 11 can use various existing digital cameras without any particular restrictions as long as it has a function of outputting the photographed surveillance image 110 as image data in a digital format so that it can be processed by the arithmetic processing unit 12.

[0046] As shown in Figure 3, when the exam monitoring system 1 is configured in an exam venue 2 where a number of examinee seats 4 (4A, 4B, 4C, ... 4Z) are arranged where each examinee 3 (3A, 3B, 3C, ... 3Z) answers their questions, the imaging unit 11 is installed in a position where it can capture at least the movement of the upper limbs 31 and the movement of the gaze 32 of each examinee 3 (3A, 3B, 3C, ... 3Z), as shown in the same figure.

[0047] More specifically, it is preferable that the photographing unit 11 is configured by combining a table-top camera 111 that is installed individually at each examinee's seat 4, and a ceiling camera 112 that can photograph the entire monitoring area in the examination venue 2 (the entire seating area of ​​all examinees 3 (3A, 3B, 3C, ... 3Z)) from an overhead view.

[0048] The table-top cameras 111 are individually installed at each examinee's seat 4 where each examinee 3 (3A, 3B, 3C, . . . 3Z) answers the questions, so that the face and upper limbs (upper body) of each examinee 3 (3A, 3B, 3C, . . . 3Z) can be photographed from the front (see Figure 4(A)).

[0049] In addition, if an information processing terminal for examinees consisting of a laptop personal computer or the like is pre-arranged for each examinee's seat 4 (4A, 4B, 4C, ... 4Z), it is preferable to have a PC built-in camera in the information processing terminal capable of capturing images in front of the information display screen function as a table-mounted camera 111.

[0050] The ceiling camera 112 is configured by a camera installed at a fixed position capable of capturing an overhead image of the entire space set as the monitored area in the examination hall 2. The ceiling camera 112 may be configured by multiple cameras in order to cover the entire monitored area as its capture range.

[0051] Moreover, it is more preferable that the ceiling camera 112 is a camera with pan-tilt-zoom function (PTZ camera) that can change the installation position and shooting area as needed. If the ceiling camera 112 is configured as a PTZ camera, it is more preferable that the camera has an automatic zoom control mode that can zoom in on the periphery of any examinee's seat 4 as needed and automatically zooms in on the examinee's seat 4 (e.g. 4A) of an examinee 3 (e.g. 3A) for which information related to the occurrence of fraudulent activity has been detected.

[0052] In this way, by combining the tabletop camera 111 arranged close to the front of the upper body of each examinee 3 seated in the examinee's seat 4 with the ceiling camera 112 arranged on the ceiling side of the examination hall 2, i.e., above the examinee 3, at a distance, to form the photographing unit 11, the former tabletop camera 111 can photograph in detail the movement of the upper limbs 31 and the line of sight 32 of the examinee 3, which are important materials for determining whether or not cheating has occurred (see FIG. 4(A)). On the other hand, the latter ceiling camera 112 can capture a bird's-eye view of prohibited items 33 that are in the blind spot of the tabletop camera 111 and the direction of the movement path of the examinee 3 who has left the examinee's seat 4 without omission (see FIG. 4(B)).

[0053] [Calculation processing unit] The arithmetic processing unit 12 is configured by an information processing device that performs arithmetic processing to detect cheating by the examinees 3 (3A, 3B, 3C, . . . 3Z) by analyzing image data of the monitoring image 110 transmitted from the photographing unit 11. The arithmetic processing unit 12 can be configured by using various information processing devices such as general-purpose computers, personal computers, tablet terminals, smartphones, etc., and programs that operate these devices. Alternatively, the arithmetic processing unit 12 can be configured by a dedicated information processing device specialized for test monitoring operations. In any of these configurations, the arithmetic processing unit 12 includes hardware such as a CPU, memory, and communication means. The arithmetic processing unit 12 configured in this way can specifically execute various operations of the test monitoring system and the test monitoring method described below by executing a computer program (such as a program for test monitoring of the present invention).

[0054] Specifically, as described above, the arithmetic processing unit 12 at least includes a motion recognition unit 121, a line of sight detection unit 122, and an improper behavior detection unit 127, and preferably further includes a heart rate detection unit 123, a prohibited item detection unit 124, a seating / leaving behavior verification unit 125, and a face authentication information acquisition unit 126. The arithmetic processing unit 12 is also connected to the photographing unit 11 and the surveillance information output unit 13 via a communication network so as to be able to transmit and receive data to and from each other.

[0055] The information processing device functioning as the arithmetic processing unit 12 may be placed in the examination venue 2 or in a nearby area, or the arithmetic processing unit 12 may be placed in a management facility in a remote location away from the examination venue 2 via a network capable of transmitting and receiving the above data. Alternatively, the exam monitoring system 1 may be configured by integrating the functions of various information processing devices distributed on the cloud as the arithmetic processing unit 12.

[0056] Furthermore, the overall configuration of the test monitoring system 1 is not limited to a configuration in which the above-mentioned sections (the photographing section 11, the arithmetic processing section 12, and the monitoring information output section 13) are arranged as separate devices, so long as each of the components as a whole can fulfill the function of the test monitoring system 1. For example, a part of the arithmetic processing section 12 can be mounted within the photographing section 11, or the test monitoring system 1 can be configured as an information processing device in which the arithmetic processing section 12 and the monitoring information output section 13 are integrated.

[0057] (motion recognition unit) The movement recognition unit 121 recognizes the movement of the examinee 3 in the monitoring image 110, particularly the movement of the upper limbs 31, by image analysis processing. Such movement recognition can be performed by various conventionally known image analysis methods, but the image analysis processing by the movement recognition unit 121 can be configured, for example, by using various machine learning type image analysis devices having neural networks (so-called "image recognition devices using deep learning technology"). Specific examples of image recognition technology using deep learning are disclosed below. "Deep Learning, Image Recognition, and Operations Research" (http: / / www.orsj.o.jp / archive2 / or60-4 / or60_4_198.pdf)

[0058] Furthermore, in order for the movement recognition unit 121 to automatically recognize the movement of the examinee 3, particularly the details of the movement of the upper limbs 31, from the monitoring image 110, a known image analysis technique called "OpenPose" can be used, for example. As disclosed by the inventor of the present application in Japanese Patent No. 6534499, by using "OpenPose", skeletal information of the examinee 3 is extracted from the monitoring image 110, and the position and speed of each feature point constituting the skeleton are analyzed, thereby making it possible to recognize the movement of the upper limbs 31 of the examinee 3 in detail.

[0059] In addition, by using image analysis technology such as "OpenPose", it is possible to recognize the movement of the upper limbs 31 of the examinee 3 and the movement of the separately detected "prohibited object 33" from the surveillance image 110 as independent information. By using such a function, it is possible to automatically detect the movement of the upper limbs 31 of the examinee 3 holding the "prohibited object 33" and the movement of the upper limbs 31 of the examinee 3 not holding the "prohibited object 33" by analyzing each of the movements of the upper limbs 31 and the "prohibited object 33" and the difference between the movements of the two. By recognizing the "movement of the examinee 3 moving the upper limbs 31 not holding the "prohibited object 33"" separately from the movement of the upper limbs 31 holding the "prohibited object 33", it is possible to reduce false positives of fraudulent acts.

[0060] It is more preferable that the movement recognition unit 121 can detect various suspicious movements other than the movements of the upper limbs 31 of the examinee 3 as "unauthorized preparatory movements" from the surveillance image 110. Here, the above-mentioned "unauthorized preparatory movements" in this specification refer to "specific movements" that are set in advance by the monitoring entity using the exam monitoring system 1 as movements that are often made prior to the examinee 3 committing a fraudulent act when the examinee 3 commits the fraudulent act. As an example, as disclosed by the inventor of the present application in Japanese Patent No. 6534499, such "specific movements" can be identified by dividing the examinee 3 in the surveillance image 110 into multiple regions (head, torso, and upper limbs) and extracting features related to the positional fluctuations of each part of the person for each of these regions. One example of a typical "unauthorized preparatory movement" by the examinee 3 is "the movement of looking around the surroundings multiple times prior to the fraudulent act."

[0061] (Gaze detection section) The gaze detection unit 122 detects the gaze movement of the examinee 3 from the monitoring image 110. Specifically, the gaze detection unit 122 can be implemented by, for example, a technical means disclosed in JP 2018-148402 A, that is, a gaze detection unit that detects the "whole head of a person" and the "position of the eyes and nose" from an image of the examinee, obtains the face direction from the "positional relationship" between them, and regards this as the gaze movement. In addition, the gaze detection unit 122 can be configured by various conventionally known gaze detection devices, programs for gaze detection applications, and the like.

[0062] In addition, in the calculation processing unit 12 that constitutes the exam monitoring system 1, by linking the operations of the action recognition unit 121 and the gaze detection unit 122, it is possible to recognize, for example, a combination of an action by examinee 3 lowering his / her hand below the work table of the examinee's seat 4 and an action of directing his / her gaze in the direction of the hand as actions that are highly likely to be cheating.

[0063] (Heart rate detection section) The heart rate detection unit 123 detects the heart rate of the examinee 3 from the monitoring image 110. Specifically, the heart rate detection unit 123 can perform image analysis using known techniques such as imaging photoplethysmography (iPPG) and remote photoplethysmography (rPPG). All of these techniques estimate the heart rate by extracting the amount of reflected light captured by a camera and changes in RGB signals. In the exam monitoring system 1, the accuracy of detecting cheating can be significantly improved by detecting the movement of the upper limbs 31 and the movement of the line of sight 32 of the examinee 3 in combination with changes in the heart rate.

[0064] (Prohibited items detection unit) The prohibited object detection unit 124 detects prohibited objects 33 present around the examinee 3 from the surveillance image 110. Detection of such objects can also be performed using the above-mentioned "image recognition device using deep learning technology." The prohibited object detection unit 124 can detect prohibited objects 33 (such as a smartphone or a wristwatch-type information processing terminal, or a piece of paper with letters or symbols on it) that have been specified in advance from the surveillance image 110 by using the "image recognition device using deep learning technology" in the same manner as above.

[0065] In addition, in the calculation processing unit 12 that constitutes the exam monitoring system 1, by linking the operations of the action recognition unit 121 and the gaze detection unit 122 with the prohibited item detection unit 124, it is also possible to recognize in detail the actions of the examinee 3 in relation to the prohibited item 33 detected by the prohibited item detection unit 124 (such as holding the prohibited item 33 with the upper limbs 31, directing the gaze 32 toward the prohibited item 33, etc.).

[0066] (Seating and Leaving Behavior Verification Division) In addition to the above-mentioned cheating, a more serious form of fraudulent behavior by examinees at a test center where many examinees gather to take a test is the so-called substitute test-taking behavior, in which an examinee leaves or sits down during the test and is replaced by another examinee. The leaving / sitting behavior verification unit 125 detects the leaving and sitting behavior of examinee 3 from the surveillance image 110, and generates an "image for verifying leaving / sitting behavior" to verify whether or not there is any fraudulent behavior (substitute test-taking behavior) associated with the leaving / sitting behavior. The "image for verifying leaving / sitting behavior" is an image that is a combination of a "specific leaving image" that is an image of examinee 3 leaving his / her seat from examinee's seat 4, and a "specific sitting image" that is an image of the first sitting down behavior after leaving his / her seat to examinee's seat 4. The image format is not particularly limited as long as it is image information that allows the test supervisor 5 to visually recognize the identity of the above-mentioned leaver and the seated person.

[0067] The leaving and sitting behavior verification unit 125 also requires processing to recognize the examinee 3 in the surveillance image 110 and recognize the movements related to leaving and sitting, and the use of the image recognition technology using the above-mentioned deep learning is an effective technical means for performing such processing automatically.

[0068] (Facial recognition information acquisition unit) The face authentication information acquisition unit 126 acquires unique face authentication information 35 of the examinee 3 from the surveillance image 110. The face authentication information acquisition unit 126 can be configured by various conventionally known face authentication information acquisition devices, programs for face authentication applications, etc. The exam monitoring system 1 equipped with the face authentication information acquisition unit 126 acquires face authentication information as unique biometric identification information of the examinee 3 to be monitored, and stores this in a storage means, so that the face authentication information can be repeatedly referred to as necessary during operation of the exam monitoring system 1 to confirm the identity of the examinee 3. This makes it possible to identify each examinee 3 with high accuracy and with very few false recognitions, even when there are a large number of examinees 3.

[0069] Furthermore, by further equipping the test monitoring system 1 with a facial recognition information acquisition unit 126, it becomes possible to automatically determine with high accuracy whether the person who is leaving their seat and the person who is sitting in the "image for verifying leaving and sitting behavior" are the same person.

[0070] (Fraud Detection Department) The misconduct detection unit 127 detects misconduct in the behavior of the examinee 3 from the monitoring image 110. When the calculation processing unit 12 constituting the exam monitoring system 1 has a minimum configuration consisting of the movement recognition unit 121, the gaze detection unit 122, and the misconduct detection unit 127, the misconduct detection unit 127 receives as an input value information consisting of a combination of the movement of the upper limbs 31 and the gaze 32 of the examinee 3, and detects misconduct by the examinee 3 by comparing this input value with a preset threshold value for determining misconduct.

[0071] If the calculation processing unit 12 constituting the exam monitoring system 1 further comprises a heart rate detection unit 123, a prohibited item detection unit 124, and a seating / leaving behavior verification unit 125, the information related to the examinee 3 detected by each of these units is used as an input value, and the degree of inappropriateness of the behavior of the examinee 3 is judged by comparing these input values ​​with predetermined thresholds. The misconduct detection unit 127 which performs such an operation can be composed of various information processing devices capable of performing information matching analysis and outputting the results.

[0072] [Monitoring information output section] The monitoring information output unit 13 is a device that outputs monitoring information related to fraudulent activities detected by the calculation processing unit 12 in a format recognizable by the exam monitor 5. The monitoring information output unit 13 constituting the exam monitoring system 1 can be placed as a tabletop personal computer on the exam monitor's seat 6 of the exam monitor 5 in the exam venue 2, or a small information processing terminal carried by the exam monitor 5 can be used as the monitoring information output unit 13.

[0073] <Test monitoring method> The "exam monitoring method" of the present invention is an exam monitoring method for detecting cheating by examinees in an exam center, and is a process that can be performed using the exam monitoring system 1 described above.

[0074] 2 is a flowchart showing an example of the flow of the test monitoring method of the present invention that can be executed using the test monitoring system 1. The test monitoring method of the present invention is a process including at least a monitoring image capturing step st1, a motion recognition step st2, a gaze detection step st4, a fraudulent behavior detection step st8, and a monitoring information output step st9. The prohibited item detection step st3, the heart rate detection step st5, the seating / leaving behavior verification step st6, and the face authentication information acquisition step st7 are not necessarily essential components, but a more preferred embodiment can be achieved by making the process include these optional components.

[0075] Furthermore, the process flow in Figure 2 is one example of the flow of the test monitoring method of the present invention, and as long as it is a flow that can utilize each function of the test monitoring system 1 to detect fraudulent behavior, the order in which each step is executed can be changed, and for example, with regard to the action recognition step st2, the prohibited item detection step st3, the gaze detection step st4, and the heart rate detection step st5, each process can be performed simultaneously in parallel, and all detection results can be input into the fraudulent behavior detection step st8.

[0076] Below, the exam monitoring method of the present invention will be described in detail with respect to an embodiment in which the exam monitoring method is carried out using an exam monitoring system 1 in which an arithmetic processing unit 12 that performs arithmetic processing to detect cheating by an examinee 3 by analyzing a surveillance image 110 is configured to include a motion recognition unit 121, a gaze detection unit 122, a heart rate detection unit 123, a prohibited item detection unit 124, a seating / leaving behavior verification unit 125, a face authentication information acquisition unit 126, and a cheating detection unit 127, i.e., a process including a surveillance image capture step st1, a motion recognition step st2, a prohibited item detection step st3, a gaze detection step st4, a heart rate detection step st5, a seating / leaving behavior verification step st6, a face authentication information acquisition step st7, a cheating detection step st8, and a surveillance information output step st9.

[0077] (Surveillance image capture step) In surveillance image capturing step st1, examinees 3 are captured by the capturing unit 11 (see Figs. 4(A) and (B)). Fig. 4(A) is a diagram showing a typical example of a surveillance image 110A captured by a tabletop camera 111 constituting the capturing unit 11, in which the face and upper limbs 31 (upper body) of each examinee 3 are captured from the front. Meanwhile, Fig. 4(B) is a diagram showing a typical example of a surveillance image 110B captured by a ceiling camera 112 also constituting the capturing unit 11, in which a prohibited item 33 present in the blind spot of the tabletop camera 111 is captured in an overhead image.

[0078] The surveillance image 110 (110A, 110B) captured in the surveillance image capturing step st1 is output as image data in a digital format and input to the arithmetic processing unit 12, where it is subjected to a predetermined arithmetic processing.

[0079] (Motion Recognition Step) In the movement recognition step st2, the movement recognition unit 121 performs image analysis processing to recognize the movement of the upper limbs 31 of the examinee 3 from the monitoring image 110 captured in the monitoring image capturing step st1. This movement recognition can be performed by various known image analysis methods, as described above.

[0080] Furthermore, in the movement recognition step st2, suspicious movements other than those of the upper limbs 31 of the examinee 3 (for example, frequent head shaking, etc.) may be detected from the surveillance image 110 as preparatory movements for fraud, and this detection result may be used to determine whether or not the movement is fraudulent.

[0081] In addition to recognizing the movement of the upper limbs 31 of the examinee 3, in the action recognition step st2, if the examinee 3 leaves his / her seat, a process of detecting this is also performed. In the action recognition step st2, if the action of the examinee 3 leaving his / her seat is detected (leaving his / her seat, YES), the process proceeds to the leaving / taking-your-seat action verification step st6. If the action of the examinee 3 leaving his / her seat is not detected (leaving his / her seat, NO), the process proceeds to the prohibited item detection step st3.

[0082] (Prohibited items detection step) In prohibited object detection step st3, the prohibited object detection unit 124 detects prohibited objects 33 present around the examinee 3 from the surveillance image 110. If a prohibited object 33 is detected in prohibited object detection step st3 (unauthorized bringing in, YES), it is preferable that a notification indicating the occurrence of unauthorized conduct (unauthorized bringing in) is immediately output by the surveillance information output unit 13 in surveillance information output step st9. If a prohibited object 33 is not detected (unauthorized bringing in, NO), the process proceeds to line of sight detection step st4.

[0083] (Gaze detection step) In gaze detection step st4, the gaze detection unit 122 detects the gaze movement of the examinee 3 from the monitoring image 110. When detecting the change in the gaze movement, a minimum threshold value of the gaze movement change to be detected may be predefined based on a combination of the gaze movement change angle and the number of changes within a unit time, and the gaze movement may be detected only when the gaze movement change exceeds this threshold value.

[0084] (Heart rate detection step) In the heart rate detection step st5, the heart rate detection unit 123 detects the heart rate of the examinee 3 from the monitoring image 110. When detecting the heart rate, a threshold value for the minimum amount of fluctuation in the heart rate to be detected may be predefined based on a combination of the amount of fluctuation in the heart rate and the number of fluctuations within a unit time, and the heart rate movement may be detected only when the heart rate fluctuates beyond this threshold value.

[0085] (Step to verify leaving and sitting behavior) In the leaving / taking seat action verification step st6, the leaving / taking seat action verification unit 125 detects the leaving / taking seat action (the leaving and taking seat action of the examinee 3) from the surveillance image 110 in order to verify whether the leaving and taking seat action of the examinee 3 is a cheating action (taking an exam in place of another examinee). In this step, specifically, an "image for verifying leaving / taking seat action" is generated, which is a combination of a "specific leaving image" which is an image of the leaving action of the examinee 3A from the examinee's seat 4 (e.g. 4A) and a "specific sitting image" which is an image of the first sitting action after the leaving action on the examinee's seat 4A. The "image for verifying leaving / taking seat action" may be immediately output from the surveillance information output unit 13 so that the test supervisor 5 can visually confirm whether cheating has occurred regarding the leaving / taking seat action, but by going through a procedure for automatically determining the identity of the person who has left the seat and the person who has sat down in the face authentication information acquisition step st7, the monitoring work burden of the test supervisor 5 can be further reduced.

[0086] (Facial recognition information acquisition step) In a face authentication information acquisition step st7, the face authentication information acquisition unit 126 acquires unique face authentication information of the examinee 3 from the surveillance image 110. Then, if the face authentication information of the absent person and the seated person in the above-mentioned "image for verifying leaving / taking seat behavior" does not match, it is preferable that in a surveillance information output step st9, the surveillance information output unit 13 immediately outputs a notice indicating the occurrence of fraudulent behavior (taking the test on behalf of another person).

[0087] In the facial recognition information acquisition step st7, the facial recognition information acquired by the facial recognition information acquisition unit 126 as the unique facial recognition information of each examinee is preferably registered in a memory means within the calculation processing unit 12 as identification information of each examinee (test answerer), so that it can be repeatedly referred to as necessary during execution of the test monitoring method, not only for verifying the actions of leaving and sitting at their seats, but also for confirming the identity of each examinee.

[0088] (Fraud detection step) In the misconduct detection step st8, the misconduct detection unit 127 detects misconduct by the examinee 3. The misconduct is detected by using as input values ​​information consisting of a combination of "movement of the upper limbs 31 of the examinee 3" recognized in the movement recognition step st2, "movement of the gaze 32 of the examinee 3" detected in the gaze detection step st4, and "fluctuation in the heart rate of the examinee 3" detected in the heart rate detection step st5 (hereinafter referred to as "information regarding the examinee's movements, etc."), and comparing this input value with a threshold value for determining misconduct to detect the misconduct of the examinee 3.

[0089] The misconduct detection unit 127 determines whether or not misconduct has occurred by comparing the above-mentioned "information on the examinee's actions, etc." with "misconduct judgment criteria (thresholds)" registered in advance in a storage means. The misconduct detection unit 127 can be configured with a storage means for storing such "misconduct judgment criteria (thresholds)" and various information processing devices and programs capable of simple information matching analysis. Table 1 below shows some specific examples of "misconduct judgment criteria (thresholds)."

[0090] [Table 1]

[0091] As described above, if the misconduct detection unit 127 detects misconduct in the misconduct detection step st8 with respect to various actions of the examinee 3 (misconduct? YES), the process proceeds to the monitoring information output step st9. On the other hand, if no misconduct is detected (misconduct? NO), the process returns to the monitoring image capture step st1, and the subsequent monitoring processes are repeated.

[0092] (Monitoring information output step) In the monitoring information output step st9, the monitoring information is output to the monitoring information output unit 13. Here, the monitoring information is information indicating that a specific examinee is committing fraud, or that the possibility of fraud is greater than a certain level, or that the situation is such that fraud is likely to occur, and the format of this monitoring information is not limited to a specific format as long as the test observer 5 can identify which monitored person needs to be dealt with. It is preferable that a frame line of a specific color that distinguishes the monitored person from other objects, skeletal information of the monitored person whose movement state can be recognized, a sign (such as a flashing light or a warning sound) indicating that the situation is such that fraud is suspected, and the like are output from the monitoring information output unit 13 held by the test observer 5.

[0093] In addition, in the monitoring information output step st9, the calculation processing unit 12 judges whether or not to end monitoring. For example, the end of monitoring is judged when a command to end monitoring is input. If it is to be ended (S18, YES), the monitoring operation is ended, and otherwise (S18, NO), the process returns to the monitoring image capturing step st1 and monitoring is continued. [Explanation of symbols]

[0094] 1. Test monitoring system 11. Photography Department 111 Tabletop Camera 112 Ceiling Camera 110 (110A, 110B) Surveillance image 12 Processing unit 121 Motion recognition section 122 Line of sight detection unit 123 Heart rate detector 124 Prohibited items detection unit 125 Seating and Leaving Behavior Verification Division 126 Face Recognition Information Acquisition Unit 127 Fraud Detection Department 13 Monitoring information output section 2. Testing venue 3(3A, 3B, 3C,...3Z) Candidates 31 Upper limb 32 line of sight 33 Prohibited items 4 (4A, 4B, 4C, ... 4Z) Candidate seats 5. Test Supervisor 6. Test supervisor seats st1 Surveillance image capture step st2 Action recognition step st3 Prohibited items detection step st4 Gaze detection step st5 Heart rate detection step st6 Step to verify seating and leaving behavior st7 Face recognition information acquisition step st8 Fraud detection step st9 Monitoring information output step

Claims

1. 1. An exam monitoring system for detecting exam-takers' cheating in an exam center, comprising: A photography team that photographs the examinees, a motion recognition unit that recognizes a motion of the upper limbs of the examinee from the monitoring image captured by the imaging unit; a gaze detection unit that detects gaze movements of the examinee from the monitoring image; a cheating detection unit that detects cheating by the examinee; a monitoring information output unit that outputs monitoring information in a format recognizable by a test supervisor when the misconduct detection unit detects misconduct; The exam monitoring system, wherein the cheating detection unit uses information consisting of a combination of the upper limb movements and the eye movement as an input value, and detects cheating by the examinee by comparing this input value with a threshold value for determining cheating.

2. a heart rate detection unit that detects a heart rate of the examinee from the monitoring image; the misconduct detection unit receives information consisting of a combination of the upper limb movement, the eye movement, and the heart rate as an input value, and detects misconduct by the examinee by comparing the input value with the threshold value; 2. The test monitor system of claim 1.

3. a prohibited object detection unit that detects prohibited objects present around the examinee from the surveillance image; When the prohibited item is detected, the monitoring information output unit outputs a notification indicating the occurrence of fraudulent activity.

3. A test monitor system according to claim 1 or 2.

4. a seat leaving / sitting behavior verification unit that detects the examinee's seat leaving behavior and seat sitting behavior from the surveillance image; the seat leaving / sitting action verification unit generates an image for verifying seat leaving / sitting actions, which is a combination of a specific seat leaving image, which is an image of the examinee's seat leaving action, and a specific seating image, which is an image of the examinee's first seating action after the examinee's seat leaving action, on the examinee's seat; The surveillance information output unit outputs the image for verifying the sitting and leaving behavior.

3. A test monitor system according to claim 1 or 2.

5. a face authentication information acquisition unit that acquires unique face authentication information of the examinee from the surveillance image; When the facial authentication information of the seated person and the seated person in the seating / leaving behavior verification image does not match, the monitoring information output unit outputs a notification indicating the occurrence of fraudulent behavior.

5. The test monitor system of claim 4.

6. The photographing unit is configured to include a table-mounted camera that is individually installed at each examinee's seat where each examinee answers questions and can photograph the upper body of each examinee from the front, and a ceiling camera that can photograph an overhead view of the entire surveillance area of ​​the examination hall.

3. A test monitor system according to claim 1 or 2.

7. the tabletop camera is a PC built-in camera built into a personal computer for each examinee that is individually installed at each examinee's seat where each examinee answers the questions; 7. The test monitor system of claim 6.

8. 1. A method for monitoring an exam for detecting exam-takers' misconduct in an exam center, comprising: a monitoring image capturing step in which an image capturing unit captures an image of the examinee; a movement recognition step in which a movement recognition unit recognizes a movement of the upper limbs of the examinee from the monitoring image captured in the monitoring image capturing step; a gaze detection step in which a gaze detection unit detects gaze movement of the examinee from the monitoring image; a cheating detection step in which a cheating detection unit detects cheating by the examinee; a monitoring information output step in which a monitoring information output unit outputs the monitoring information related to the fraudulent act in a format recognizable by a test invigilator; In the misconduct detection step, information consisting of a combination of the upper limb movement and the gaze movement is used as an input value, and the input value is compared with a threshold value for determining misconduct to detect the misconduct. Test monitoring methods.

9. The method further includes a heart rate detection step of detecting a heart rate of the examinee from the monitoring image, In the misconduct detection step, information consisting of a combination of the upper limb movement, the gaze movement, and the heart rate is used as an input value, and the input value is compared with a threshold value for determining misconduct to detect the misconduct.

9. The method of claim 8.

10. a prohibited object detection unit detecting a prohibited object present around the examinee from the surveillance image; When the prohibited item is detected, a notification indicating the occurrence of fraudulent activity is output in the monitoring information output step.

10. A method of monitoring a test as claimed in claim 8 or 9.

11. The method further includes a seat leaving / sitting behavior verification step of detecting the examinee's seat leaving and seat sitting behavior from the monitoring image, In the step of verifying the act of leaving or taking a seat, a verification image for leaving or taking a seat is generated, the verification image being a combination of a specific leaving image, which is an image of the act of leaving the examinee's seat from the examinee's seat, and a specific sitting image, which is an image of the first sitting down action after the act of leaving the seat to the examinee's seat; In the monitoring information output step, the image for verifying the sitting / leaving behavior is output.

10. A method of monitoring a test as claimed in claim 8 or 9.

12. The method further includes a face authentication information acquisition step of acquiring unique face authentication information of the examinee from the surveillance image by a face authentication information acquisition unit, When the facial authentication information of the seated person and the seated person in the seating behavior verification image does not match, a notification indicating the occurrence of fraudulent activity is output in the monitoring information output step.

12. The method of claim 11.

13. the imaging unit is configured to include a table-mounted camera that is installed at each examinee's seat where each examinee answers questions and is capable of imaging the upper body of each examinee from the front, and a ceiling camera that is capable of imaging an overhead view of the entire surveillance area of ​​the examination hall, In the monitoring image capturing step, the tabletop camera captures an image of the face and upper body of the examinee.

10. A method of monitoring a test as claimed in claim 8 or 9.

14. the tabletop camera is a PC built-in camera built into a personal computer for each examinee that is individually installed at each examinee's seat where each examinee answers the questions; 14. The method of claim 13.

15. 9. The test monitoring method according to claim 8, further comprising causing an information processing device to execute the action recognition step, the gaze detection step, the fraudulent behavior detection step, and the monitoring information output step. A program for monitoring exams to detect test-takers' cheating in the exam room.

Citation Information

Patent Citations

  • Examination room anti-cheating monitoring system

    CN109816951A

  • Online examination anti-cheating implementation method based on AI face recognition technology

    CN113657300A

  • Monitoring camera system for examination site

    JP2014064186A

  • Online examination support device and program

    JP2022029113A

  • Information processing apparatus and information processing program

    JP2023036273A

Cited By

  • Upper roller, glass plate manufacturing device and glass plate manufacturing method

    CN121651647A