Monitoring system and monitoring method

Abstract

To reduce processing load required for detecting a fraud in a monitoring system which detects a fraud using face authentication.SOLUTION: A monitoring system 1 includes: a storage unit 34 which stores, in advance, face information of participants; imaging apparatuses 5, 15 capable of imaging each of the participants; a processor 31 which determines whether a fraud exists or not on the basis of the images obtained by the imaging apparatuses; and a notification unit 27 which gives information on a result determined by the processor. The processor is configured to: determine whether each of the participants has fraudulently acted or not on the basis of the images captured by the imaging apparatus; perform, when a fraud is determined, face authentication so as to collate an image of a participant determined to have fraudulently acted with face information of the participant determined to have fraudulently acted; and determine, when the collation fails in the face authentication, that there is a possibility of impersonation.SELECTED DRAWING: Figure 11

Description

[Technical field]

[0001] The present disclosure relates to a monitoring system and method for monitoring participants in an examination, training session, or the like. [Background technology]

[0002] In recent years, tests have come to be conducted using computers and the Internet. Tests that use computers are called Computer Based Testing (hereafter referred to as CBT), and tests that use the Internet are called Internet Based Testing (hereafter referred to as IBT).

[0003] Conventionally, there is known a test monitoring system that can automatically detect cheating with high detection accuracy when conducting a CBT (see Patent Document 1). Also, conventionally, there is known a test system that can efficiently perform confirmation work, including identity verification, for starting a test in an IBT (see Patent Document 2). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2022-052144 [Patent Document 2] JP 2022-162517 A Summary of the Invention [Problem to be solved by the invention]

[0005] In addition to cheating, other types of fraudulent acts include impersonation, in which a person takes the test pretending to be the test-taker. For example, behavior recognition technology is used to detect cheating, and face recognition is used to detect impersonation.

[0006] In the conventional testing system disclosed in Patent Document 1 above, first, the identity of the examinee is confirmed using facial recognition technology, and then behavioral recognition technology is used to detect when the examinee leaves their work position.

[0007] However, in the examination system disclosed in Patent Document 1, the identity of the examinee is repeatedly determined using face recognition technology regardless of whether the examinee has left the seat. Therefore, in the examination system disclosed in Patent Document 1, there is a risk that the processing load required to detect cheating will be large.

[0008] Therefore, a main objective of the present disclosure is to reduce the processing load required to detect fraudulent activity in a monitoring system and monitoring method that detects fraudulent activity using face recognition. [Means for solving the problem]

[0009] The surveillance system disclosed herein includes a storage device that pre-stores facial information of participants, an imaging device capable of capturing an image of each of the participants, a processor that determines whether or not there is suspicious activity based on the image captured by the imaging device, and a notification device that notifies the processor of the determination result, wherein the processor determines whether or not there is suspicious activity for each of the participants based on the image captured by the imaging device, and when it determines that suspicious activity has occurred, performs facial authentication to compare the image of the participant for whom it has been determined that suspicious activity has occurred with the facial information of the participant for whom it has been determined that suspicious activity has occurred, and when a match is not made in the facial authentication, it determines that there may have been an act of impersonation.

[0010] The surveillance method disclosed herein is executed by a surveillance system including a storage device that pre-stores facial information of participants, an imaging device capable of capturing an image of each of the participants, a processor that determines whether or not suspicious activity has occurred based on the image captured by the imaging device, and a notification device that notifies the processor of the determination result, wherein the processor determines whether or not suspicious activity has occurred for each of the participants based on the image captured by the imaging device, and when it determines that suspicious activity has occurred, performs facial authentication to compare the image of the participant for whom it has been determined that suspicious activity has occurred with the facial information of the participant for whom it has been determined that suspicious activity has occurred, and when a match is not made in the facial authentication, it determines that there may have been an act of impersonation. Effect of the Invention

[0011] According to the present disclosure, in a monitoring system and monitoring method that detects fraudulent activity using face recognition, it is possible to reduce the processing load required for detecting fraudulent activity. [Brief description of the drawings]

[0012] [Figure 1] Overall configuration of the monitoring system [Diagram 2] Functional block diagram of the monitoring system [Diagram 3] Example of monitoring screen [Figure 4] (A) Examples of icon (frame) display when the suspicious behavior detection flag and face recognition NG flag are both 0, (B) when the suspicious behavior detection flag is 1, and (C) when the face recognition NG flag is 1. [Diagram 5] (A) Modified examples of icon (frame) display when the suspicious behavior detection flag is 1 and the face recognition NG flag is 0, (B) when the suspicious behavior detection flag is 0 and the face recognition NG flag is 1, and (C) when both the suspicious behavior detection flag and the face recognition NG flag are 1. [Figure 6] Example of the monitoring screen displayed after an input is made to the sort button [Figure 7] Example of a pop-up when suspicious activity is detected [Figure 8] An example of a pop-up message to warn a candidate who has been detected engaging in suspicious behavior [Figure 9] Example of a popup when spoofing is detected [Figure 10] Flow diagram showing the suspicious behavior detection process [Figure 11] Flow diagram showing the process of detecting spoofing DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] A first invention made to solve the above problems is a surveillance system comprising a storage device that pre-stores facial information of participants, an imaging device capable of imaging each of the participants, a processor that determines whether or not suspicious activity has occurred based on the image acquired by the imaging device, and a notification device that notifies the processor of the determination result, wherein the processor determines whether or not suspicious activity has occurred for each of the participants based on the image acquired by the imaging device, and when it determines that suspicious activity has occurred, performs facial authentication to compare the image of the participant for whom it has been determined that suspicious activity has occurred with the facial information of the participant for whom it has been determined that suspicious activity has occurred, and when a match is not made in the facial authentication, it determines that there is a possibility of impersonation activity.

[0014] According to this, face recognition is performed when it is determined that a suspicious act has occurred. Therefore, the number of face recognitions can be reduced compared to a case where face recognition is repeatedly performed and the presence or absence of a suspicious act is determined when face recognition is not successful. Therefore, in a surveillance system that detects a fraudulent act using face recognition, it is possible to reduce the processing load required for detecting the fraudulent act.

[0015] In addition, in a second invention, the notification device is provided with a display device that displays a screen for a supervisor, and the processor is configured to cause the display device to display an icon corresponding to the participant in a color corresponding to the judgment result.

[0016] This allows the supervisor to easily understand the results of the evaluation of the participants.

[0017] In addition, the third invention is configured such that the processor causes the display device to display the icon corresponding to the participant determined to have engaged in the suspicious activity in a first color, and to display the icon corresponding to the participant determined to have engaged in the impersonation activity in a second color.

[0018] This allows the supervisor to easily grasp which participants are determined to have engaged in suspicious behavior and which participants are determined to have possibly engaged in impersonation.

[0019] In addition, a fourth invention is configured such that the processor displays on the display device the icon corresponding to the participant for whom it has been determined that the suspicious activity has occurred and for whom it has been determined that the impersonation activity has occurred, alternating between the first color and the second color.

[0020] This allows the supervisor to easily grasp the results of the judgment regarding suspicious behavior of each participant and whether or not there has been any impersonation.

[0021] In addition, a fifth invention is configured to include an input device that accepts input to the icon from the supervisor, and the processor is configured to display information related to the participant corresponding to the icon on the display device when the input device accepts input to the icon.

[0022] This allows the supervisor to easily grasp information relating to the participants.

[0023] In addition, a sixth aspect of the present invention is configured such that, when it is determined that the suspicious activity has occurred, the processor obtains a type of the suspicious activity and causes the notification device to notify the type of the suspicious activity.

[0024] According to this, when a participant is judged to have engaged in suspicious behavior, the supervisor can easily grasp the details of the suspicious behavior that the participant has been judged to have engaged in.

[0025] In addition, a seventh invention is configured in such a way that the storage device stores a suspicious behavior detection flag set for each of the participants, and the processor sets the suspicious behavior detection flags of all of the participants to a first value before the start of the test, and if it determines that suspicious behavior has occurred after the start of the test, it sets the suspicious behavior detection flag of the participant for whom it has been determined that suspicious behavior has occurred to a second value, and when the suspicious behavior detection flag becomes the second value, it performs the face authentication of the participant whose suspicious behavior detection flag has become the second value.

[0026] According to this, when it is determined that a suspicious act has occurred, it is possible to configure so as to perform face authentication on the participant who is determined to have engaged in the suspicious act.

[0027] In addition, an eighth invention is configured to include an input device that accepts input from a supervisor, and the processor sets the suspicious activity detection flag corresponding to the participant to the first value based on the input to the input device.

[0028] According to this, the supervisor can set the suspicious activity detection flag to the first value by inputting information into the input device.

[0029] In addition, a ninth invention includes an input device that accepts input from a supervisor, and the processor is configured to transmit warning information to a terminal corresponding to the participant determined to have engaged in suspicious activity based on the input to the input device.

[0030] This makes it possible to give warnings to participants who are determined to have engaged in suspicious behavior.

[0031] In addition, a tenth invention is configured such that the processor, based on an input to the input device, transmits the warning information simultaneously to terminals corresponding to the participants in which it has been determined that the suspicious activity has occurred.

[0032] This makes it possible to simultaneously warn participants who are determined to have engaged in suspicious behavior.

[0033] In an eleventh aspect of the present invention, the processor is configured to receive, at the input device, a setting input for setting the alert information.

[0034] According to this, the supervisor can set warning information by inputting information into the input device, which allows the supervisor to set, for example, the content to be taken against a participant who is determined to have engaged in suspicious behavior.

[0035] The twelfth invention also includes an input device that accepts input from a supervisor, and the processor is configured to accept, at the input device, a determination by the supervisor as to whether or not the participant determined to have engaged in suspicious activity has engaged in fraudulent activity, and to store the determination result in the storage device.

[0036] According to this, the supervisor's determination result regarding the suspicious activity can be stored in the storage device, and thus the determination result that is useful for improving the accuracy of the suspicious activity determination can be held in the storage device.

[0037] In addition, a thirteenth invention is configured such that the storage device stores a learning model for determining whether or not suspicious activity has occurred, and the processor updates the learning model based on the determination result stored in the storage device.

[0038] According to this, the learning model is updated based on the supervisor's certification of suspicious behavior, which can improve the accuracy of suspicious behavior determination.

[0039] A fourteenth invention made to solve the above problem is a surveillance method executed by a surveillance system including a storage device that prestores facial information of participants, an imaging device capable of imaging each of the participants, a processor that determines whether or not there has been suspicious activity based on the image acquired by the imaging device, and a notification device that notifies the processor of the determination result, wherein the processor determines whether or not there has been suspicious activity for each of the participants based on the image captured by the imaging device, and when it determines that there has been suspicious activity, performs facial authentication to compare the image of the participant for which it has been determined that there has been suspicious activity with the facial information of the participant for which it has been determined that there has been suspicious activity, and when a match is not made in the facial authentication, it determines that there may have been an act of impersonation.

[0040] According to this, when it is determined that a suspicious act has occurred, face authentication is performed. Therefore, the number of face authentications can be reduced compared to a case where face authentication is repeatedly performed and the presence or absence of a suspicious act is determined when face authentication is not successful. Therefore, in a monitoring method that detects a fraudulent act using face authentication, it is possible to reduce the processing load required for detecting the fraudulent act.

[0041] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0042] The monitoring system is a system for implementing a monitoring method for monitoring and supervising participants in an examination or training session to prevent them from committing fraudulent acts, and is incorporated into various systems such as computer-based testing (CBT) and training sessions, Internet-based testing (IBT) and training sessions, etc. Below, an embodiment in which the monitoring system is incorporated into a system for performing IBT will be described.

[0043] FIG. 1 shows an overall configuration diagram of a monitoring system 1 according to the first embodiment. The monitoring system 1 comprises an examinee terminal 2, a supervisor terminal 3, and a management server 4. Each examinee terminal 2 is used by one participant (i.e., an examinee) who takes part in the examination. The supervisor terminal 3 is used by a supervisor (also called an examination proctor) who monitors whether or not there is any cheating. The management server 4 is managed by an administrator who administers the examination. Administrators who administer the examination may include, for example, various educational institutions and companies that administer qualification examinations.

[0044] The monitoring system 1 may be applied to a CBT in which examinees gather in a designated test venue such as a classroom. In that case, the monitoring system 1 may include an imaging device (hereinafter, on-site imaging device 5) that captures images of the site where the test is held (i.e., the test venue). The on-site imaging device 5 may be composed of one or more cameras 5A installed on the ceiling or the like of the test venue.

[0045] 1, the examinee terminal 2, the supervisor terminal 3, and the management server 4 are each connected to a wide area network (WAN) 6 such as the Internet. If the exam management system includes an on-site imaging device 5, the on-site imaging device 5 should also be connected to the WAN 6.

[0046] (Exam taker's device) 1, the examinee terminal 2 may include one or more processors 11 (CPU, MPU, etc.), a RAM 12 (Random Access Memory), a ROM 13 (Read Only Memory), storage 14 including a Solid State Drive (SSD) or a Hard Disk Drive (HDD), and a communication interface 15 (I / F). The examinee terminal 2 is connected to the management server 4 via the communication interface 15.

[0047] The examinee terminal 2 may further include an imaging device 16, a notification device 17 for notifying the examinee, and an input device 18 for receiving input from the examinee. The notification device 17 may be composed of a monitor 17A, a speaker 17B, etc. The input device 18 may be composed of a keyboard 18A, a mouse 18B, etc. The notification device 17 and the input device 18 may be composed of one or more touch panel displays.

[0048] The imaging device 16 is composed of a video camera 16A equipped with known hardware. The imaging device 16 generates a captured image of an imaging range determined for each imaging device 16.

[0049] The imaging device 16 is provided, for example, above the monitor 17A, and its imaging range is preset to be the area in front of the monitor 17A. As a result, when an examinee stands in front of the monitor 17A and faces the monitor 17A, an image including the examinee's face is captured by the imaging device 16.

[0050] When the examinee makes a specified input to the examinee terminal 2 before the start of the exam, the examinee terminal 2 transmits in real time the captured images acquired by the imaging device 16 to the management server 4. When the exam is over and the examinee makes a specified input, transmission of the captured images from the examinee terminal 2 to the management server 4 ends.

[0051] After the test starts, the examinee terminal 2 notifies the examinee from the notification device 17 based on various information transmitted from the management server 4 via the WAN 6. The information transmitted from the management server 4 may include warning information for the examinee. When the examinee terminal 2 receives the warning information from the management server 4, the notification device 17 issues a notification corresponding to the warning information to warn the examinee. At this time, the notification issued by the notification device 17 may include, for example, displaying a pop-up (not shown) on the monitor 17A to discourage the examinee from engaging in any behavior that may be suspected of cheating, or emitting a warning sound from the speaker 17B to encourage the examinee to concentrate on the test.

[0052] Additionally, the information transmitted from the management server 4 may include information related to the test questions. In that case, the examinee terminal 2 may receive input related to the answers from the examinee via the input device 18 and transmit the input answers to the management server 4.

[0053] (Supervisor terminal) 1, the supervisor terminal 3 may include one or more processors 21 (CPU, MPU, etc.), a RAM 22 (Random Access Memory), a ROM 23 (Read Only Memory), a storage 24 including a Solid State Drive (SSD) and a Hard Disk Drive (HDD), and a communication interface 25 (I / F), etc. The supervisor terminal 3 is connected to the management server 4 via the communication interface 25.

[0054] 1, the supervisor terminal 3 may further include a notification device 27 for notifying the supervisor, and an input device 28 for receiving input from the supervisor. The notification device 27 may be configured with a monitor 27A as a display device for displaying a screen to the supervisor, a speaker 27B for notifying the supervisor by voice, etc. The input device 28 may be configured with a keyboard 28A, a mouse 28B, a touch pad 28C, etc. The notification device 27 and the input device 28 may be configured with one or more touch panel displays.

[0055] (Management Server) 1, the management server 4 may include one or more processors 31 (CPU, MPU, etc.), a RAM 32 (Random Access Memory), a ROM 33 (Read Only Memory), a storage 34 including a Solid State Drive (SSD) and a Hard Disk Drive (HDD), and a communication interface 35 (I / F), etc. The management server 4 is connected to the management server 4 via the communication interface 35.

[0056] The management server 4 detects suspicious behavior or impersonation by the examinee based on the captured images sent from the examinee terminal 2. If suspicious behavior or impersonation is detected, the management server 4 notifies the supervisor terminal 3 of the fact.

[0057] Suspicious behavior may include leaving the seat, carrying a smartphone, etc. Other suspicious behavior may include the examinee looking around (looking away) at a part other than the screen of the monitor 17A, staring at a part other than the screen of the monitor 17A, opening or looking at a book such as a reference book, etc.

[0058] Impersonation is when someone other than the test-taker pretends to be the test-taker and takes the test.

[0059] 2, the management server 4 includes, as functional units, a storage unit 41, a captured image acquisition unit 42, a suspicious activity detection unit 43, and an impersonation activity detection unit 44. In this embodiment, the management server 4 further includes an icon display setting unit 45, a certification result receiving unit 46, and a model update unit 47.

[0060] The memory unit 41 is composed of the storage 34, and records and holds information for various processes executed by the processor 31. The information recorded and held in the memory unit 41 includes, for example, an examinee information table 41A. In the examinee information table 41A, an examinee ID for identifying each examinee is recorded in association with personal information related to the examinee. The personal information included in the examinee information table 41A includes, for example, name, affiliation, and facial information. The facial information may be, for example, an ID photo (face photo) attached to an examination ticket or the like, or may be facial information extracted in advance from a face photo or the like. The personal information related to the examinee is registered in advance before the start of the test, and is stored in the memory unit 41 (storage 34) in advance as the examinee information table 41A before the start of the test.

[0061] The information recorded and stored in the memory unit 41 further includes a learning model 41B, a history image 41C, a suspicious behavior detection flag 41D, and a face authentication NG flag 41E. The history image 41C, the suspicious behavior detection flag 41D, and the face authentication NG flag 41E are each associated with an examinee ID for each examinee, and are recorded and stored in the memory unit 41.

[0062] The learning model 41B is a learned learning model for recognizing the behavior of the examinee using the captured images and videos acquired by the imaging device 16. The learning model 41B is used for recognizing the behavior of the examinee, particularly for detecting suspicious behavior. The learning model 41B is stored in the storage unit 41 before the start of the test. The suspicious behavior detection flag 41D is a flag indicating whether or not a suspicious behavior has been detected in the examinee (i.e., the monitoring system 1 has determined that the examinee has engaged in a suspicious behavior). The suspicious behavior detection flag 41D is set to a first value before the start of the test, and is changed to a second value when a suspicious behavior is detected (i.e., it has been determined that a suspicious behavior has occurred). The first value and the second value may be numerical values ​​or true / false values. In this embodiment, the first value is set to 0, and the second value is set to 1. Thus, the suspicious behavior detection flag 41D is set to 0 before the start of the test and is stored in the storage unit 41.

[0063] The face authentication NG flag 41E is a flag indicating whether or not an act of impersonation of an examinee has been detected (i.e., the monitoring system 1 has determined that an act of impersonation of an examinee has occurred, more specifically, that there is a possibility that an act of impersonation has occurred). The face authentication NG flag 41E is set to a third value before the start of the test, and is changed to a fourth value when an act of impersonation is detected (i.e., it has been determined that there is a possibility that an act of impersonation of an examinee has occurred, or that the examinee has engaged in suspicious behavior). In this embodiment, the third value is set to 0, and the fourth value is set to 1. Thus, the face authentication NG flag 41E is set to 0 before the start of the test and is stored in the storage unit 41.

[0064] The photographed image acquisition unit 42 acquires in real time the captured images captured by the imaging device 16 from each examinee terminal 2. Furthermore, the photographed image acquisition unit 42 associates the captured images acquired in real time with the examinee ID for each examinee and stores them in the memory unit 41 as history images 41C. At the same time, the photographed image acquisition unit 42 transmits the captured images of each examinee acquired by the imaging device 16 to the supervisor terminal 3 in real time. When the supervisor terminal 3 acquires the captured images of each examinee from the management server 4, it displays a monitoring screen 48 on the monitor 27A, which is the notification device 27.

[0065] 3 shows a monitoring screen 48. A corresponding icon 49 is set for each examinee. In this embodiment, the icon 49 corresponding to each examinee is a rectangular frame, and a captured image of the corresponding examinee is fitted into the frame. The captured image acquisition unit 42 acquires captured images from the imaging device 16 in real time and transmits them to the supervisor terminal 3, so that the image within the icon 49 is updated in real time.

[0066] The suspicious activity detection unit 43 executes a suspicious activity detection process for detecting suspicious activity for all examinees based on the captured images acquired by the captured image acquisition unit 42. The suspicious activity detection unit 43 is configured by the processor 31 cooperating with the memory and storage 34 to execute a predetermined program.

[0067] The suspicious behavior detection process is a process for detecting suspicious behavior of an examinee using a captured image, and the presence or absence of suspicious behavior is determined based on the detection result. In this embodiment, the suspicious behavior detection unit 43 detects suspicious behavior of an examinee captured in a captured image using the learning model 41B stored in the storage unit 41. In this embodiment, the suspicious behavior detection unit 43 can further acquire the type of suspicious behavior, such as the examinee leaving his / her seat, carrying a smartphone, or looking away, based on the learning model 41B.

[0068] When the suspicious behavior detection unit 43 determines based on the detection result that an examinee has engaged in suspicious behavior (that is, suspicious behavior is detected in the examinee), it sets the suspicious behavior detection flag 41D corresponding to that examinee to 1.

[0069] The impersonation detection unit 44 executes an impersonation detection process for detecting impersonation for a specific examinee or for all examinees, using the captured image acquired by the photographed image acquisition unit 42. The impersonation detection unit 44 includes a timing setting unit 51, a face detection unit 52, and a face matching unit 53.

[0070] The timing setting unit 51 sets the timing for executing the impersonation detection process and the examinees who are to be subject to the impersonation detection process. The timing setting unit 51 can be realized by the processor 31 working in cooperation with memories such as the RAM 32 and the ROM 33 and / or the storage 34.

[0071] The timing setting unit 51 sets the timing for executing the impersonation detection process for all examinees (hereinafter, all detection timing) and the timing for executing the impersonation detection process for a specific examinee (hereinafter, individual detection timing). The impersonation detection process is executed by the face detection unit 52 and the face matching unit 53 working together.

[0072] The timing setting unit 51 sets the all detection timing so that the impersonation detection process is executed for all examinees at predetermined time intervals (for example, every 5 to 10 minutes). When the timing setting unit 51 sets the all detection timing, the face detection unit 52 and the face matching unit 53 execute the impersonation detection process for all examinees at the set timing.

[0073] The timing setting unit 51 may set the overall detection timing using a timer provided for each test to be conducted. The timing setting unit 51 may set a timer for each examinee, start the timers for all examinees at the same time, and reset them at predetermined intervals to set the overall detection timing.

[0074] The timing setting unit 51 sets the individual detection timing when the suspicious activity detection flag 41D becomes 1 for any of all the examinees. Specifically, the timing setting unit 51 sets the individual detection timing such that when the suspicious activity detection flag 41D becomes 1 for any of the examinees, the impersonation detection process is executed for the examinee whose suspicious activity detection flag 41D becomes 1. Once the timing setting unit 51 sets the individual detection timing, when the suspicious activity detection flag 41D becomes 1, the face detection unit 52 and the face matching unit 53 work together to execute the impersonation detection process for the examinee whose suspicious activity detection flag 41D becomes 1.

[0075] In the impersonation detection process, the face detection unit 52 and the face matching unit 53 work together to match (i.e., match) the captured image of the target examinee with the face information of the examinee stored in the storage unit 41, that is, to perform so-called face authentication, thereby detecting an act of another person taking an exam by pretending to be the examinee, that is, an act of impersonation. That is, the presence or absence of an act of impersonation is determined based on the result of face authentication.

[0076] The face detection unit 52 can be realized by the processor 31 working in cooperation with memories such as the RAM 32 and the ROM 33 and / or the storage 34. When the timing setting unit 51 instructs the face detection unit 52 to execute the impersonation detection process, the face detection unit 52 detects a face area from the captured image of the examinee who has been instructed to execute the impersonation detection process. The face detection unit 52 then extracts facial features from the captured image, associates the examinee ID of the examinee who has been instructed to execute the impersonation detection process with the extracted facial features, and outputs the associated data to the face matching unit 53.

[0077] The face matching unit 53 can be realized by the processor 31 working in cooperation with memories such as the RAM 32 and the ROM 33 and / or the storage 34.

[0078] When the face matching unit 53 acquires the facial feature amount and the corresponding examinee ID from the face detection unit 52, it refers to the examinee information table 41A stored and held in advance in the storage unit 41, and acquires the facial information corresponding to the examinee ID. Then, the face matching unit 53 collates the facial feature amount acquired from the face detection unit 52 with the acquired facial information to determine whether they match.

[0079] When facial feature amounts preregistered before the start of the test are recorded in the examinee information table 41A, the face matching section 53 may perform face matching by matching the prerecorded facial feature amounts with the facial feature amounts acquired from the captured image by the face detection section 52. When a facial photograph preregistered before the start of the test is recorded in the examinee information table 41A, the face matching section 53 may perform face matching by matching the facial feature amounts acquired from the preregistered facial photograph by the face matching section 53 with the facial feature amounts acquired from the captured image by the face matching section 53.

[0080] The face matching unit 53 compares (face matching) the facial features acquired by the face detection unit 52 based on the captured image with the facial information previously stored in the memory unit 41, and if the two do not match (i.e., face authentication cannot be performed normally), it determines that there may have been an act of impersonation (i.e., impersonation has been detected), and sets the face authentication NG flag 41E of that examinee to 1.

[0081] Here, the face matching unit 53 sets the face authentication NG flag 41E to 1 (second value) even when the quality of the captured image is not sufficient for face matching and face authentication is not possible. For example, the quality of the captured image may be insufficient for face matching when the examinee looks down unnaturally (e.g., when cheating). In this case, face authentication is not possible and the face matching unit 53 sets the face authentication NG flag 41E to 1. Therefore, the case where the face authentication NG flag 41E is 1 may include not only the case where impersonation has occurred, but also the case where face authentication is not possible even though impersonation has not occurred.

[0082] The icon display setting unit 45 can be realized by the processor 31 working in cooperation with memories such as the RAM 32 and the ROM 33 and / or the storage 34. The icon display setting unit 45 sets the display of the icons 49 corresponding to each examinee in real time based on the suspicious behavior detection flag 41D and the face authentication NG flag 41E, and notifies the supervisor terminal 3. The supervisor terminal 3 displays the icons 49 corresponding to each examinee in accordance with the display settings made by the icon display setting unit 45.

[0083] This allows the supervisor to know whether or not there is an examinee who has been detected to be engaging in suspicious behavior or impersonation, based on the display mode of the icon 49. Therefore, the icon display setting unit 45 functions, in a sense, as a notification unit for the supervisor.

[0084] The icon display setting unit 45 may set the display so that the thickness of the frame of the icon 49 becomes thicker in the order of when the suspicious behavior detection flag 41D and the face authentication NG flag 41E are both 0, when the suspicious behavior detection flag 41D is 1, and when the face authentication NG flag 41E is 1. Fig. 4(A) shows a display example of the icon 49 when the suspicious behavior detection flag 41D and the face authentication NG flag 41E are both 0, Fig. 4(B) shows a display example of the icon 49 when the suspicious behavior detection flag 41D is 1, and Fig. 4(C ... and the face authentication NG flag 41E are both 0,

[0085] When the suspicious behavior detection flag 41D and the face authentication NG flag 41E are both 1, the icon display setting unit 45 may perform the same display setting as when the suspicious behavior detection flag 41D is 0 and the face authentication NG flag 41E is 1. In addition, when the suspicious behavior detection flag 41D and the face authentication NG flag 41E are both 1, the icon display setting unit 45 may perform display setting such that the frame of the icon 49 is thicker than when the suspicious behavior detection flag 41D is 0 and the face authentication NG flag 41E is 1.

[0086] The icon display setting unit 45 may be configured to set the color of the icon 49 based on the suspicious behavior detection flag 41D or the face authentication NG flag 41E. This allows the icon 49 for the examinee to be displayed in a color corresponding to the detection result of the suspicious behavior or the impersonation. Figures 5(A) to (C) show modified examples of the display setting of the icon 49 when the color of the icon 49 is set based on the suspicious behavior detection flag 41D or the face authentication NG flag 41E.

[0087] When an examinee's suspicious behavior detection flag 41D and face authentication NG flag 41E are both 0, the icon display setting section 45 sets the colour of the icon 49 (frame) corresponding to that examinee to the standard colour (black).

[0088] When an examinee's suspicious behavior detection flag 41D is 1 and the face authentication NG flag 41E is 0, the icon display setting unit 45 sets the color of the icon 49 (frame) corresponding to that examinee to a first color (yellow) indicating a warning. Fig. 5(A) shows an example of how the icon 49 is displayed when an examinee's suspicious behavior detection flag 41D is 1 and the face authentication NG flag 41E is 0. At this time, the icon 49 (frame) is displayed in the first color.

[0089] When an examinee's suspicious behavior detection flag 41D is 0 and the face authentication NG flag 41E is 1, the icon display setting unit 45 sets the color of the icon 49 (frame) corresponding to that examinee to a second color (red) indicating a warning. Fig. 5(B) shows an example of how the icon 49 is displayed when an examinee's suspicious behavior detection flag 41D is 0 and the face authentication NG flag 41E is 1. At this time, the icon 49 (frame) is displayed in the second color.

[0090] When an examinee's suspicious behavior detection flag 41D and face authentication NG flag 41E are both 1, the icon display setting unit 45 sets the display so that the icon 49 (frame) corresponding to that examinee is alternately colored in a first color (yellow) and a second color (red). Fig. 5(C) shows an example of the display of the icon 49 when an examinee's suspicious behavior detection flag 41D is 1 and face authentication NG flag 41E is 1. At this time, the icon 49 (frame) is alternately displayed in the first color and the second color.

[0091] 5(A) to 5(C), the supervisor can easily recognize which examinees have been detected as engaging in suspicious behavior and which examinees have been detected as engaging in impersonation, based on the display mode of the icon 49. In this embodiment, particularly when the suspicious behavior detection flag 41D and the face authentication NG flag 41E are both 1, the color of the icon 49 alternates between the first color and the second color, allowing the supervisor to easily recognize that a suspicious behavior or an impersonation behavior has been detected.

[0092] Additionally, the icon display setting unit 45 may set the display so that the icon 49 is colored in both the first color and the second color when the examinee's suspicious behavior detection flag 41D and face authentication NG flag 41E are both 1. Additionally, the icon display setting unit 45 may make the icon 49 blink in the first color, the second color, or another color when the examinee's suspicious behavior detection flag 41D and / or face authentication NG flag 41E are both 1.

[0093] 6, the monitoring screen 48 is provided with a sort button 55. The sort button 55 is used to sort the examinees based on the suspicious behavior detection flag 41D and the face authentication NG flag 41E. When the supervisor presses the sort button 55, examinees whose face authentication NG flag 41E is 1 and whose suspicious behavior detection flag 41D is 1, examinees whose face authentication NG flag 41E is 1, examinees whose suspicious behavior detection flag 41D is 1, and examinees whose face authentication NG flag 41E is 1 and whose suspicious behavior detection flag 41D is 0 are displayed on the monitor 27A in the listed order.

[0094] The monitoring screen 48 is further provided with a simultaneous warning button 56. The simultaneous warning button 56 is used to issue a warning or alert to examinees whose suspicious behavior detection flag 41D is set to 1 via the examinee terminals 2. When the supervisor clicks the simultaneous warning button 56 using the input device 28 (touchpad 28C) of the supervisor terminal 3, the management server 4 is notified of this, and the processor 31 of the management server 4 transmits warning information simultaneously to all examinee terminals 2 of examinees who have been determined to have engaged in suspicious behavior. This causes the notification device 17 of the examinee terminal 2 to issue a warning or alert. The notification device 17 of the examinee terminal 2 can issue a warning or alert, for example, by displaying on the monitor 17A of the examinee terminal 2, or by emitting an alarm sound from the speaker 17B.

[0095] The monitoring screen 48 is further provided with a simultaneous release button 57. When the supervisor clicks the simultaneous release button 57 using the input device 28 (touchpad 28C) of the supervisor terminal 3, a message to that effect is sent from the supervisor terminal 3 to the management server 4, and the processor 31 of the management server 4 resets the suspicious behavior detection flags 41D and / or the face authentication NG flags 41E of all the examinees to 0.

[0096] When the supervisor makes an input (double-clicks in this embodiment) to icon 49 on monitoring screen 48, supervisor terminal 3 transmits a certification acceptance signal to management server 4. The certification acceptance signal includes information related to the examinee corresponding to the input (double-clicked) icon 49 (in this embodiment, the examinee ID). In this embodiment, even when the captured image displayed inside icon 49 is double-clicked, supervisor terminal 3 and / or management server 4 consider that an input has been made to icon 49, and supervisor terminal 3 transmits a certification acceptance signal including the examinee ID of the examinee to management server 4. When certification result acceptance unit 46 of management server 4 receives the certification acceptance signal, it displays a pop-up 60 on monitor 27A of supervisor terminal 3 for accepting the supervisor's certification result for the detection result.

[0097] Figures 7, 8 and 9 show examples of a pop-up 60 displayed to accept the certification result. Figures 7 and 8 show an example of the pop-up 60 when a suspicious activity is detected, and Figure 9 shows an example of the pop-up 60 when an impersonation activity is detected.

[0098] The pop-up 60 displays information related to the corresponding examinee, such as personal information and a captured image. The personal information included in the pop-up 60 may include, for example, a name display field 61 that displays the name of the corresponding examinee. By including information related to the examinee in the pop-up 60 in this manner, the supervisor can easily grasp information about the examinee whose suspicious behavior or impersonation has been detected. Furthermore, the pop-up 60 may include an image display field 62 that displays the examinee's history image 41C, a slider 63 for changing the time at which the history image 41C is displayed, a play button 64 for playing the history image 41C as a video, a stop button 65 for stopping the played history image 41C, a rewind button 66 for rewinding the time of playback, and the like. By moving the slider 63, the supervisor may be able to check the history image 41C of the examinee for a predetermined time period (e.g., ±5 seconds) including the time at which the suspicious behavior was determined.

[0099] The pop-up 60 may include a display field 68 for displaying the time when the suspicious activity was detected and the type of suspicious activity detected by the suspicious activity detection unit 43. Providing the display field 68 in the pop-up 60 makes it possible to notify the supervisor of the type of suspicious activity that was detected. In addition, the pop-up 60 may preferably include the time when the impersonation activity was detected, etc.

[0100] The certification result receiving unit 46 receives the certification result of the supervisor regarding the detected suspicious behavior or impersonation behavior. The certification result receiving unit 46 can be realized by the processor 31 cooperating with the memory such as the RAM 32 and the ROM 33 and / or the storage 34.

[0101] 7, if suspicious behavior is detected in an examinee, the determination result receiving unit 46 generates a pop-up 60 including a suspicious behavior determination button 70 for receiving the supervisor's judgment as to whether or not suspicious behavior has occurred, and displays the pop-up 60 on the supervisor terminal 3. The supervisor checks the history image 41C to determine whether or not suspicious behavior has occurred, and if suspicious behavior has occurred, enters the information by clicking the suspicious behavior determination button 70.

[0102] In this embodiment, when it is determined that no suspicious activity has occurred, the determination result receiving unit 46 generates a pop-up 60 including a button for the supervisor to input (hereinafter, suspicious activity denial button 71), and displays the pop-up 60 on the supervisor terminal 3. When there is input into the suspicious activity denial button 71, the determination result receiving unit 46 sets (i.e. resets) the suspicious activity detection flag 41D of the corresponding examinee to 0.

[0103] When the suspicious activity determination button 70 is pressed, the determination result receiving unit 46 may display a pop-up 60 for receiving a request to issue a warning to the examinee who has been determined to have engaged in suspicious activity, on the monitor 27A of the supervisor terminal 3. Figure 8 shows an example of the pop-up 60 for receiving a request to issue a warning.

[0104] As shown in Fig. 8, the pop-up 60 may include a setting input field 75 for warning information to be sent to the examinee terminal 2 when a warning is issued. This allows the certification result receiving unit 46 to receive setting input for setting the warning information at the supervisor terminal 3. In the example shown in Fig. 9, the setting input field 75 is composed of a check box 76 for the action content and an input field 77 for inputting text to be displayed on the examinee terminal 2 of the examinee who has been certified as having engaged in suspicious activity. The pop-up 60 is provided with an execute button 78 for causing the management server 4 to send warning information to the corresponding examinee terminal 2 and for issuing a warning to the examinee.

[0105] 9, if impersonation of an examinee is detected, the certification result receiving unit 46 generates a pop-up 60 including an impersonation confirmation button 80 for receiving the supervisor's determination as to whether or not impersonation has occurred, and causes the supervisor terminal 3 to display the pop-up 60. The supervisor checks the history image 41C to determine whether or not impersonation has occurred. If the supervisor determines that impersonation has occurred, he or she inputs the information by clicking the impersonation confirmation button 80.

[0106] In this embodiment, when it is determined that no impersonation has occurred, the certification result receiving unit 46 generates a pop-up 60 including a button for the supervisor to input information (hereinafter, the impersonation denial button 81), and displays the pop-up 60 on the supervisor terminal 3. When there is input into the impersonation denial button 81, the certification result receiving unit 46 sets (i.e., resets) the face authentication NG flag 41E of the corresponding examinee to 0.

[0107] The pop-up 60 displayed on the supervisor terminal 3 when an impersonation act is detected may include, in the same way as when a suspicious act is detected, a slider 63 for changing the time, a play button 64 for playing the history image 41C, a stop button 65 for stopping the history image 41C, a rewind button 66 for rewinding the time being played, etc. Also, in the same way as when a suspicious act is detected, the pop-up 60 may be provided with a display field 68 indicating the time when the impersonation act was detected by the impersonation act detection unit 44.

[0108] In addition, the pop-up 60 displayed on the supervisor terminal 3 when an impersonation attempt is detected may display a display field 82 (e.g., a facial photo or the like) of the examinee's facial information stored in the storage 34, and an image display field 62 displaying the history image 41C side by side. Additionally, the pop-up 60 may be provided with a score display field 84 that displays the degree of match between the facial information and the facial image captured in the captured image as a score.

[0109] The determination result receiving unit 46 acquires the determination result regarding the presence or absence of suspicious activity or the presence or absence of impersonation activity received at the supervisor terminal 3, and stores it in the memory unit 41 (storage 34).

[0110] The model update unit 47 updates the learning model 41B based on the certification result stored in the memory unit 41 (storage 34). The model update unit 47 can be realized by the processor 31 working in cooperation with memories such as the RAM 32 and the ROM 33 and the storage 34. In this embodiment, the model update unit 47 may update the learning model 41B by re-learning at an appropriate timing based on the certification result stored in the storage 34 and the history image 41C.

[0111] Next, a description will be given of the suspicious behavior detection process and the impersonation detection process executed by the processor 31 of the management server 4. Before the start of the exam, the processor 31 of the management server 4 sets the suspicious behavior detection flag 41D and the face authentication NG flag 41E of all examinees stored in the storage 34 to 0. When the exam start time arrives, captured images are constantly acquired from each examinee terminal 2, and the suspicious behavior detection process and the impersonation detection process are simultaneously executed in parallel for each examinee.

[0112] A flow diagram of the suspicious behavior detection process is shown in Fig. 10. After the start of the test, the processor 31 of the management server 4 executes the suspicious behavior detection process for each examinee.

[0113] 10, in the first step S101 of the suspicious activity detection process, the processor 31 acquires a video for detecting a suspicious activity (a video for detection). The processor 31 may generate the video for detection by, for example, extracting a portion of a predetermined time width from the history video stored in the storage 34. When the acquisition of the video for detection is completed, the processor 31 executes step S102.

[0114] In S102, the processor 31 detects suspicious behavior based on the acquired detection video and determines whether or not there is a suspicious behavior. At this time, the processor 31 may detect the suspicious behavior using the learning model 41B stored in the storage 34. Also, at this time, when a suspicious behavior is detected, the processor 31 may obtain the type of the suspicious behavior.

[0115] In step S102, if processor 31 determines based on the detection result that a suspicious act has occurred, processor 31 executes step S103, and if processor 31 determines that a suspicious act has not occurred, processor 31 executes step S101.

[0116] In step S103, the processor 31 sets the suspicious activity detection flag 41D to 1. After the setting of the suspicious activity detection flag 41D is completed, the processor 31 executes step S104.

[0117] In step S104, processor 31 determines whether or not there has been an input from the supervisor to set suspicious activity detection flag 41D to 0 (for example, an input to all clear button 57 in FIG. 3 or the no suspicious activity button in FIG. 7). When there has been an input to set suspicious activity detection flag 41D to 0, processor 31 executes step S105. When there has been no input to set suspicious activity detection flag 41D to 0, processor 31 executes step S104 and waits until there is an input to set suspicious activity detection flag 41D to 0.

[0118] In step S105, the processor 31 sets the suspicious activity detection flag 41D to 0. After completing the setting, the processor 31 executes step S106.

[0119] In step S106, the processor 31 determines whether the test end time has arrived (or the test end time has passed). If the test end time has arrived (or has passed), the processor 31 ends the suspicious behavior detection process, and if the test end time has not yet arrived, the processor 31 executes step S101.

[0120] A flow diagram of the impersonation detection process is shown in Fig. 11. After the start of the test, the processor 31 executes the impersonation detection process for each examinee.

[0121] 11, in the first step S201 of the spoofing detection process, the processor 31 starts a timer. After the timer has been started, the processor 31 executes step S202.

[0122] In S202, processor 31 acquires the value of suspicious activity detection flag 41D of the examinee who is the target of the currently executed impersonation activity detection process. Then, processor 31 executes step S203 when the acquired suspicious activity detection flag 41D is 1, and executes step S204 when the suspicious activity detection flag 41D is 0.

[0123] In S203, the processor 31 acquires a detection image to be used for face detection and face authentication. The detection image may be a single image newly acquired by the imaging device 16 when S203 is executed, or may be an image cut out from the video acquired in step S101. The processor 31 may cut out a plurality of images from the video acquired in step S101, and select one of the cut images to acquire the detection image. When acquisition of the detection image is completed, the processor 31 executes S205.

[0124] In step S204, processor 31 determines whether a predetermined time (e.g., 5 minutes) has elapsed since the timer was started, that is, whether the timer is up. If it is determined that the timer is up, step S206 is executed, and if the timer is not up, step S202 is executed.

[0125] In step S205, the processor 31 detects a face portion from the face image for detection. In this embodiment, the processor 31 extracts a face portion from the face image for detection and acquires a face feature amount. When the detection of the face portion is completed, the processor 31 executes S207.

[0126] In step S206, the processor 31 resets the timer and starts it again. When the timer has been started, the processor 31 executes step S203.

[0127] In step S207, the processor 31 acquires facial information of the examinee who is the target of the impersonation detection process from the storage 34. The processor 31 then performs facial authentication by comparing the acquired facial information with the detection result acquired in step S205 (i.e., performing facial matching). If the processor 31 acquires facial features in step S205, it is preferable that the processor 31 compares the facial features with the facial information and determines whether or not facial authentication is possible.

[0128] If the face information cannot be matched with the detection result of step S205 (i.e., face authentication cannot be performed), processor 31 executes step S208, and if the face information can be matched (face authentication can be performed), processor 31 executes step S202.

[0129] In S208, the processor 31 sets the face authentication NG flag 41E of the examinee who is the target of the impersonation act detection process to 1. When the face authentication flag setting is completed, the processor 31 executes step S209.

[0130] In step S209, processor 31 determines whether there has been an input from the supervisor to set (reset) face authentication NG flag 41E to 0. Processor 31 may determine that there has been an input to set the flag to 0 when there has been an input to, for example, all cancel button 57 in FIG. 3 or the no impersonation button in FIG. 9. When there has been an input to set face authentication NG flag 41E to 0, processor 31 executes step S210. When there has been no input to set face authentication NG flag 41E to 0, processor 31 executes step S209 and waits until there is an input to set face authentication NG flag 41E to 0.

[0131] In step S210, processor 31 sets face authentication NG flag 41E of the examinee who is the target of the impersonation detection process to 0. After setting face authentication NG flag 41E, processor 31 executes step S211.

[0132] In step S211, the processor 31 determines whether the test end time has arrived (or the test end time has passed). If the test end time has arrived (or passed), the processor 31 ends the suspicious behavior detection process, and if the test end time has not yet arrived, the processor 31 executes step S202.

[0133] Next, a description will be given of the operation and effects of the monitoring system 1. Before the start of a test, the suspicious behavior detection flag 41D and the face authentication NG flag 41E of each examinee are both set to 0.

[0134] When the test starts, the processor 31 of the management server 4 performs the suspicious behavior detection process and the impersonation detection process in parallel for each examinee. In addition, the processor 31 of the management server 4 sets the display of the icon 49 based on the suspicious behavior detection flag 41D and the face authentication NG flag 41E from the start of the test to the end of the test, and notifies the supervisor terminal 3.

[0135] When a suspicious act is committed by a specific examinee, the processor 31 of the management server 4 detects the suspicious act based on the captured image of the examinee (Yes in S101). The processor 31 of the management server 4 then sets the suspicious act detection flag 41D of the examinee who committed the suspicious act to 1 (S102). When the suspicious act detection flag 41D becomes 1, the processor 31 of the management server 4 performs display settings for the icon 49 so that the thickness of the icon 49 (frame) of the examinee who committed the suspicious act is thicker than usual. This changes the display mode of the icon 49 of the examinee who committed the suspicious act, allowing the supervisor to easily understand that a suspicious act has been detected.

[0136] When the suspicious behavior detection flag 41D becomes 1, the processor 31 of the management server 4 determines that the suspicious behavior detection flag 41D has become 1 in the impersonation detection process (Yes in S202), and performs facial authentication using the captured image and facial information of the examinee for whom suspicious behavior was detected (S204).

[0137] If face authentication is not possible (No in S204), face authentication NG flag 41E becomes 1 (S206), and it is determined that there is a possibility that impersonation has occurred. When face authentication NG flag 41E becomes 1, the display setting of icon 49 is performed so that the thickness of the icon 49 (frame) of the examinee who has engaged in suspicious behavior becomes thicker than when the suspicious behavior was detected. This changes the display mode of icon 49 of the examinee who has been determined to have engaged in possible impersonation, so that the supervisor can easily understand that it has been determined that there is a possibility that impersonation has occurred.

[0138] In this way, in the monitoring system 1 of the present disclosure, when a suspicious act is detected, face recognition is performed to determine whether or not there is a possibility of impersonation. This makes it possible to reduce the frequency of periodic face recognition, and to reduce the processing load of the processor 31 required for detecting fraudulent acts.

[0139] As described above, the embodiments have been described as examples of the technology disclosed in this application. However, the technology in this disclosure is not limited to these, and can be applied to embodiments in which modifications, substitutions, additions, omissions, etc. are made. In addition, it is also possible to combine the components described in the above embodiments to create new embodiments.

[0140] In the above embodiment, an example has been described in which the monitoring system 1 is used to monitor examinees participating in an examination, but this is not limited to this embodiment, and the monitoring system 1 can also be applied to a system for monitoring participants in various seminars, training courses, etc.

[0141] In addition, in the above embodiment, the display mode of the frame of the icon 49 was configured to be set based on two flags, the suspicious activity detection flag 41D and the face authentication NG flag 41E, but the present invention is not limited to this mode.

[0142] The processor 31 of the management server 4 may set the frame of the icon 49 to a second color (red) when an action such as lifting the smartphone or looking at the smartphone screen is detected based on the captured image. In detail, the processor 31 of the management server 4 may store a suspicious action level corresponding to the type of suspicious action for each examinee, and obtain the suspicious action level of the examinee based on the captured image and the learning model 41B. The processor 31 of the management server 4 may then set the display mode of the icon 49 based on the obtained suspicious action level.

[0143] In the above embodiment, the color of the icon 49 is configured to be the first color (yellow) when a suspicious act is detected, regardless of the number of suspicious acts, but the color of the icon 49 may be configured to change depending on the number of suspicious acts detected. For example, the color of the icon 49 may be configured to change depending on the number of times the suspicious act is detected. Specifically, the color of the icon 49 may be configured to change from the first color (yellow) to the second color (red) when a suspicious act is detected several times.

[0144] In the above embodiment, the monitoring screen 48 is configured to display the captured image in the icon 49 (square frame) in real time, but the present invention is not limited to this. The icon 49 may be a square with a filled interior. Information related to the examinee, such as the examinee number, the examinee's name, and the seat number, may be displayed inside or near the icon 49. Also, the monitoring screen 48 may be configured to display a setting screen for setting the display of the icon 49 by performing a predetermined input on the monitoring screen 48.

[0145] Also, a configuration may be made such that a setting screen for setting the type of suspicious behavior to be extracted is displayed by making a predetermined input on the monitoring screen 48. The type of suspicious behavior may be set by inputting into a check box (not shown). This makes it possible to change the type of behavior recognized as suspicious behavior, for example, when the monitoring system 1 is used for checking the degree of understanding in learning where textbooks, etc. can be referred to, and when the monitoring system 1 is used for exams, etc. where textbooks, etc. cannot be referred to.

[0146] Additionally, the examinee terminal 2 may be provided with a microphone, and the management server 4 may be configured to detect suspicious behavior or impersonation using the voice captured by the microphone. By using a microphone, for example, cheating or someone other than the examinee answering a question can be detected. The management server 4 may be configured to detect suspicious behavior or impersonation based only on the captured images captured by the on-site imaging device 5, or may be configured to detect suspicious behavior or impersonation by using the on-site imaging device 5, imaging device 16, and microphone in combination.

[0147] In the above embodiment, the processor 31 is configured to set the suspicious activity detection flag 41D to 1 after determining that a suspicious activity has occurred, execute step S104, and wait until a reset input is received, but this is not limited to the above embodiment. The processor 31 may continue to detect suspicious activities after setting the suspicious activity detection flag 41D to 1. The processor 31 may also record the presence or absence of a suspicious activity in the storage unit 41 as a detection log together with the time when the suspicious activity was detected.

[0148] In the above embodiment, the timer is started in step S201 or S206, and when a predetermined time (e.g., 5 minutes) has elapsed (Yes in S204), a detection image is acquired (S203), but the system may be configured to acquire a detection image and perform face recognition every time a predetermined time has elapsed since the start of the test. Specifically, the system may be configured to perform face recognition at timings such as 15 minutes, 30 minutes, and 45 minutes after the start of the test.

[0149] Similarly, in the above embodiment, when face authentication is not successful (No in S207), the processor 31 sets the face authentication NG flag 41E to 1 (S208) and then waits until a reset input is received (S209), but this is not limited to the above embodiment. After setting the face authentication NG flag 41E to 1, the processor 31 may continue to detect an impersonation act. In addition, the processor 31 may record the presence or absence of an impersonation act in the storage unit 41 as a detection log together with the time when the impersonation act was detected.

[0150] The processor 31 of the management server 4 may be configured to, when the same type of suspicious behavior is detected in a number of examinees equal to or greater than a predetermined threshold in step S101, maintain the suspicious behavior detection flags 41D at 0, rather than setting the suspicious behavior detection flags 41D of the examinees in which the same type of suspicious behavior was detected in step S102 to 1. This makes it possible to prevent, for example, when there are many restless examinees at the start of an exam, the suspicious behavior detection flags 41D from all being set to 1 for those examinees. [Industrial Applicability]

[0151] The surveillance system according to the present disclosure is a surveillance system and a surveillance method that detects fraudulent activity using facial recognition, and is useful as a surveillance system and a surveillance method that can reduce the processing load required to detect fraudulent activity. [Explanation of symbols]

[0152] 1: Surveillance system 2: Exam taker's device 3: Supervisor terminal 4: Management server 5: On-site imaging device 5A: Camera 11: Processor 12: RAM 13: ROM 14: Storage 15: Communication interface 16: Imaging device 16A: Video camera 17: Notification device 17A:Monitor 17B: Speaker 18: Input device 18A: Keyboard 18B: Mouse 21: Processor 22: RAM 23: ROM 24: Storage 25: Communication interface 27: Notification device 27A: Monitor (display device) 27B: Speaker 28: Input device 28A: Keyboard 28B: Mouse 28C:Touchpad 31: Processor 32: RAM 33: ROM 34: Storage 35: Communication interface 41: Storage section 41A: Exam taker information table 41B: Learning model 41C: Historical images 41D: Suspicious activity detection flag 41E: Face recognition NG flag 42: Image capture unit 43: Suspicious Activity Detection Unit 44: Spoofing detection unit 45: Icon display settings section 46: Certification Result Reception Department 47: Model Update Section 48: Monitoring screen 49: Icon 51: Timing setting section 52: Face detection unit 53: Face matching unit 55: Sort button 56: General Attention Button 57: Cancel all at once button 60: Popup 61: Name display field 62: Image display column 63: Slider 64: Play button 65: Stop button 66: Rewind button 68:Display field 70: Suspicious activity confirmation button 71: Suspicious activity denial button 75: Setting input field 76: Checkbox 77: Input field 78: Execute button 80: Impersonation confirmation button 81: Impersonation denial button 82:Display field 84: Score display area

Claims

1. 1. A surveillance system comprising: A storage device that stores face information of participants in advance; An imaging device capable of imaging each of the participants; A processor that determines whether or not there is any suspicious activity based on the image captured by the imaging device; a notification device that notifies the determination result of the processor, The processor, determining whether or not there is any suspicious behavior among the participants based on the images captured by the imaging device; When it is determined that the suspicious behavior has occurred, a face authentication is performed to compare the image of the participant who is determined to have engaged in the suspicious behavior with the face information of the participant who is determined to have engaged in the suspicious behavior; The monitoring system determines that there is a possibility of impersonation when a match cannot be made in the face authentication.

2. the notification device includes a display device that displays a screen for a supervisor; The monitoring system according to claim 1 , wherein the processor causes the display device to display an icon corresponding to the participant in a color corresponding to the determination result.

3. The surveillance system described in claim 2, wherein the processor causes the display device to display the icon corresponding to the participant who is determined to have engaged in suspicious activity in a first color, and to display the icon corresponding to the participant who is determined to have engaged in impersonation activity in a second color.

4. The surveillance system described in claim 3, wherein the processor displays on the display device the icon corresponding to a participant who has been determined to have engaged in suspicious activity and who has been determined to have engaged in impersonation activity, alternating between the first color and the second color.

5. an input device for receiving an input from the supervisor with respect to the icon; A monitoring system as described in any one of claims 2 to 4, wherein the processor, when receiving input to the icon in the input device, causes the display device to display information relating to the participant corresponding to the icon.

6. The monitoring system according to claim 1 , wherein when the processor determines that a suspicious activity has occurred, the processor obtains a type of the suspicious activity and causes the notification device to notify the type of the suspicious activity.

7. The storage device stores a suspicious behavior detection flag set for each of the participants, The processor, setting the suspicious activity detection flags of all of the participants to a first value before the start of a test; If it is determined that the suspicious behavior has occurred after the start of the test, the suspicious behavior detection flag of the participant determined to have engaged in the suspicious behavior is set to a second value; The surveillance system according to claim 1 , wherein when the suspicious behavior detection flag becomes the second value, face authentication is performed on the participant whose suspicious behavior detection flag has become the second value.

8. an input device for receiving input from a supervisor; The surveillance system according to claim 7 , wherein the processor sets the suspicious activity detection flag corresponding to the participant to the first value based on an input to the input device.

9. an input device for receiving input from a supervisor; The surveillance system according to claim 1 , wherein the processor transmits warning information to a terminal corresponding to the participant determined to have engaged in the suspicious activity, based on an input to the input device.

10. The surveillance system according to claim 9 , wherein the processor is configured to broadcast the warning information to terminals corresponding to the participants in which it has been determined that the suspicious activity has occurred, based on an input to the input device.

11. 11. The monitoring system according to claim 9, wherein the processor receives a setting input for setting the warning information from the input device.

12. an input device for receiving input from a supervisor; The monitoring system of claim 1, wherein the processor accepts, at the input device, a determination by the supervisor as to whether or not the participant determined to have engaged in suspicious activity has engaged in fraudulent activity, and stores the determination result in the storage device.

13. the storage device stores a learning model for determining whether or not the suspicious activity exists; The monitoring system of claim 12 , wherein the processor updates the learning model based on the recognition results stored in the storage device.

14. 1. A monitoring method, comprising: The method is executed by a surveillance system including: a storage device that stores face information of participants in advance; an imaging device that can image each of the participants; a processor that determines whether or not there is a suspicious activity based on the image acquired by the imaging device; and a notification device that notifies the result of the determination by the processor; The processor, determining whether or not the suspicious behavior of each of the participants has occurred based on the image captured by the imaging device; When it is determined that the suspicious behavior has occurred, a face authentication is performed to compare the image of the participant who is determined to have engaged in the suspicious behavior with the face information of the participant who is determined to have engaged in the suspicious behavior; The monitoring method determines that there is a possibility of impersonation when a match cannot be made in the face authentication.

Images