A Multi-Perspective Method for Recognizing Cheating Behavior in Paperless Online Examinations

By combining multi-view visual sensors with transfer learning and network models, the problems of monitoring blind spots and inaccurate behavior classification in online exam cheating detection are solved, enabling detailed identification and real-time evidence collection of online exam cheating behavior, and supporting intelligent proctoring for online paperless exams.

CN119152552BActive Publication Date: 2026-11-17NANTONG UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410425700.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2023-06-14
Filing Date
2024-04-10
Publication Date
2026-11-17
Estimated Expiration
2044-04-10

AI Technical Summary

Technical Problem

Existing online exam cheating detection methods suffer from incomplete monitoring scope, blind spots, lack of visual images showing the direction of the examinee's gaze, and insufficient detail and comprehensiveness in behavior classification, leading to inaccurate judgment.

Method used

Employing multi-view visual sensors, this system combines PC screen cameras, side-view cameras, and head-mounted cameras with transfer learning and specific network models to construct modules for recognizing line of sight, desktop environment objects, and abnormal targets. A multilayer perceptron network is then built to discriminate behaviors, enabling detailed classification and real-time identification of cheating behaviors in online examinations.

Benefits of technology

It covers areas where cheating may occur during exams, enables detailed classification and identification of different types of cheating behaviors, provides real-time online image capture and evidence collection, and supports intelligent proctoring for online paperless exams.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119152552B_ABST
    Figure CN119152552B_ABST
Patent Text Reader

Abstract

This invention discloses a multi-view paperless online examination cheating behavior detection method. Specifically, step 1 involves acquiring and preprocessing a candidate's facial image using a PC screen camera, and then constructing a candidate's gaze direction recognition module using transfer learning combined with a SwingTransformer network. Step 2 involves acquiring an examination desktop environment image using a side-view camera, and constructing a desktop environment object recognition module using transfer learning combined with a YOLOX network. Step 3 involves acquiring a candidate's first-person view image using a head-mounted camera, and constructing an abnormal target recognition module from the candidate's first-person view using transfer learning combined with YOLOX. Step 4 involves constructing a suspected cheating behavior classification module for standard online paperless examinations based on the above detection results, identifying the type of suspected cheating by the candidate in the online examination, and capturing cheating evidence images from three perspectives. This invention can automatically detect common abnormal examination behaviors during paperless online examinations, providing invigilators with objective invigilation evidence and reducing labor costs.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to a multi-view paperless online examination cheating behavior identification method, belonging to the field of multi-view visual sensor data fusion technology. Background Technology

[0002] Since the late 20th century, online education has emerged with technological advancements. To maintain normal operations, schools at all levels have shifted their teaching online, driving the rapid development of online education and creating a pressing need for efficient online examination proctoring. However, the development of online examinations has also raised issues of exam integrity. Therefore, timely, efficient, and automated online examinations, as a form of behavioral identification, are of great significance to the development of online education.

[0003] Currently, research on online paperless exam cheating detection generally focuses on a single perspective, mainly falling into two categories: "Single-view cheating detection": Detecting online exam cheating behavior by detecting gaze points after calibration and using facial information for identity verification under the main view image; "Dual-view cheating detection": Estimating gaze direction under the main view image and detecting environmental objects and estimating examinee posture under the side view image, combining multi-view information to judge exam behavior.

[0004] In summary, existing online exam cheating detection methods suffer from the following problems: ① The monitoring range of online exam cheating detection is incomplete, with blind spots and a lack of visual field images showing the candidate's line of sight; ② The classification of online exam cheating behaviors is not detailed or comprehensive enough, and the behavior identification is inaccurate. To address these problems, this invention proposes a method for identifying and collecting evidence of cheating behaviors in online paperless exams based on multi-view visual sensors. Summary of the Invention

[0005] To address the problems existing in the prior art, this invention provides a method for identifying and collecting evidence of cheating behavior in online paperless examinations using a multi-view visual sensor, thereby solving the aforementioned technical problems.

[0006] To achieve the above objectives, the technical solution adopted by this invention is: a multi-view paperless online examination cheating behavior identification method, comprising the following steps:

[0007] Step S1: Obtain and preprocess the candidate's facial image through the PC screen camera (first view). Utilize transfer learning combined with the SwingTransformer network to construct a candidate's gaze direction recognition module to detect abnormal gaze directions (peeping) of the candidate from the first view.

[0008] Step S2: Obtain an image of the exam desktop environment through a side-view camera (side view). Utilize transfer learning combined with the YOLOX network to construct a desktop environment item recognition module to detect common exam cheating-related items on the desktop.

[0009] Step S3: Obtain the candidate's first-person view image through a head-mounted camera (first-person perspective), and use transfer learning combined with the YOLOX network to construct an abnormal target recognition module under the candidate's first-person perspective to detect common cheating targets;

[0010] Step S4: Based on the detection results of steps S1, S2, and S3, construct a classification module for suspected cheating behavior in online paperless standard examinations, identify the types of suspected cheating by candidates in online examinations, and capture cheating evidence images from three perspectives.

[0011] Furthermore, step S1 specifically includes:

[0012] S11: First, collect the candidate's main view image based on the PC screen camera, and use the feature points numbered 1, 8, 15, and 20 in the HOG algorithm of the Dlib function to preprocess the image to obtain the facial image;

[0013] S12: Next, establish a two-dimensional coordinate axis with the center of the screen as the origin, and divide each viewing direction with 45° as the threshold. The viewing directions include nine directions: upper left, straight left, lower left, straight up, center, straight down, upper right, straight right, and lower right.

[0014] S13: Finally, an optimal model for gaze direction recognition is established using transfer learning based on the SWIN-T pre-trained weights of the Swin Transformer network. The input is the preprocessed facial image, and the output is the confidence score of each gaze direction in the image. A candidate gaze direction recognition module is constructed to detect abnormal gaze directions (peeping) of candidates from the main viewpoint.

[0015] Furthermore, step S2 specifically includes:

[0016] S21: First, a dataset of unusual items in the desktop environment is collected based on a side-view camera, including three common unusual items in online paperless exams: books, paper, and electronic devices.

[0017] S22: Secondly, an optimal model for detecting abnormal items on the desktop is established using transfer learning based on YOLOX-s pre-trained weights. The output is the bounding box and confidence score of each abnormal item. A desktop environment item recognition module is constructed to detect common exam cheating-related items on the desktop.

[0018] Furthermore, step S3 specifically includes:

[0019] S31: First, the hardware for collecting first-person perspective data of examinees is based on a self-assembling head-mounted bracket and a USB camera. The first-person perspective abnormal target dataset is obtained based on the head-mounted camera, which includes four categories of online paperless examination targets: people, books, paper and electronic devices.

[0020] S32: Secondly, an optimal target detection model based on YOLOX-s pre-trained weights of YOLOX is established using transfer learning. The output is the bounding box and confidence score of the target. An abnormal target recognition module is constructed in the first-person perspective of the candidate to detect common cheating targets.

[0021] Furthermore, step S4 specifically includes:

[0022] S41: First, perform MAX-MIN normalization based on the obtained gaze direction detection confidence score. The specific method is as follows:

[0023]

[0024] Where, p θi This represents the confidence level (probability) of the candidate's line of sight in nine directions from the candidate's primary perspective. θmax p represents the maximum confidence level among the nine line-of-sight directions. θmin The minimum confidence (probability) among the nine gaze directions. After image normalization, the confidence value x for the i-th gaze direction. i Located between (0, 1);

[0025] S42: Next, using the output of the candidate's line of sight recognition module described in step S1 and the output of the desktop environment object recognition module described in step S2 as inputs, a "front view-side view" joint cheating behavior discrimination network based on multilayer perceptron is constructed.

[0026] S43: Again, using the output of the candidate's line of sight recognition module described in step S1 and the output of the candidate's first-person perspective abnormal target recognition module described in step S3 as input, construct a candidate "main view-first view" joint cheating behavior discrimination network based on multilayer perceptron.

[0027] S44: Finally, based on the output results of the two multilayer perceptrons mentioned above, a classification module for suspected cheating behavior in online paperless standard examinations is constructed to identify 13 common behaviors of suspected cheating in online examinations and capture cheating evidence images from three perspectives.

[0028] Furthermore, step S42 specifically includes:

[0029] S42-1 is a cheating behavior discrimination network based on a multilayer perceptron with a "first-view-second-view" joint perspective. The network consists of an input layer, a hidden layer, and an output layer.

[0030] The input layer S42-2 has 12 neurons. The first 9 neurons are the normalized values ​​of the 9 gaze direction recognition results from step S1, as described in step S41. These include: probability x1 for the upper left visual field, probability x2 for the left frontal visual field, probability x3 for the lower left visual field, probability x4 for the upper frontal visual field, probability x5 for the center visual field, probability x6 for the lower frontal visual field, probability x7 for the upper right visual field, probability x8 for the right frontal visual field, and probability x9 for the lower right visual field. The last 3 neurons represent the confidence levels of the three abnormal items output from step S2, including: confidence level for a book x... 10 Paper confidence level x 11 Confidence level of electronic devices x 12 ;

[0031] The number of neurons in the S42-3 hidden layer is the same as that in the output layer, which is 12 neurons.

[0032] The S42-4 output layer has 9 neurons, representing 9 types of cheating behaviors performed near the desktop. These include: looking at paper (y1) at or below the desktop; looking at books (y2) at or below the desktop; looking at electronic devices (y3) looking at paper behind the computer; looking at books behind the computer; looking at electronic devices behind the computer; looking at multiple prohibited items (y7) looking at multiple prohibited items behind the computer; and normal behavior from a side view (y9).

[0033] S42-5 uses the backpropagation method to train and obtain the optimal weight values ​​of the three-layer perceptron.

[0034] Furthermore, step S43 specifically includes:

[0035] S43-1 is a cheating behavior discrimination network based on multilayer perceptron with a "first-person perspective" and a "main view - first-person perspective" structure, consisting of three layers: an input layer, a hidden layer, and an output layer.

[0036] The input layer S43-2 has 13 neurons. The first 9 neurons are the normalized values ​​of the 9 gaze direction recognition results from step S1, as described in step S41. These include: probability x1 for the upper left visual field, probability x2 for the left frontal visual field, probability x3 for the lower left visual field, probability x4 for the upper frontal visual field, probability x5 for the center visual field, probability x6 for the lower frontal visual field, probability x7 for the upper right visual field, probability x8 for the right frontal visual field, and probability x9 for the lower right visual field. The last 4 neurons represent the four abnormal target confidence levels output from step S3, including: book confidence level x... 10 Paper confidence level x 11 Confidence level of electronic devices x 12 Human confidence level x 13 ;

[0037] The number of neurons in the S43-3 hidden layer is the same as that in the output layer, which is 13 neurons;

[0038] The S43-4 output layer has four neurons, representing four types of cheating behavior occurring near the desktop. These include: assisting the cheater in holding up a book (y1), assisting the cheater in holding up paper (y2), assisting the cheater in holding up an electronic device (y3), and the student exhibiting normal behavior from a first-person perspective (y4).

[0039] S43-5 uses the backpropagation method to train and obtain the optimal weight values ​​of the three-layer perceptron.

[0040] The beneficial effects of the present invention are: by using the method of the present invention, (1) the PC screen camera, the desktop top-down camera, and the wearable camera on the candidate's head are used to cover the area where cheating may occur during the exam; (2) an online cheating judgment classification network is constructed to classify and identify different types of cheating behaviors in detail; (3) real-time online capture and evidence collection is carried out to provide effective intelligent invigilation technology support for online paperless exams. Attached Figure Description

[0041] Figure 1 This is a schematic diagram of the method flow of the present invention;

[0042] Figure 2 (a) is a schematic diagram of the main view image before Dlib processing according to the present invention. Figure 2 (b) is a schematic diagram of the main view image after Dlib processing according to the present invention;

[0043] Figure 3 This is a schematic diagram of the SwinTransformer network of the present invention;

[0044] Figure 4 This is a schematic diagram of the YOLOX network of the present invention;

[0045] Figure 5 This is a schematic diagram of the paperless standard examination suspected cheating behavior classification module of the present invention;

[0046] Figure 6 This is a schematic diagram showing the placement of the three-view vision sensor of the present invention;

[0047] Figure 7 (a) is a schematic diagram of the "side-down view cheating behavior recognition network" structure of the present invention. Figure 7 (b) is a schematic diagram of the "Candidate First-Perspective Cheating Behavior Recognition Network" structure of the present invention;

[0048] Figure 8This is a schematic diagram illustrating an example of abnormal behavior capture and evidence collection from three perspectives according to the present invention. In the diagram, (a) a piece of paper appears, looking towards the desktop and below it; (b) a piece of paper appears, looking towards the desktop and below it; (c) a book appears, looking towards the desktop and below it; (d) an electronic device appears, looking towards the desktop and below it; (e) a book and a collection of objects appear, looking behind the computer; (f) a piece of paper appears, looking behind the computer; (g) a book appears, looking behind the computer; (h) a book and a piece of paper appear, looking behind the computer; (i) a third-party assisted cheating device; (j) a third-party assisted cheating book; and (k) a third-party assisted cheating paper. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. However, it should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the scope of the invention.

[0050] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention.

[0051] like Figure 1 As shown, this invention is a multi-view paperless online examination cheating behavior identification method, including the following steps:

[0052] Step S1: Obtain and preprocess the candidate's facial image through the PC screen camera (first view). Utilize transfer learning combined with the SwingTransformer network to construct a candidate's gaze direction recognition module to detect abnormal gaze directions (peeping) of the candidate from the first view.

[0053] Specifically, S11: First, collect the candidate's main view image based on the PC screen camera, and use the feature points numbered 1, 8, 15, and 20 in the Dlib function HOG algorithm to preprocess the image to obtain the facial image;

[0054] S12: Next, establish a two-dimensional coordinate axis with the center of the screen as the origin, and divide each viewing direction with 45° as the threshold. The viewing directions include nine directions: upper left, straight left, lower left, straight up, center, straight down, upper right, straight right, and lower right.

[0055] S13: Finally, an optimal model for gaze direction recognition is established using transfer learning based on the SWIN-T pre-trained weights of the SWIN Transformer network. The input is the preprocessed facial image, and the output is the confidence score of each gaze direction in the image. This constructs a candidate gaze direction recognition module to detect abnormal gaze directions (peeping) of candidates from the main viewpoint.

[0056] Step S2: Obtain an image of the exam desktop environment through a side-view camera (side view). Utilize transfer learning combined with the YOLOX network to construct a desktop environment item recognition module to detect common exam cheating-related items on the desktop.

[0057] Specifically, S21: First, a dataset of unusual items in the desktop environment is collected based on a side-view camera, including three common unusual items in online paperless exams: books, paper, and electronic devices;

[0058] S22: Secondly, an optimal model for detecting abnormal items on the desktop is established using transfer learning based on YOLOX-s pre-trained weights. The output is the bounding box and confidence score of each abnormal item. A desktop environment item recognition module is constructed to detect common exam cheating-related items on the desktop.

[0059] Step S3: Obtain the candidate's first-person view image through a head-mounted camera (first-person perspective), and use transfer learning combined with the YOLOX network to construct an abnormal target recognition module under the candidate's first-person perspective to detect common cheating targets;

[0060] Specifically, S31: First, the hardware for collecting first-person perspective data of examinees is based on a self-assembled head-mounted bracket and a USB camera. The first-person perspective abnormal target dataset is obtained based on the head-mounted camera, which includes four categories of online paperless examination targets: people, books, paper and electronic devices.

[0061] S32: Secondly, an optimal target detection model based on YOLOX-s pre-trained weights of YOLOX is established using transfer learning. The output is the bounding box and confidence score of the target. An abnormal target recognition module is constructed in the first-person perspective of the candidate to detect common cheating targets.

[0062] Step S4: Based on the detection results of steps S1, S2, and S3, construct a classification module for suspected cheating behavior in online paperless standard examinations, identify the types of suspected cheating by candidates in online examinations, and capture cheating evidence images from three perspectives.

[0063] Specifically, step S4 includes:

[0064] S41: First, perform MAX-MIN normalization based on the obtained gaze direction detection confidence score. The specific method is as follows:

[0065]

[0066] Where, p θi This represents the confidence level (probability) of the candidate's line of sight in nine directions from the candidate's primary perspective. θmax p represents the maximum confidence level among the nine line-of-sight directions. θmin The minimum confidence (probability) among the nine gaze directions. After image normalization, the confidence value x for the i-th gaze direction. i Located between (0, 1);

[0067] S42: Next, using the output of the candidate's line of sight recognition module described in step S1 and the output of the desktop environment object recognition module described in step S2 as inputs, a "front view-side view" joint cheating behavior discrimination network based on multilayer perceptron is constructed.

[0068] S43: Again, using the output of the candidate's line of sight recognition module described in step S1 and the output of the candidate's first-person perspective abnormal target recognition module described in step S3 as input, construct a candidate "main view-first view" joint cheating behavior discrimination network based on multilayer perceptron.

[0069] S44: Finally, based on the output results of the two multilayer perceptrons mentioned above, a classification module for suspected cheating behavior in online paperless standard examinations is constructed to identify 13 common behaviors of suspected cheating in online examinations and capture cheating evidence images from three perspectives.

[0070] In this preferred embodiment, step S42 specifically involves:

[0071] S42-1 is a cheating behavior discrimination network based on a multilayer perceptron with a "first-view-second-view" joint perspective. The network consists of an input layer, a hidden layer, and an output layer.

[0072] The input layer S42-2 has 12 neurons. The first 9 neurons are the normalized values ​​of the 9 gaze direction recognition results from step S1, as described in step S41. These include: probability x1 for the upper left visual field, probability x2 for the left frontal visual field, probability x3 for the lower left visual field, probability x4 for the upper frontal visual field, probability x5 for the center visual field, probability x6 for the lower frontal visual field, probability x7 for the upper right visual field, probability x8 for the right frontal visual field, and probability x9 for the lower right visual field. The last 3 neurons represent the confidence levels of the three abnormal items output from step S2, including: confidence level for a book x... 10 Paper confidence level x 11 Confidence level of electronic devices x 12 ;

[0073] The number of neurons in the S42-3 hidden layer is the same as that in the output layer, which is 12 neurons.

[0074] The S42-4 output layer has 9 neurons, representing 9 types of cheating behaviors performed near the desktop. These include: looking at paper (y1) at or below the desktop; looking at books (y2) at or below the desktop; looking at electronic devices (y3) looking at paper behind the computer; looking at books behind the computer; looking at electronic devices behind the computer; looking at multiple prohibited items (y7) looking at multiple prohibited items behind the computer; and normal behavior from a side view (y9).

[0075] S42-5 uses the backpropagation method to train and obtain the optimal weight values ​​of the three-layer perceptron.

[0076] In this preferred embodiment, step S43 specifically includes:

[0077] S43-1 is a cheating behavior discrimination network based on multilayer perceptron with a "first-person perspective" and a "main view - first-person perspective" structure, consisting of three layers: an input layer, a hidden layer, and an output layer.

[0078] The input layer S43-2 has 13 neurons. The first 9 neurons are the normalized values ​​of the 9 gaze direction recognition results from step S1, as described in step S41. These include: probability x1 for the upper left visual field, probability x2 for the left frontal visual field, probability x3 for the lower left visual field, probability x4 for the upper frontal visual field, probability x5 for the center visual field, probability x6 for the lower frontal visual field, probability x7 for the upper right visual field, probability x8 for the right frontal visual field, and probability x9 for the lower right visual field. The last 4 neurons represent the four abnormal target confidence levels output from step S3, including: book confidence level x... 10 Paper confidence level x 11 Confidence level of electronic devices x 12 Human confidence level x 13 ;

[0079] The number of neurons in the S43-3 hidden layer is the same as that in the output layer, which is 13 neurons.

[0080] The S43-4 output layer has four neurons, representing four types of cheating behavior occurring near the desktop. These include: assisting the cheater in holding up a book (y1), assisting the cheater in holding up paper (y2), assisting the cheater in holding up an electronic device (y3), and the student exhibiting normal behavior from a first-person perspective (y4).

[0081] S43-5 uses the backpropagation method to train and obtain the optimal weight values ​​of the three-layer perceptron.

[0082] Example:

[0083] Step 1) Obtain and preprocess the candidate's facial image using the PC screen camera (first-person view). Then, using transfer learning combined with a SwinTransformer network, construct a candidate gaze direction recognition module to detect abnormal gaze directions (peeping) from the first-person view. The specific content is as follows:

[0084] First, images from the candidate's first-person perspective are collected using the PC screen's webcam, such as... Figure 2 As shown in (a), the facial image is obtained by preprocessing the image using feature points numbered 1, 8, 15, and 20 in the HOG algorithm of the Dlib function, as follows: Figure 2 As shown in (b);

[0085] Secondly, a two-dimensional coordinate axis is established with the center of the screen as the origin, and each viewing direction is divided with 45° as the threshold. The viewing directions include nine directions: upper left, straight left, lower left, upper, center, lower, upper right, straight right, and lower right.

[0086] Finally, an optimal model for gaze direction recognition is established using transfer learning based on the SWIN-T pre-trained weights of the Swin Transformer network. The Swin Transformer network is as follows: Figure 3 As shown, the input is a preprocessed facial image, and the output is the confidence score of each gaze direction in the image. A candidate gaze direction recognition module is constructed to detect abnormal gaze directions (peeking) of candidates from the main viewpoint.

[0087] Step 2) Obtain an image of the exam desktop environment using a side-view camera. Utilize transfer learning combined with a YOLOX network to construct a desktop environment item recognition module to detect common exam cheating-related items on the desktop. The specific content is as follows:

[0088] First, a dataset of unusual items in the desktop environment was collected based on a side-view camera, including three common types of unusual items in online paperless exams: books, paper, and electronic devices.

[0089] Secondly, an optimal model for detecting abnormal items on a desktop is established using transfer learning based on YOLOX-s pre-trained weights. The YOLOX network is as follows: Figure 4 As shown, the output is the bounding box and confidence score of each abnormal item. A desktop environment item recognition module is constructed to detect common exam cheating-related items on the desktop.

[0090] Step 3) Obtain the candidate's first-person perspective image using a head-mounted camera (first-person view), and construct an abnormal target recognition module from the candidate's first-person perspective using transfer learning to detect common cheating aids; the specific content is as follows:

[0091] First, the hardware for collecting first-person perspective data from test takers is based on a self-assembling head-mounted bracket and a USB camera. The head-mounted camera is used to acquire a dataset of abnormal targets from the first-person perspective, which includes four categories of online paperless test targets: people, books, paper, and electronic devices.

[0092] Secondly, an optimal target detection model based on YOLOX-s pre-trained weights from the perspective of examinees is established using transfer learning. The output is the bounding box and confidence score of the target. An abnormal target recognition module is constructed from the perspective of examinees to detect common targets used for cheating.

[0093] Step 4) Based on the detection results of steps 1, 2, and 3, construct an online paperless standard examination suspected cheating behavior classification module, such as... Figure 5 As shown, to identify the types of suspected cheating during online exams, images of cheating evidence are captured from three different perspectives, such as... Figure 6 As shown; its specific content is as follows:

[0094] First, MAX-MIN normalization is performed based on the obtained confidence scores of the gaze direction.

[0095] Secondly, a side-view cheating behavior discrimination network based on a multilayer perceptron is constructed, such as... Figure 7 As shown in (a), the structure includes an input layer, hidden layers, and an output layer. The input layer has 12 neurons, representing the following probabilities: upper left visual field (x1), left frontal visual field (x2), lower left visual field (x3), upper frontal visual field (x4), central visual field (x5), lower frontal visual field (x6), upper right visual field (x7), right frontal visual field (x8), lower right visual field (x9), and book confidence level (x). 10 Paper confidence level x 11 Confidence level of electronic devices x 12 The output layer has 9 neurons, including: looking at the paper below the desktop (y1), looking at the book below the desktop (y2), looking at the electronic device below the desktop (y3), looking at the paper behind the computer (y4), looking at the book behind the computer (y5), looking at the electronic device behind the computer (y6), looking at the desktop and seeing multiple prohibited items (y7), looking at the back of the computer and seeing multiple prohibited items (y8), and normal behavior from a side view (y9); the optimal weight values ​​are obtained through training.

[0096] Secondly, a network for detecting cheating behavior from the first-person perspective of test takers based on a multilayer perceptron is constructed, such as... Figure 7As shown in (b), the structure consists of three layers: an input layer, a hidden layer, and an output layer. The input layer contains 13 neurons, representing the following probabilities: x1 for the upper left visual field, x2 for the left frontal visual field, x3 for the lower left visual field, x4 for the upper frontal visual field, x5 for the central visual field, x6 for the lower frontal visual field, x7 for the upper right visual field, x8 for the right frontal visual field, x9 for the lower right visual field, and x1 for the book confidence level. 10 Paper confidence level x 11 Confidence level of electronic devices x 12 Human confidence level x 13 The output layer has four neurons: y1 (assisting the cheater in holding a book), y2 (assisting the cheater in holding a piece of paper), y3 (assisting the cheater in holding an electronic device), and y4 (the examinee's first-person perspective shows normal behavior). The optimal weight values ​​are obtained through training.

[0097] Finally, based on the outputs of two multilayer perceptrons, a classification module for suspected cheating behavior in online paperless standardized examinations is constructed to identify the types of suspected cheating by examinees in online examinations, capture cheating evidence from three perspectives, and capture evidence images of common abnormal behaviors, such as... Figure 8 As shown.

[0098] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions or improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A multi-view paperless online examination cheating behavior identification method, characterized in that, Includes the following steps: Step S1: Obtain and preprocess the candidate's facial image through the PC screen camera. Utilize transfer learning combined with the SwinTransformer network to construct a candidate's gaze direction recognition module to detect abnormal gaze directions of the candidate from the main viewpoint. Step S2: Obtain an image of the exam desktop environment using a side-view camera. Utilize transfer learning combined with a YOLOX network to construct a desktop environment item recognition module to detect exam cheating-related items on the desktop. Step S3: Obtain the candidate's first-person perspective image using a head-mounted camera, and construct an abnormal target recognition module from the candidate's first-person perspective using transfer learning combined with a YOLOX network to detect targets that assist in cheating; Step S4: Based on the detection results of steps S1, S2, and S3, construct a classification module for suspected cheating behavior in online paperless standard examinations, identify the types of suspected cheating by candidates in online examinations, and capture cheating evidence images from three perspectives. Specifically, step S4 involves: S41: First, perform MAX-MIN normalization based on the obtained gaze direction detection confidence score. The specific method is as follows: in, It is the confidence level of the candidate's line of sight in nine directions from the main perspective; The maximum confidence level among the nine line-of-sight directions. The minimum confidence level among the nine viewing directions; after image normalization, the [missing information] is... i Confidence value for each line of sight direction Located between (0, 1); S42: Next, using the output of the candidate's line of sight recognition module described in step S1 and the output of the desktop environment object recognition module described in step S2 as input, a "front view-side view" joint cheating behavior discrimination network based on multilayer perceptron is constructed; S43: Again, using the output of the candidate's gaze direction recognition module described in step S1 and the output of the candidate's first-person perspective abnormal target recognition module described in step S3 as input, construct a candidate "first-person perspective" joint cheating behavior discrimination network based on multilayer perceptron; S44: Finally, based on the output results of the two multilayer perceptrons mentioned above, a classification module for suspected cheating behavior in online paperless standard examinations is constructed to identify 13 types of suspected cheating behaviors of candidates in online examinations and capture cheating evidence images from three perspectives.

2. The method for identifying cheating behavior in a multi-view paperless online examination according to claim 1, characterized in that, Specifically, step S1 involves: S11: First, collect the candidate's main view image based on the PC screen camera, and use the feature points numbered 1, 8, 15, and 20 in the HOG algorithm of the Dlib function to preprocess the image to obtain the facial image; S12: Next, establish a two-dimensional coordinate axis with the center of the screen as the origin, and divide each viewing direction with 45° as the threshold. The viewing directions include nine directions: upper left, straight left, lower left, straight up, center, straight down, upper right, straight right, and lower right. S13: Finally, an optimal model for gaze direction recognition is established using transfer learning based on the SWIN-T pre-trained weights of the Swin Transformer network. The input is the preprocessed facial image, and the output is the confidence score of each gaze direction in the image. A candidate gaze direction recognition module is constructed to detect abnormal gaze directions of candidates from the main viewpoint.

3. The method for identifying cheating behavior in a multi-view paperless online examination according to claim 1, characterized in that, Specifically, step S2 involves: S21: First, a dataset of abnormal items in the desktop environment is collected based on a side-view camera, including three categories of abnormal items for online paperless exams: books, paper, and electronic devices; S22: Secondly, an optimal model for detecting abnormal items on the desktop is established using transfer learning based on YOLOX-s pre-trained weights. The output is the bounding box and confidence score of each abnormal item. A desktop environment item recognition module is constructed to detect items related to cheating in desktop exams.

4. The method for identifying cheating behavior in a multi-view paperless online examination according to claim 1, characterized in that, Specifically, step S3 involves: S31: First, the hardware for collecting first-person perspective data of examinees is based on a self-assembled head-mounted bracket and a USB camera. The first-person perspective abnormal target dataset is obtained based on the head-mounted camera, which includes four categories of online paperless examination targets: people, books, paper and electronic devices. S32: Secondly, an optimal model for first-person view target detection is established by using transfer learning based on YOLOX-s pre-trained weights. The output is the bounding box and confidence score of the target. An abnormal target recognition module is constructed in the first-person view of the candidate to detect targets that assist in cheating.

5. The method for identifying cheating behavior in a multi-view paperless online examination according to claim 1, characterized in that, Specifically, step S42 involves: S42-1 is a cheating behavior discrimination network based on a multilayer perceptron with a "front-view-side-view" joint perspective. The network includes an input layer, a hidden layer, and an output layer. The input layer S42-2 has 12 neurons. The first 9 neurons are the normalized values ​​of the 9 gaze direction recognition results from step S1, as described in step S41; they include: probability x1 for the upper left visual field, probability x2 for the left frontal visual field, probability x3 for the lower left visual field, probability x4 for the upper frontal visual field, probability x5 for the center visual field, probability x6 for the lower frontal visual field, probability x7 for the upper right visual field, probability x8 for the right frontal visual field, and probability x9 for the lower right visual field. The last 3 neurons are the confidence scores for the three abnormal items output from step S2, including: confidence score for a book x1 for the book. 10 Paper confidence level x 11 Confidence level of electronic devices x 12 ; The number of neurons in the S42-3 hidden layer is the same as that in the output layer, which includes 12 neurons; The S42-4 output layer has 9 neurons, representing 9 types of cheating behaviors performed near the desktop; including: looking at the paper below the desktop (y1), looking at the book below the desktop (y2), looking at the electronic device below the desktop (y3), looking at the paper behind the computer (y4), looking at the book behind the computer (y5), looking at the electronic device behind the computer (y6), looking at the desktop and seeing multiple illegal items (y7), looking at the back of the computer and seeing multiple illegal items (y8), and normal behavior from the side view (y9). S42-5 uses the backpropagation method to train and obtain the optimal weight values ​​of the three-layer perceptron.

6. The method for identifying cheating behavior in a multi-view paperless online examination according to claim 1, characterized in that, Specifically, step S43 involves: The S43-1 is a cheating behavior discrimination network based on a multilayer perceptron with a "first-person perspective" and a "main view" perspective. It consists of three layers: an input layer, a hidden layer, and an output layer. The input layer S43-2 has 13 neurons. The first 9 neurons are the normalized values ​​of the 9 gaze direction recognition results from step S1, as described in step S41, including: probability x1 for the upper left visual field, probability x2 for the left frontal visual field, probability x3 for the lower left visual field, probability x4 for the upper frontal visual field, probability x5 for the center visual field, probability x6 for the lower frontal visual field, probability x7 for the upper right visual field, probability x8 for the right frontal visual field, and probability x9 for the lower right visual field. The last 4 neurons are the confidence scores of the four abnormal targets output from step S3, including: confidence score for the book x1 for the book. 10 Paper confidence level x 11 Confidence level of electronic devices x 12 Human confidence level x 13 ; The number of neurons in the S43-3 single hidden layer is the same as that in the output layer, which includes 13 neurons; The S43-4 output layer has four neurons, representing four types of cheating behavior that occur near the desktop: assisting the cheater in holding up a book (y1), assisting the cheater in holding up paper (y2), assisting the cheater in holding up an electronic device (y3), and the candidate's first-person perspective behavior being normal (y4). S43-5 uses the backpropagation method to train and obtain the optimal weight values ​​of the three-layer perceptron.