Method for detecting cheating behaviors of examinees in examination room based on improved YOLOv8

Abstract

The invention relates to the technical field of computer vision, in particular to an examination room examinee cheating behavior detection method based on improved YOLOv8. The invention provides a CEYOLOV8 model, and aims to remarkably improve the cheating behavior recognition capability. According to the model, the OfficientViT is used as a backbone network to replace a backbone structure in traditional YOLOv8, so that more efficient feature extraction is realized. The CSPHet is introduced to replace a C2F structure in the YOLOv8, so that the expression ability and the calculation efficiency of the CEYOLOV8 are further enhanced. Besides, MPDIOU is adopted as a new loss function, so that the training process of the model is optimized, and the accuracy and robustness of cheating behavior recognition are improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method for detecting cheating behavior of examinees in an examination room based on improved YOLOv8. Background Art

[0002] In today's rapidly developing society, the demand for high-quality professional talents in various industries is becoming more and more urgent, and one of the most common and effective ways to select talents is to objectively evaluate their professional abilities through examinations. Examination invigilation is the key to ensuring that the examinations are conducted in an orderly manner. At present, examination invigilation mainly adopts traditional offline invigilation and online invigilation with camera shooting. Offline examinations are highly dependent on invigilators, which is time-consuming and labor-intensive. In addition, when faced with large-scale or decentralized examination scenarios, the limitations of manual invigilation become more obvious. Although camera invigilation reduces manpower, its effect in preventing cheating is relatively poor.

[0003] With the advancement of information and communication technology and education technology, the application of artificial intelligence (AI) technology, especially computer vision technology in monitoring systems, has gradually attracted attention. Real-time detection and identification of abnormal behaviors in exams through AI technology provides technical support for fair exams. However, a comprehensive data set is a prerequisite for cheating detection and identification. Existing cheating detection networks are often limited to a few common behaviors in identifying cheating types. "Smart Online Exam Proctoring Assist for Cheating Detection. Advanced Data Mining and Applications" uses a CNN network to judge the three indicators of the examinee's head orientation, eye direction and mouth opening to determine whether there is cheating behavior. However, the data sets of these studies are not comprehensive enough, and are limited by technical bottlenecks, and often need to be indirectly reflected by other indicators. Therefore, the present invention designs a relatively comprehensive data set, covering five very representative cheating actions such as turning the head, passing notes, leaving the table with both hands, leaving the seat and bending over. In addition, in order to compare with normal exam behavior, the detection of normal behavior is also added to the data set. The data set is collected in different scenes and camera angles, which enhances the diversity and robustness of the data.

[0004] In previous studies, the examination system mainly used traditional vision-based machine learning methods to detect cheating behavior of candidates. "Machine Learning Based Approach to Exam Cheating Detection" uses recursive neural networks combined with anomaly detection algorithms to identify abnormal scores in final exams. Experimental results show that this method has achieved a high accuracy rate in detecting exam cheating cases. However, traditional machine learning methods rely on manual features and cannot learn the hierarchical representation of objects from data, which affects the efficiency and effectiveness of the system. Summary of the invention

[0005] The purpose of the present invention is to solve the above problems and propose a method for detecting cheating behaviors of examinees in an examination room based on improved YOLOv8, which is used to accurately capture various cheating behaviors in the examination.

[0006] In order to achieve the above-mentioned purpose, the present invention adopts the following technical solutions:

[0007] A method for detecting cheating behavior of examinees in an examination room based on improved YOLOv8, comprising the following steps:

[0008] Step 1: Collect and annotate the image data of candidates’ cheating behaviors, including turning the head, bending over, leaving the seat, leaving both hands on the table, and passing notes. Consider the normal examination posture and pre-process the images to generate different types of cheating behavior data sets.

[0009] Step 2: Randomly divide different types of cheating behavior data sets into training sets and validation sets according to proportion;

[0010] Step 3: Replace the C2f module in YOLOv8 with the CSPHet structure, which reduces the number of parameters and calculations while enhancing the feature expression capability. In addition, EfficientViT is introduced as the backbone network of YOLOv8 to achieve global receptive field and multi-scale feature extraction, and build an algorithm for detecting cheating behavior in the examination room based on the improved YOLOv8.

[0011] Step 4: Use the improved YOLOv8 model to train the training set and the validation set to obtain the optimal cheating behavior detection model for test takers;

[0012] Step 5: Input the cheating behavior pictures of the candidates to be tested into the optimal cheating behavior detection model to obtain the detection results.

[0013] Furthermore, in step one, the preprocessing process is to rotate, flip, add noise, and change the brightness of the image data of the examinee's cheating behavior to achieve a data enhancement effect.

[0014] Furthermore, in the step one, the labeling process is to use the labelimg tool to label the pre-processed pictures of the examinees' cheating behavior in yolo format to obtain different types of examinees' cheating behavior data sets.

[0015] Furthermore, in the step 2, python code is used to divide the picture data set of the examinee's cheating behavior into a training set and a validation set according to a set ratio of 8:2, wherein the training set will be used to train the model, and the validation set is used for the training process and the performance of the model.

[0016] Furthermore, in step 3, the network selects YOLOv8 as the benchmark model, and:

[0017] S1. Use the CSPHet structure to replace the C2f module in YOLOV8. CSPHet reduces the number of parameters and calculations through heterogeneous convolution while enhancing the feature expression capability;

[0018] S2. EfficientViT is introduced as the backbone network for improving YOLOV8. With the help of its multi-scale linear attention mechanism, the global receptive field and multi-scale feature extraction are effectively realized.

[0019] Furthermore, in step 4, the model is trained using the prepared training data set to maximize the accuracy of detecting the examinee's cheating behavior. The training steps are:

[0020] P1. Forward propagation: During each iteration of model training, the image data in the training set will be forward propagated through the neural network; the image input will be processed by multiple convolutional layers and activation function layers; in each layer, the input data will be weighted, activated, and converted, so as to gradually extract more abstract and advanced feature information;

[0021] P2. Loss calculation: Calculate the loss value between the model's predicted box and the true box, including target classification error, target position error, and target confidence error. Here, MPDIoU is used as a new loss function to optimize the model's training process and improve the accuracy and robustness of cheating behavior recognition. The formula is as follows:

[0022]

[0023]

[0024]

[0025] PD=KL(P red ||P gt ) (4)

[0026]

[0027] Among them, P Pred (x, y) represents the probability density at the two-dimensional coordinate (x, y), P gt (x, y) represents the true probability density at the two-dimensional coordinate (x, y), Area of ​​Overlap is the overlapping area of ​​the predicted bounding box and the true bounding box, Area of ​​Union is the area of ​​the joint area, (u x ,u y )and are the coordinates of the center points of the predicted bounding box and the true bounding box, respectively. σ is the standard deviation of the distribution. PD is the penalty distance, which is the Euclidean distance between the center points of the predicted box and the true box. KL divergence is used to calculate the error between the true value and the predicted value.

[0028] Furthermore, in order to evaluate the performance of the improved model, accuracy, recall, and average precision are used as evaluation indicators. At the same time, in order to evaluate the complexity of the model, floating-point operations and parameters are used as evaluation indicators. The calculation formulas of the relevant evaluation indicators are as follows:

[0029]

[0030]

[0031]

[0032]

[0033] Among them, Precision is the accuracy rate, which indicates the ratio of correctly detected positive samples to all samples detected as positive; Recall is the recall rate, which indicates the ratio of correctly detected positive samples to all actual positive samples; mAP is the average detection accuracy of all categories, which indicates the comprehensive performance of the model in the entire data set; Accuracy is used to measure the proportion of correct detections of the model in all category predictions, TP is the number of true positive examples, that is, the number of samples that the model correctly classifies as positive when they are actually positive; FP is the number of positive examples, that is, the number of samples that the model mistakenly classifies as positive when they are actually negative. TN is the number of true negative examples, that is, the number of samples that the model correctly classifies as negative when they are actually negative. FN is the number of false negative examples, that is, the number of samples that the model mistakenly classifies as negative when they are actually positive.

[0034] The present invention introduces Grad-CAM technology; Grad-CAM generates a heat map of the image areas that are most sensitive to target prediction by calculating the gradients of each layer of convolution during the model prediction process. This mapping shows the key areas that the model focuses on during the detection process, which helps explain how the model recognizes targets from the input image.

[0035] The present invention designs and implements a test cheating detection visualization system with a user interface, which provides an intuitive and clear user interface and enables users to input pictures and obtain target detection results.

[0036] The method for detecting cheating behavior of examinees in an examination room based on improved YOLOv8 described in the present invention adopts the above technical solution and has the following technical effects compared with the prior art:

[0037] The present invention proposes the CEYOLOV8 model, which aims to significantly improve the ability to identify cheating behaviors. The model uses EfficientViT as the backbone network to replace the backbone structure in the traditional YOLOv8 to achieve more efficient feature extraction. By introducing CSPHet to replace the C2F structure in YOLOv8, CEYOLOV8 is further enhanced in terms of expression ability and computational efficiency. In addition, MPDIoU is used as a new loss function to optimize the training process of the model and improve the accuracy and robustness of cheating behavior identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a schematic diagram of the steps of the detection method of the present invention;

[0039] Figure 2 This is the network structure diagram of the CEYOLOv8 model of the present invention;

[0040] Figure 3 This is the network structure diagram of the EfficientViT module introduced in the present invention;

[0041] Figure 4 The network structure diagram of the CSPHet module introduced in the present invention;

[0042] Figure 5 The training curves of each model at mAP50 and mAP95 are shown below;

[0043] Figure 6 The image thermal map generated by the Grad-CAM technology introduced in the present invention;

[0044] Figure 7 They are the confusion matrices of YOLOv8, YOLOv8-CSPHet, YOLOv8-EfficientVit, and CEYOLOv8 respectively;

[0045] Figure 8 The detection results of the CEYOLOv8 model of the present invention on 6 cheating actions;

[0046] Fig. 9 Diagram of a visualization system for exam cheating detection with user interface. DETAILED DESCRIPTION

[0047] The present invention is further explained below in detail with reference to the accompanying drawings so that those skilled in the art can more deeply understand the present invention and be able to implement it. However, the following reference examples are only used to explain the present invention and are not intended to limit the present invention.

[0048] like Figure 1-4 As shown in FIG. 1 , a method for detecting cheating behavior of examinees in an examination room based on improved YOLOv8 is shown. The specific steps are as follows:

[0049] Step 1: Collect and annotate the cheating image data of examinees, and pre-process the images to generate a cheating behavior detection dataset for examinees in the examination room; Step 2: Use the processed dataset as the input information of the CEYOLOv8 model; Step 3: Introduce EfficientViT as the backbone network of YOLOV8 and perform feature extraction to obtain feature maps of different scales; Step 4: Replace the C2f in the Neck part with CSPHet, and perform multi-scale feature fusion on the feature maps of different scales in the Neck part to obtain the fused feature map. The core of CSPHet is Hetconv, which adopts the idea of ​​heterogeneous convolution. In the same layer of convolution operation, different convolution kernels are used to process the input features. This mixed convolution operation can effectively maintain or even enhance the representation ability of features while reducing computing resources; Step 5: Obtain the fused feature map, and use the detection platform Head to perform convolution operation on the fused feature map to obtain the final recognition result.

[0050] In step one, the preprocessing process is to rotate, flip, add noise, and change the brightness of the image data of the examinee's cheating behavior to achieve the effect of data enhancement.

[0051] In step one, the labeling process is to use the labelimg tool to label the preprocessed pictures of the examinees' cheating behavior in yolo format to obtain different types of examinees' cheating behavior data sets.

[0052] In step 2, use Python code to divide the image dataset of candidates’ cheating behavior into training set and validation set according to the set ratio of 8:2. The training set will be used to train the model, and the validation set will be used for the training process and model performance.

[0053] In step 3, the network selects YOLOv8 as the baseline model, and:

[0054] S1. Use the CSPHet structure to replace the C2f module in YOLOV8. CSPHet reduces the number of parameters and calculations through heterogeneous convolution while enhancing the feature expression capability;

[0055] S2. EfficientViT is introduced as the backbone network for improving YOLOV8. With the help of its multi-scale linear attention mechanism, the global receptive field and multi-scale feature extraction are effectively realized.

[0056] In step 4, the model is trained using the prepared training data set to maximize the accuracy of detecting cheating behavior of examinees. The training steps are:

[0057] P1. Forward propagation: During each iteration of model training, the image data in the training set will be forward propagated through the neural network; the image input will be processed by multiple convolutional layers and activation function layers; in each layer, the input data will be weighted, activated, and converted, so as to gradually extract more abstract and advanced feature information;

[0058] P2. Loss calculation: Calculate the loss value between the model's predicted box and the true box, including target classification error, target position error, and target confidence error. Here, MPDIoU is used as a new loss function to optimize the model's training process and improve the accuracy and robustness of cheating behavior recognition. The formula is as follows:

[0059]

[0060]

[0061]

[0062] PD ＝ KL(P red ||P gt ) (4)

[0063]

[0064] Among them, P Pred (x, y) represents the probability density at the two-dimensional coordinate (x, y), P gt (x, y) represents the true probability density at the two-dimensional coordinate (x, y), Area of ​​Overlap is the overlapping area of ​​the predicted bounding box and the true bounding box, Area of ​​Union is the area of ​​the joint area, (u x ,u y )and are the coordinates of the center points of the predicted bounding box and the true bounding box, respectively. σ is the standard deviation of the distribution. PD is the penalty distance, which is the Euclidean distance between the center points of the predicted box and the true box. KL divergence is used to calculate the error between the true value and the predicted value.

[0065] To evaluate the performance of the improved model, accuracy, recall, and average precision are used as evaluation indicators. To evaluate the complexity of the model, floating-point operations and parameters are used as evaluation indicators. The calculation formulas for the relevant evaluation indicators are as follows:

[0066]

[0067]

[0068]

[0069]

[0070] Among them, Precision is the accuracy rate, which indicates the ratio of correctly detected positive samples to all samples detected as positive; Recall is the recall rate, which indicates the ratio of correctly detected positive samples to all actual positive samples; mAP is the average detection accuracy of all categories, which indicates the comprehensive performance of the model in the entire data set; Accuracy is used to measure the proportion of correct detections of the model in all category predictions, TP is the number of true positive examples, that is, the number of samples that the model correctly classifies as positive when they are actually positive; FP is the number of positive examples, that is, the number of samples that the model mistakenly classifies as positive when they are actually negative. TN is the number of true negative examples, that is, the number of samples that the model correctly classifies as negative when they are actually negative. FN is the number of false negative examples, that is, the number of samples that the model mistakenly classifies as negative when they are actually positive.

[0071] The present invention introduces Grad-CAM technology; Grad-CAM generates a heat map of the image areas that are most sensitive to target prediction by calculating the gradients of each layer of convolution during the model prediction process. This mapping shows the key areas that the model focuses on during the detection process, which helps explain how the model recognizes targets from the input image.

[0072] This embodiment tests the cheating behavior of examinees in the examination room through specific experiments to verify the beneficial effects of the method of the present invention:

[0073] This experiment was conducted on a computer equipped with an AMD Core 7945HX processor and an Nvidia RTX4060 graphics processor (8GB memory). The deep learning framework was PyTorch 12.1, the programming platform was PyCharm, the programming language was Python 3.9, and all comparison algorithms were performed in the same environment.

[0074] Experimental settings: YOLOv8 is used as the baseline model during training. The specific experimental environment parameter settings are shown in Table 1.

[0075] Table 1 Training parameter settings

[0076]

[0077] Ablation experiment: In order to verify the effectiveness of the two improved modules CSPHet and EfficientVIT proposed in this paper, corresponding ablation experiments were designed, and the number of parameters, calculation amount (GFLOPs), mAP50 and mAP95 were used as evaluation indicators, where the IoU threshold of mAP was 0.5.

[0078] 1. According to Figure 5 As shown in the figure, from the training curve of mAP50, it can be seen that the original YOLOv8 has a slight fluctuation around 50 rounds, but all models can converge quickly around 125 rounds. From the training curve of mAP95, it can be seen that all models can converge around 250 rounds.

[0079] 2. In order to further illustrate the effectiveness of the CEYOLOv8 proposed in this invention, a comparative analysis was conducted with the original YOLOv8 as the baseline model. As can be seen from Table 2, the CSPHet module significantly reduces the number of parameters (-28.2%) and the amount of calculation (-34.1%), but maintains or even enhances the ability to characterize features. The results show that it improves mAP50 and mAP95 by 1.72% and 0.13% respectively.

[0080] 3. Although the number of parameters and computational complexity of using EfficientVIT as the feature extraction module has increased, it has also brought effective performance improvements. Its accuracy on mAP50 has increased by 1.51%, and its accuracy on the more stringent indicator mAP95 has increased by 0.93%.

[0081] 4. The proposed CEYOLOv8 takes into account the feature extraction capability and lightweight performance of CSPHet, and the global receptive field and effective feature screening capability of EfficientVit, and finally achieves the best effect. The results show that although the number of parameters of CEYolov8 increases by 22.9, its mAP50 and mAP95 are improved by 1.72% and 1.65% respectively, and the amount of calculation increases by only 7.3%.

[0082] Table 2 Ablation experiment

[0083]

[0084]

[0085] In order to better understand the decision-making process of the model, the present invention introduces the Grad-CAM technology. Figure 6As shown in the figure, Grad-CAM generates a heat map of the image area that is most sensitive to target prediction by calculating the gradient of each layer of convolution during the model prediction process.

[0086] Although YOLOv8 has a high accuracy, its focus area is not obvious; YOLOv8-CSPHet can shift the model's attention to the main features to a certain extent. YOLOv8-EfficientVit can truly focus the model's attention on the target to be detected; CEYOLOv8 proposed in this invention takes into account the advantages of CSPHet and EfficientVit and achieves better results in feature mapping.

[0087] from Figure 7 It can be seen from the confusion matrix that the accuracy of CEYOLOv8 in the four cheating actions of turning the head, putting both hands under the table, bending over, and leaving is 7%, 4%, 1%, and 2% higher than the original version, respectively. It only drops by 1% and 4% in normal, and passing the note, respectively. The overall accuracy is higher than YOLOv8. CEYOLOv8 detection results Figure 8 shown.

[0088] To further illustrate the reason for choosing the minimum model n of YOLOv8 as the baseline network improvement, a series of comparative experiments were conducted.

[0089] 1. From Table 3, we can see that although YOLOv8x achieves the highest accuracy of 94.6% on mAP50, it is only 0.21% higher than the proposed CEYOLOv8, and its mAP95, a more stringent indicator, is 1.2% lower than the network, and its parameter volume and computational complexity increase by 2164% and 3048% respectively compared with the baseline;

[0090] 2. YOLOv8s and YOLOv8m not only increase the number of parameters and computation by 270%, 759%, 249%, 865% respectively compared with YOLOv8n, but also their performance in mAP50 and mAP95 is not as good as the proposed network.

[0091] 3. As a new V8 framework network, although YOLOv10 has reduced the number of parameters and computation by 23.6% and 18.3% respectively compared with baseline, its mAP50 has only increased by 0.65%, and Map95 has even decreased by 2.2%, indicating that its performance on the cheating dataset is not particularly outstanding.

[0092] 4. Therefore, the present invention still uses YOLOv8n as the baseline to improve the network CEYOLOv8. Although the final network has only 22.9% and 7.3% increase in parameters and calculation respectively, it improves by 1.72% on mAP50 and achieves the highest accuracy of 73.8% on mAP95.

[0093] Table 3 Comparative test

[0094]

[0095] The present invention designs and implements a test cheating detection visualization system with a user interface, such as Fig. 9 The system integrates the CEYolov8 model and provides an intuitive and clear user interface, allowing users to input pictures and obtain target detection results.

[0096] The interface mainly uses Python's pyside2 code library, and uses Qtdesigner to write UI interface components including Text components, Button components, Label components, canvas components, etc., and adds actual functions to these components. By instantiating UI components and implementing various functional embedding of components through function calls, the following functions are mainly included:

[0097] 1. Use the Load Picture button to load the picture to be identified and present it on the canvas;

[0098] 2. Click the Detect button to call the trained CEYolov8 model to detect the loaded image and display the recognition results on the canvas;

[0099] 3. The data recording function records the image paths corresponding to all recognized images, and can save them to a text file through the export button for subsequent use.

[0100] The specific implementation scheme described above further describes in detail the purpose, technical scheme and beneficial effects of the present invention. It should be understood that the above is only a specific implementation scheme of the present invention and is not intended to limit the scope of the present invention. Any equivalent changes and modifications made by any technician in the field without departing from the concept and principle of the present invention should fall within the scope of protection of the present invention.

Claims

1. A method for detecting cheating behavior of examinees in an examination room based on improved YOLOv8, characterized in that: The following steps are involved: Step 1: Collect and annotate the image data of candidates’ cheating behaviors, including turning the head, bending over, leaving the seat, leaving both hands on the table, and passing notes. Consider the normal examination posture and pre-process the images to generate different types of cheating behavior data sets. Step 2: Randomly divide different types of cheating behavior data sets into training sets and validation sets according to proportion; Step 3: Replace the C2f module in YOLOv8 with the CSPHet structure, which reduces the number of parameters and calculations while enhancing the feature expression capability. In addition, EfficientViT is introduced as the backbone network of YOLOv8 to achieve global receptive field and multi-scale feature extraction, and build an algorithm for detecting cheating behavior in the examination room based on the improved YOLOv8. Step 4: Use the improved YOLOv8 model to train the training set and the validation set to obtain the optimal cheating behavior detection model for test takers; Step 5: Input the cheating behavior pictures of the candidates to be tested into the optimal cheating behavior detection model to obtain the detection results.

2. According to claim 1, a method for detecting cheating behavior of examinees in an examination room based on improved YOLOv8 is characterized in that: In the step 1, the preprocessing process is to rotate, flip, add noise, and change the brightness of the image data of the examinee's cheating behavior to achieve the effect of data enhancement.

3. According to claim 2, a method for detecting cheating behavior of examinees in an examination room based on improved YOLOv8 is characterized in that: In the step 1, the labeling process is to use the labelimg tool to label the pre-processed pictures of the examinees' cheating behavior in yolo format to obtain different types of examinees' cheating behavior data sets.

4. According to claim 3, a method for detecting cheating behavior of examinees in an examination room based on improved YOLOv8 is characterized in that: In the step 2, Python code is used to divide the image dataset of examinees' cheating behavior into a training set and a validation set according to a set ratio of 8:2, wherein the training set will be used to train the model, and the validation set will be used for the training process and the performance of the model.

5. According to claim 4, a method for detecting cheating behavior of examinees in an examination room based on improved YOLOv8 is characterized in that: In step 3, the network selects YOLOv8 as the benchmark model, and: S1. Use the CSPHet structure to replace the C2f module in YOLOV8. CSPHet reduces the number of parameters and calculations through heterogeneous convolution while enhancing the feature expression capability; S2. EfficientViT is introduced as the backbone network for improving YOLOV8. With the help of its multi-scale linear attention mechanism, the global receptive field and multi-scale feature extraction are effectively realized.

6. The method for detecting cheating behavior of examinees in an examination room based on improved YOLOv8 according to claim 5, characterized in that: In step 4, the model is trained using the prepared training data set to maximize the accuracy of detecting the examinee's cheating behavior. The training steps are: P1. Forward propagation: During each iteration of model training, the image data in the training set will be forward propagated through the neural network; the image input will be processed by multiple convolutional layers and activation function layers; In each layer, the input data undergoes weighting, activation, and transformation operations to gradually extract more abstract and advanced feature information; P2. Loss calculation: Calculate the loss value between the model's predicted box and the true box, including target classification error, target position error, and target confidence error. Here, MPDIoU is used as a new loss function to optimize the model's training process and improve the accuracy and robustness of cheating behavior recognition. The formula is as follows: PD=KL(P red ||P gt ) (4) Among them, P Pred (x, y) represents the probability density at the two-dimensional coordinate (x, y), P gt (x, y) represents the true probability density at the two-dimensional coordinate (x, y), Area of ​​Overlap is the overlapping area of ​​the predicted bounding box and the true bounding box, Area of ​​Union is the area of ​​the joint area, (u x ,u y )and are the coordinates of the center points of the predicted bounding box and the true bounding box, respectively. σ is the standard deviation of the distribution. PD is the penalty distance, which is the Euclidean distance between the center points of the predicted box and the true box. KL divergence is used to calculate the error between the true value and the predicted value.

Images