Design Method of Examination Room Abnormality Judgment System Based on Machine Vision

Abstract

The present invention proposes a design method for an examination room anomaly judgment system based on machine vision. First, analyze the resolution and frame rate of the video stream to determine the operating environment of the examination room anomaly judgment system, and then select the operating frequency of the system. Secondly, in order to obtain the coordinates of the Anchor boxes of the students and the image patches, use YOLO to perform target detection on the personnel in the examination room and classify the students and teachers in the examination room. Thirdly, in order to judge whether the current frame behavior of the student is abnormal, use AJNet to perform anomaly detection on the obtained student image patches. Finally, label the target detection and pose recognition data sets required for the examination room anomaly judgment, train the system model, and compare the results predicted by the trained model with the actual abnormal situation in the video examination room to judge the completion of the tasks of the target. Compared with other examination room anomaly judgment methods, the method of the present invention has the best performance in terms of accuracy, and the processing speed meets the real-time requirements for different operating environments, with obvious advantages.

Description

Technical Field

[0001] The present invention belongs to the technical fields of machine learning and computer vision, and particularly relates to a design method for an abnormal situation judgment system in an examination room based on machine vision. Background Art

[0002] Currently, for the abnormal behaviors of students in the examination room, in addition to the inspection by teachers in the examination room, it is more necessary to comprehensively judge the abnormal behaviors in the examination room through technical means. However, the current surveillance videos in the examination room cannot effectively detect the abnormal behaviors occurring in the examination room, and still require manual observation and discrimination of the surveillance videos, which results in low efficiency in judging the abnormal behaviors of students in the examination room. One of the important means to solve the above problems is to explore and implement a real-time automatic abnormal situation judgment system in the examination room to improve the efficiency of abnormal situation judgment in the examination room. Summary of the Invention

[0003] Therefore, the purpose of the present invention is to provide a design method for an abnormal situation judgment system in an examination room based on machine vision. First, analyze the resolution and frame rate of the video stream to determine the operating environment of the abnormal situation judgment system in the examination room, and then select the operating frequency of the system. Secondly, in order to obtain the coordinates of the Anchor boxes of students and image patches, use YOLO to perform object detection on the personnel in the examination room and classify the students and teachers in the examination room. Thirdly, in order to judge whether the behavior of the student in the current frame is abnormal, use AJNet to perform abnormal detection on the obtained student image patches. Finally, label the object detection and pose recognition data sets required for abnormal situation judgment in the examination room, train the system model, and compare the results predicted by the trained model with the actual abnormal situations in the video examination room to judge the completion of the tasks of the targets. Based on the above steps, deploy the hardware servers, communication transmission processes, and feedback processes required for abnormal situation detection in the examination room. Compared with other methods for judging abnormal situations in the examination room, the method of the present invention has the best performance in terms of accuracy, and the processing speed meets the real-time requirements for different operating environments, with obvious advantages.

[0004] It can ensure real-time operating speed and high judgment accuracy in different examination room monitoring devices to meet different application scenarios and examination room requirements.

[0005] The technical solution adopted by the present invention to solve its technical problems is as follows:

[0006] A design method for an abnormal situation judgment system in an examination room based on machine vision, characterized by including the following steps:

[0007] Step S1: Determine the operating environment of the abnormal situation judgment system in the examination room, analyze the resolution and frame rate of the video stream, and then select the operating configuration of the system;

[0008] Step S2: Intercept discriminant frames from the input video stream, use the YOLO object detection network to perform object detection on the people in the examination room, distinguish students and teachers in the examination room, and obtain the anchor box coordinates and image patches of the students;

[0009] Step S3: Use the AJNet anomaly judgment network, use the data output in Step S2 to judge whether the behavior of the student is abnormal in the current frame, feedback the result and end the current frame, and start the system operation of the next frame;

[0010] Step S4: Make the dataset required for the examination room anomaly judgment system, compare the results predicted by the training model with the actual abnormal conditions of the video examination room, and test the running time of the model to verify the reliability of the system.

[0011] Furthermore, Step S1 specifically includes the following steps:

[0012] Step S11: Obtain the monitoring video stream of the examination room, and analyze the resolution and frame rate of the monitoring video stream; according to the video resolution size and frame rate, calculate the appropriate running frequency corresponding to the real-time running system in the current monitoring video environment, that is: the number of frames processed per second α in the video stream, average sample α frames from the video stream, and the system running time is τ;

[0013] Step S12: Determine the running frequencies of the YOLO object detection network and the AJNet anomaly judgment network according to Step S11;

[0014] Step S13: Input the obtained video stream into the examination room anomaly judgment system according to the rules of Step S11.

[0015] Furthermore, Step S2 specifically includes the following steps:

[0016] Step S211: When the running time τ ≤ 5s, or when the running time τ > 5s and the current frame is the first frame of the total α frames in the current second, use the YOLO algorithm to perform data enhancement and adaptive image scaling on the input image in Step S21;

[0017] Step S212: The image processed by Step S211 passes through the CSP structure, uses the FPN feature pyramid to obtain three enhanced features, and finally uses the Yolo Head to obtain the prediction result;

[0018] Step S213: After Steps S211 to S212, obtain the predicted target classification result of YOLO for the current frame; assume that the probability p(x1) = 1 when a certain target is classified as a student in a non-current frame, the classification probability in the current frame is p(x2), and when the target is a student, p(x2) = 1, and when the target is a teacher or others, p(x2) = 0; assume that a certain target is classified as a student a total of x times in the total α frames of the previous second, calculate the actual target classification result of the first frame of the current second as When p ≥ 80%, classify the current frame of the target as a student;

[0019] Perform the above calculations on all targets in the current frame;

[0020] Step S22: When the running time τ > 5s and the current frame is not the first frame among the total α frames in the current second, perform the Kalman tracking algorithm based on the targets obtained by YOLO detection to track the identified student targets;

[0021] Step S23: Save the student target information obtained in Step S21 or Step S22, including the corresponding anchor box coordinates and image patches.

[0022] Furthermore, Step S3 specifically includes the following steps:

[0023] Step S31: Use the output data obtained in Step S2 as the input data of the AJNet anomaly judgment network, and obtain the anomaly judgment result of the student targets in the current frame through AJNet; Assume that the probability p(y1) = 1 when a certain target is judged as abnormal in a non-current second, the probability of judging whether it is abnormal in the current second is p(y2), and p(y2) = 1 when abnormal and p(y2) = 0 when normal; Assume that a certain target is judged as abnormal a total of i times among the total α frames in the previous second, then the probability of being judged as abnormal in the previous second is When p l ≥ 50%, record the judgment result of the target in the previous second as abnormal;

[0024] Perform the same analysis for the previous 5 seconds. Let the number of times the judgment result of the target is abnormal in the previous 5 seconds be y, then calculate the actual anomaly judgment result of the target in the current frame as When p ≥ 70%, determine the anomaly judgment result of the current frame of the target as abnormal;

[0025] Perform the above calculations on all targets in the current frame;

[0026] Step S32: Set the anchor box of the abnormal target to red and give a warning, set the anchor box of the normal target to green and do not give a warning, and display the anomaly judgment result of the student targets in the current frame in the output video stream;

[0027] The system proceeds to the next step of analysis. Assume that when the running time τ ≤ 5s or the running time τ > 5s and the current frame is the first frame among the total α frames in the current second, return to Step S21 and execute in sequence; Assume that the running time τ > 5s and the current frame is not the first frame among the total α frames in the current second, return to Step S22 and execute in sequence; Repeat the above execution operations until the examination room anomaly judgment terminates.

[0028] Furthermore, step S4 specifically includes the following steps:

[0029] Step S41: Make the target detection dataset required for the judgment of abnormal situations in the examination room. Label the positive samples as students and the negative samples as teachers, and train the YOLO target detection network with this dataset; make the abnormal situation judgment dataset required for the judgment of abnormal situations in the examination room. Label the positive samples as normal and the negative samples as behaviorally abnormal, and train the AJNet abnormal situation judgment network with this dataset;

[0030] Step S42: Introduce CIOU_Loss and DIOU_nms as the training losses of the YOLO target detection model, evaluate the model training process in real time, and save the training model and data in real time;

[0031] Step S43: Introduce Binary Cross-Entropy (BCE) Loss as the training loss of the AJNet abnormal situation judgment network, evaluate the model training process in real time, and save the training model and data in real time.

[0032] Step S44: Introduce the running time to evaluate the processing speed of the examination room abnormal situation judgment system.

[0033] In steps S1 to S4, it is recommended that the basic configuration of the accessories is not lower than Table 1:

[0034]

[0035]

[0036] The monitoring video data of the examination room is obtained by the local camera and the abnormal behavior detection is carried out locally, and the results are fed back to the invigilator. The video data is transmitted to the monitoring center and stored in the disk array or the cloud.

[0037] When an abnormal behavior is recognized, the device in the invigilator's hand will receive a message reminder, which can be set to vibration feedback. At the same time, the picture of the object with abnormal behavior is transmitted to the invigilator. The invigilator needs to focus on observing the abnormal object and impose a penalty after confirming that it has cheating behavior.

[0038] For the thresholds involved in judging students and teachers and judging whether there are abnormalities in steps S2 and S3, they can all be adjusted according to the actual situation to obtain a more rapid prompt or a more accurate judgment.

[0039] Compared with the prior art, the present invention and its preferred solutions perform best in terms of accuracy performance, and the processing speed meets the real-time requirements for different operating environments, with obvious advantages. It effectively improves the efficiency of the examination room abnormal situation judgment and has a very broad application prospect. Brief Description of the Drawings

[0040] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments:

[0041] Figure 1 It is the overall working flow chart in the embodiment of the present invention.

[0042] Figure 2 It is the YOLO object recognition and AJNet anomaly detection network model diagram in the embodiment of the present invention.

[0043] Figure 3 It is an example of an unprocessed image and a processed image in the embodiment of the present invention.

[0044] Figure 4 It is the overall system flow chart in the embodiment of the present invention. Specific Embodiments

[0045] To make the features and advantages of this patent more obvious and understandable, specific embodiments are given below for detailed description as follows:

[0046] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used in this specification have the same meaning as commonly understood by those of ordinary skill in the technical field to which this application belongs.

[0047] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0048] As Figures 1 - 4 shown, this embodiment provides a design method for an examination room anomaly judgment system based on machine vision, and the overall working process is as Figure 1As shown in the figure, it includes the following steps: Step S1: Determine the environment in which the examination room anomaly judgment system operates, analyze the resolution and frame rate of the video stream, and then select the configuration for the system to operate; Step S2: Intercept discriminant frames from the input video stream, use the YOLO object detection network to perform object detection on the people in the examination room, distinguish between students and teachers in the examination room, and obtain the coordinates of the student's anchor boxes and image patches; Step S3: Use the AJNet anomaly judgment network, use the data output in Step S2 to judge whether the behavior of the student is abnormal in the current frame, feedback the result and end the current frame, and start the operation of the system for the next frame; Step S4: Make the dataset required for the examination room anomaly judgment system, compare the results predicted by the training model with the abnormal conditions of the real video examination room, and test the running time of the model to verify the reliability of the system.

[0049] In this embodiment, in Step S1, analyze the resolution and frame rate of the video stream, determine the environment in which the examination room anomaly judgment system operates, and then select the configuration for the system to operate, which specifically includes the following steps:

[0050] Step S11: Obtain the monitoring video stream of the examination room and analyze the resolution and frame rate of the monitoring video stream. According to the video resolution size and frame rate, calculate the appropriate running frequency corresponding to the real-time operation system in the current monitoring video environment, that is, the number of frames α processed per second in the video stream, and evenly sample α frames from the video stream, and the system running time is τ.

[0051] Step S12: Determine the running frequencies of the YOLO object detection network and the AJNet anomaly judgment network according to S11.

[0052] Step S13: Input the obtained video stream into the examination room anomaly judgment system according to the rules of S11.

[0053] The YOLO object detection network model is as Figure 2 shown: In this embodiment, the YOLO object detection network is used to perform object detection on the people in the examination room. Step S2: Use the YOLO object detection network to perform object detection on the people in the examination room, distinguish between students and teachers in the examination room, and obtain the coordinates of the student's anchor boxes and image patches, which specifically includes the following steps:

[0054] Step S211: When the running time τ ≤ 5 s (seconds) or when the running time τ > 5 s (seconds) and the current frame is the first frame of the total α frames in the current second, use the YOLO algorithm to perform data enhancement and adaptive image scaling on the input image in S21;

[0055] Step S212: The image processed by S211 passes through the CSP structure, uses the FPN feature pyramid to obtain three enhanced features, and finally uses the YOLO Head to obtain the prediction result.

[0056] Step S213: After steps S211 to S212, obtain the predicted target classification result of YOLO for the current frame. Assume that the probability p(x1) = 1 when a certain target is classified as a student in a non-current frame, the classification probability in the current frame is p(x2), and when the target is a student, p(x2) = 1, and when the target is a teacher or others, p(x2) = 0. Assume that in a total of α frames in the previous second, a certain target was classified as a student x times in total. Calculate the actual target classification result of the first frame in the current second as When p ≥ 80%, determine that the classification of the target in the current frame is a student. Perform the above calculation for all targets in the current frame.

[0057] Step S22: When the running time τ > 5 s (seconds) and the current frame is not the first frame among the total α frames in the current second, perform the Kalman tracking algorithm based on the targets detected by YOLO to track the identified student targets.

[0058] Step S23: Save the student target information obtained in step S21 or S22, and save the coordinates of its corresponding anchor boxes and the image patches.

[0059] The AJNet anomaly judgment network model is as Figure 2 shown: In this embodiment, the AJNet anomaly judgment network is used to perform pose recognition on the students in the examination room. Step S3: Use the AJNet anomaly judgment network to perform pose recognition on the image patches obtained in step S2, judge whether the behavior of the students in the current frame is abnormal, and end the current frame to start the system operation of the next frame, which specifically includes the following steps:

[0060] Step S31: Use the output data obtained in step S2 as the input data of the AJNet anomaly judgment network, and obtain the anomaly judgment result of the student targets in the current frame through AJNet. Assume that the probability p(y1) = 1 when a certain target is judged to be abnormal in a non-current second, the probability of judging whether it is abnormal in the current second is p(y2), and when it is abnormal, p(y2) = 1, and when it is normal, p(y2) = 0. Assume that in a total of α frames in the previous second, a certain target was judged to be abnormal i times in total, then the probability of being judged to be abnormal in the previous second is When p l ≥ 50%, record the judgment result of the target in the previous second as abnormal. Perform the same analysis for the previous 5 seconds. Let the number of times the judgment result of the target is abnormal in the previous 5 seconds be y, then calculate the actual anomaly judgment result of the target in the current frame as When p ≥ 70%, determine that the anomaly judgment result of the target in the current frame is abnormal. Perform the above calculation for all targets in the current frame.

[0061] Step S32: Set the anchor boxes of abnormal targets to red and issue a warning, set the anchor boxes of normal targets to green and do not issue a warning, and display the abnormal judgment result of the current frame of student targets in the output video stream. The system proceeds to the next step of analysis. Assume that when the running time τ ≤ 5 s (seconds) or the running time τ > 5 s (seconds) and the current frame is the first frame among a total of α frames in the current second, then return to step S21 and execute in sequence; assume that the running time τ > 5 s (seconds) and the current frame is not the first frame among a total of α frames in the current second, then return to step S22 and execute in sequence. Repeat the above execution operations until the examination room abnormal judgment terminates.

[0062] Furthermore, compare the results predicted by the trained model with the actual abnormal situations in the examination room video, and test the running time of the model to verify the reliability of the system. Figure 3 For the unprocessed images and processed images in the embodiment. Specifically, it includes the following steps:

[0063] Step S41: Make a target detection dataset required for examination room abnormal judgment. Label the positive samples as students and the negative samples as teachers, and the ratio of the two labeled data is close to 1:1. Train the YOLO target detection network with this dataset; make an abnormal judgment dataset required for examination room abnormal judgment. Label the positive samples as normal and the negative samples as behaviorally abnormal, and the ratio of the two labeled data is close to 1:1. Train the AJNet abnormal judgment network with this dataset.

[0064] Step S42: Introduce CIOU_Loss and DIOU_nms as the training losses of the YOLO target detection model, evaluate the model training process in real time, and save the trained model and data in real time.

[0065] Step S43: Introduce Binary Cross-Entropy (BCE) Loss as the training loss of the AJNet abnormal judgment network, evaluate the model training process in real time, and save the trained model and data in real time.

[0066] Step S44: Introduce the running time to evaluate the processing speed of the examination room abnormal judgment system.

[0067] In steps S1 to S4, it is recommended that the basic configuration of the accessories is not lower than Table 1:

[0068]

[0069] The monitoring video data of the examination room is obtained by the local camera and abnormal behavior detection is performed locally, and the results are fed back to the invigilator. The video data is transmitted to the monitoring center and stored in the disk array or the cloud.

[0070] When an abnormal behavior is recognized, the device in the invigilator's hand will receive a message reminder, which can be set to vibrate feedback, and at the same time, the picture of the object with abnormal behavior will be transmitted to the invigilator. The invigilator needs to focus on observing the abnormal object and impose a penalty after confirming that it has cheated.

[0071] For the thresholds involved in judging students and teachers and judging whether there is an abnormality in steps S2 and S3, they can all be adjusted according to the actual situation to obtain a more rapid prompt or a more accurate judgment.

[0072] The above is only a preferred embodiment of the present invention, and it is not a limitation of the present invention in other forms. Any person skilled in the art may use the disclosed technical content to make changes or modifications into equivalent embodiments with equivalent changes. However, any simple modification, equivalent change, and modification made to the above embodiments based on the technical essence of the present invention without departing from the technical solution content of the present invention still fall within the protection scope of the technical solution of the present invention.

[0073] This patent is not limited to the above best implementation manner. Anyone inspired by this patent can come up with other various forms of design methods for the abnormal judgment system in the examination room based on machine vision. All equal changes and modifications made according to the scope of the patent application of the present invention shall fall within the scope covered by this patent.

Claims

1. A design method for an abnormal situation judgment system in an examination room based on machine vision, characterized in that, It includes the following steps: Step S1: Determine the environment in which the examination room anomaly judgment system runs, analyze the resolution and frame rate of the video stream, and then select the configuration for system operation; Step S2: Intercept discriminant frames from the input video stream, use the YOLO object detection network to perform object detection on the people in the examination room, distinguish students and teachers in the examination room, and obtain the anchor box coordinates and image patches of the students; Step S3: Use the AJNet anomaly judgment network to judge whether the behavior of the student in the current frame is abnormal by using the data output in Step S2, feedback the result and end the current frame, and start the system operation of the next frame; Step S4: Make the data set required for the examination room anomaly judgment system, compare the results predicted by the training model with the actual abnormal situation of the video examination room, and test the running time of the model to verify the reliability of the system; Step S3 specifically includes the following steps: Step S31: Use the output data obtained in step S2 as the input data of the AJNet anomaly judgment network, and obtain the anomaly judgment result of the student target in the current frame through AJNet; assume that the probability p(y1)=1 when a certain target is judged as abnormal in a non-current second, the probability of judging whether it is abnormal in the current second is p(y2), and when it is abnormal, p(y2)=1, and when it is normal, p(y2)=0; assume that in a total of α frames in the previous second, a certain target is judged as abnormal a total of i times, then the probability of being judged as abnormal in the previous second is When p l ≥50%, record the judgment result of this target in the previous second as abnormal; Perform the same analysis on the first 5 seconds. Let the number of times the target is judged as abnormal in the first 5 seconds be y, then calculate the actual abnormal judgment result of the target in the current frame as When p≥70%, determine that the abnormal judgment result of the current frame of the target is abnormal; Execute the above calculations for all the targets in the current frame; Step S32: Set the anchor box of the abnormal target to red and give a warning, set the anchor box of the normal target to green and do not give a warning, and display the anomaly judgment result of the student target in the current frame in the output video stream; The system performs the next step of analysis. Assume that when the running time τ ≤ 5s or the running time τ > 5s and the current frame is the first frame of the total α frames in the current second, return to Step S21 and execute in sequence; assume that the running time τ > 5s and the current frame is not the first frame of the total α frames in the current second, return to Step S22 and execute in sequence; repeat the above execution operations until the examination room anomaly judgment terminates.

2. The design method of the examination room anomaly judgment system based on machine vision according to claim 1, characterized in that: Step S1 specifically includes the following steps: Step S11: Obtain the monitoring video stream of the examination room and analyze the resolution and frame rate of the monitoring video stream; according to the video resolution size and frame rate, calculate the appropriate running frequency for the real-time operation system in the current monitoring video environment, that is: the number of frames α processed per second in the video stream, average sample α frames of the video stream, and the system running time is τ; Step S12: Determine the running frequencies of the YOLO object detection network and the AJNet anomaly judgment network according to Step S11; Step S13: Input the obtained video stream into the examination room anomaly judgment system according to the rules of Step S11.

3. The design method of the examination room anomaly judgment system based on machine vision according to claim 2, wherein, Step S2 specifically includes the following steps: Step S211: When the running time τ ≤ 5s, or when the running time τ > 5s and the current frame is the first frame of the total α frames in the current second, use the YOLO object detection network to perform data enhancement and adaptive image scaling on the input image in Step S21; Step S212: The image processed by Step S211 passes through the CSP structure, uses the FPN feature pyramid to obtain three enhanced features, and finally uses Yolo Head to obtain the prediction result; Step S213: After steps S211 to S212, obtain the predicted target classification result of the current frame by the YOLO object detection network; assume that the probability p(x1) = 1 when a certain target is classified as a student in a non-current frame, the classification probability in the current frame is p(x2), and when the target is a student, p(x2) = 1, and when the target is a teacher or others, p(x2) = 0; assume that in a total of α frames in the previous second, a certain target was classified as a student a total of x times, and calculate the actual target classification result of the first frame of the current second as When p ≥ 80%, determine that the classification of the target in the current frame is a student; Execute the above calculations for all the targets in the current frame; Step S22: When the running time τ > 5s and the current frame is not the first frame of the total α frames in the current second, perform the Kalman tracking algorithm on the targets detected by the YOLO object detection network to track the identified student targets; Step S23: Save the student target information obtained in Step S21 or Step S22, including the corresponding anchor box coordinates and image patches.

4. The design method of the examination room anomaly judgment system based on machine vision according to claim 1, characterized in that: Step S4 specifically includes the following steps: Step S41: Make the target detection data set required for the examination room anomaly judgment, label the positive samples as students and the negative samples as teachers, and train the YOLO target detection network with this data set; make the anomaly judgment data set required for the examination room anomaly judgment, label the positive samples as normal and the negative samples as behavior anomalies, and train the AJNet anomaly judgment network with this data set; Step S42: Introduce CIOU_Loss and DIOU_nms as the training losses of the YOLO target detection model, conduct real-time evaluation on the model training process, and save the training model and data in real time; Step S43: Introduce Binary Cross-Entropy Loss as the training loss of the AJNet anomaly judgment network, conduct real-time evaluation on the model training process, and save the training model and data in real time; Step S44: Introduce the running time to evaluate the processing speed of the examination room anomaly judgment system.

Images