An examination supervision method and supervision system based on image and edge computing

By adopting edge computing and deep learning image classification technology in the examination supervision system, we collect and analyze candidate behavior images in real time and conduct reviews in combination with voting algorithms, delay and bandwidth problems in the cloud computing model are solved, and efficient and fair examination supervision is achieved. Reduced labor and hardware costs.

CN114663971BActive Publication Date: 2024-11-01SHANDONG XINSONG IND SOFTWARE RES INST CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202011428642.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-12-07
Publication Date
2024-11-01
Estimated Expiration
2040-12-07

AI Technical Summary

Technical Problem

Under the traditional cloud computing model, the examination supervision system has problems such as high latency, network instability and low bandwidth, making it difficult to collect and analyze candidate behavior images in real time, resulting in low supervision efficiency and high cost.

Method used

Using the examination supervision method based on image classification and edge computing, the candidate's images are collected in real time through edge supervision equipment, the behavior classification and judgment are used to combine the voting algorithm for review, and the data is finally summarized to the examination center server to realize automated supervision and manage.

Benefits of technology

Real-time analysis and fair supervision of candidate behaviors are achieved, labor costs are reduced, and the omissions in subjective judgments are avoided by the invigilator, a fair examination environment is provided, and the burden of hardware and network transmission is reduced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114663971B_ABST
    Figure CN114663971B_ABST
Patent Text Reader

Abstract

The present invention discloses an examination supervision system and supervision method based on image classification and edge computing, wherein the examination supervision system comprises: an examination center server cluster, at least one examination center central server connected to the examination center server cluster, at least one examination room server connected to the examination center central server, and at least one edge supervision device connected to the examination room server. The present invention can make judgments on the violations of examinees in real time, realize the automated supervision and management of computer-based examinations, avoid supervision omissions caused by the subjective judgment ability of invigilators and other human factors, and provide examinees with a fair and just examination environment; at the same time, the use of the examination supervision system can reduce the number of invigilators required for the examination, greatly saving labor costs.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of video processing, and specifically relates to an examination supervision method and a supervision system based on image classification and edge computing. Background Art

[0002] Image classification technology is an image processing method that uses computers to perform quantitative analysis on images and classify images or each pixel or area in the image according to the different features contained in the image. It can replace human visual judgment to a certain extent. Image classification based on traditional machine vision technology can be divided into three types: texture-based, shape-based, and spatial relationship-based. The core is to manually design algorithms to extract feature information from images. It needs to be customized for specific detection targets and application scenarios. The research and development is difficult and has poor versatility. Image classification technology based on machine learning, according to the original data and annotated data, the computer can automatically learn the feature information in the image, which greatly reduces the difficulty of the image classification task. It also has good versatility, so that the examination supervision system can be easily applied to different examination environments.

[0003] Edge computing is a concept proposed in recent years. It provides cloud services and IT environment services for application developers and service providers at the edge of the network. The goal is to provide computing, storage and network bandwidth close to data input or users. The important problem that edge computing aims to solve is the high latency, network instability and low bandwidth problems that exist in the traditional cloud computing model. Since the examination monitoring system needs to collect the behavioral images of candidates in the examination in real time and perform a large amount of calculation and analysis on the images, the bandwidth requirements are high, and the use of cloud computing will cause high latency. At the same time, in recent years, edge computing hardware devices developed around deep learning applications have become increasingly rich and perfect, which can reduce costs while improving computing power. Therefore, compared with cloud computing technology, edge computing technology is more suitable for application in the examination supervision system of the present invention. Summary of the invention

[0004] A first aspect of an embodiment of the present invention provides an examination supervision method based on image classification and edge computing, comprising the following steps:

[0005] S100: The examination room server receives the test start signal from the central server of the examination center, triggers the edge supervision device to collect the test image of the examinee in real time, and classifies the examinee's behavior in the collected image through the image classification algorithm based on deep learning, and determines whether the examinee's behavior is qualified, and then saves the captured violation behavior image and the judgment result. After the test is over, the edge supervision device sends all the images during the test and the corresponding judgment result data to the corresponding examination room server;

[0006] S200: The examination room server checks the data sent by each edge monitoring device, checks the data integrity, and packages and sends it to the examination center central server;

[0007] S300: The central server of the test center collects and reviews the data sent by all the test center servers, and re-identifies the violation behavior of each image based on the deep learning image classification algorithm, and saves it together with the identification results of the edge supervision device. After the review work is completed, the supervision data of this test center for this test is packaged and sent to the server cluster of the test center;

[0008] S400: The supervision data of the central servers of all test sites are aggregated, and each image is re-identified based on the deep learning image classification algorithm to determine whether the behavior is in violation of the rules. The judgment result and the two judgment results of the central server of the test site and the edge supervision device are finally determined through a voting algorithm to determine whether the image is in violation of the rules, and then to determine whether the examinee has violated the rules. After all the review work is completed, the server cluster of the test center sends all the supervision data to the test score evaluation system as the basis for the examinee's test score.

[0009] In the embodiment provided by the present invention, the image classification algorithm adopts the ResNet convolutional neural network in deep learning.

[0010] Furthermore, in step S100, image acquisition includes the following steps:

[0011] S101: The edge supervision device initializes a timer, then captures and collects images, and classifies the images through an image classification algorithm to determine whether there is a violation;

[0012] S102: If there is no violation, determine whether the timer is full;

[0013] S1021: If it is full, reset the timer, save the data, and determine whether the test is over;

[0014] S1022: If the number of test time is not full, directly determine whether the test is over;

[0015] S103: If there is a violation, the data is directly saved to determine whether the test is over;

[0016] S104: If the test is not over, return to recapture the image; if the test is over, save the supervision data and send it to the test site server.

[0017] It includes: an examination center server cluster, at least one examination site central server connected to the examination center server cluster, at least one examination room server connected to the examination site central server, and at least one edge monitoring device connected to the examination room server.

[0018] In the embodiment provided by the present invention, the examination room server, the examination center central server, and the examination center server cluster use the Apache server framework to build the server.

[0019] In the embodiment provided by the present invention, the edge supervision device and the examination room server are connected via Gigabit Ethernet, and the examination room server and the examination center central server, as well as the examination center central server and the examination center server cluster are connected via Gigabit Ethernet, or 10 Gigabit Ethernet.

[0020] In the embodiment provided by the present invention, the image classification algorithm of the edge supervision device adopts ResNet-18 or ResNet-34 structure. The image classification algorithm of the test center central server adopts ResNet-101 network structure. The image classification algorithm of the test center server cluster adopts ResNet-152 network structure.

[0021] Compared with the prior art, the present invention can analyze the behavior of examinees in the examination in real time, upload the images of illegal behavior and randomly collected images to the cloud for archiving and review, thereby realizing the automated supervision and management of computer-based examinations, thereby avoiding the supervision omissions caused by the subjective judgment ability of the invigilators and other human factors, and providing examinees with a fair and just examination environment. At the same time, the use of the examination supervision system can reduce the number of invigilators required for the examination, greatly saving labor costs.

[0022] Other advantages, objectives and features of the present invention will be embodied in part through the following description, and in part will be understood by those skilled in the art through study and practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0024] Figure 1 The figure is a schematic diagram of the composition of the examination supervision system according to the embodiment of the present invention.

[0025] Figure 2 This is the workflow of the edge supervision device according to the embodiment of the present invention. DETAILED DESCRIPTION

[0026] The present invention is further described in detail below in conjunction with the accompanying drawings so that those skilled in the art can implement the invention with reference to the description.

[0027] It should be understood that terms such as “having”, “including” and “comprising” used herein do not specify the existence or addition of one or more other elements or combinations thereof.

[0028] The present invention provides an examination supervision method and an examination supervision system based on image classification and edge computing, which can judge illegal behaviors such as making phone calls, transferring answers, and peeping at others during the examination. In particular, the present invention is mainly used in various occasions that require the use of computers for computer-based examinations, such as driving license examinations, computer level 2 examinations, IELTS, CPA examinations, etc.

[0029] The test supervision system of the present invention is composed of four parts: edge supervision equipment, test site server, test site central server, and test center server cluster;

[0030] Among them, the edge supervision device is responsible for making real-time judgments on the candidates' violations; the examination room server is responsible for connecting to one or more edge supervision devices in nearby examination rooms; the examination center central server is responsible for caching and reviewing all supervision data of this examination center; the examination center server cluster is responsible for storing and reviewing the supervision data of all examination centers in this examination.

[0031] All server devices involved in the examination supervision system can reuse the server devices belonging to the examination system to save hardware costs and improve system integration.

[0032] The present invention adopts edge computing technology, that is, the real-time monitoring program runs on the local edge device, and the server side only performs the review process. This is different from the current mainstream cloud computing model. Due to the limitations of server computing power and network communication capabilities, it is almost impossible to achieve real-time supervision of each candidate through cloud computing. Therefore, edge devices (generally embedded devices) are added on the basis of cloud computing. On the one hand, it shares the computing pressure of the server, and on the other hand, it also reduces the burden of network transmission, making real-time monitoring possible.

[0033] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0034] like Figure 1 As shown, the present invention provides an examination supervision system based on image classification and edge computing, including: an examination center server cluster 4, at least one examination site central server 3 connected to the examination center server cluster 4, at least one examination room server 2 connected to the examination site central server 3, and at least one edge supervision device 1 connected to the examination room server 2.

[0035] In the embodiment provided by the present invention, the examination room server 2, the examination center central server 3, and the examination center server cluster 4 use the Apache server framework to build server-side software to achieve data transmission.

[0036] The MySQL relational database management system is used to realize the storage and query functions of regulatory data.

[0037] Furthermore, the edge supervision device 1 and the examination room server 2 are connected via Gigabit Ethernet, the examination room server 2 and the examination center central server 3, and the examination center central server 3 and the examination center server cluster 4 are connected via Gigabit Ethernet, or 10 Gigabit Ethernet.

[0038] Edge supervision device 1 uses a USB camera with a resolution of 2 million pixels as an image acquisition device and a Jetson Nano edge AI device as a processor to control the USB camera to collect images of candidates and analyze and process the images.

[0039] On the edge detection device, due to limited computing and storage resources, the image classification algorithm adopts a lower-depth ResNet-18 or ResNet-34 structure; in the test center central server 3 and the test center server cluster 4, GPU devices can be added to accelerate the execution of the image classification algorithm, so a larger network structure can be adopted. For example, the ResNet-101 network structure is used on the test center central server 3, and the larger ResNet-152 network structure is used on the test center server cluster 4.

[0040] The use of different depth structures of the ResNet convolutional network system in the edge detection device, the test site central server 3 and the test center server cluster 4 balances hardware resources and actual application requirements, while also avoiding simple repeated calculations of image classification, making the review results and the final voting algorithm highly credible.

[0041] In addition, the present invention also provides an examination supervision method based on image classification and edge computing, comprising the following steps:

[0042] S100: the examination room server 2 receives the test start signal from the examination center central server 3, triggers the edge, and the edge supervision device 1 collects the test image of the examinee in real time, and classifies the examinee's behavior in the collected image through the image classification algorithm based on deep learning, and judges whether the examinee's behavior is qualified, and then saves the captured violation behavior image and the judgment result. After the test is over, the edge supervision device 1 sends all the images during the test and the corresponding judgment result data to the corresponding examination room server 2;

[0043] Specifically, the edge monitoring device 1 collects images of the examinee during the examination in real time, and classifies each frame of the collected image through deep learning-based image classification technology to determine whether the examinee's behavior is qualified or illegal, such as making phone calls, passing answers, and peeping at others, and saves the captured illegal behavior images and the judgment results. To avoid missed judgments, the edge monitoring device 1 will randomly save multiple images that have been judged as qualified behaviors. After the examinee hands in the paper, the edge monitoring device 1 sends all the images of the examinee during the examination and the corresponding judgment result data to the examination room server 2 to which it belongs.

[0044] Further, image acquisition includes the following steps: Figure 2 As shown:

[0045] S101: the edge supervision device 1 initializes a timer, then takes pictures to collect images, and classifies the images through an image classification algorithm to determine whether there is a violation;

[0046] S102: If there is no violation, determine whether the timer is full;

[0047] S1021: If it is full, reset the timer, save the data, and determine whether the test is over;

[0048] S1022: If the number of test time is not full, directly determine whether the test is over;

[0049] S103: If there is a violation, the data is directly saved to determine whether the test is over;

[0050] S104: If the test is not finished, return to re-capture the image; if the test is finished, save the supervision data and send it to the test room server 2.

[0051] S200: the examination room server 2 checks the data sent by each edge monitoring device 1, checks the data integrity, and packages and sends it to the examination center central server 3;

[0052] Furthermore, the design idea of ​​image acquisition is to monitor the behavior of candidates in real time, so taking photos to obtain images is carried out continuously throughout the entire examination process. Not every image obtained needs to be saved in the database, because there are many candidates and the amount of data is large, so the obtained images will be put into the cache first. When the cached image is judged by the system to have violated the rules, the image will be saved in the database as evidence. In order to avoid errors in the results of real-time judgment by edge devices, some images should be randomly extracted from the cache and saved in the database as a record. The timer provides a condition for this random saving. For example, the timer period is 20 minutes, that is, a signal will be sent every 20 minutes (indicated by "timer full" in the figure). After receiving the signal, the system will save the pictures in the cache to the database.

[0053] Specifically, the examination room server 2 is connected to one or more edge monitoring devices 1 in the adjacent examination room, receives the start signal from the examination center central server 3 at the beginning of the examination, and triggers the edge monitoring device 1 to start working. During the examination, the examination room server 2 will poll and detect the working status of each edge monitoring device 1 to ensure the normal operation of the system. After the examination, the examination room server 2 checks the data sent by each edge monitoring device 1, checks the data integrity, and packages it and sends it to the examination center central server 3.

[0054] S300: The central server 3 of the test center is responsible for caching and reviewing all the supervision data in the test center. At the beginning of the test, it sends a start signal to the test center server 2. The central server 3 of the test center aggregates the data sent by all the test center servers 2 and reviews them. It also re-identifies the violation behavior of each image based on the deep learning image classification algorithm and saves it together with the identification result of the edge supervision device 1. After the review work is completed, the supervision data of this test center for this test is packaged and sent to the test center server cluster 4;

[0055] S400: The test center server cluster 4 is responsible for storing and reviewing the supervision data of all test sites in this test. After the test, the supervision data of all test site central servers 3 are summarized, and each image is re-judged again based on the deep learning image classification algorithm to determine whether the behavior is in violation of the rules. The judgment result and the two judgment results of the test site central server 3 and the edge supervision device 1 are finally determined through a voting algorithm to determine whether the image is in violation of the rules, and then determine whether the examinee has violated the rules. After all the review work is completed, the test center server cluster 4 sends all the supervision data to the test score evaluation system as the basis for the examinee's test score.

[0056] In the embodiment provided by the present invention, the image classification algorithm adopts the ResNet convolutional neural network in deep learning. The ResNet convolutional neural image classification algorithm adds a residual mechanism on the basis of the ordinary convolutional neural network, which can effectively suppress the gradient diffusion effect, thereby increasing the depth of the image classification algorithm and greatly improving the learning ability of the image classification algorithm.

[0057] Although the embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and the embodiments. It can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily realized. Therefore, without departing from the general concept defined by the claims and equivalent scope, the present invention is not limited to the specific details and the illustrations shown and described here.

Claims

1. An examination supervision method based on image classification and edge computing, characterized in that: The following steps are involved: S100: The examination room server receives the test start signal from the central server of the examination center, triggers the edge supervision device to collect the test image of the examinee in real time, and classifies the examinee's behavior in the collected image through the image classification algorithm based on deep learning, and determines whether the examinee's behavior is qualified, and then saves the captured violation behavior image and the judgment result. After the test is over, the edge supervision device sends all the images during the test and the corresponding judgment result data to the corresponding examination room server; S200: The examination room server checks the data sent by each edge monitoring device, checks the data integrity, and packages and sends it to the examination center central server; S300: The central server of the test center collects and reviews the data sent by all the test center servers, and re-identifies the violation behavior of each image based on the deep learning image classification algorithm, and saves it together with the identification results of the edge supervision device. After the review work is completed, the supervision data of this test center for this test is packaged and sent to the server cluster of the test center; S400: The supervision data of the central servers of all test sites are aggregated, and each image is re-identified based on the deep learning image classification algorithm to determine whether the behavior is in violation of the rules. The judgment result and the two judgment results of the central server of the test site and the edge supervision device are finally determined through a voting algorithm to determine whether the image is in violation of the rules, and then to determine whether the examinee has violated the rules. After all the review work is completed, the server cluster of the test center sends all the supervision data to the test score evaluation system as the basis for the examinee's test score.

2. The test supervision method based on image classification and edge computing as claimed in claim 1, characterized in that: The image classification algorithm adopts the ResNet convolutional neural network in deep learning.

3. The test supervision method based on image classification and edge computing as claimed in claim 1, characterized in that: In step S100, image acquisition includes the following steps: S101: The edge supervision device initializes a timer, then captures and collects images, and classifies the images through an image classification algorithm to determine whether there is a violation; S102: If there is no violation, determine whether the timer is full; S1021: If it is full, reset the timer, save the data, and determine whether the test is over; S1022: If it is not full, directly determine whether the test is over; S103: If there is a violation, the data is directly saved to determine whether the test is over; S104: If the test is not over, return to recapture the image; if the test is over, save the supervision data and send it to the test site server.

4. An examination supervision system according to any one of claims 1 to 3, characterized in that: include: An examination center server cluster, at least one examination center central server connected to the examination center server cluster, at least one examination room server connected to the examination center central server, and at least one edge supervision device connected to the examination room server; The central server of the test center is used to collect and review the data sent by all the test center servers, and re-identify the violations of each image based on the deep learning image classification algorithm, and save it together with the identification results of the edge supervision device. After the review work is completed, the supervision data of this test center will be packaged and sent to the server cluster of the test center; The examination room server is used to receive the test start signal from the central server of the examination center, trigger the edge supervision device to collect the test images of the examinees in real time, and classify the examinee behavior in the collected images through the image classification algorithm based on deep learning, and judge whether the examinee behavior is qualified, and then save the captured violation behavior image and the judgment result. After the test is over, it receives all the images during the test and the corresponding judgment result data sent by the edge supervision device; The edge supervision device is used to send all images and corresponding judgment result data during the examination to the corresponding examination room server after the examination is completed; The central server cluster of the test center summarizes the supervision data of all central servers of the test centers, and re-determines whether the behavior of each image is in violation of the rules based on the deep learning image classification algorithm. The judgment result and the two judgment results of the central server of the test center and the edge supervision device are finally determined through a voting algorithm to determine whether the image is in violation of the rules, and then determine whether the candidate has violated the rules. After all the review work is completed, the server cluster of the test center sends all the supervision data to the test score evaluation system as the basis for the candidate's test score.

5. The examination supervision system of the examination supervision method based on image classification and edge computing according to claim 4 is characterized in that: The examination room server, the examination center central server, and the examination center server cluster use the Apache server framework to build the server.

6. The examination supervision system of the examination supervision method based on image classification and edge computing according to claim 4, characterized in that: The edge supervision device and the examination room server are connected via Gigabit Ethernet, and the examination room server and the examination center central server, as well as the examination center central server and the examination center server cluster are connected via Gigabit Ethernet or 10 Gigabit Ethernet.

7. The examination supervision system of the examination supervision method based on image classification and edge computing according to claim 4 is characterized in that: The image classification algorithm of the edge supervision device adopts the ResNet-18 or ResNet-34 structure.

8. The examination supervision system of the examination supervision method based on image classification and edge computing according to claim 4 is characterized in that: The image classification algorithm of the central server of the test center adopts the ResNet-101 network structure.

9. The examination supervision system of the examination supervision method based on image classification and edge computing according to claim 4, characterized in that: The image classification algorithm of the test center server cluster adopts the ResNet-152 network structure.