A digital twin and intelligent recognition system for examination rooms based on a large visual model

Abstract

The present application provides a digital twin and intelligent recognition system for an examination room based on a visual large model. The present application performs digital twin mapping through a digital twin mapping module to obtain a digital twin virtual examination room; captures the object behavior actions in the digital twin virtual examination room through a violation judgment module and constructs an examination room monitoring array; extracts the behavior constraint features of the digital twin virtual examination room based on the examination room monitoring array, and scores the behavior constraint features through machine learning; when the behavior constraint score is lower than a preset threshold, compares the object behavior actions with the preset violation behavior standard to achieve violation behavior determination; after determining that there is violation information through a violation record archiving module, the corresponding violation video clip is returned to the system interaction interface, and the violation video clip is automatically uploaded to a cloud storage center for archiving, thereby achieving abnormal identification of examinee behavior and evidence retention, improving examination security and achieving comprehensive control of remote examination room situations.

Description

Technical Field

[0001] The present application relates to the field of intelligent recognition technology, and more specifically, to an examination room digital twin and intelligent recognition system based on a large visual model. Background Art

[0002] Currently, examination room monitoring mainly relies on a combination of manual invigilation and video surveillance. Usually, two invigilators are arranged before and after the examination room, and there are also mobile invigilators to conduct random inspections. The video surveillance of the entire examination room will be uploaded to the Education Bureau in real time and monitored by the Education Bureau's remote monitors. This monitoring method can ensure examination room discipline to a certain extent, but there is also a problem of heavy reliance on manual labor.

[0003] Although manual invigilation is currently more efficient and accurate, it is difficult to manage invigilators in large-scale examinations and has high labor costs. Although the Education Bureau has remote video surveillance, the existing system lacks effective evidence preservation for violations. The examination room status needs to rely on manual video review, which has low coverage and cannot effectively identify key content. The Education Bureau cannot fully grasp the examination room situation in real time. Therefore, using existing surveillance videos, candidate information and examination room data to achieve intelligent invigilation has become an urgent problem to be solved. Summary of the invention

[0004] The present application provides an examination room digital twin and intelligent recognition system based on a visual large model. By constructing a digital twin examination room and performing behavioral constraint scoring on examination room objects, it is possible to identify abnormalities in examinees' behavior and retain evidence, thereby improving examination security and achieving comprehensive control of remote examination room situations.

[0005] In a first aspect, the present application provides a digital twin and intelligent recognition system of an examination room based on a large visual model, the recognition system comprising:

[0006] A video acquisition module is used to acquire the examination room surveillance video, and perform examination room object positioning and identity information matching based on the examination room surveillance video;

[0007] A digital twin mapping module is used to extract the key feature points of the examination room in the examination room monitoring video, and then perform digital twin mapping according to the key feature points of the examination room to obtain a digital twin virtual examination room;

[0008] A violation judgment module is used to capture the behavior of objects in the digital twin virtual examination room based on a visual large model algorithm, and to construct an examination room monitoring array according to the captured object behavior;

[0009] The violation judgment module is further used to extract the behavior constraint features of the digital twin virtual examination room based on the examination room monitoring array, and score the behavior constraint features through machine learning to obtain a behavior constraint score;

[0010] The violation judgment module is further configured to compare the behavior of the object in the digital twin virtual examination room with the preset violation standard to determine the violation when the behavior constraint score is lower than a preset threshold value;

[0011] The violation record archiving module is used to return the corresponding violation video clip to the system interaction interface after determining whether there is violation information, and automatically upload the violation video clip to the cloud storage center for archiving.

[0012] In this embodiment, before performing examination room object positioning and identity information matching based on the examination room surveillance video, the method further includes: performing noise reduction preprocessing on the examination room surveillance video.

[0013] In this embodiment, the examination room object positioning and identity information matching based on the examination room surveillance video specifically includes:

[0014] Extracting video frames from the examination room surveillance video to obtain multiple examination room video frames;

[0015] Using a target detection algorithm to locate objects in the examination room respectively in multiple examination room video frames, the examination room objects including students and invigilators;

[0016] The facial recognition algorithm is used to extract the facial features of the test subjects for comparison and matching with the identity information in the database.

[0017] In this embodiment, the SIFT feature extraction algorithm is used to extract the key feature points of the examination room in the examination room monitoring video.

[0018] In this embodiment, digital twin mapping is performed according to the key feature points of the examination room to obtain a digital twin virtual examination room, which specifically includes:

[0019] Obtain key feature points of the examination room and a virtual 3D model of the examination room;

[0020] According to the virtual three-dimensional model of the examination room, three-dimensional model mapping is performed on the key feature points of each examination room to obtain a digital twin virtual examination room.

[0021] In this embodiment, the OpenPose visual large model algorithm is used to capture the behavior and actions of objects in the digital twin virtual examination room.

[0022] In this embodiment, in the process of automatically uploading the illegal video clip to the cloud storage center for archiving, it is uploaded to the cloud storage center through an API, and the file path, naming format and file access permission are set.

[0023] In this embodiment, the behavior constraint feature is scored by machine learning to obtain a behavior constraint score specifically including:

[0024] Obtaining a standard examination room surveillance video, and extracting standard behavior constraint features based on the standard examination room surveillance video;

[0025] Determining the cosine similarity between the feature vector in the standard behavior constraint feature and the feature vector in the behavior constraint feature to be scored;

[0026] Constructing a machine learning comparison vector according to the standard behavior constraint feature, and constructing a scoring feature vector according to the behavior constraint feature to be scored;

[0027] A support vector machine is used to perform scoring based on the cosine similarity, the scoring feature vector and the machine learning comparison vector to obtain a final behavior constraint score.

[0028] In this embodiment, the behavior of the object in the digital twin virtual examination room is compared with the preset violation standard, and the violation determination is implemented specifically including:

[0029] Get the current timestamp;

[0030] Extracting the behavior actions of the objects in the digital twin virtual examination room according to the current timestamp to obtain the skeleton behavior information corresponding to each examination room object;

[0031] Get preset violation behavior standards;

[0032] The skeleton behavior information corresponding to each examination room object is compared with the preset violation behavior standard, and the examination room object whose skeleton behavior information exceeds the violation behavior standard is determined as a violation behavior object.

[0033] In this embodiment, after determining that there is illegal information, returning the corresponding illegal video clip to the system interaction interface specifically includes:

[0034] After determining that there is violation information, obtain the timestamps corresponding to the start and end of the violation behavior;

[0035] The system backend extracts the illegal behavior segments from the examination room surveillance video according to the timestamps corresponding to the start and end of the illegal behavior, and saves them as independent video files;

[0036] After the back-end video clips are generated, the front-end interface is notified through polling that new illegal video clips have been generated, and a dynamic list is generated in the system interactive interface to display the video clips of each illegal behavior.

[0037] The technical solution provided by the embodiments disclosed in this application has the following beneficial effects:

[0038] The video acquisition module is used to obtain the examination room surveillance video, and the examination room object positioning and identity information matching are performed based on the examination room surveillance video; after the key feature points of the examination room in the examination room surveillance video are extracted through the digital twin mapping module, digital twin mapping is performed based on the key feature points of the examination room to obtain a digital twin virtual examination room; the violation judgment module is used to capture the object behavior actions in the digital twin virtual examination room based on the visual large model algorithm, and an examination room monitoring array is constructed based on the captured object behavior actions; the behavior constraint features of the digital twin virtual examination room are extracted based on the examination room monitoring array, and the behavior constraint features are scored through machine learning to obtain a behavior constraint score; when the behavior constraint score is lower than the preset threshold, the object behavior actions in the digital twin virtual examination room are compared with the preset violation behavior standard to realize the violation behavior judgment; after determining that there is violation information through the violation record archiving module, the corresponding violation video clip is returned to the system interaction interface, and the violation video clip is automatically uploaded to the cloud storage center for archiving.

[0039] It can be seen that in this application, by comparing with the candidate identity information database, the legality of the candidate's identity is ensured, and violations such as substitution are prevented. The key feature points of the examination room, that is, the dynamic information such as the candidate's posture and gesture, are identified from the monitoring video to reflect their behavioral characteristics. The key feature points are mapped to the virtual environment to create a digital twin virtual examination room, thereby simulating the spatial layout of the actual examination room and providing a basis for subsequent analysis; then a monitoring array is formed based on the captured object behavior information, and the candidate's behavior pattern is analyzed through the dimensions of space and time. The system also extracts the candidate's behavioral constraint characteristics, and then scores the behavioral constraint characteristics through the trained machine learning model to achieve a quantitative evaluation of the candidate's behavior, and sets a preset threshold to promptly identify candidates with abnormal behavior. Through digital twin technology and advanced computer vision algorithms, real-time monitoring of the entire examination room is achieved, and dynamic information in the examination room can be grasped at any time to reduce the probability of violations. With the help of a machine learning scoring system for behavioral characteristics, abnormal behavior of candidates can be quickly identified to ensure the fairness of the examination.

[0040] To sum up, the technical solution adopted in this application can build a digital twin examination room and perform behavioral constraint scoring on examination room objects, realize abnormal identification of examinees' behavior and retain evidence, improve examination security and achieve comprehensive control of remote examination room situations. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0042] Figure 1 It is a module structure diagram of the examination room digital twin and intelligent recognition system based on the visual big model provided by this application;

[0043] Figure 2 It is a schematic diagram of the process of locating objects in the examination room and matching identity information according to the application;

[0044] Figure 3 It is a flowchart of abnormal behavior identification for digital twin examinations provided in this application. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0046] The embodiment of the present application provides an examination room digital twin and intelligent recognition system based on a visual big model, the core of which is to obtain the examination room monitoring video through a video acquisition module, and locate the examination room objects and match the identity information based on the examination room monitoring video; after extracting the key feature points of the examination room in the examination room monitoring video through the digital twin mapping module, digital twin mapping is performed according to the key feature points of the examination room to obtain a digital twin virtual examination room; the object behavior actions in the digital twin virtual examination room are captured based on the visual big model algorithm through the violation judgment module, and an examination room monitoring array is constructed according to the captured object behavior actions; the behavior constraint features of the digital twin virtual examination room are extracted based on the examination room monitoring array, and the behavior constraint features are scored through machine learning to obtain a behavior constraint score; when the behavior constraint score is lower than a preset threshold, the object behavior actions in the digital twin virtual examination room are compared with the preset violation behavior standard to realize violation behavior judgment; after determining that there is violation information through the violation record archiving module, the corresponding violation video clip is returned to the system interaction interface, and the violation video clip is automatically uploaded to the cloud storage center for archiving. The above solution can be used to build a digital twin examination room and conduct behavioral constraint scoring on examination room subjects, realize abnormal identification of examinees' behavior and retain evidence, improve examination security and achieve comprehensive control of remote examination room situations.

[0047] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods. Figure 1As shown, this figure is a module structure diagram of a digital twin and intelligent recognition system of an examination room based on a visual large model according to this embodiment of the present application. The recognition system includes: a video acquisition module 100, a digital twin mapping module 200, a violation judgment module 300 and a violation record archiving module 400, which are respectively described as follows:

[0048] The video acquisition module 100 is used to acquire the examination room monitoring video, and perform examination room object positioning and identity information matching based on the examination room monitoring video.

[0049] In this embodiment, a surveillance camera is used to collect the examination room surveillance video;

[0050] In this embodiment, before locating the examination room objects and matching the identity information based on the examination room surveillance video, it also includes: performing noise reduction preprocessing on the examination room surveillance video. In specific implementation, the mean filtering method can be used to perform noise reduction preprocessing on the examination room surveillance video, that is, by taking the average value of the neighboring pixels for each pixel, the random noise of the image is reduced.

[0051] refer to Figure 2 The figure is a schematic diagram of the process of locating an object in an examination room and matching identity information in some embodiments of the present application. In this embodiment, locating an object in an examination room and matching identity information based on the examination room monitoring video specifically includes:

[0052] In step S11, video frames are extracted from the examination room monitoring video to obtain a plurality of examination room video frames;

[0053] In step S12, a target detection algorithm is used to locate objects in the examination room respectively on the multiple examination room video frames, where the objects in the examination room include students and invigilators;

[0054] In step S13, facial features of the test subject are extracted and compared using a facial recognition algorithm, and matched with the identity information in a database.

[0055] In specific implementation, the YOLO target detection algorithm can be used to locate the students and invigilators in the examination room in each frame to realize the positioning of the objects in the examination room. The FaceNet facial recognition algorithm can be used to extract the facial features of the objects and match them with the identity information in the database to realize the identity information matching of the objects in the examination room.

[0056] The digital twin mapping module 200 is used to extract the key feature points of the examination room in the examination room monitoring video, and then perform digital twin mapping based on the key feature points of the examination room to obtain a digital twin virtual examination room.

[0057] It should be noted that the key feature points of the examination room include: candidate position feature points and examination environment feature points, wherein the candidate position feature points include face feature points, body joint points, hand feature points and sight direction points; the examination environment feature points include: desk and chair and test paper edge points, seat layout feature points and entrance and exit position feature points; in this embodiment, the SIFT feature extraction algorithm is used to extract the key feature points of the examination room in the examination room monitoring video. The SIFT feature extraction algorithm is an algorithm widely used for image key feature point extraction. It can be used to extract stable feature points in the examination room monitoring video for object positioning, behavior analysis and scene matching. The SIFT feature extraction algorithm has scale invariance and rotation invariance, which is suitable for the examination room monitoring video to maintain stability under environmental changes such as lighting, viewing angle and noise.

[0058] In this embodiment, digital twin mapping is performed according to the key feature points of the examination room to obtain a digital twin virtual examination room, which specifically includes:

[0059] Obtain key feature points of the examination room and a virtual 3D model of the examination room;

[0060] According to the virtual three-dimensional model of the examination room, three-dimensional model mapping is performed on the key feature points of each examination room to obtain a digital twin virtual examination room.

[0061] In this embodiment, after obtaining the precise dimensions of the examination room space such as length, width, and height, including the physical locations of tables, chairs, passages, walls, monitoring equipment, etc., Blender modeling software is used to create a three-dimensional model based on the actual examination room, including all fixed objects in the examination room, and fixed points in the examination room (corners, edges of fixed tables and chairs, etc.) are selected as reference points to define a three-dimensional coordinate system so that the coordinate mapping between the actual examination room and the virtual model is consistent.

[0062] In this embodiment, the three-dimensional model is mapped to the key feature points of each examination room according to the virtual three-dimensional model of the examination room. In the process of obtaining the digital twin virtual examination room, the two-dimensional plane coordinates in the video are converted into three-dimensional space coordinates through depth information, and the camera projection matrix can be used for coordinate transformation; then the extracted features of the examinee, objects, invigilator, etc. are matched with the objects in the three-dimensional model to ensure that each actual feature point finds a corresponding position in the virtual examination room model. Finally, the three-dimensional coordinates of each key feature point in the actual examination room are obtained by calculation, and corresponded with the three-dimensional coordinates in the virtual model to establish a mapping relationship between the actual space and the virtual model. In addition, the intrinsic and extrinsic parameters of the camera can be corrected through calibration technology (usually using Zhang Zhengyou calibration method) to reduce errors in the coordinate conversion process and complete the three-dimensional model mapping.

[0063] A violation judgment module 300 is used to capture the behavior of objects in the digital twin virtual examination room based on a visual large model algorithm, and to construct an examination room monitoring array according to the captured object behavior;

[0064] In this embodiment, the OpenPose visual large model algorithm can be used to capture the behavior and actions of objects in the digital twin virtual examination room, that is, the behavior and actions of objects in the digital twin virtual examination room are captured by real-time tracking the movements of the examination objects in the examination room, and at the same time, the captured key point data (limbs, head position) of the examination objects are used as the basic feature input of the object behavior and actions. It should be noted that the OpenPose visual large model algorithm is an open source computer vision algorithm based on deep learning, which is commonly used for human posture estimation. The algorithm can detect human key points in real time from images or videos, including the positions of joints, hands, face and feet, and provide skeleton information of the object based on the key points, so that the posture and action of the person can be recognized and tracked.

[0065] In this embodiment, constructing an examination room monitoring array according to captured object behaviors specifically includes:

[0066] Label the examination room objects according to spatial order;

[0067] Perform skeleton restoration on the object behavior actions corresponding to each label of the examination room object, and obtain the skeleton behavior information corresponding to each label of the examination room object;

[0068] Based on the skeleton behavior information, the behavior coordination degree is extracted to obtain the skeleton behavior coordination degree corresponding to the examination room objects of each label, and all the skeleton behavior coordination degrees are sequentially organized into an examination room monitoring array according to the label order.

[0069] In this embodiment, a multi-view camera or a depth camera can be combined to obtain the depth information of the object, and the two-dimensional skeleton points can be converted into three-dimensional coordinates to achieve the restoration of the skeleton in three-dimensional space. Three-dimensional reconstruction can be performed by triangulation or depth mapping methods to obtain the skeleton behavior information, which includes: the three-dimensional coordinates of the skeleton extraction key points of the examination room object (head, shoulder, elbow, knee, etc.), and the motion information (speed and acceleration) of the skeleton extraction key points; and then obtain standard skeleton behavior comparison information, which is collected according to the standard examination action demonstration process performed by the experimenter in the standard environment examination room, and the skeleton behavior coordination degree is obtained according to the feature similarity between the skeleton behavior comparison information and the skeleton behavior information of the examination room object. In specific implementation, a comparison coordinate system is constructed based on the spatial center of the examination room object as the origin, and each skeleton extraction key point of the examination room object in the comparison coordinate system is normalized according to the three-dimensional coordinates of the skeleton extraction key point in the skeleton behavior comparison information, and then the skeleton behavior coordination degree of the examination room object is obtained according to the ratio between the position deviation of each skeleton extraction key point and the preset comparison threshold.

[0070] In specific implementation, the examination room monitoring array is a feature matrix composed of the skeleton behavior coordination degrees corresponding to each examination room line according to the spatial position coordinates of the examination room. For example, when the number of seats in the examination room is N×M, the examination room monitoring array is a feature matrix composed of N×M skeleton behavior coordination degrees according to the spatial positions, that is, according to the spatial position of each seat, the corresponding skeleton behavior coordination degree is filled into the feature matrix. By constructing the feature matrix of the examination room monitoring array, the behavior of each object in the examination room can be effectively quantitatively analyzed to provide data support for examination room management and supervision. At the same time, the feature matrix can also be used as input data for training machine learning models to predict potential violations in the examination room or identify behavior patterns, which greatly enhances the intensity of examination room monitoring and improves the accuracy of identifying violations in the examination room.

[0071] The violation judgment module 300 is also used to extract the behavior constraint features of the digital twin virtual examination room based on the examination room monitoring array, and score the behavior constraint features through machine learning to obtain a behavior constraint score.

[0072] In this embodiment, the behavior constraint feature extraction based on the examination room monitoring array includes: obtaining the examination room monitoring array, and obtaining the behavior constraint feature by performing singular value decomposition on the examination room monitoring array, wherein the eigenvalue or eigenvector of the examination room monitoring array can be used as the behavior constraint feature. It should be noted that the eigenvalue or eigenvector in the behavior constraint feature represents the main behavior pattern and action distribution of the monitored object.

[0073] In this embodiment, the behavior constraint features are scored by machine learning, and the behavior constraint score obtained specifically includes:

[0074] Obtaining a standard examination room surveillance video, and extracting standard behavior constraint features based on the standard examination room surveillance video;

[0075] Determining the cosine similarity between the feature vector in the standard behavior constraint feature and the feature vector in the behavior constraint feature to be scored;

[0076] Constructing a machine learning comparison vector according to the standard behavior constraint feature, and constructing a scoring feature vector according to the behavior constraint feature to be scored;

[0077] A support vector machine is used to perform scoring based on the cosine similarity, the scoring feature vector and the machine learning comparison vector to obtain a final behavior constraint score. In specific implementation, after integrating the cosine similarity, the scoring feature vector and the machine learning comparison vector into a feature set, a support vector machine (SVM) model is used to train the relationship between the scoring features and the machine learning comparison vector, and a Gaussian kernel function is selected to improve the classification performance, and then the trained SVM model is used to score the new features to be scored to obtain the final behavior constraint score. It should be noted that the score output by the support vector machine model can be used to determine whether the behavior to be scored complies with the regulations, thereby realizing the behavior constraint scoring of the examination subjects, and according to the scoring results, a threshold can be set to determine whether there is a violation and take corresponding measures.

[0078] In specific implementation, the standard examination room monitoring video is an examination room video recorded in advance in a standard venue with no violation of regulations. Extracting standard behavior constraint features based on the standard examination room monitoring video specifically includes: capturing behavioral actions based on the standard examination room monitoring video, constructing a standard examination room monitoring array according to the captured object behavior actions, and extracting standard behavior constraint features based on the standard examination room monitoring array.

[0079] In this embodiment, the machine learning comparison vector may be a mean vector of feature vectors in the standard behavior constraint feature, and the scoring feature vector may be a mean vector of feature vectors in the behavior constraint feature to be scored.

[0080] The violation judgment module 300 is also used to compare the behavior of the object in the digital twin virtual examination room with the preset violation standard when the behavior constraint score is lower than a preset threshold value to realize violation judgment.

[0081] In this embodiment, the threshold of the behavior restriction score is set to 60.

[0082] In this embodiment, the behavior of the object in the digital twin virtual examination room is compared with the preset violation standard, and the violation determination is implemented specifically including:

[0083] Get the current timestamp;

[0084] Extracting the behavior actions of the objects in the digital twin virtual examination room according to the current timestamp to obtain the skeleton behavior information corresponding to each examination room object;

[0085] Get preset violation behavior standards;

[0086] The skeleton behavior information corresponding to each examination room object is compared with the preset violation behavior standard, and the examination room object whose skeleton behavior information exceeds the violation behavior standard is determined as a violation behavior object.

[0087] In this embodiment, the skeleton behavior information of the examinee and the invigilator can be detected based on the behavior analysis algorithm of the visual large model, 40 predefined violation behaviors can be identified and marked, and whether the examination room object has violated the rules can be determined based on the comparison results.

[0088] The violation record archiving module 400 is used to return the corresponding violation video clip to the system interaction interface after determining whether there is violation information, and automatically upload the violation video clip to the cloud storage center for archiving.

[0089] In this embodiment, after determining that there is illegal information, returning the corresponding illegal video clip to the system interaction interface specifically includes:

[0090] After determining that there is violation information, obtain the timestamps corresponding to the start and end of the violation behavior;

[0091] The system backend extracts the illegal behavior segments from the examination room surveillance video according to the timestamps corresponding to the start and end of the illegal behavior, and saves them as independent video files;

[0092] After the back-end video clips are generated, the front-end interface is notified through polling that new illegal video clips have been generated, and a dynamic list is generated in the system interactive interface to display the video clips of each illegal behavior.

[0093] In this embodiment, in the process of automatically uploading the illegal video clip to the cloud storage center for archiving, it is uploaded to the cloud storage center through an API, and the file path, naming format and file access permission are set.

[0094] Digital twin is a technology that simulates and analyzes the real-world environment through virtual models. In the digital twin and intelligent recognition system of the examination room based on the visual big model provided by this application, the digital twin examination room is the core, which receives and processes data from different sources to achieve real-time monitoring of the examination room environment and identification of abnormal behavior. In this embodiment, reference Figure 3 As shown, Figure 3 This is a flowchart of the application for identifying abnormal behavior in digital twin examinations. The following is a detailed description of the flowchart:

[0095] Data input includes examination room video data, examination room information, candidate information and invigilator information, which are described as follows:

[0096] Examination room video data: Video stream from the examination room surveillance camera, used to capture the situation inside the examination room.

[0097] Examination room information: including examination room layout, seating arrangement and other basic information.

[0098] Candidate information: personal information of the candidate, used for identity verification and behavior analysis.

[0099] Invigilator information: The invigilator’s personal information, used to monitor whether his / her behavior complies with the regulations.

[0100] Digital twin examination room: It is the core part of the system, which integrates all input data to create a virtual examination room model. This model can reflect the actual situation of the examination room in real time, including the behavior of candidates and invigilators.

[0101] Display: The system displays the real-time status of the digital twin examination room to the monitoring personnel, allowing them to intuitively understand the situation in the examination room.

[0102] Abnormal behavior identification: The system identifies abnormal behavior by analyzing video data and the behavior patterns of candidates and invigilators, including cheating by candidates or invigilator misconduct.

[0103] Storage: Identified abnormal behaviors are recorded and stored in the system for subsequent review and analysis.

[0104] Abnormal behavior of candidates and invigilators: The system can distinguish and record abnormal behavior of candidates and invigilators, which helps to take targeted measures.

[0105] It can be seen that the present application provides a digital twin and intelligent recognition system for an examination room based on a visual large model, which can obtain the examination room monitoring video through the video acquisition module, and locate the examination room objects and match the identity information based on the examination room monitoring video; after extracting the key feature points of the examination room in the examination room monitoring video through the digital twin mapping module, digital twin mapping is performed according to the key feature points of the examination room to obtain a digital twin virtual examination room; the object behavior actions in the digital twin virtual examination room are captured based on the visual large model algorithm through the violation judgment module, and an examination room monitoring array is constructed according to the captured object behavior actions; the behavior constraint features of the digital twin virtual examination room are extracted based on the examination room monitoring array, and the behavior constraint features are scored through machine learning to obtain a behavior constraint score; when the behavior constraint score is lower than the preset threshold, the object behavior actions in the digital twin virtual examination room are compared with the preset violation behavior standard to realize the violation behavior judgment; after determining that there is violation information through the violation record archiving module, the corresponding violation video clip is returned to the system interaction interface, and the violation video clip is automatically uploaded to the cloud storage center for archiving. The above solution can be used to build a digital twin examination room and conduct behavioral constraint scoring on examination room subjects, realize abnormal identification of examinees' behavior and retain evidence, improve examination security and achieve comprehensive control of remote examination room situations.

[0106] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0107] A person skilled in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, the storage medium including a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically-erasable programmable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0108] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

Claims

1. A digital twin and intelligent recognition system for examination rooms based on a large visual model, characterized in that: The identification system comprises: A video acquisition module is used to acquire the examination room surveillance video, and perform examination room object positioning and identity information matching based on the examination room surveillance video; A digital twin mapping module is used to extract the key feature points of the examination room in the examination room monitoring video, and then perform digital twin mapping according to the key feature points of the examination room to obtain a digital twin virtual examination room; A violation judgment module is used to capture the behavior of objects in the digital twin virtual examination room based on a visual large model algorithm, and to construct an examination room monitoring array according to the captured object behavior; The violation judgment module is further used to extract the behavior constraint features of the digital twin virtual examination room based on the examination room monitoring array, and score the behavior constraint features through machine learning to obtain a behavior constraint score; The violation judgment module is further configured to compare the behavior of the object in the digital twin virtual examination room with the preset violation standard to determine the violation when the behavior constraint score is lower than a preset threshold value; The violation record archiving module is used to return the corresponding violation video clip to the system interaction interface after determining whether there is violation information, and automatically upload the violation video clip to the cloud storage center for archiving; Among them, building an examination room monitoring array based on captured object behaviors specifically includes: Label the examination room objects according to spatial order; Perform skeleton restoration on the object behavior actions corresponding to each label of the examination room object, and obtain the skeleton behavior information corresponding to each label of the examination room object; Based on the skeleton behavior information, the behavior coordination degree is extracted to obtain the skeleton behavior coordination degree corresponding to each test room object of each label, and all the skeleton behavior coordination degrees are sequentially formed into a test room monitoring array according to the label order; Extracting behavior constraint features based on the examination room monitoring array includes: acquiring the examination room monitoring array, and obtaining behavior constraint features by performing singular value decomposition on the examination room monitoring array, wherein the behavior constraint features represent the behavior pattern and action distribution of the monitored object.

2. The digital twin and intelligent recognition system of the examination room based on the visual large model as claimed in claim 1, characterized in that: Before positioning the examination room object and matching the identity information based on the examination room surveillance video, the method also includes: performing noise reduction preprocessing on the examination room surveillance video.

3. The digital twin and intelligent recognition system of the examination room based on the visual large model as claimed in claim 1, characterized in that: The examination room object positioning and identity information matching based on the examination room surveillance video specifically includes: Extracting video frames from the examination room surveillance video to obtain multiple examination room video frames; Using a target detection algorithm to locate objects in the examination room respectively in multiple examination room video frames, the examination room objects including students and invigilators; The facial recognition algorithm is used to extract the facial features of the test subjects for comparison and matching with the identity information in the database.

4. The digital twin and intelligent recognition system of the examination room based on the visual large model as claimed in claim 1, characterized in that: The SIFT feature extraction algorithm is used to extract the key feature points of the examination room in the examination room monitoring video.

5. The digital twin and intelligent recognition system of the examination room based on the visual large model as claimed in claim 1, characterized in that: Digital twin mapping is performed based on the key feature points of the examination room, and the digital twin virtual examination room includes: Obtain key feature points of the examination room and a virtual 3D model of the examination room; According to the virtual three-dimensional model of the examination room, three-dimensional model mapping is performed on the key feature points of each examination room to obtain a digital twin virtual examination room.

6. The digital twin and intelligent recognition system of the examination room based on the visual large model as claimed in claim 1, characterized in that: The OpenPose visual large model algorithm is used to capture the behavior and actions of objects in the digital twin virtual examination room.

7. The digital twin and intelligent recognition system of the examination room based on the visual large model as claimed in claim 1, characterized in that: In the process of automatically uploading the illegal video clips to the cloud storage center for archiving, the video clips are uploaded to the cloud storage center through the API, and the file path, naming format and file access permissions are set.

8. The digital twin and intelligent recognition system of the examination room based on the visual large model as claimed in claim 1, characterized in that: The behavior constraint features are scored by machine learning to obtain a behavior constraint score including: Obtaining a standard examination room surveillance video, and extracting standard behavior constraint features based on the standard examination room surveillance video; Determining the cosine similarity between the feature vector in the standard behavior constraint feature and the feature vector in the behavior constraint feature to be scored; Constructing a machine learning comparison vector according to the standard behavior constraint feature, and constructing a scoring feature vector according to the behavior constraint feature to be scored; A support vector machine is used to perform scoring based on the cosine similarity, the scoring feature vector and the machine learning comparison vector to obtain a final behavior constraint score.

9. The digital twin and intelligent recognition system of the examination room based on the visual large model as claimed in claim 1, characterized in that: Comparing the behavior of the object in the digital twin virtual examination room with the preset violation standards to determine the violation specifically includes: Get the current timestamp; Extracting the behavior actions of the objects in the digital twin virtual examination room according to the current timestamp to obtain the skeleton behavior information corresponding to each examination room object; Get preset violation behavior standards; The skeleton behavior information corresponding to each examination room object is compared with the preset violation behavior standard, and the examination room object whose skeleton behavior information exceeds the violation behavior standard is determined as a violation behavior object.

10. The digital twin and intelligent recognition system of the examination room based on the visual large model as claimed in claim 1, characterized in that: After determining that there is illegal information, the corresponding illegal video clip is returned to the system interaction interface, specifically including: After determining that there is violation information, obtain the timestamps corresponding to the start and end of the violation behavior; The system backend extracts the illegal behavior segments from the examination room surveillance video according to the timestamps corresponding to the start and end of the illegal behavior, and saves them as independent video files; After the back-end video clips are generated, the front-end interface is notified through polling that new illegal video clips have been generated, and a dynamic list is generated in the system interactive interface to display the video clips of each illegal behavior.

Images