Examinee abnormal behavior recognition method, electronic equipment, storage medium and computer program product

Abstract

The invention discloses an examinee abnormal behavior recognition method, electronic equipment, a storage medium and a computer program product, and the method comprises the steps: determining a behavior feature vector of a first examinee based on a first video and the facial image feature of the first examinee in a first examination room; inputting the behavior feature vector of the first examinee into a first model to obtain first information; wherein the first video represents videos shot in a plurality of different directions in the first examination room, and the behavior feature vector of the first examinee at least represents the occurrence frequency and the occurrence duration of the first examinee in the first video; the first model is used for predicting whether the behavior of the corresponding examinee is abnormal or not based on the behavior feature vector of the examinee, and the first information indicates that the behavior of the first examinee is abnormal or normal.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method for identifying abnormal behavior of examinees, an electronic device, a storage medium and a computer program product. Background Art

[0002] In the related art, monitoring of cheating behavior in the examination room requires manual screening of the examination room monitoring video input into the examination room cheating behavior recognition network model, or the analysis results of the examinee's operating behavior on the input and output devices and the input content need to be input into the cheating recognition model to identify the cheating behavior in the examination room. There is a lack of methods for automatically identifying abnormal behavior of examinees in the examination room and the efficiency of identifying cheating behavior in the examination room is not high. Summary of the invention

[0003] To solve related technical problems, the embodiments of the present application provide a method for identifying abnormal behavior of examinees, an electronic device, a storage medium and a computer program product.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] The present application provides a method for identifying abnormal behavior of examinees, the method comprising:

[0006] determining a behavior feature vector of the first examinee based on a first video and facial image features of the first examinee in the first examination room, wherein the first video represents a video shot at multiple different positions in the first examination room, and the behavior feature vector of the first examinee represents at least the number of times and duration of appearance of the first examinee in the first video;

[0007] The behavior feature vector of the first examinee is input into a first model to obtain first information; wherein the first model is used to predict whether the behavior of the corresponding examinee is abnormal based on the examinee's behavior feature vector, and the first information indicates whether the behavior of the first examinee is abnormal or normal.

[0008] In the above solution, before determining the behavior feature vector of the first examinee based on the first video and the facial image features of the first examinee in the first examination room, the method further includes:

[0009] Determine, based on the second video and facial image features of one or more second examinees in the second examination room, behavioral feature vectors of the one or more second examinees, wherein the second video represents a video shot at multiple different positions in the second examination room, and the behavioral feature vector of the second examinee represents at least the number of appearances and the duration of appearance of the corresponding second examinee in the second video;

[0010] The first model is trained based on the one or more behavioral feature vectors of the second examinees until a set convergence condition is reached.

[0011] In the above scheme, the multiple different directions in the examination room include at least four directions of front, back, left and right in the examination room; the examinee's behavioral feature vector represents the number of times and duration of the examinee's appearance in the video taken at each direction in the examination room, as well as the number and / or duration of the examinee's non-appearance in the video taken at all directions in the examination room.

[0012] In the above scheme, the behavioral feature vector of the examinee is determined, including:

[0013] Extract frames from the video at set time intervals to obtain multiple pictures;

[0014] Based on the facial image features of the examinee and the multiple pictures, determine all first serial numbers and / or all second serial numbers corresponding to the examinee, wherein the first serial number represents the serial number of the picture containing the examinee's facial image among the multiple pictures, and the second serial number represents the serial number of the picture not containing the examinee's facial image among the multiple pictures;

[0015] Based on all first serial numbers and / or all second serial numbers corresponding to the examinee, determine the number of times and duration of appearance of the examinee in the plurality of pictures, and / or the number of times and duration of absence of the examinee in the plurality of pictures;

[0016] Determine the examinee's behavioral feature vector based on one or more of the following:

[0017] The number of times and duration of the candidate's appearance in the multiple images;

[0018] The number of times and duration that the candidate is absent from the multiple images.

[0019] In the above solution, the training of the first model based on the behavioral feature vectors of the one or more second examinees includes:

[0020] Inputting the behavior feature vectors of the one or more second examinees into the first model to obtain second information of the one or more second examinees, the second information indicating whether the predicted behavior of the second examinees is abnormal or normal;

[0021] A first gradient is calculated by using a back propagation algorithm, the second information, and the third information; the first gradient is a gradient of a loss function with respect to a parameter of the first model, and the third information indicates whether the behavior of the marked second examinee is abnormal or normal;

[0022] Based on a first parameter, a first learning rate and the first gradient, update the parameters of the first model; the first parameter is an initial parameter of the first model or a current model parameter; the first learning rate is a learning rate used in the N+1th round of training, and is determined based on at least a second learning rate used in the Nth round of training, where N is a positive integer and is less than or equal to a set total number of training rounds.

[0023] In the above solution, updating the parameters of the first model based on the first parameter, the first learning rate and the first gradient includes:

[0024] Determine the first learning rate based on the second learning rate, the set learning rate adjustment factor and the first gradient, and based on the set learning rate decay rate and the value of N;

[0025] Based on the difference between the first parameter and a first value, the parameters of the first model are updated, where the first value represents the product of the first learning rate and the first gradient.

[0026] In the above scheme, determining the first learning rate based on the second learning rate, the set learning rate adjustment factor and the first gradient, and based on the set learning rate decay rate and the value of N includes:

[0027] Determine a second value based on the set learning rate adjustment factor, the set hyperparameter and the first gradient;

[0028] Determine a third value based on the set learning rate decay rate and the value of N;

[0029] The first learning rate is determined based on the second learning rate, the second value, and the third value.

[0030] In the above solution, the first learning rate is negatively correlated with the absolute value of the first gradient.

[0031] In the above solution, the third value is calculated based on the set learning rate decay rate as the base and the value of N as the exponent; and / or

[0032] The first learning rate is the product of the second learning rate, the second value, and the third value.

[0033] The embodiment of the present application also provides an electronic device, including a processor and a communication interface; wherein,

[0034] The processor is configured to determine a behavior feature vector of the first examinee based on a first video and a facial image feature of the first examinee in a first examination room, wherein the first video represents a video captured at multiple positions in the first examination room, and the behavior feature vector of the first examinee represents at least the number of times and duration of appearance of the first examinee in the first video;

[0035] The behavior feature vector of the first examinee is input into a first model to obtain first information; wherein the first model is used to predict whether the behavior of the corresponding examinee is abnormal based on the examinee's behavior feature vector, and the first information indicates whether the behavior of the first examinee is abnormal or normal.

[0036] An embodiment of the present application further provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above methods are implemented.

[0037] An embodiment of the present application also provides a computer program product, including a computer program, which implements the steps of any of the above methods when executed by a processor.

[0038] In the method for identifying abnormal behavior of examinees, electronic equipment, storage medium and computer program product provided in the embodiments of the present application, based on the facial image features of the first video and the first examinee in the first examination room, the behavioral feature vector of the first examinee is determined; the behavioral feature vector of the first examinee is input into the first model to obtain first information; wherein, the first video represents a video shot from multiple different directions in the first examination room, and the behavioral feature vector of the first examinee at least represents the number of appearances and the duration of the appearance of the first examinee in the first video; the first model is used to predict whether the behavior of the corresponding examinee is abnormal based on the behavioral feature vector of the examinee, and the first information indicates whether the behavior of the first examinee is abnormal or normal. The above scheme, by inputting the behavioral feature vector of the examinee in the examination room into the first model to obtain the first information, realizes the automatic identification of abnormal behavior of examinees in the examination room, improves the efficiency of identifying abnormal behavior of examinees, and the videos shot from different directions can identify most of the abnormal behaviors of examinees, thereby further improving the accuracy of identifying abnormal behavior of examinees. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a flow chart of a method for identifying abnormal behavior of examinees according to an embodiment of the present application;

[0040] Figure 2 This is a schematic diagram of the structure of a device for identifying abnormal behavior of examinees according to an embodiment of the present application;

[0041] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0042] In the current educational examination process, invigilators need to closely monitor the behavior of candidates to ensure the fairness and impartiality of the examination. However, since there are too many candidates in the examination room, it is difficult for the monitoring personnel to fully and real-time monitor the behavior of each candidate. Therefore, a system that can automatically identify abnormal behavior of candidates is needed.

[0043] There are currently two methods for identifying cheating in exams:

[0044] (I) First, collect the video data of the examination room monitoring, and manually screen and pre-process the data to produce the examination room behavior data set; then design the examination room cheating behavior recognition network model based on the monitoring video stream, including the spatial information extraction channel, the temporal information extraction channel and the spatiotemporal attention mechanism feature fusion module; then train the examination room cheating behavior recognition network to obtain the trained examination room cheating behavior recognition network model; finally, save the trained examination room cheating behavior recognition network model and deploy it to the actual system for real-time examination room cheating behavior recognition in the monitoring video stream.

[0045] (ii) firstly detect through the captured image that the deflection angle of the facial normal of the candidate to be identified exceeds a preset threshold, intercept multiple images of the candidate to be identified within a preset time before and after the time point of the captured image, and analyze the synergy between the collected operation behavior of the I / O device of the candidate to be identified and the input content; then substitute the synergy analysis result into a pre-trained cheating identification model to obtain the output result of the cheating identification model; if the output result of the cheating identification model indicates that the behavior of the candidate to be identified is abnormal, then output a cheating detection signal to the display screen of the candidate to be identified, and judge whether the candidate to be identified has suspected cheating behavior based on the output cheating detection signal.

[0046] It can be seen that the current cheating identification method in the examination room cannot automatically identify abnormal behavior of offline candidates, and there is still room for improvement in the training speed, stability, and parameter adjustment complexity of the cheating behavior identification model, and the efficiency of cheating behavior identification in the examination room is not high.

[0047] Based on this, in various embodiments of the present application, based on the first video and the facial image features of the first examinee in the first examination room, the behavioral feature vector of the first examinee is determined; the behavioral feature vector of the first examinee is input into the first model to obtain first information; wherein, the first video represents a video shot from multiple different positions in the first examination room, and the behavioral feature vector of the first examinee at least represents the number of appearances and the duration of the appearance of the first examinee in the first video; the first model is used to predict whether the behavior of the corresponding examinee is abnormal based on the examinee's behavioral feature vector, and the first information indicates whether the behavior of the first examinee is abnormal or normal. The above scheme, by inputting the behavioral feature vector of the examinee in the examination room into the first model to obtain the first information, realizes the automatic recognition of the abnormal behavior of the examinee in the examination room, improves the efficiency of the recognition of the examinee's abnormal behavior, and the videos shot from different positions can recognize the abnormal behavior of most examinees, thereby further improving the accuracy of the recognition of the examinee's abnormal behavior.

[0048] The present application is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0049] The present application embodiment provides a method for identifying abnormal behavior of examinees, which is applied to electronic devices, and the electronic devices include servers and / or terminals. Figure 1 As shown, the method includes:

[0050] Step 101: Determine a behavior feature vector of a first examinee in a first examination room based on a first video and facial image features of the first examinee.

[0051] The first video represents a video shot at multiple different positions in the first examination room, and the behavior feature vector of the first examinee at least represents the number of times and duration of appearance of the first examinee in the first video.

[0052] Here, before the exam begins, a clear frontal facial image of each first examinee in the first examination room is captured by a photographic device, and the frontal facial image is input into the second model to obtain facial image features of all first examinees in the first examination room; the obtained facial image features of all first examinees in the first examination room can be stored; then, based on the facial image features of the first examinees in the first examination room, face recognition is performed on the first examinees appearing in the first videos at different orientations, and the number of times and duration of the first examinees appearing in the first videos at different orientations are determined, thereby determining the behavioral feature vector of the first examinee. Storing facial image features may be storing the examinee's identification and the corresponding facial image features as a set of data in a database and / or a disk of an electronic device; for example, numbering each first examinee, and at least in the first examination room, the number of each first examinee is unique, and storing the first examinee's number and the corresponding facial image features in the student table of a MySQL database, that is, the student table contains two fields, namely, the first examinee's number and the first examinee's facial image features, and the corresponding field names may be id and embedding; in addition, the first examinee's number and the corresponding facial image features may also be associated with the examination room, for example, the student table may also include the examination room number.

[0053] The second model can be understood as a pre-trained face recognition model or face feature learning model, for example, the FaceNet model; the training set of the FaceNet model can include the MS-Celeb-1M data set. The triplet loss function of the second model can be the first loss function L = max(d(a,p)-d(a,n),0), where d(a,p) represents the distance between the anchor point (also called Anchor) and the positive sample (also called Positive), and d(a,n) represents the distance between the anchor point and the negative sample (Negative). When using the first loss function, there may be a situation where the difference between d(a,p) and d(a,n) is very small, and the actual relationship between the anchor point, positive sample and negative sample is clear enough, but the model will still try to further adjust the distance. Based on this, a hyperparameter can be introduced on the basis of the first loss function to prevent the model from overfitting and reduce unnecessary computational overhead. That is, the second model can also use the second loss function L = max(d(a,p) - d(a,n) + α, 0), where α is a hyperparameter used to set a threshold and boundary to further clearly distinguish whether the distance between the anchor point and the positive sample is close enough, and whether the distance between the anchor point and the negative sample is far enough; between d(a,p) and d(a,n) When the difference between d(a,p) and d(a,n) is less than or equal to α, the second model will try to further adjust the distance to ensure that the convergence condition is met. When the difference between d(a,p) and d(a,n) is greater than α, the second model will not make too many adjustments. The introduction of hyperparameters makes the second model more flexible in processing samples with very close distances, and can accurately control the feature vector distance between different face images, reduce misrecognition, and ensure that the similarity of feature vectors of different face images of the same person is large enough, and the similarity of feature vectors of face images of different people is small enough, so that the determined facial image features of the second candidate are more accurate. At the same time, the performance of the second model for specific tasks can be optimized by adjusting the value of α.

[0054] It should be noted that cameras need to be set up at multiple different positions in the examination room to capture the behavior of the examinees in the examination room. The start time and / or end time of all cameras in an examination room are the same. The cameras can also record the images and timestamps of the examinees, thereby forming continuous video stream data. The cameras can be numbered. For example, camera 1, camera 2, camera 3 and camera 4 are set up in the first examination room, and the timestamps of all cameras are synchronized; the first examination room is any examination room where cameras are set up at multiple different positions; the cameras set up at multiple different positions can clearly capture the faces of the examinees in the examination room when they look at the corresponding positions of the cameras. The multiple different positions include one or more of the four positions of front, back, left and right, and can also include one or more of the cross-shaped positions, which include eight positions of east, south, west, north, southeast, northeast, southwest and northwest, or can also be determined according to the actual structure and size of the examination room. The first video can also be understood as a video segment of a set length, in which one candidate corresponds to one behavioral feature vector; the video segment can be the most recent video segment of a set length taken by the camera, for example, it can be the most recent 1-minute video segment taken by the camera; the video segment can also be captured instantly, for example, the video segment can be captured instantly when the camera is shooting, until the acquired video segment reaches 1 minute in length, and then the next video segment (first video) is acquired.

[0055] In this embodiment, based on videos from different directions and facial image features of the examinee, a behavioral feature vector of the examinee is obtained, which can identify most abnormal behaviors of the examinee, such as the examinee looking back, looking left and right, and lowering the head for a long time.

[0056] Step 102: Input the behavioral feature vector of the first examinee into a first model to obtain first information.

[0057] The first model is used to predict whether the behavior of the corresponding examinee is abnormal based on the examinee's behavior feature vector, and the first information indicates whether the behavior of the first examinee is abnormal or normal.

[0058] Here, the first model can be understood as a candidate abnormal behavior recognition model. By inputting the determined behavior feature vector of the first candidate into the first model, the first information can be obtained quickly and timely; when the first information indicates that the behavior of the first candidate is abnormal, a prompt or notification can also be output to instruct the invigilator to conduct further verification. The first model can be a logistic regression model; the first information can be understood as the candidate abnormal behavior recognition result or prediction result, which can be expressed as the calculated probability of normal and / or abnormal behavior of the candidate. When the probability of normal behavior of the candidate is greater than the third threshold, the candidate's behavior is considered normal, and / or, when the probability of abnormal behavior of the candidate is greater than the fourth threshold, the candidate's behavior is considered abnormal. The third threshold and the fourth threshold are greater than 0.5, and the specific values ​​are set according to the specific situation.

[0059] Before step 101, it is necessary to train a first model. Based on this, in one embodiment, before determining the behavior feature vector of the first examinee based on the first video and the facial image features of the first examinee in the first examination room, the method further includes:

[0060] Determine, based on the second video and facial image features of one or more second examinees in the second examination room, behavioral feature vectors of the one or more second examinees, wherein the second video represents a video shot at multiple different positions in the second examination room, and the behavioral feature vector of the second examinee represents at least the number of appearances and the duration of appearance of the corresponding second examinee in the second video;

[0061] The first model is trained based on the one or more behavioral feature vectors of the second examinees until a set convergence condition is reached.

[0062] Here, the second video can be understood as a video segment of a set duration, and the video segment can be obtained by segmenting the captured video; for example, the video captured by each camera in the examination room is segmented at one-minute time intervals starting from the same starting time. If the duration of the video segment in the last segmented interval is less than one minute, the video segment in the last segmented interval can be removed to obtain one or more video segments (second videos) with a duration of one minute. The one or more second videos are sorted and numbered in ascending order of the starting time. For example, the one or more second video segments obtained are [t1, t2, …, t n ], and then store the one or more second videos, as well as the corresponding numbers, camera numbers, start times and end times according to the start time and end time of the one or more second videos obtained by the timestamp record.

[0063] The method for determining the behavior feature vectors of one or more second examinees based on the second video and the facial image features of one or more second examinees in the second examination room is the same as the method for determining the behavior feature vectors of the first examinee based on the first video and the facial image features of the first examinee in the first examination room. There can be one or more second examination rooms, one second examination room can correspond to multiple videos shot in different directions, one second video corresponds to one or more second examinees, and one second examinee corresponds to one behavior feature vector.

[0064] The behavioral feature vectors of one or more second examinees can be used as a training set for the first model, and the true labels corresponding to the behavioral feature vectors of one or more second examinees can be obtained by annotating the behaviors of the second examinees in the second videos at different locations corresponding to the second examination room; for example, the annotation team can cross-annotate the behaviors of one or more second examinees in combination with the second videos at multiple locations to obtain the true labels corresponding to the behavioral feature vectors of the one or more second examinees, thereby ensuring the accuracy and consistency of the annotation results. The annotation team can include multiple experienced invigilators and machine learning experts. When annotating, examinees with obvious abnormal behaviors can be annotated and / or examinees with obvious no abnormal behaviors can be annotated, while the remaining examinees are not annotated; as shown in Table 1, the annotation results of some examinees in the second videos in multiple second examination rooms.

[0065] Table 1 The true labeling results corresponding to the examinee's behavior feature vector

[0066] Examination room number Start time Second candidate number Annotation results i <![CDATA[t1]]> ac123 0 i <![CDATA[t2]]> ac215 1 j <![CDATA[t3]]> ac618 0 j <![CDATA[t4]]> df234 1 k <![CDATA[t5]]> eg233 0 h <![CDATA[t6]]> eg236 0 w <![CDATA[t7]]> rf256 1

[0067] Among them, the second row indicates that in the second video with the starting time t1 shot from multiple angles in the second examination room i, there is a second candidate ac123 with obvious no abnormal behavior, and the behavior of the second candidate in the second video with the starting time t1 is marked as 0, indicating that the probability of the second candidate's abnormal behavior in the second video with the starting time t1 is 0; the third row indicates that in the second video with the starting time t2 shot from multiple angles in the second examination room i, there is a second candidate ac215 with obvious abnormal behavior, and the behavior of the second candidate in the second video with the starting time t2 is marked as 1, indicating The fourth row indicates that in the second video with the starting time t3 taken from multiple angles in the second examination room j, there is a second candidate ac618 who obviously has no abnormal behavior. The behavior of the second candidate in the second video with the starting time t3 is marked as 0, indicating that the probability of the second candidate's abnormal behavior in the second video with the starting time t3 is 0. The fifth row indicates that in the second video with the starting time t4 taken from multiple angles in the second examination room j, there is a second candidate df234 who obviously has abnormal behavior. The behavior of the second candidate in the second video with the starting time t4 is marked as 1, indicating that the probability of the second candidate's abnormal behavior in the second video with the starting time t4 is 1; the sixth row indicates that in the second video with the starting time t5 taken from multiple angles in the second examination room k, there is a second candidate eg233 who obviously has no abnormal behavior, and the behavior of the second candidate in the second video with the starting time t5 is marked as 0, indicating that the probability of the second candidate's abnormal behavior in the second video with the starting time t5 is 0; the seventh row indicates that in the second video with the starting time t6 taken from multiple angles in the second examination room h, there is a second candidate eg233 who obviously has no abnormal behavior. In the second video, there is a second candidate eg236 who obviously has no abnormal behavior. The behavior of the second candidate in the second video with the starting time t6 is marked as 0, indicating that the probability of the second candidate's abnormal behavior in the second video with the starting time t6 is 0; the eighth row indicates that in the second video with the starting time t7 shot from multiple angles in the second examination room w, there is a second candidate eg256 who obviously has abnormal behavior. The behavior of the second candidate in the second video with the starting time t7 is marked as 1, indicating that the probability of the second candidate's abnormal behavior in the second video with the starting time t7 is 1.

[0068] The behavioral feature vectors of one or more second examinees in the second video and the corresponding true labels are used to train the first model, including: inputting the behavioral feature vectors of one or more second examinees in the second video into the first model to obtain abnormal behavior prediction results of one or more second examinees in the second video, and obtaining a loss value based on the abnormal behavior prediction results of one or more second examinees in the second video and the corresponding true labels, as well as the loss function of the first model; training the first model according to the loss value until a set convergence condition is reached. The set convergence condition may include: the loss value is less than a set threshold, and / or the training reaches a maximum number of iterations or reaches a set round. When the set convergence condition is reached, it indicates that the first model has ended training or the training is completed.

[0069] In order to identify most abnormal behaviors of examinees and improve the accuracy of identifying abnormal behaviors of examinees, in one embodiment, the multiple different directions in the examination room include at least four directions of front, back, left and right in the examination room; the examinee's behavior feature vector represents the number of appearances and duration of the examinee's behavior in the video taken at each direction in the examination room, as well as the number and / or duration of the examinee's behavior not appearing in the video taken at all directions in the examination room.

[0070] Here, the multiple different directions in the examination room include at least the front, back, left, and right directions in the examination room, and may also include the front left, back left, front right, and back right directions in the examination room. The specific directions may be determined according to the structure and size of the actual examination room. For example, a camera is installed at the front, back, left, and right of the examination room, respectively, wherein the front camera can clearly capture the face of the examinee in the normal answering process in the examination room, the back camera can clearly capture the face of the examinee when the examinee looks back, the left camera can clearly capture the face of the examinee when the examinee looks left, and the right camera can clearly capture the face of the examinee when the examinee looks right.

[0071] The examinee's behavioral feature vector corresponds to the video shot at each position in the examination room; for example, when the multiple different positions in the examination room include the front, back, left and right positions of the examination room, the examinee's behavioral feature vector may include the front duration, the back duration, the left duration, the right duration, the front number of times, the back number of times, the left number of times, the right number of times, as well as the duration and number of times that do not appear in the video segments of the front, back, left and right positions.

[0072] In order to improve the efficiency and accuracy of identifying abnormal behaviors of examinees, in one embodiment, determining the behavioral feature vector of the examinee includes:

[0073] Extract frames from the video at set time intervals to obtain multiple pictures;

[0074] Based on the facial image features of the examinee and the multiple pictures, determine all first serial numbers and / or all second serial numbers corresponding to the examinee, wherein the first serial number represents the serial number of the picture containing the examinee's facial image among the multiple pictures, and the second serial number represents the serial number of the picture not containing the examinee's facial image among the multiple pictures;

[0075] Based on all first serial numbers and / or all second serial numbers corresponding to the examinee, determine the number of times and duration of appearance of the examinee in the plurality of pictures, and / or the number of times and duration of absence of the examinee in the plurality of pictures;

[0076] Determine the examinee's behavioral feature vector based on one or more of the following:

[0077] The number of times and duration of the candidate's appearance in the multiple images;

[0078] The number of times and duration that the candidate is absent from the multiple images.

[0079] Here, after the video is subjected to frame extraction processing at a set time interval, the extracted pictures can be numbered according to the timestamp of the pictures or the time corresponding to the pictures in the video; the video includes the first video and / or the second video; the set time interval can be 1 second or 2 seconds, which is not specifically limited here; the frame extraction processing can be performed through a video processing library or software, for example, using the computer vision processing open source software library (OpenCV, Open Source Computer Vision Library) to read the video captured by the camera in real time, and perform frame extraction processing on the read video, so that one frame can be extracted per second, one frame corresponds to one picture, and the video obtained by shooting in one direction can correspond to one OpenCV library. For example, the first video or the second video is a video segment with a duration of 1 minute. The first video or the second video is subjected to frame extraction processing at a set time interval of 1 second, and 60 pictures can be obtained, and the timestamps corresponding to the 60 pictures can also be obtained; the obtained multiple pictures can be sorted from front to back according to the timestamps to obtain a sequence of multiple pictures [pic1, pic2,…, pic m ].

[0080] When multiple images are obtained, the face image in each image is detected and extracted. For example, OpenCV is used to read multiple images, and the pre-trained Haar cascade classifier provided by OpenCV is used to detect and extract the faces in the multiple images; the face image extracted from each image is input into the second model to obtain the feature vector corresponding to the face image; the feature vector corresponding to the face image is matched with the facial image features of the examinee in the corresponding examination room; the cosine similarity can be used to determine whether it matches. When the cosine similarity is greater than or equal to the set threshold, the examinee is considered to be the examinee corresponding to the face image. When there are multiple examinees whose corresponding cosine similarities are greater than or equal to the set threshold, the examinee with the largest cosine similarity is considered to be the examinee. The examinee corresponding to the degree is the examinee corresponding to the face picture. The threshold value can be determined according to the actual situation. For example, the threshold value is set to 0.8. According to the matching of the face pictures in multiple pictures and the facial image features of the examinee, all the first serial numbers and / or all the second serial numbers corresponding to the examinee are determined. Specifically, when the face picture matches the facial image features of the examinee, it means that the examinee appears in the picture corresponding to the face picture. The first serial number of the examinee includes the serial number of the picture corresponding to the face picture. When there is no face picture matching the facial image features of a examinee in a picture, it means that the examinee does not appear in the picture. It may be that the examinee lowers his head or covers his face with his hands. The second serial number of the examinee includes the serial number of the picture.

[0081] For example, any face picture extracted from any one of the multiple pictures in the second video is input into the second model to obtain a feature vector corresponding to the face picture, recorded as emb1; the feature vector corresponding to the face picture is matched with the facial image features of all second candidates in the examination room corresponding to the stored second video; for example, the cosine similarity between emb1 and the facial image features of each second candidate in the embedding1 field of the stored student1 table is calculated; when the cosine similarity between the facial image features of a second candidate in emb1 and the student1 table is greater than 0.8, the first serial number of the second candidate includes the number of the picture corresponding to the face picture; when the cosine similarity between the facial image features of multiple second candidates in emb1 and the student1 table is greater than 0.8, the first serial number of the second candidate corresponding to the maximum cosine similarity includes the number of the picture corresponding to the face picture; when all the first serial numbers of the second candidate are determined, all the first serial numbers of the second candidate are removed from the serial numbers of multiple pictures in the second video to obtain all the second serial numbers of the second candidate.

[0082] For another example, all face images extracted from any one of the multiple images in the second video are input into the second model to obtain the feature vector corresponding to all the face images in the image; the facial image features of a second candidate in the examination room corresponding to the stored second video are matched with the feature vector corresponding to all the face images in the image; for example, the cosine similarities of the feature vectors corresponding to the facial image features of the second candidate and all the face images in the image are calculated respectively; when the cosine similarities of the feature vectors corresponding to the facial image features of the second candidate and all the face images in the image are less than 0.8, the second serial number of the second candidate includes the serial number of the image; when all the second serial numbers of the second candidate are determined, all the second serial numbers of the second candidate are removed from the serial numbers of the multiple images in the second video to obtain all the first serial numbers of the second candidate.

[0083] Then, the duration of the candidate's appearance in multiple pictures and / or the duration of his / her absence is determined by the number of all first serial numbers and / or second serial numbers corresponding to the candidate; the duration of the candidate's appearance in multiple pictures and / or the duration of his / her absence can be understood as the duration of the candidate's appearance in the video and / or the duration of his / her absence; for example, when the set time interval is one second, the candidate appears once in a picture, which is regarded as the candidate's appearance in the video for one second, that is, the number of seconds the candidate appears in the video is the number of all first serial numbers corresponding to the candidate, and the duration of the candidate's absence in the video can be determined based on the duration of the candidate's appearance in the video; for another example, when the set time interval is one second, the number of seconds the candidate does not appear in the video is the number of all second serial numbers corresponding to the candidate, and the duration of the candidate's appearance in the video can be determined based on the duration of the candidate's absence in the video. Similarly, the duration of the candidate's appearance and / or absence in the videos of various directions can be determined.

[0084] The number of times the examinee appears and / or does not appear in multiple pictures is determined by the number of interruptions of all first serial numbers and / or the number of interruptions of all second serial numbers corresponding to the examinee; the number of times the examinee appears and / or does not appear in multiple pictures can be understood as the number of times the examinee appears and / or does not appear in the video; specifically, the number of interruptions of the first serial number and / or the second serial number can be determined in order from front to back, the number of times the examinee appears in the video is the number of interruptions of the corresponding first serial number plus 1, or the number of interruptions of the second serial number; the number of times the examinee does not appear in the video is the number of interruptions of the corresponding first serial number, or the number of interruptions of the second serial number minus 1. For example, all first serial numbers corresponding to the examinee include all serial numbers except serial number 6 of multiple pictures, that is, the examinee appears in the 5th picture of the video, does not appear in the 6th picture, and appears in the 7th picture, then the number of times the examinee appears in the video is 2. Similarly, the number of times the examinee appears and / or does not appear in the videos of all directions can be determined.

[0085] Based on the same method of determining the duration of the examinee's appearance and / or non-appearance in the video, and the number of times the examinee appears and / or does not appear in the video, the duration and number of times the examinee appears in the videos of each orientation, and the duration and number of times the examinee does not appear in the videos of all orientations can be determined, that is, the examinee's behavior feature vector is determined. As shown in Table 2, in the video segment with the start time t1 shot by the four cameras at the front, back, left, and right in the examination room i, the duration of each examinee's appearance in the video segment shot by the cameras at each orientation, and the duration of their non-appearance in the video segments at all orientations.

[0086] Table 2 Behavior feature vectors related to duration of each examinee in the video segment starting at time t1

[0087]

[0088] Among them, the front duration refers to the time the candidate appears in the video shot in front of the examination room; the back duration refers to the time the candidate appears in the video shot from the back of the examination room; the left duration refers to the time the candidate appears in the video shot from the left of the examination room; the right duration refers to the time the candidate appears in the video shot from the right of the examination room; the non-appearance duration refers to the time the candidate does not appear in the videos shot from the front, back, left and right of the examination room.

[0089] As shown in Table 3, in the video segment with the starting time t1 shot by the four cameras at the front, back, left and right in the examination room i, the number of times each candidate appears in the video segments shot by cameras in each direction, and the number of times he or she does not appear in the video segments in all directions.

[0090] Table 3 Behavior feature vectors related to the number of times each examinee appears in the video segment starting at time t1

[0091]

[0092]

[0093] Among them, the front number indicates the number of times the candidate appears in the video shot in front of the candidate in the examination room; the back number indicates the number of times the candidate appears in the video shot behind the candidate in the examination room; the left number indicates the number of times the candidate appears in the video shot on the left side of the examination room; the right number indicates the number of times the candidate appears in the video shot on the right side of the examination room; the non-appearance number indicates the number of times the candidate does not appear in the videos shot from the front, back, left and right of the examination room.

[0094] It should be noted that, during the model training process, the above Tables 1, 2 and 3 can be associated according to the three fields of examination room number, start time and candidate number to obtain a general table, which includes at least 14 fields: examination room number, start time, candidate number, front duration, back duration, left duration, right duration, non-appearance duration, front number of times, back number of times, left number of times, right number of times, non-appearance number of times and marking results; the general table can be understood as the training set of the first model, and any row of data in the general table except the examination room number, start time and candidate number can be used as a training data for the first model.

[0095] In order to obtain a model for identifying abnormal behavior of examinees, realize automatic identification of abnormal behavior of examinees, and improve the efficiency of identifying cheating behavior in the examination room, in one embodiment, training the first model based on the behavior feature vectors of the one or more second examinees includes:

[0096] Inputting the behavior feature vectors of the one or more second examinees into the first model to obtain second information of the one or more second examinees, the second information indicating whether the predicted behavior of the second examinees is abnormal or normal;

[0097] A first gradient is calculated by using a back propagation algorithm, the second information, and the third information; the first gradient is a gradient of a loss function with respect to a parameter of the first model, and the third information indicates whether the behavior of the marked second examinee is abnormal or normal;

[0098] Based on a first parameter, a first learning rate and the first gradient, update the parameters of the first model; the first parameter is an initial parameter of the first model or a current model parameter; the first learning rate is a learning rate used in the N+1th round of training, and is determined based on at least a second learning rate used in the Nth round of training, where N is a positive integer and is less than or equal to a set total number of training rounds.

[0099] Here, in the training phase of the first model, the first model is trained based on the training set; for example, the back propagation algorithm is used to adjust the parameters in the first model. Specifically, the behavioral feature vectors of one or more second examinees in the training set are input into the first model to obtain the second information of one or more second examinees; for example, a training sample in the training set can be expressed as (x i ,y i ), where x i represents the behavioral feature vector of the second examinee, y i represents the third information of the second examinee; the third information can be understood as the real label of the second examinee, that is, the real label corresponding to the behavioral feature vector of the second examinee, which can be obtained by the annotation team by cross-annotating the behaviors of one or more second examinees in multiple second videos of different orientations, as shown in Table 1. The second information can be understood as the predicted value or predicted result, which can be expressed as h θ (x i ), where θ represents the parameters in the first model, and h represents the calculation function in the first model; then, the first gradient is calculated through the back propagation algorithm, the second information and the third information, that is, the partial derivative of the loss function with respect to the model parameters is calculated, and the second information and the third information are substituted into the partial derivative to obtain the gradient value (first gradient); then, based on the first parameter, the first learning rate and the first gradient, the parameters of the first model are updated, that is, based on the first parameter, the first learning rate, the first gradient, and the value of the current parameter, the updated parameter value is calculated. The loss function may be a cross entropy loss function. The current model parameters refer to the parameters of the first model after the last round of parameter update, that is, the parameters of the first model after the Nth round of training.

[0100] In order to dynamically adjust the learning rate, optimize the training process of the model, and improve the performance and stability of the model, in one embodiment, updating the parameters of the first model based on the first parameter, the first learning rate, and the first gradient includes:

[0101] Determine the first learning rate based on the second learning rate, the set learning rate adjustment factor and the first gradient, and based on the set learning rate decay rate and the value of N;

[0102] Based on the difference between the first parameter and a first value, the parameters of the first model are updated, where the first value represents the product of the first learning rate and the first gradient.

[0103] Here, based on the difference between the first parameter and the first value, the parameters of the first model are updated; for example, based on the formula Update the parameters of the first model, where θ N+1represents the new parameters calculated in this round of parameter update, N+1 represents the round of parameter update in this round, N represents the round of parameter update in the previous round, θ N represents the parameters used in the current first model or the parameters of the first model after the last round of parameter update, α N+1 Indicates the learning rate used in this round of parameter update, that is, the first learning rate, Represents the first gradient. Before starting to train the first model, the first parameter, the second learning rate, the set learning rate adjustment factor, and the learning rate decay rate are initialized. The first learning rate is calculated based on the second learning rate, the set learning rate adjustment factor and the first gradient, as well as the set learning rate decay rate and the value of N, that is, the first learning rate can be dynamically adjusted based on the learning rate used in the previous round of training to better adapt to changes in data distribution and model complexity. Such a dynamic adjustment method can be understood as an adaptive learning rate. Since the first model is a logistic regression model, the training method using the adaptive learning rate can also be called an adaptive learning rate logistic regression algorithm (ALR-LR, Adaptive Learning Rate-Logistic Regression).

[0104] In this embodiment, the learning rate can be automatically adjusted according to the needs of different rounds, so that the loss function can converge within a smaller number of iterations, thereby improving the training speed. In addition, the logistic regression algorithm may experience oscillations or fail to converge during training due to improper setting of the learning rate under a fixed learning rate. By adjusting the parameters with an adaptive learning rate, the problem of oscillations or failure to converge can be better dealt with, thereby improving the stability of the training; and compared with the logistic regression algorithm with a fixed learning rate, the adaptive learning rate logistic regression algorithm requires fewer parameters to be adjusted, making the parameter adjustment process simpler.

[0105] In order to accurately calculate the adaptive learning rate, make the model converge faster, have better stability, and simplify the parameter adjustment process, in one embodiment, the first learning rate is determined based on the second learning rate, the set learning rate adjustment factor and the first gradient, and the set learning rate decay rate and the value of N, including:

[0106] Determine a second value based on the set learning rate adjustment factor, the set hyperparameter and the first gradient;

[0107] Determine a third value based on the set learning rate decay rate and the value of N;

[0108] The first learning rate is determined based on the second learning rate, the second value, and the third value.

[0109] Here, the second value is determined based on the set learning rate adjustment factor, the set hyperparameter and the first gradient; for example, it can be based on the formula Determine a second value, where β represents a set learning rate adjustment factor, which can be set according to actual conditions, and k represents a set hyperparameter used to control the sensitivity of the gradient to the learning rate adjustment. represents the first gradient, and e represents a constant, which can be the base of the natural logarithm function, the Euler number, or the Napier constant.

[0110] Based on the set learning rate decay rate and the value of N, a third value is determined, where the third value represents the decay rate of the learning rate during the parameter update of this round (N+1 round); for example, the third value can be based on the formula γ N or γN Determine a third value, where γ represents the set learning rate decay rate, which can be set according to actual conditions, and N represents the number of rounds of parameter updates in the previous round. The learning rate decay rate can further improve the convergence speed, enhance the model stability, and prevent the model from overfitting the training set.

[0111] Based on the second learning rate, the second value and the third value, the first learning rate is determined, so that the learning rate can be dynamically adjusted according to the training situation of the model to improve the model performance and stability; for example, based on the product of the three, that is, the formula Determine the first learning rate, where α N+1 Characterize the first learning rate, α N Characterize the second learning rate, Characterize the second value, γ N Characterizes the third value.

[0112] In one embodiment, the first learning rate is negatively correlated with the absolute value of the first gradient.

[0113] Here, the first learning rate is negatively correlated with the absolute value of the first gradient, that is, when the absolute value of the first gradient is large, the first learning rate is small to avoid excessive update step size; when the absolute value of the first gradient is small, the learning rate is large to accelerate convergence; that is, the first learning rate can be dynamically adjusted by the absolute value of the first gradient; for example, the calculation formula of the first learning rate can be expressed as

[0114] In order to further improve the convergence speed, enhance the model stability, and prevent the model from overfitting the training set, in one embodiment, the third value is calculated based on the set learning rate decay rate as the base and the value of N as the exponent; and / or

[0115] The first learning rate is the product of the second learning rate, the second value, and the third value.

[0116] Here, the third value is calculated based on the set learning rate decay rate and the value of N as the exponent, that is, the calculation formula of the third value is γ N The first learning rate is the product of the second learning rate, the second value and the third value. That is, the calculation formula of the first learning rate is Using the first learning rate to update the parameters reduces the number of parameters that need to be adjusted, making the training process of the first model more stable while increasing the training speed.

[0117] In order to implement the method of the embodiment of the present application, the embodiment of the present application also provides a device for identifying abnormal behavior of examinees, such as Figure 2 As shown, the device comprises:

[0118] A first determining unit 201 is configured to determine a behavior feature vector of a first examinee based on a first video and a facial image feature of the first examinee in a first examination room, wherein the first video represents a video shot at multiple different positions in the first examination room, and the behavior feature vector of the first examinee represents at least the number of times and duration of appearance of the first examinee in the first video;

[0119] The prediction unit 202 is used to input the behavioral feature vector of the first examinee into a first model to obtain first information; wherein the first model is used to predict whether the behavior of the corresponding examinee is abnormal based on the examinee's behavioral feature vector, and the first information indicates whether the behavior of the first examinee is abnormal or normal.

[0120] In one embodiment, the device further comprises:

[0121] A second determining unit is used to determine a behavior feature vector of the one or more second examinees based on the second video and facial image features of the one or more second examinees in the second examination room, wherein the second video represents a video shot at multiple different positions in the second examination room, and the behavior feature vector of the second examinee represents at least the number of appearances and the duration of appearance of the corresponding second examinee in the second video;

[0122] A training unit is used to train the first model based on the behavioral feature vectors of the one or more second examinees until a set convergence condition is reached.

[0123] In one embodiment, the multiple different directions in the examination room include at least four directions of front, back, left and right in the examination room; the examinee's behavioral feature vector represents the number of times and duration of the examinee's appearance in the video taken at each direction in the examination room, as well as the number and / or duration of the examinee's non-appearance in the video taken at all directions in the examination room.

[0124] In one embodiment, the first determination unit 201 and the second determination unit are specifically configured to perform frame extraction processing on the video at a set time interval to obtain multiple pictures;

[0125] Based on the facial image features of the examinee and the multiple pictures, determine all first serial numbers and / or all second serial numbers corresponding to the examinee, wherein the first serial number represents the serial number of the picture containing the examinee's facial image among the multiple pictures, and the second serial number represents the serial number of the picture not containing the examinee's facial image among the multiple pictures;

[0126] Based on all first serial numbers and / or all second serial numbers corresponding to the examinee, determine the number of times and duration of appearance of the examinee in the plurality of pictures, and / or the number of times and duration of absence of the examinee in the plurality of pictures;

[0127] Determine the examinee's behavioral feature vector based on one or more of the following:

[0128] The number of times and duration of the candidate's appearance in the multiple images;

[0129] The number of times and duration that the candidate is absent from the multiple images.

[0130] In one embodiment, the training unit is specifically configured to input the behavior feature vectors of the one or more second examinees into the first model to obtain second information of the one or more second examinees, the second information indicating whether the predicted behavior of the second examinees is abnormal or normal;

[0131] A first gradient is calculated by using a back propagation algorithm, the second information, and the third information; the first gradient is a gradient of a loss function with respect to a parameter of the first model, and the third information indicates whether the behavior of the marked second examinee is abnormal or normal;

[0132] Based on a first parameter, a first learning rate and the first gradient, update the parameters of the first model; the first parameter is an initial parameter of the first model or a current model parameter; the first learning rate is a learning rate used in the N+1th round of training, and is determined based on at least a second learning rate used in the Nth round of training, where N is a positive integer and is less than or equal to a set total number of training rounds.

[0133] In one embodiment, the training unit is specifically used to determine the first learning rate based on the second learning rate, the set learning rate adjustment factor and the first gradient, and based on the set learning rate decay rate and the value of N;

[0134] Based on the difference between the first parameter and a first value, the parameters of the first model are updated, where the first value represents the product of the first learning rate and the first gradient.

[0135] In one embodiment, the training unit is specifically configured to determine a second value based on the set learning rate adjustment factor, the set hyperparameter, and the first gradient;

[0136] Determine a third value based on the set learning rate decay rate and the value of N;

[0137] The first learning rate is determined based on the second learning rate, the second value, and the third value.

[0138] In one embodiment, the first learning rate is negatively correlated with the absolute value of the first gradient.

[0139] In one embodiment, the third value is calculated based on the set learning rate decay rate as the base and the value of N as the exponent; and / or

[0140] The first learning rate is the product of the second learning rate, the second value, and the third value.

[0141] In practical applications, the first determination unit 201, the prediction unit 202, the second determination unit and the training unit can be implemented by a processor in the device for identifying abnormal behavior of examinees.

[0142] It should be noted that: the above embodiment provides an abnormal behavior identification device for examinees, and only uses the division of the above program modules as an example to illustrate when performing abnormal behavior identification of examinees. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device is divided into different program modules to complete all or part of the processing described above. In addition, the abnormal behavior identification device for examinees provided in the above embodiment and the abnormal behavior identification method embodiment of examinees belong to the same concept. The specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0143] Based on the hardware implementation of the above program modules, the embodiment of the present application also provides an electronic device, such as Figure 3 As shown, the electronic device 300 includes:

[0144] The communication interface 301 can exchange information with other devices or network nodes.

[0145] The processor 302 is connected to the communication interface 301 to implement information interaction with other devices or network nodes, and is used to execute the methods provided by one or more technical solutions of the above embodiments when running a computer program.

[0146] The memory 303 is used to store computer programs that can be executed on the processor 302 .

[0147] Specifically, the processor 302 is configured to determine a behavior feature vector of the first examinee based on a first video and a facial image feature of the first examinee in the first examination room, wherein the first video represents a video captured at multiple positions in the first examination room, and the behavior feature vector of the first examinee represents at least the number of times and duration of appearance of the first examinee in the first video;

[0148] The behavior feature vector of the first examinee is input into a first model to obtain first information; wherein the first model is used to predict whether the behavior of the corresponding examinee is abnormal based on the examinee's behavior feature vector, and the first information indicates whether the behavior of the first examinee is abnormal or normal.

[0149] In one embodiment, the processor 302 is specifically configured to determine the behavior feature vectors of the one or more second examinees based on the second video and the facial image features of the one or more second examinees in the second examination room, wherein the second video represents a video captured at multiple different positions in the second examination room, and the behavior feature vector of the second examinee represents at least the number of appearances and the duration of appearance of the corresponding second examinee in the second video;

[0150] The first model is trained based on the one or more behavioral feature vectors of the second examinees until a set convergence condition is reached.

[0151] In one embodiment, the multiple different directions in the examination room include at least four directions of front, back, left and right in the examination room; the examinee's behavioral feature vector represents the number of times and duration of the examinee's appearance in the video taken at each direction in the examination room, as well as the number and / or duration of the examinee's non-appearance in the video taken at all directions in the examination room.

[0152] In one embodiment, the processor 302 is specifically configured to extract frames from the video at a set time interval to obtain a plurality of images;

[0153] Based on the facial image features of the examinee and the multiple pictures, determine all first serial numbers and / or all second serial numbers corresponding to the examinee, wherein the first serial number represents the serial number of the picture containing the examinee's facial image among the multiple pictures, and the second serial number represents the serial number of the picture not containing the examinee's facial image among the multiple pictures;

[0154] Based on all first serial numbers and / or all second serial numbers corresponding to the examinee, determine the number of times and duration of appearance of the examinee in the plurality of pictures, and / or the number of times and duration of absence of the examinee in the plurality of pictures;

[0155] Determine the examinee's behavioral feature vector based on one or more of the following:

[0156] The number of times and duration of the candidate's appearance in the multiple images;

[0157] The number of times and duration that the candidate is absent from the multiple images.

[0158] In one embodiment, the processor 302 is specifically configured to input the behavior feature vectors of the one or more second examinees into the first model to obtain second information of the one or more second examinees, wherein the second information indicates whether the predicted behavior of the second examinees is abnormal or normal;

[0159] A first gradient is calculated by using a back propagation algorithm, the second information, and the third information; the first gradient is a gradient of a loss function with respect to a parameter of the first model, and the third information indicates whether the behavior of the marked second examinee is abnormal or normal;

[0160] Based on a first parameter, a first learning rate and the first gradient, update the parameters of the first model; the first parameter is an initial parameter of the first model or a current model parameter; the first learning rate is a learning rate used in the N+1th round of training, and is determined based on at least a second learning rate used in the Nth round of training, where N is a positive integer and is less than or equal to a set total number of training rounds.

[0161] In one embodiment, the processor 302 is specifically configured to determine the first learning rate based on the second learning rate, the set learning rate adjustment factor and the first gradient, and based on the set learning rate decay rate and the value of N;

[0162] Based on the difference between the first parameter and a first value, the parameters of the first model are updated, where the first value represents the product of the first learning rate and the first gradient.

[0163] In one embodiment, the processor 302 is specifically configured to determine a second value based on the set learning rate adjustment factor, the set hyperparameter and the first gradient;

[0164] Determine a third value based on the set learning rate decay rate and the value of N;

[0165] The first learning rate is determined based on the second learning rate, the second value, and the third value.

[0166] In one embodiment, the first learning rate is negatively correlated with the absolute value of the first gradient.

[0167] In one embodiment, the third value is calculated based on the set learning rate decay rate as the base and the value of N as the exponent; and / or the first learning rate is the product of the second learning rate, the second value and the third value.

[0168] It should be noted that the specific processing process of the processor 302 can be understood by referring to the above method.

[0169] Of course, in actual application, the various components in the electronic device 300 are coupled together through the bus system 304. It can be understood that the bus system 304 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 304 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Figure 3 Various buses are labeled as bus system 304 .

[0170] The memory 303 in the embodiment of the present application is used to store various types of data to support the operation of the electronic device 300. Examples of such data include: any computer program used to operate on the electronic device 300.

[0171] The method disclosed in the above embodiment of the present application can be applied to the processor 302, or implemented by the processor 302. The processor 302 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 302. The above-mentioned processor 302 may be a general-purpose processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 302 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor may be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the present application, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in the memory 303, and the processor 302 reads the information in the memory 303 and completes the steps of the above method in combination with its hardware.

[0172] In an exemplary embodiment, the electronic device 300 can be implemented by one or more application specific integrated circuits (ASIC), DSP, programmable logic device (PLD), field programmable gate array (FPGA), general processor, controller, microcontroller (MCU), microprocessor, or other electronic components to execute the aforementioned method.

[0173] It can be understood that the memory (memory 303) of the embodiment of the present application can be a volatile memory or a non-volatile memory, and can also include both a volatile memory and a non-volatile memory. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), and direct RAM bus random access memory (DRRAM, Direct Rambus Random Access Memory).The memory described in the embodiments of the present application is intended to include, but is not limited to, these and any other suitable types of memory.

[0174] In an exemplary embodiment, the present application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, for example, a memory 303 storing a computer program, and the computer program can be executed by a processor 302 of an electronic device 300 to complete the steps of any of the aforementioned methods. The computer-readable storage medium can be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface storage, optical disk, or CD-ROM.

[0175] Illustratively, an embodiment of the present application further provides a computer program product, including a computer program, and the computer program can be executed by the processor 302 of the electronic device 300 to complete the steps of any of the aforementioned methods.

[0176] It should be noted that "first", "second", etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. The term "and / or" herein is merely a description of an association relationship of associated objects, indicating that three relationships may exist, for example, A and or B, which may represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the technical solutions described in the embodiments of the present application may be combined arbitrarily without conflict. The above is only a preferred embodiment of the present application and is not intended to limit the scope of protection of the present application.

Claims

1. A method for identifying abnormal behavior of examinees, characterized in that: The method comprises: determining a behavior feature vector of the first examinee based on a first video and facial image features of the first examinee in the first examination room, wherein the first video represents a video shot at multiple different positions in the first examination room, and the behavior feature vector of the first examinee represents at least the number of times and duration of appearance of the first examinee in the first video; The behavior feature vector of the first examinee is input into a first model to obtain first information; wherein the first model is used to predict whether the behavior of the corresponding examinee is abnormal based on the examinee's behavior feature vector, and the first information indicates whether the behavior of the first examinee is abnormal or normal.

2. The method for identifying abnormal behavior of examinees according to claim 1, characterized in that: Before determining the behavior feature vector of the first examinee based on the first video and the facial image features of the first examinee in the first examination room, the method further includes: Determine, based on the second video and facial image features of one or more second examinees in the second examination room, behavioral feature vectors of the one or more second examinees, wherein the second video represents a video shot at multiple different positions in the second examination room, and the behavioral feature vector of the second examinee represents at least the number of appearances and the duration of appearance of the corresponding second examinee in the second video; The first model is trained based on the one or more behavioral feature vectors of the second examinees until a set convergence condition is reached.

3. The method for identifying abnormal behavior of examinees according to claim 1 or 2, characterized in that: The multiple different directions in the examination room include at least four directions of front, back, left and right in the examination room; the examinee's behavioral feature vector represents the number of times and duration of the examinee's appearance in the video taken from each direction in the examination room, as well as the number of times and / or duration of the examinee's non-appearance in the video taken from all directions in the examination room.

4. The method for identifying abnormal behavior of examinees according to claim 1 or 2, characterized in that: Determine the examinee's behavioral feature vector, including: Extract frames from the video at set time intervals to obtain multiple pictures; Based on the facial image features of the examinee and the multiple pictures, determine all first serial numbers and / or all second serial numbers corresponding to the examinee, wherein the first serial number represents the serial number of the picture containing the examinee's facial image among the multiple pictures, and the second serial number represents the serial number of the picture not containing the examinee's facial image among the multiple pictures; Based on all first serial numbers and / or all second serial numbers corresponding to the examinee, determine the number of times and duration of appearance of the examinee in the plurality of pictures, and / or the number of times and duration of absence of the examinee in the plurality of pictures; Determine the examinee's behavioral feature vector based on one or more of the following: The number of times and duration of the candidate's appearance in the multiple images; The number of times and duration that the candidate is absent from the multiple images.

5. The method for identifying abnormal behavior of examinees according to claim 2, characterized in that: The training of the first model based on the behavioral feature vectors of the one or more second examinees includes: Inputting the behavior feature vectors of the one or more second examinees into the first model to obtain second information of the one or more second examinees, the second information indicating whether the predicted behavior of the second examinees is abnormal or normal; A first gradient is calculated by using a back propagation algorithm, the second information, and the third information; the first gradient is a gradient of a loss function with respect to a parameter of the first model, and the third information indicates whether the behavior of the marked second examinee is abnormal or normal; Based on a first parameter, a first learning rate and the first gradient, update the parameters of the first model; the first parameter is an initial parameter of the first model or a current model parameter; the first learning rate is a learning rate used in the N+1th round of training, and is determined based on at least a second learning rate used in the Nth round of training, where N is a positive integer and is less than or equal to a set total number of training rounds.

6. The method for identifying abnormal behavior of examinees according to claim 5, characterized in that: The updating of the parameters of the first model based on the first parameter, the first learning rate and the first gradient includes: Determine the first learning rate based on the second learning rate, the set learning rate adjustment factor and the first gradient, and based on the set learning rate decay rate and the value of N; Based on the difference between the first parameter and a first value, the parameters of the first model are updated, where the first value represents the product of the first learning rate and the first gradient.

7. The method for identifying abnormal behavior of examinees according to claim 6, characterized in that: The determining the first learning rate based on the second learning rate, the set learning rate adjustment factor and the first gradient, and based on the set learning rate decay rate and the value of N, includes: Determine a second value based on the set learning rate adjustment factor, the set hyperparameter and the first gradient; Determine a third value based on the set learning rate decay rate and the value of N; The first learning rate is determined based on the second learning rate, the second value, and the third value.

8. The method for identifying abnormal behavior of examinees according to any one of claims 5 to 7, characterized in that: The first learning rate is negatively correlated with an absolute value of the first gradient.

9. The method for identifying abnormal behavior of examinees according to claim 7, characterized in that: The third value is calculated by taking the set learning rate decay rate as the base and the value of N as the exponent; and / or The first learning rate is the product of the second learning rate, the second value, and the third value.

10. An electronic device, characterized in that: include: Processor and communication interface; wherein, The processor is configured to determine a behavior feature vector of the first examinee based on a first video and a facial image feature of the first examinee in a first examination room, wherein the first video represents a video captured at multiple positions in the first examination room, and the behavior feature vector of the first examinee represents at least the number of times and duration of appearance of the first examinee in the first video; The behavior feature vector of the first examinee is input into a first model to obtain first information; wherein the first model is used to predict whether the behavior of the corresponding examinee is abnormal based on the examinee's behavior feature vector, and the first information indicates whether the behavior of the first examinee is abnormal or normal.

11. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

12. A computer program product, comprising a computer program, characterized in that The computer program implements the steps of the method according to any one of claims 1 to 9 when executed by a processor.

Images