Abnormal behavior detection method and device, equipment, storage medium and program product

Abstract

The embodiment of the invention discloses an abnormal behavior detection method and device, equipment, a storage medium and a program product, and the method comprises the steps: obtaining a pre-trained inspection scene abnormal behavior detection model which is obtained based on balanced processing data training, the equalization processing data is obtained by adjusting the input vector corresponding to each image according to the proportion of the number of each category detection frame of each image in pre-acquired training data to the number of all category detection frames in each image; and inputting the to-be-detected image into a pre-trained inspection scene abnormal behavior detection model to obtain a detection result of the to-be-detected image. The inspection scene abnormal behavior detection model is trained through the balanced processing data, and the detection precision of the trained inspection scene abnormal behavior detection model can be improved.

Description

Technical Field

[0001] This application belongs to the field of Artificial Intelligence (AI), and specifically relates to an abnormal behavior detection method, device, equipment, storage medium, and program product. Background Technology

[0002] In related technologies, abnormal behavior detection in examination monitoring scenarios involves analyzing various abnormal behaviors that invigilators and examinees may exhibit during the examination, such as standing or raising their hands. The industry typically uses object detection algorithms to detect objects in single-frame images and determine the abnormal behavior associated with the detection box. However, the actual probability of abnormal behaviors occurring in examination monitoring scenarios is extremely low. Therefore, when constructing examination monitoring data, it is necessary to specify a large number of people in the same scenario to simulate abnormal behaviors in order to increase the richness of abnormal behaviors in the data. This can result in too many abnormal behaviors in a single image, leading to a severe imbalance between the categories of abnormal and normal behaviors during model training. This, in turn, reduces the detection accuracy of the abnormal behavior detection model in examination monitoring scenarios. Summary of the Invention

[0003] To address the issue of low detection accuracy in abnormal behavior detection models for examination scenarios in related technologies, this application proposes an abnormal behavior detection method, apparatus, device, storage medium, and program product.

[0004] This application provides an abnormal behavior detection method, the method comprising:

[0005] A pre-trained abnormal behavior detection model for invigilation scenarios is obtained, wherein the abnormal behavior detection model for invigilation scenarios is trained based on balanced processing data, and the balanced processing data is obtained by adjusting the input vector corresponding to each image according to the ratio of the number of each category detection box in each image to the number of all category detection boxes in each image in the pre-acquired training data.

[0006] The image to be detected is input into a pre-trained abnormal behavior detection model for examination scenarios to obtain the detection result of the image to be detected.

[0007] In some embodiments, the training method for the abnormal behavior detection model in the examination scene includes: determining the category label of each image; determining the masking method of each image based on the category label of each image and the ratio of the number of each category detection box to the total number of all category detection boxes in each image, wherein the masking method of each image is used to characterize the method of masking detection boxes of at least one category in the image; adjusting the input vector corresponding to each image according to the masking method of each image to obtain the balanced processing data; and training the abnormal behavior detection model in the examination scene based on the balanced processing data.

[0008] In some embodiments, determining the masking method for each image based on the category label of each image and the ratio of each category detection box in each image to the total number of all category detection boxes in each image includes: determining the target category of the detection boxes in each image that need to be masked based on the category label of each image; determining the number of detection boxes of the target category that need to be masked in each image based on the target category of the detection boxes in each image and the ratio of each category detection box in each image to the total number of all category detection boxes in each image; and determining the masking method for each image based on the number of detection boxes of the target category that need to be masked in each image.

[0009] In some embodiments, determining the number of target category detection boxes requiring masking in each image based on the target category of the detection boxes requiring masking in each image and the ratio of the number of each category detection boxes in each image to the number of all category detection boxes in each image includes: determining the ratio of the number of each category detection boxes to the number of all category target boxes in a batch of images to which each image belongs; and determining the number of target category detection boxes requiring masking in each image based on the target category of the detection boxes requiring masking in each image, the ratio of the number of each category detection boxes in each image to the number of all category target boxes in each image, and the ratio of the number of each category detection boxes to the number of all category target boxes in a batch of images to which each image belongs.

[0010] In some embodiments, determining the category label of each image includes: determining the category label of each image based on the ratio of the number of each category detection box in each image to the number of all category detection boxes in each image.

[0011] In some embodiments, the category label for each image includes at least one of the following: pre-exam state, normal state during the exam, abnormal state during the exam, and abnormal state at the end of the exam.

[0012] In some embodiments, each image includes at least one of the following: a detection box for the normal behavior category, a detection box for the abnormal behavior category, and a detection box for the empty table category.

[0013] In some embodiments, the input vector includes a memory vector obtained by the encoder in the abnormal behavior detection model of the examination scene encoding each image; the training method of the abnormal behavior detection model of the examination scene includes: performing dimensional and shape transformations on the memory vector corresponding to each image to obtain a predicted category vector for each image; calculating the nuclear norm number for the predicted category vector of each image; determining the loss of the abnormal behavior detection model of the examination scene based on the nuclear norm number corresponding to the predicted category vector of each image; and training the abnormal behavior detection model of the examination scene based on the loss of the abnormal behavior detection model of the examination scene.

[0014] In some embodiments, determining the loss of the abnormal behavior detection model for the examination scene based on the nuclear norm number corresponding to the predicted category vector of each image includes: determining the negative of the nuclear norm number as a first loss; determining the loss of the abnormal behavior detection model for the examination scene, wherein the loss of the abnormal behavior detection model for the examination scene includes at least the first loss.

[0015] This application embodiment also provides an abnormal behavior detection device, the device comprising:

[0016] The processing module is used to acquire a pre-trained abnormal behavior detection model for the examination scene. The abnormal behavior detection model for the examination scene is trained based on balanced processing data. The balanced processing data is obtained by adjusting the input vector corresponding to each image according to the ratio of the number of each category detection box in each image to the number of all category detection boxes in each image in the pre-acquired training data.

[0017] The detection module is used to input the image to be detected into a pre-trained abnormal behavior detection model for examination scenarios, and obtain the detection result of the image to be detected.

[0018] This application also provides an electronic device, which includes a processor and a memory for storing a computer program that can run on the processor; wherein the processor is used to run the computer program to perform any of the above-described abnormal behavior detection methods.

[0019] This application also provides a computer storage medium storing a computer program that, when executed by a processor, implements any of the above-described abnormal behavior detection methods.

[0020] This application also provides a computer program product, including a computer program that, when executed by a processor, implements any of the above-described abnormal behavior detection methods.

[0021] As can be seen, when training the abnormal behavior detection model for examination supervision scenarios, the data can be balanced based on the ratio of the number of detection boxes of each category in each image to the total number of detection boxes of all categories in each image. Then, the abnormal behavior detection model can be trained using this balanced data. Since the ratio of the number of detection boxes of each category in each image to the total number of detection boxes of all categories in each image accurately reflects the ratio of abnormal behavior data to normal behavior data in the examination supervision scenario, training the abnormal behavior detection model by balancing the data can improve the detection accuracy of the trained model. Attached Figure Description

[0022] Figure 1A An image provided in the relevant technology indicating the existence of cheating behavior;

[0023] Figure 1B This is another image provided in the related technology showing the existence of cheating behavior;

[0024] Figure 1C This is yet another image provided in the related technology indicating the existence of cheating behavior;

[0025] Figure 1D An image representing normal exam behavior provided in related technologies;

[0026] Figure 1E This is another image provided in the related technology to represent normal examination behavior;

[0027] Figure 2 This is a flowchart of the abnormal behavior detection method according to an embodiment of this application;

[0028] Figure 3A This is a schematic diagram of the data processing flow performed by the encoder in an embodiment of this application;

[0029] Figure 3B This is a schematic diagram of the data processing flow executed by the query selection module in an embodiment of this application;

[0030] Figure 3C This is a schematic diagram of the data processing flow performed by the decoder in an embodiment of this application;

[0031] Figure 3D This is a schematic diagram of the data processing flow executed by the post-processing module in an embodiment of this application;

[0032] Figure 4 A schematic diagram illustrating the data processing flow performed by the comparison equalization module provided in this application embodiment;

[0033] Figure 5 A schematic diagram of the data processing flow performed by the diversity perception and selection module provided in the embodiments of this application;

[0034] Figure 6 This is a schematic diagram of the abnormal behavior detection device according to an embodiment of this application;

[0035] Figure 7 This is a schematic diagram of the composition structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0036] In scenarios involving abnormal behavior detection during invigilation, common object detection (OD) models include YOLO (You Only Look Once)-v8. These models, based on a backbone network, feature enhancement network, and detection head, employ an anchor-free approach to achieve image classification, object detection, and instance segmentation. By analyzing video streams frame by frame, they identify abnormal behaviors of test-takers. The core idea of ​​the YOLO algorithm is to treat object detection as a regression task, predicting the location and category of objects in an image through a single forward propagation, achieving fast and efficient object detection. Its loss function considers bounding box coordinate prediction errors, confidence prediction errors, and classification prediction errors, optimizing the model through appropriate weight settings.

[0037] However, the actual probability of abnormal behavior occurring in inspection scenarios is extremely low. Therefore, when constructing inspection data, it is necessary to specify a large number of people to simulate abnormal behavior in the same scenario to increase the richness of abnormal behavior in the data. This will result in too many abnormal behaviors in the same image, leading to a severe imbalance between abnormal and normal behavior categories during model training, which will reduce the detection accuracy of the abnormal behavior detection model in inspection scenarios. Object detection models such as YOLO-v8 usually use the Focal Loss function to constrain the class imbalance problem, but when the class imbalance is too severe and there are too many types of abnormal behavior, it is difficult to fundamentally solve the problem of severe imbalance of various abnormal behaviors.

[0038] Additionally, placing hands under the table and burying one's head are typical cheating actions, which are easily confused with normal behavior in an exam setting. Figures 1A to 1C Images that indicate cheating (i.e., hands under the table and head down) can be used to... Figures 1A to 1C The label reads "Hands under the table and head down"; Figure 1D and Figure 1E Images representing normal behavior can be... Figure 1D and Figure 1E The label is denoted as "normal exam". In practical applications, images of cheating behavior can easily be confused with images corresponding to normal behavior. To determine the behavior category represented by different images, it is necessary to judge based on the degree to which the examinee is burying their head. Considering the issues of judging the degree of burying the head, the difficulty in unifying the estimation standard of the degree of burying the head due to the different orientations of the examinee's seat relative to the camera, and the limited capacity and modeling method of the YOLO model, it is generally difficult to distinguish such subtle movements.

[0039] To address the aforementioned problems in related technologies, this application proposes a technical solution that relates to computer vision (CV) and multimodal large-scale models within artificial intelligence technologies. This application can be implemented based on a pre-trained abnormal behavior detection model for examination scenarios. This abnormal behavior detection model includes at least an encoder, a query selection module, a decoder, and a post-processing module. For example, the abnormal behavior detection model further includes a contrastive balancer module. During training of the abnormal behavior detection model, the data input to the decoder is the data processed by the contrastive balancer module.

[0040] The embodiments of this application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the embodiments provided herein are merely illustrative of the embodiments of this application and are not intended to limit the embodiments of this application. Furthermore, the embodiments provided below are some embodiments for implementing this application, and not all embodiments for implementing this application. Unless otherwise specified, the technical solutions described in the embodiments of this application can be implemented in any combination.

[0041] It should be noted that, in the embodiments of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a method or apparatus that includes a list of elements includes not only the elements expressly described, but also other elements not expressly listed, or elements inherent to implementing the method or apparatus. Without further limitations, an element defined by the phrase "comprising a..." does not exclude the presence of other related elements (e.g., steps in the method or units in the apparatus, such as portions of circuitry, processors, programs, or software, etc.) in the method or apparatus that includes that element.

[0042] The abnormal behavior detection method provided in this application includes a series of steps, but the abnormal behavior detection method provided in this application is not limited to the steps described. Similarly, the abnormal behavior detection device provided in this application includes a series of modules, but the device provided in this application is not limited to the modules explicitly described, and may also include modules that need to be set up for obtaining relevant information or processing based on information.

[0043] Figure 2 This is a flowchart of the abnormal behavior detection method according to an embodiment of this application, such as... Figure 2 As shown, the process includes:

[0044] Step 201: Obtain the pre-trained abnormal behavior detection model for the examination scene. The abnormal behavior detection model for the examination scene is trained based on balanced processing data. The balanced processing data is obtained by adjusting the input vector corresponding to each image according to the ratio of the number of detection boxes of each category in each image to the number of detection boxes of all categories in each image in the pre-acquired training data.

[0045] For example, each image in the training data includes at least one of the following: detection boxes for normal behavior, detection boxes for abnormal behavior, and detection boxes for empty tables. The category number and label of each type of detection box can be determined according to a preset mapping relationship. For example, the label of the empty table detection box can be denoted as EMPTY_DESK, and the category number of the empty table detection box is 1; the label of the normal behavior detection box can be denoted as SITTING; and the category number of the normal behavior detection box is 3. In each image of the training data, the detection boxes other than those for normal behavior and empty tables are detection boxes for abnormal behavior.

[0046] In some embodiments, refer to Figure 3A Each image in the training data can be processed by the image feature extractor of the backbone network to obtain multi-layered image features. These image features can be reshaped into image feature vectors of shape [batch_size, num_features, hidden_dim], denoted as src_flatten. After processing src_flatten by the encoder and query selection module, the input vector corresponding to each image can be obtained. In this embodiment, batch_size represents the number of images processed in a batch, num_features represents the number of feature points, and hidden_dim represents the vector dimension corresponding to the feature points.

[0047] For example, the positional embeddings corresponding to the image features of the above-mentioned multi-layered structures can be obtained according to the PositionEmbeddingSineHW in the DINO (DETR with Improved deNoising AnchOr Boxes) algorithm. The shape of the positional embedding is [batch_size, num_queries, hidden_dim], and the positional embedding can be denoted as lvl_pos_embed_flatten. In this embodiment, num_queries represents the number of feature points to be selected.

[0048] Reference Figure 3A The image feature vector `src_flatten` and the position vector `lvl_pos_embed_flatten` are input into the encoder. After processing by N encoder layers, the encoded memory vector and position vector `pos` are obtained, where N is an integer greater than 1. Here, the encoder layers can be implemented with reference to the network structure of a standard Transformer. For ease of description, an application scenario example can be used for illustration. In this application scenario example, the shapes of the image feature vector `src_flatten` and the position vector `lvl_pos_embed_flatten` are both [5, 9297, 256], and the shapes of the encoded memory vector and position vector are also [5, 9297, 256].

[0049] For each image, the memory vector (i.e., the encoded memory vector) is subjected to dimensionality and shape transformation to obtain the predicted class vector for each image. For example, the memory vector corresponding to each image can be transformed in dimension through a linear layer of a neural network to obtain an unselected class vector. This unselected class vector can be denoted as `enc_outputs_class_unselected`. In the above application scenario embodiment, the shape of this unselected class vector is [5, 9297, 13], where 13 represents the total number of corresponding classes.

[0050] For example, the query selection module described above can adopt the diversity-aware query selection module proposed in the embodiments of this application. The category vector enc_outputs_class_unselected that has not been selected by category will be input into the diversity-aware query selection module. In the diversity-aware query selection module, the torch.topk function can be used to process enc_outputs_class_unselected, thereby sorting the last dimension vector representing the category probability and selecting the index vector corresponding to the topk categories. Here, the index vector corresponding to the topk categories can be denoted as topk_proposals, and the shape of topk_proposals is [batch_size, num_queries], where topk = num_queries, which represents the number of feature points to be selected. For example, when topk = num_queries = 300, 300 feature points need to be selected from 9297 feature points.

[0051] Indexing the memory vectors using the index vector `topk_proposals`, the 300 feature points with the highest class scores can be selected from the 9297 feature points in each batch of images. The feature vector corresponding to the 300 feature points with the highest class scores is denoted as `tgt_undetach`, and the shape of `tgt_undetach` is [5, 300, 256]. `tgt_undetach` is the memory vector selected based on the index vector.

[0052] After selecting the memory vector based on the index vector, the reference target bounding box vector can be selected based on the index vector. The input vector for each image includes the memory vector selected based on the index vector and the reference target bounding box vector selected based on the index vector.

[0053] For example, refer to Figure 3BEach image's memory vector is reshaped to reconstruct a multi-scale feature vector (multi_scale_features). The multi_scale_features data format includes multi-size information, and its shape is [level, batch_size, hidden_dim, level_w, level_h], where level represents the number of feature layers used, batch_size represents the number of images processed in a batch, hidden_dim represents the vector dimension corresponding to the feature point, level_w represents the width of the feature vector of the corresponding feature layer, and level_h represents the height of the feature vector of the corresponding feature layer. In the above application scenario embodiment, reshaping the multi_scale_features flattens it, resulting in the src vector, which is identical to the memory vector corresponding to the multi-scale feature vector. In the above application scenario embodiment, the shape of the src vector is [5, 9297, 256].

[0054] The src vector is processed by the gen_encoder_output_proposals module and can be copied into an output_memory vector, which is identical to the src vector. The output_memory vector is then subjected to dimensionality reduction using a multilayer perceptron (MLP) to obtain an offset vector representing the coordinate offset; for the above application scenario embodiment, the shape of the offset vector is [5, 9297, 3].

[0055] A linear transformation is performed on the src vector to reduce the dimensionality of its latent features, resulting in a vector representing the anchor point, which can be denoted as output_proposals. In the above application scenario, the shape of the vector representing the anchor point is [5, 9297, 4].

[0056] The offset vector is summed with the vector representing the anchor point to obtain the unselected prediction box vector enc_outputs_coord_unselected; for the above application scenario embodiment, the shape of the unselected prediction box vector enc_outputs_coord_unselected is [5,9297,4].

[0057] The unselected prediction bounding box vectors enc_outputs_coord_unselected can be filtered based on the index vector topk_proposals to obtain the reference target bounding box vector refpoint_embed_undetach selected based on the index vector; for the above application scenario embodiment, the shape of the reference target bounding box vector refpoint_embed_undetach is [5,300,4].

[0058] For example, refer to Figure 3B Furthermore, the maximum size feature of the multi-scale feature vector multi_scale_features can be upsampled by a factor of 1 to obtain a mask vector, which can be denoted as mask_features. In the above application scenario embodiment, the shape of the maximum size feature of the multi-scale feature vector multi_scale_features is [5,256,92,76], and the shape of the mask vector mask_features is [5,256,184,152].

[0059] For example, refer to Figure 3C The memory vector tgt_undetach selected based on the index vector and the reference target box vector refpoint_embed_undetach selected based on the index vector can be input into the contrast denoising data preparation module (prepare_for_dn). The contrast denoising data preparation module processes the memory vector tgt_undetach and the reference target box vector refpoint_embed_undetach to obtain the noisy target box vector input_query_bbox and the noisy label vector input_query_label. In the above application scenario embodiment, the shape of the noisy target box vector input_query_bbox is [5,93,256], and the shape of the noisy label vector input_query_label is [5,93,4].

[0060] The above-mentioned tgt vector is concatenated with the noisy target bounding box vector input_query_bbox to obtain the input_tgt vector. The above-mentioned refpoint_embed vector is concatenated with the noisy label vector input_query_label to obtain the input_refpoint_embed vector. The input_tgt vector and the input_refpoint_embed vector together form the input vector of the decoder. In the above application scenario embodiment, the shape of the input_tgt vector is [5,393,256], and the shape of the input_refpoint_embed vector is [5,393,4].

[0061] The input_tgt vector and input_refpoint_embed vector are processed through an M-layer decoding layer, and the output of each decoding layer is retained to obtain the hidden layer vector hs. Here, M is a set integer greater than 1, for example, M = 9. The shape of the hidden layer vector hs is [layer_nb, batch_size, num_tgt_dn_queries, hidden_dim], where layer_nb represents the number of decoding layers, and num_tgt_dn_queries is the number of feature points corresponding to the input_tgt vector or input_refpoint_embed vector; that is, num_tgt_dn_queries represents the number of feature points after concatenation. In the above application scenario embodiment, the shape of the hidden layer vector hs is [9, 5, 393, 256].

[0062] For example, refer to Figure 3DIn the post-processing module, the hidden layer vector hs and the input_tgt vector are transformed using the forward_prediction_heads module, which is composed of MLP, to obtain a prediction mask vector of shape [layer_nb, batch_size, num_tgt_dn_queries, mask_w, mask_h], a prediction class vector of shape [layer_nb, batch_size, num_tgt_dn_queries, class_num], and a prediction class vector of shape [layer_nb, batch_size, num_tgt_dn_queries, class_num]. The predicted score vector is defined as [h_size, num_tgt_dn_queries, 1], where mask_w represents the width of the feature map corresponding to the mask vector, mask_h represents the height of the feature map corresponding to the mask vector, class_num is the total number of categories of the detection boxes, the predicted mask vector obtained in the post-processing module can be denoted as predictions_mask, the predicted class vector obtained in the post-processing module can be denoted as predictions_class, and the predicted score vector obtained in the post-processing module can be denoted as predictions_score. For the above application scenario embodiment, the shape of predictions_mask is [10, 5, 393, 136, 248], the shape of predictions_class is [10, 5, 393, 13], and the shape of predictions_score is [10, 5, 393, 1]. For example, the hidden layer vector hs and the input_tgt vector can be transformed by the forward_prediction_heads module, which is composed of MLP, based on the mask vector mask_features output by the diversity perception selection module.

[0063] Reference Figure 3D The input_refpoint_embed vector can be merged with the reference vectors output from the decoder's hidden layer to obtain a merged vector list. In the above application scenario embodiment, the merged vector list includes 10 vectors with shapes [5, 393, 4]. The merged vector list is processed by the prediction box vector determination module (pred_box module) to obtain prediction box vectors with shapes [layer_nb, batch_size, num_tgt_dn_queries, 4], which can be denoted as out_boxes; in the above application scenario embodiment, the shape of the prediction box vectors is [10, 5, 393, 4].

[0064] Reference Figure 3DThe predictions_mask vector, predictions_class vector, predictions_score vector, out_boxes vector, and a preset pad_size parameter can be input into the post-processing submodule. For example, the value of the pad_size parameter is 93. In the post-processing submodule, the predictions_mask vector, predictions_class vector, predictions_score vector, and out_boxes vector are split according to the pad_size parameter to obtain the contrast denoising branch vector set and the decoding branch vector set. The set of contrast denoising branch vectors includes a first class vector of shape [5,93,13], a first score vector of shape [5,93,11], a first predicted box vector of shape [5,93,4], and a first mask vector of shape [5,93,184,152]. Here, the first class vector can be denoted as pred_logits_dn, the first score vector can be denoted as pred_scores_dn, the first predicted box vector can be denoted as pred_boxes_dn, and the first mask vector can be denoted as pred_masks_dn. The decoding branch vector set includes a second class vector of shape [5,93,13], a second score vector of shape [5,93,11], a second predicted box vector of shape [5,93,4], and a second mask vector of shape [5,93,184,152]. Here, the second class vector can be denoted as pred_logits, the second score vector can be denoted as pred_scores, the second predicted box vector can be denoted as pred_boxes, and the second mask vector can be denoted as pred_masks.

[0065] Step 202: Input the image to be detected into the pre-trained abnormal behavior detection model for the examination scene to obtain the detection result of the image to be detected.

[0066] In practical applications, steps 201 to 202 can be implemented based on a processor, which can be at least one of the following: Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), Central Processing Unit (CPU), Controller, Microcontroller, and Microprocessor.

[0067] As can be seen, when training the abnormal behavior detection model for examination supervision scenarios, the data can be balanced based on the ratio of the number of detection boxes of each category in each image to the total number of detection boxes of all categories in each image. Then, the abnormal behavior detection model can be trained using this balanced data. Since the ratio of the number of detection boxes of each category in each image to the total number of detection boxes of all categories in each image accurately reflects the ratio of abnormal behavior data to normal behavior data in the examination supervision scenario, training the abnormal behavior detection model by balancing the data can improve the detection accuracy of the trained model.

[0068] In some embodiments of this application, the training method for the abnormal behavior detection model in examination monitoring scenarios includes:

[0069] Determine the category label for each image;

[0070] The masking method for each image is determined based on the category label of each image and the ratio of the number of detection boxes for each category to the total number of detection boxes for all categories in each image. The masking method for each image is used to characterize the way the number of detection boxes for at least one category in the image is masked.

[0071] Based on the masking method of each image, the input vector corresponding to each image is adjusted to obtain the balanced processing data;

[0072] The abnormal behavior detection model for the examination supervision scene is trained based on the balanced processing data.

[0073] In some embodiments, the category label for each image includes at least one of the following: pre-exam state, normal state during the exam, abnormal state during the exam, and abnormal state at the end of the exam.

[0074] In some embodiments, the process of determining the category label of each image may include: determining the category label of each image based on the ratio of the number of each category detection box in each image to the total number of category detection boxes in each image.

[0075] In some embodiments, if all the detection boxes in an image are for empty tables, the image is labeled as a pre-exam state; if the number of normal behavior detection boxes is the largest in the image, the image is labeled as a normal state during the exam; if the number of abnormal behavior detection boxes is the largest in the image, the image is labeled as an abnormal state during the exam; if there are both normal and abnormal behavior detection boxes in the image, but the number of empty tables is the largest, the image is labeled as an abnormal state at the end of the exam.

[0076] In one example, the batch contains 5 images, and the categories of the bounding boxes contained in different images may differ. For example, the 5 images in the batch can be denoted as batch_id_0, batch_id_1, batch_id_2, batch_id_3, and batch_id_4, respectively. Figure 4 The contrast equalization module can determine the category label of each image. `batch_id_0` contains 30 bounding boxes for empty tables, corresponding to an empty classroom before the exam. `batch_id_1` and `batch_id_2` each contain 24 bounding boxes for normal behavior, 4 bounding boxes for abnormal behavior, and 3 bounding boxes for empty tables. Since the probability of abnormal behavior occurring during the actual exam is low, `batch_id_1` and `batch_id_2` both correspond to a normal state during the exam. `batch_id_3` contains 5 bounding boxes for normal behavior. The batch_id_3 contains checkboxes for 25 categories of abnormal behavior and 1 checkbox for an empty table. Given the actual scenario shown in batch_id_3, it can be assumed that collective cheating occurred during the exam. Therefore, the category label for batch_id_3 is determined as an abnormal state during the exam, which facilitates subsequent theoretical analysis. Batch_id_4 contains checkboxes for 5 categories of normal behavior, 4 categories of abnormal behavior, and a large number of empty tables. Considering that a small number of abnormal behaviors may occur near the end of the exam, the category label for batch_id_4 can be determined as an abnormal state at the end of the exam.

[0077] As can be seen, the embodiments of this application can reasonably determine the category label of each image based on the ratio of the number of each category detection box in each image to the number of all category detection boxes in each image.

[0078] In this embodiment of the application, for each image in each batch of images in the training data, the ratio of the number of detection boxes of each category to the number of detection boxes of all categories can be counted. For example, the ratio of the number of detection boxes of the normal behavior category to the number of detection boxes of all categories can be recorded as x_i, where i is the image number; the ratio of the number of detection boxes of the abnormal behavior category to the number of detection boxes of all categories can be recorded as y_i; and the ratio of the number of detection boxes of the empty table category to the number of detection boxes of all categories can be recorded as z_i.

[0079] In some embodiments, the number of detection boxes in an image that needs to be masked can be determined based on the category label of each image and the ratio of the number of detection boxes of each category in each image to the total number of detection boxes of all categories in each image, and the masking method for each image can be determined based on this number.

[0080] In some embodiments, the masking method for each image can be represented by a mask vector focal_mask. That is, the focal_mask for each image can be determined based on the class label of each image and the ratio of the number of detection boxes for each class to the total number of detection boxes for all classes in each image. Then, the focal_mask for each image can be combined with the input vector for each image to obtain the contrast-equalized output vector output_tensor. The equalization processing data includes the contrast-equalized output vectors for each image in the training data.

[0081] For example, for detection boxes that need to be masked, the corresponding element in the mask vector is 0, and for detection boxes that need to be retained (detection boxes that do not need to be masked), the corresponding element in the mask vector is 1.

[0082] As can be seen, the embodiments of this application can reasonably determine the masking method for the detection boxes of each type of image based on the category label of each image and the ratio of the number of each category detection box in each image to the number of all category detection boxes in each image. This is beneficial to obtain balanced processing data that matches the actual scene shown in each type of image. Based on this balanced processing data, the training accuracy of the abnormal behavior detection model in the examination scene can be improved.

[0083] In some embodiments of this application, the process of determining the masking method for each image based on the category label of each image and the ratio of the number of detection boxes for each category in each image to the number of target boxes for all categories in each image includes:

[0084] Based on the category label of each image, determine the target category of the detection box that needs to be masked in each image;

[0085] The number of target category detection boxes that need to be masked in each image is determined based on the target category of the detection boxes that need to be masked in each image, and the ratio of the number of detection boxes of each category in each image to the number of target boxes of all categories in each image.

[0086] The masking method for each image is determined based on the number of detection boxes for the target categories that require masking in each image.

[0087] In some embodiments, if the image's category label is determined to be the pre-exam state, then the target category of the detection boxes in the image that need to be masked is determined to be the empty desk category; if the image's category label is determined to be the normal state during the exam, then the target categories of the detection boxes in the image that need to be masked include the normal behavior category and the empty desk category; if the image's category label is determined to be the abnormal state during the exam, then the target categories of the detection boxes in the image that need to be masked include the abnormal behavior category and the normal behavior category; if the image's category label is determined to be the abnormal state at the end of the exam, then the target categories of the detection boxes in the image that need to be masked include the empty desk category and the normal behavior category.

[0088] In some embodiments, the ratio of the number of detection boxes of each category to the number of target boxes of all categories in a batch of images to which each image belongs can be counted, and the number of detection boxes of the target category to be masked in each image can be determined based on the target category of the detection boxes that need to be masked in each image, the ratio of the number of detection boxes of each category to the number of detection boxes of all categories in each image, and the ratio of the number of detection boxes of each category to the number of target boxes of all categories in a batch of images to which each image belongs.

[0089] In one example, the five images in the batch processing can be denoted as b_0, b_1, b_2, b_3, and b_4. The contrast equalization module can determine the category label for each image. b_0 contains three bounding boxes for empty tables, so its category label is determined as the pre-exam state. b_1 contains two bounding boxes for normal behavior, four bounding boxes for abnormal behavior, and one bounding box for an empty table, so its category label is determined as the normal state during the exam. b_2 contains two bounding boxes for normal behavior... The detection boxes for categories b_0, b_1, b_2, b_3, b_4, b_4, b_5, and b_6 are: 1 detection box for the normal behavior category, 4 detection boxes for the abnormal behavior category, and 1 detection box for the empty table category. The category label for b_3 is defined as the abnormal behavior category during the exam. The category label for b_4 is defined as the abnormal behavior category at the end of the exam. It can be seen that in b_0, b_1, b_2, b_3, and b_4, the number of detection boxes for the normal behavior category (x), the number of detection boxes for the abnormal behavior category (y), and the number of detection boxes for the empty table category (z) are 8, 18, and 8 respectively. The ratio of x to all category detection boxes is 8 / 34, the ratio of y to all category detection boxes is 18 / 34, and the ratio of z to all category detection boxes is 8 / 34. In another example, x, y, and z are 58, 37, and 59 respectively. Therefore, the ratio of x to all class detection boxes is 58 / 154, the ratio of y to all class detection boxes is 37 / 154, and the ratio of z to all class detection boxes is 59 / 154.

[0090] In some embodiments, if the category label of an image is "pre-exam status", then the masking method for that image is determined to be: randomly retained. The detection boxes for the empty table category are defined, where bn_i represents the number of categories of detection boxes in the image, and z_min and scale are preset values, for example, z_min = 1, scale = 0.5. In the embodiments of this application, This indicates rounding down to the nearest integer.

[0091] If the image's category label is "normal state during the exam," then the masking method for that image is determined as follows: randomly retain. Detection boxes for each normal behavior category and The detection boxes for the category of empty tables; where y_min represents the minimum number of detection boxes for the pre-defined abnormal behavior category, for example, y_min equals 1 or 3.

[0092] If the image's category label is "abnormal state during the exam," then the masking method for that image is determined as follows: randomly retain... Detection boxes for each abnormal behavior category and Detection boxes for each of the normal behavior categories, where...

[0093] If the image's category label is "exam failure status," then the masking method for that image is determined as follows: randomly retain... The detection box for each category of empty tables and A detection box for each normal behavior category.

[0094] In this embodiment, the memory vector tgt_undetach selected based on the index vector and the reference target box vector refpoint_embed_undetach selected based on the index vector can be input to the contrast equalization module. In the contrast equalization module, the input vector is analyzed and processed according to the data equalization rules. That is, the regions that do not need to be focused on can be masked according to the masking method described above, and only the abnormal behaviors that need to be focused on and the normal behaviors or empty tables of the corresponding equalization quantity can be retained to obtain the mask vector focal_mask corresponding to each image. Then, tgt_undetach and refpoint_embed_undetach can be multiplied by the mask vector focal_mask respectively to obtain the corresponding tgt vector and refpoint_embed vector respectively. For example, in the above application scenario embodiment, the shapes of the tgt vector and refpoint_embed vector are [5,300,256] and [5,300,4], respectively.

[0095] In this embodiment, the data distribution of the invigilation scenario can be analyzed in the contrast balancing module. Through contrast balancing, the problem of severe imbalance of detection boxes of various categories in the training data can be solved. At the same time, since the labels of various detection boxes have undergone contrast balancing, subtle abnormal actions are more easily perceived by the abnormal behavior detection model in the invigilation scenario, thereby improving the model's ability to perceive subtle abnormal behaviors.

[0096] In some embodiments, an abnormal behavior detection model for examination scenarios can be trained based on a matching module. The matching module takes a second category vector `pred_logits`, a second predicted bounding box vector `pred_boxes`, and a target vector `target` including labels and target boxes as input. It employs the Hungarian algorithm to perform a one-to-one matching of the indices of the feature points in the predicted vectors with the indices of the target vectors. The corresponding index matching is represented by `indices`. Here, the target vector `target` includes the aforementioned `tgt` vector and `refpoint_embed` vector. For example, for each image in each batch of images, `indices` consists of `min(num_queries, num_target_boxes)` (index_i, index_j) tuples, where `num_queries` represents the number of feature points selected in the query selection module, `num_target_boxes` represents the number of detection boxes in the image, `index_i` represents the index of the feature point in the predicted vector that can match the target vector, and `index_j` represents the index of the specified target in the target vector that can be matched by the feature points in the predicted vector.

[0097] For the Decoder Part, the labels and bounding boxes of the target can be transformed according to the index matching indices and num_queries to construct the target class vector target_classes_one_hot with shape [5,300,13] and the bounding box vector target_boxes with shape [5,300,4]. The cross-entropy loss is determined according to the second class vector pred_logits and the target class vector target_classes_one_hot, and the generalized intersection-over-union (GIOU) loss is determined according to the second predicted box vector pred_boxes and the bounding box vector target_boxes.

[0098] For the contrastive denoising branch (CDN Part), we can assume that dn_num = 3, indicating that 3 contrastive denoising groups are used. The predicted class vector of the i-th group in the 3 contrastive denoising groups is denoted as pred_logits_dn_i, and the predicted box vector of the i-th group in the 3 contrastive denoising groups is denoted as pred_boxes_dn_i; i represents the number of contrastive denoising groups, i∈[1,2,3]. For example, the shape of pred_logits_dn_i is [5,93 / 3,13] = [5,31,13], and the shape of pred_boxes_dn_i is [5,93 / 3,4] = [5,31,3].

[0099] The shape of the target class vector `target_class_dn_i` corresponding to each group is the same as the shape of `pred_logits_dn_i`, and the target indices correspond one-to-one. Similarly, the shape of the target box vector `target_boxes_dn_i` corresponding to each group is the same as the shape of `pred_boxes_dn_i`, and the target indices correspond one-to-one. The cross-entropy loss is determined based on the first class vector `pred_logits_dn` and the target class vector `target_classes_one_hot`, and the GIOU loss is determined based on the first predicted box vector `pred_boxes_dn` and the target box vector `target_boxes`.

[0100] It should be noted that the contrast equalization module, the contrast denoising data preparation module, the contrast denoising branch and its loss function are only used in the training phase of the abnormal behavior detection model in the examination scene, and do not play a role in the testing and application phases of the abnormal behavior detection model in the examination scene.

[0101] In some embodiments of this application, the input vector mentioned above includes the memory vector obtained by the encoder in the abnormal behavior detection model of the examination scene, which encodes each image.

[0102] Accordingly, the training methods for the abnormal behavior detection model in the examination supervision scenario include:

[0103] The memory vector corresponding to each image is subjected to dimensionality and shape transformation to obtain the predicted category vector for each image;

[0104] Calculate the nuclear norm for the predicted category vector of each image;

[0105] The loss of the abnormal behavior detection model in the examination scene is determined based on the nuclear norm corresponding to the predicted category vector of each image.

[0106] The abnormal behavior detection model for the examination scene is trained based on the encoder loss.

[0107] For example, refer to Figure 5The input to the diversity-aware selection module includes the memory vector obtained by the encoder for each image and the topk scalar. The memory vector obtained by the encoder for each image is the aforementioned encoded memory vector. In the above application scenario embodiment, the shape of the encoded memory vector is [5, 9297, 256], and the value of the topk scalar is 300. The function of the diversity-aware selection module is to select the top 300 feature points from the second dimension of the encoded memory vector in descending order of category scores. When the number of images in the batch is 5, an index vector topk_proposals with a shape of [5, 300] can be obtained. The index vector topk_proposals serves as the basis for subsequently deriving tgt_undetach and the reference target box vector refpoint_embed_undetach.

[0108] For example, refer to Figure 5 The encoded memory vector can be dimensionally transformed using the linear layers of a neural network to obtain unselected class vectors. These unselected class vectors can be denoted as `enc_outputs_class_unselected`. In the above application scenario, the shape of the unselected class vector is [5, 9297, 13], where 13 represents the total number of corresponding classes. The unselected class vector is then transformed to obtain the predicted class vector for each image. This predicted class vector can also be denoted as `enc_outputs_class_unselected`, and in the above application scenario, its shape is [5*9297, 13]. This predicted class vector can be used as input data for calculating the loss and the nuclear norm loss function `loss_bnm`.

[0109] In some embodiments, the negative of the nuclear norm corresponding to the predicted category vector of each image can be determined as the first loss, and the loss of the abnormal behavior detection model in the examination scene includes at least the first loss.

[0110] Those skilled in the art will know that the nuclear norm of matrix A is ||A|| * Let A represent the sum of the singular values ​​of matrix A; the F-norm of matrix A represents the square root of the sum of the squares of the elements of matrix A; when the F-norm of matrix A is ||A|| F When ≤1, the convex approximation of the rank rank(A) of matrix A is the nuclear norm ||A|| of matrix A. * According to relevant theories, in model training, for a batch of training data with a batch size of batch_size, when... When, the convex approximation of rank(A) is:

[0111] Considering that the rank(A) of matrix A can be used to estimate the number of predicted classes in a batch of data, i.e., it can be used to reflect the richness of predicted classes, the F norm of matrix A, ||A||, can be used to estimate the number of predicted classes in a batch of data. F As we approach the supremum, the nuclear norm of matrix A is ||A|| * It can characterize the richness of predicted categories.

[0112] Therefore, the nuclear norm is calculated for the predicted category vector of each image, and the negative value is used as part of the global loss of the abnormal behavior detection model in the examination scene. The training objective of the abnormal behavior detection model in the examination scene is to minimize the first loss. This can increase the diversity of predicted categories and avoid selecting high-frequency categories for the top k feature points while ignoring low-frequency categories with long tail distribution.

[0113] It can be seen that by introducing the nuclear norm into the abnormal behavior detection model in the examination supervision scenario to impose a loss constraint on the memory vector of the encoder output, the predictive diversity of the encoder output vector can be increased, and attention to long-tail categories can be increased.

[0114] In summary, this application proposes an abnormal behavior detection scheme for examination scenarios based on cross-temporal contrastive learning. This scheme can flexibly introduce a diversity perception selection module for the encoder to constrain the feature vector in advance, and introduce a contrast equalization module for the decoder to balance the data and form a mask that affects the feature vector of the decoder. In contrast, related technologies are based on Convolutional Neural Networks (CNN) for modeling, which has a relatively simple modeling method and is difficult to flexibly solve the data balance problem.

[0115] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0116] Figure 6 This is a schematic diagram of the abnormal behavior detection device according to an embodiment of this application, as shown below. Figure 6 As shown, the device includes:

[0117] The processing module 601 is used to acquire a pre-trained abnormal behavior detection model for the examination scene. The abnormal behavior detection model for the examination scene is trained based on balanced processing data. The balanced processing data is obtained by adjusting the input vector corresponding to each image according to the ratio of the number of each category detection box in each image to the number of all category detection boxes in each image in the pre-acquired training data.

[0118] The detection module 602 is used to input the image to be detected into a pre-trained abnormal behavior detection model for examination scenes, and obtain the detection result of the image to be detected.

[0119] In some embodiments, the training method for the abnormal behavior detection model in the examination supervision scenario includes:

[0120] Determine the category label for each image;

[0121] The masking method for each image is determined based on the category label of each image and the ratio of the number of detection boxes of each category in each image to the total number of detection boxes of all categories in each image. The masking method for each image is used to characterize the method of masking detection boxes of at least one category in the image.

[0122] Based on the masking method of each image, the input vector corresponding to each image is adjusted to obtain the equalization processing data;

[0123] The abnormal behavior detection model for the examination scene is trained based on the balanced processing data.

[0124] In some embodiments, determining the masking method for each image based on the category label of each image and the ratio of each category detection box in each image to the total number of category detection boxes in each image includes:

[0125] Based on the category label of each image, determine the target category of the detection box that needs to be masked in each image;

[0126] The number of target category detection boxes requiring masking in each image is determined based on the target category of the detection boxes that need to be masked in each image, and the ratio of the number of each category detection boxes in each image to the number of all categories detection boxes in each image.

[0127] The masking method for each image is determined based on the number of detection boxes for the target categories that require masking in each image.

[0128] In some embodiments, determining the number of target category detection boxes requiring masking in each image based on the target category of the detection boxes requiring masking in each image and the ratio of each category detection box in each image to the total number of all categories detection boxes in each image includes:

[0129] Determine the ratio of the number of detection boxes for each category to the total number of target boxes for all categories within the batch of images to which each image belongs;

[0130] The number of target category detection boxes requiring masking in each image is determined based on the target category of the detection boxes that need to be masked in each image, the ratio of the number of detection boxes of each category in each image to the number of target boxes of all categories in each image, and the ratio of the number of detection boxes of each category to the number of target boxes of all categories in a batch of images to which each image belongs.

[0131] In some embodiments, determining the category label of each image includes: determining the category label of each image based on the ratio of the number of each category detection box in each image to the number of all category detection boxes in each image.

[0132] In some embodiments, the category label for each image includes at least one of the following: pre-exam state, normal state during the exam, abnormal state during the exam, and abnormal state at the end of the exam.

[0133] In some embodiments, each image includes at least one of the following: a detection box for the normal behavior category, a detection box for the abnormal behavior category, and a detection box for the empty table category.

[0134] In some embodiments, the input vector includes a memory vector obtained by the encoder in the abnormal behavior detection model of the examination scene encoding each image;

[0135] The training method for the abnormal behavior detection model in the examination supervision scenario includes:

[0136] The memory vector corresponding to each image is subjected to dimensional and shape transformations to obtain the predicted category vector for each image;

[0137] Calculate the nuclear norm for the predicted category vector of each image;

[0138] The loss of the abnormal behavior detection model in the examination scene is determined based on the nuclear norm number corresponding to the predicted category vector of each image.

[0139] The abnormal behavior detection model for the examination scene is trained based on the loss of the abnormal behavior detection model for the examination scene.

[0140] In some embodiments, the kernel norm number corresponding to the predicted category vector of each image is used to determine the loss of the abnormal behavior detection model in the examination scene, including:

[0141] The negative of the nuclear norm is determined as the first loss;

[0142] Determine the loss of the abnormal behavior detection model in the examination scene, wherein the loss of the abnormal behavior detection model in the examination scene includes at least the first loss.

[0143] In practical applications, the processing module 601 and the detection module 602 can be implemented based on a processor.

[0144] It should be noted that the description of the above device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0145] It should be noted that, in the embodiments of this application, if the above methods are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a terminal, server, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0146] Correspondingly, this application embodiment further provides a computer program product, the computer program product including computer executable instructions, which are used to implement any of the abnormal behavior detection methods provided in this application embodiment.

[0147] Accordingly, this application embodiment further provides a computer storage medium storing computer-executable instructions, which are used to implement any of the abnormal behavior detection methods provided in the above embodiments.

[0148] This application also provides an electronic device. Figure 7 This is a schematic diagram of the composition structure of an electronic device provided in an embodiment of this application, as shown below. Figure 7 As shown, the electronic device 70 may include:

[0149] Memory 701 is used to store executable instructions;

[0150] The processor 702 is used to implement any of the above-mentioned abnormal behavior detection methods when executing executable instructions stored in the memory 701.

[0151] The processor 702 mentioned above can be at least one of ASIC, DSP, DSPD, PLD, FPGA, CPU, controller, microcontroller, and microprocessor.

[0152] The aforementioned computer-readable storage medium and memory 702 may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM), etc.; or it may be various terminals that include one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0153] In some embodiments, the functions or modules of the apparatus provided in this application can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0154] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0155] The methods disclosed in the various method embodiments provided in this application can be arbitrarily combined to obtain new method embodiments without conflict.

[0156] The features disclosed in the various product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0157] The features disclosed in the various method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0158] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0159] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims. All of these forms are within the protection scope of this application.

Claims

1. A method for detecting abnormal behavior, characterized in that, The method includes: A pre-trained abnormal behavior detection model for invigilation scenarios is obtained, wherein the abnormal behavior detection model for invigilation scenarios is trained based on balanced processing data, and the balanced processing data is obtained by adjusting the input vector corresponding to each image according to the ratio of the number of each category detection box in each image to the number of all category detection boxes in each image in the pre-acquired training data. The image to be detected is input into a pre-trained abnormal behavior detection model for examination scenarios to obtain the detection result of the image to be detected.

2. The method according to claim 1, characterized in that, The training method for the abnormal behavior detection model in the examination supervision scenario includes: Determine the category label for each image; The masking method for each image is determined based on the category label of each image and the ratio of the number of detection boxes of each category in each image to the total number of detection boxes of all categories in each image. The masking method for each image is used to characterize the method of masking detection boxes of at least one category in the image. Based on the masking method of each image, the input vector corresponding to each image is adjusted to obtain the equalization processing data; The abnormal behavior detection model for the examination scene is trained based on the balanced processing data.

3. The method according to claim 2, characterized in that, The step of determining the masking method for each image based on the category label of each image and the ratio of the number of detection boxes for each category in each image to the total number of detection boxes for all categories in each image includes: Based on the category label of each image, determine the target category of the detection box that needs to be masked in each image; The number of target category detection boxes requiring masking in each image is determined based on the target category of the detection boxes that need to be masked in each image, and the ratio of the number of each category detection boxes in each image to the number of all categories detection boxes in each image. The masking method for each image is determined based on the number of detection boxes for the target categories that require masking in each image.

4. The method according to claim 3, characterized in that, The step of determining the number of target category detection boxes requiring masking in each image based on the target category of the detection boxes requiring masking in each image, and the ratio of each category of detection boxes to the total number of all categories of detection boxes in each image, includes: Determine the ratio of the number of detection boxes for each category to the total number of target boxes for all categories within the batch of images to which each image belongs; The number of target category detection boxes requiring masking in each image is determined based on the target category of the detection boxes that need to be masked in each image, the ratio of the number of detection boxes of each category in each image to the number of target boxes of all categories in each image, and the ratio of the number of detection boxes of each category to the number of target boxes of all categories in a batch of images to which each image belongs.

5. The method according to claim 2, characterized in that, Determining the category label of each image includes: determining the category label of each image based on the ratio of the number of each category detection box in each image to the number of all category detection boxes in each image.

6. The method according to any one of claims 2 to 5, characterized in that, The category label for each image includes at least one of the following: pre-exam state, normal state during the exam, abnormal state during the exam, and abnormal state at the end of the exam.

7. The method according to any one of claims 1 to 5, characterized in that, Each image includes at least one of the following: a detection box for normal behavior, a detection box for abnormal behavior, and a detection box for an empty table.

8. The method according to claim 1, characterized in that, The input vector includes the memory vector obtained by the encoder in the abnormal behavior detection model of the examination scene, which encodes each image. The training method for the abnormal behavior detection model in the examination supervision scenario includes: The memory vector corresponding to each image is subjected to dimensional and shape transformations to obtain the predicted category vector for each image; Calculate the nuclear norm for the predicted category vector of each image; The loss of the abnormal behavior detection model in the examination scene is determined based on the nuclear norm number corresponding to the predicted category vector of each image. The abnormal behavior detection model for the examination scene is trained based on the loss of the abnormal behavior detection model for the examination scene.

9. The method according to claim 8, characterized in that, The step of determining the loss of the abnormal behavior detection model for the examination scene based on the nuclear norm corresponding to the predicted category vector of each image includes: The negative of the nuclear norm is determined as the first loss; Determine the loss of the abnormal behavior detection model in the examination scene, wherein the loss of the abnormal behavior detection model in the examination scene includes at least the first loss.

10. An abnormal behavior detection device, characterized in that, The device includes: The processing module is used to acquire a pre-trained abnormal behavior detection model for the examination scene. The abnormal behavior detection model for the examination scene is trained based on balanced processing data. The balanced processing data is obtained by adjusting the input vector corresponding to each image according to the ratio of the number of each category detection box in each image to the number of all category detection boxes in each image in the pre-acquired training data. The detection module is used to input the image to be detected into a pre-trained abnormal behavior detection model for examination scenarios, and obtain the detection result of the image to be detected.

11. An electronic device, characterized in that, The electronic device includes a processor and a memory for storing computer programs capable of running on the processor; wherein, The processor is used to run the computer program to perform the abnormal behavior detection method according to any one of claims 1 to 9.

12. A computer storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the abnormal behavior detection method according to any one of claims 1 to 9.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the abnormal behavior detection method according to any one of claims 1 to 9.

Images