Abnormal behavior determination method, device, computer equipment and storage medium

Abstract

The present application relates to an abnormal behavior determination method, device, computer equipment and storage medium. Based on body movement detection technology, the method includes: obtaining classroom videos collected by a camera device, inputting each classroom video frame of the obtained classroom video into a human movement recognition model and a concentration recognition model, determining the abnormal movement probability and concentration probability of each student, and storing the classroom video frame, the abnormal movement probability of each student in the classroom video frame, and the concentration probability of each student in the classroom video frame in a target database. When an abnormal behavior determination instruction carrying a target time period is received, multiple classroom video frames corresponding to the target time period are determined from the target database to obtain continuous video frames, and based on the abnormal movement probability set of each student corresponding to the continuous video frames and the concentration probability set of each student, the abnormal behavior student is determined, thereby improving the accuracy of abnormal behavior identification of each student.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, apparatus, computer device and storage medium for determining abnormal behavior. Background Art

[0002] Based on the business situation of each company, some training courses are often carried out to enable trainees to better acquire useful knowledge. During the training process, in order to reasonably restrain the trainees, the attendance of the trainees who need to participate in the training and the learning situation of each trainee will be recorded.

[0003] At present, the way to record the attendance and learning status of students is to install video equipment in the classroom and analyze the pictures taken in the classroom based on face recognition technology. The main purpose is to collect pictures of the classroom, recognize the faces of students in the pictures, and judge whether the face images are complete to determine whether the students are listening carefully. However, whether the face is complete does not mean that the students are listening carefully, and it is easy to be misjudged, so the accuracy rate is low. Summary of the invention

[0004] Based on this, it is necessary to provide a method, device, computer equipment and storage medium for determining abnormal behavior that can improve the accuracy in order to solve the above technical problems.

[0005] A method for determining abnormal behavior, the method comprising:

[0006] Obtain classroom videos captured by video equipment;

[0007] Inputting each classroom video frame of the classroom video into a human action recognition model to perform human action analysis, and determining the abnormal probability of each student's action in each classroom video frame;

[0008] Inputting each classroom video frame of the classroom video into a concentration recognition model to determine the concentration probability of each student in each classroom video frame;

[0009] The classroom video frame, the probability of abnormal movements of each student in the classroom video frame, and the probability of concentration of each student are correspondingly stored in a target database;

[0010] When receiving an abnormal behavior determination instruction carrying a target time period, a plurality of classroom video frames corresponding to the target time period are determined from the target database to obtain continuous video frames;

[0011] Based on the abnormal action probability set of each of the students corresponding to the continuous video frames and the concentration probability set of each of the students, the students with abnormal behavior are determined.

[0012] In one embodiment, after the step of determining the abnormal behavior student based on the abnormal action probability set of each student corresponding to the continuous video frames and the concentration probability set of each student, the step further includes:

[0013] Obtain the student information and person-in-charge information of the student with abnormal behavior;

[0014] According to the information of each student and the information of the person in charge, the corresponding classroom abnormality data is sent to the corresponding terminal.

[0015] In one embodiment, the step of determining students with abnormal behavior based on the abnormal action probability set of each student corresponding to the continuous video frames and the concentration probability set of each student includes:

[0016] Based on a set of abnormal motion probabilities of each of the students corresponding to the continuous video frames and a set of concentration probabilities of each of the students;

[0017] Analyze the action abnormality probability set and the concentration probability set of each student based on a preset threshold, and determine the number of classroom abnormalities of each student exceeding the preset threshold within the target time period;

[0018] Students whose abnormal number of classroom behaviors exceeds a preset value are determined as students with abnormal behavior.

[0019] In one embodiment, the step of inputting each classroom video frame of the classroom video into a human motion recognition model to perform human motion analysis and determine the probability of abnormal motion of each student includes:

[0020] Performing face recognition on each student in each classroom video frame of the classroom video to determine a label for each student in each classroom video frame;

[0021] Extracting human features from each classroom video frame based on each student label in each classroom video frame to obtain human features of each student corresponding to each classroom video frame;

[0022] The human body features of each student corresponding to each classroom video frame are input into a human motion recognition model to perform human motion analysis, and the abnormal probability of the motion of each student in each classroom video frame is determined.

[0023] In one embodiment, the step of inputting each classroom video frame of the classroom video into a concentration recognition model to determine the concentration probability of each student includes:

[0024] Dividing the preset areas in each of the classroom video frames of the classroom video to obtain a set of face images corresponding to each of the classroom video frames;

[0025] The face image set corresponding to each classroom video frame is input into a concentration recognition model to perform concentration analysis, so as to determine the concentration probability of each student in each classroom video frame.

[0026] In one embodiment, the step of dividing the preset face area in each of the classroom video frames of the classroom video to obtain a corresponding face image set in each of the classroom video frames includes:

[0027] Divide the preset area in each classroom video frame of the classroom video into regions to obtain corresponding regional images in each classroom video frame;

[0028] Performing color analysis on the corresponding regional images in each classroom video frame to determine the distribution of each human body region in each classroom video frame;

[0029] The facial image position analysis is performed according to the distribution of each of the human body regions in each of the classroom video frames to obtain the facial image set corresponding to each of the classroom video frames.

[0030] In one embodiment, the method further comprises:

[0031] Acquire all video frames stored in the target database during the training process, and obtain a set of abnormal action probabilities of each trainee corresponding to all video frames, and a set of concentration probabilities of each trainee;

[0032] The students are evaluated based on the abnormal action probability set of each student and the concentration probability set of each student to obtain an evaluation score for each student.

[0033] An abnormal behavior determination device, the device comprising:

[0034] The classroom video frame acquisition module is used to obtain the classroom video captured by the camera device;

[0035] An abnormal motion probability determination module is used to input each classroom video frame of the classroom video into a human motion recognition model to perform human motion analysis, and determine the abnormal motion probability of each student in each classroom video frame;

[0036] A concentration probability determination module, used to input each classroom video frame of the classroom video into a concentration recognition model to determine the concentration probability of each student in each classroom video frame;

[0037] A storage module, used to store the classroom video frame, the abnormal motion probability of each student in the classroom video frame, and the concentration probability of each student in the classroom video frame into a target database;

[0038] A continuous video frame acquisition module is used to determine a plurality of classroom video frames corresponding to the target time period from the target database when receiving an abnormal behavior determination instruction carrying the target time period, and obtain continuous video frames;

[0039] The abnormal behavior student determination module is used to determine the abnormal behavior student based on the abnormal action probability set of each student corresponding to the continuous video frames and the concentration probability set of each student.

[0040] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method when executing the computer program.

[0041] A computer-readable storage medium stores a computer program, which implements the steps of the method when executed by a processor.

[0042] In the above-mentioned abnormal behavior determination method, device, computer equipment and storage medium, the server obtains the classroom video collected by the camera equipment, and the camera equipment shoots the classroom scene, and then each classroom video frame of the obtained classroom video is input into the human action recognition model for human action analysis to determine the abnormal movement probability of each student, so that the students with abnormal movement can be identified, and each classroom video frame of the classroom video is input into the concentration recognition model to determine the concentration probability of each student, so that the students with low concentration can be identified, and the classroom video frames, the abnormal movement probability of each student in the classroom video frames, and the concentration probability of each student are stored in the target database accordingly. When an abnormal behavior determination instruction carrying a target time period is received, multiple classroom video frames corresponding to the target time period are determined from the target database to obtain continuous video frames, and abnormal behavior students are determined based on the abnormal movement probability set of each student corresponding to the continuous video frames and the concentration probability set of each student, and the students with abnormal behavior are judged whether each student has abnormal behavior through the student's movement and concentration, thereby improving the accuracy of identifying abnormal behavior of each student. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is an application scenario diagram of an abnormal behavior determination method in an embodiment;

[0044] Figure 2 A schematic diagram of a flow chart of a method for determining abnormal behavior in one embodiment;

[0045] Figure 3 is a structural block diagram of an abnormal behavior determination device in one embodiment;

[0046] Figure 4 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0048] The abnormal behavior determination method provided in this application can be applied to Figure 1 In the application environment shown. Among them, the camera device 102 communicates with the server 104 through the network. The server 104 obtains the classroom video collected by the camera device 102; the server 104 inputs each classroom video frame of the classroom video into the human action recognition model for human action analysis, and determines the abnormal probability of the action of each student in each classroom video frame; the server 104 inputs each classroom video frame of the classroom video into the concentration recognition model to determine the concentration probability of each student in each classroom video frame; the server 104 stores the classroom video frame, the abnormal probability of the action of each student in the classroom video frame, and the concentration probability of each student in the target database; when receiving the abnormal behavior determination instruction carrying the target time period, determine the multiple classroom video frames corresponding to the target time period from the target database to obtain continuous video frames; based on the abnormal probability set of the action of each student corresponding to the continuous video frame and the concentration probability set of each student, determine the abnormal behavior student. Among them, the camera device 102 can be but not limited to various surveillance cameras, AI cameras and other devices, and the server 104 can be implemented by an independent server or a server cluster composed of multiple servers.

[0049] In one embodiment, Figure 2 As shown, a method for determining abnormal behavior is provided, and the method is applied to Figure 1 The server in the example is used to illustrate the following steps:

[0050] Step S220, obtaining the classroom video captured by the camera device.

[0051] The classroom video captured by the camera device can be acquired in real time, such as acquiring the classroom video currently captured by the camera device in real time, acquiring the classroom video in real time, and timely acquiring the class situation of the students, and analyzing the abnormal behavior of each student in real time, so as to timely remind each student to listen carefully. It can also be acquired after the classroom video is captured by the camera device and stored in the database, etc.

[0052] Step S240, input each classroom video frame of the classroom video into the human motion recognition model to perform human motion analysis, and determine the abnormal probability of the motion of each student in each classroom video frame.

[0053] Among them, the human action recognition model trained based on the human action recognition technology for the classroom video frame recognizes each human feature to perform human action analysis, judges the probability of abnormal action of each human feature, determines the abnormal action probability of each student in the classroom video frame, and records it. Among them, the human action recognition model can be obtained according to training, such as training the human action recognition model with human features when sleeping, human features when playing, human features when leaving the seat, and normal human features. When the classroom video frame is input into the human action recognition model, the human action recognition model determines the probability of abnormal action of the human features in the classroom video frame. The classroom video frame refers to a still picture of a classroom video in a certain frame. Abnormal action refers to the student's actions such as sleeping, playing, and leaving the seat during class. For example, the most common abnormal behaviors of students in the classroom include: sleeping, playing, playing with mobile phones, etc., and when students play, their body movements will inevitably exceed the normal range of activities, but the face may not be facing the camera device. By analyzing the human action of each classroom video frame, the abnormal action probability of the student can be identified, and it can also be determined as abnormal behavior based on the abnormal action probability of the student.

[0054] Step S260: input each classroom video frame of the classroom video into the concentration recognition model to determine the concentration probability of each student in each classroom video frame.

[0055] Among them, the face image in the classroom video frame is extracted, and the face image is input into the concentration recognition model. The concentration recognition model performs concentration analysis based on each face image to obtain the concentration probability of each student in each classroom video frame. For example, the video is taken at an angle, and all the students sit in different positions, so the faces taken will be different. In particular, the camera is installed on the side, and the faces taken will be very different, which is easy to identify errors. Based on the camera, multiple training sample features can be collected to train a concentration recognition model with different concentration analysis standards according to different distribution positions. Concentration refers to the degree of seriousness of students' listening in class. For example, the most common abnormal behaviors of students in class include sleeping, fighting, playing with mobile phones, etc. When students play with mobile phones, their body movements do not exceed the normal range of activities, but the face is not facing the camera, but looking at the mobile phone. By performing concentration analysis on each classroom video frame, the concentration probability of the student can be identified, and it can also be determined as abnormal behavior based on the concentration probability of the student.

[0056] Step S280, storing the classroom video frames, the abnormal movement probability of each student in the classroom video frames, and the concentration probability of each student in the classroom video frames in a corresponding manner in the target database.

[0057] Among them, the classroom video has multiple classroom video frames, and one classroom video frame corresponds to a set of abnormal action probability and concentration probability data of each student. Therefore, the classroom video frame, the abnormal action probability of each student in the classroom video frame, and the concentration probability of each student need to be stored in the target database. For example, one classroom video frame corresponds to a set of abnormal action probability information of each student and a set of concentration probability information of each student. The classroom video frame is associated with the abnormal action probability of each student in the classroom video frame and the concentration probability of each student. It is convenient for subsequent viewing and use.

[0058] Step S300, when an abnormal behavior determination instruction carrying a target time period is received, multiple classroom video frames corresponding to the target time period are determined from a target database to obtain continuous video frames.

[0059] Among them, the target time period can be set according to the needs of analyzing students with abnormal behavior. The target time period can be a specified time period, such as: the target time period of 10:00-10:01 on March 1, 2019. When it is necessary to determine the students with abnormal behavior in real time, the target time period can also be determined according to the time point of the last determination of the students with abnormal behavior, such as: the time point of the last determination of the students with abnormal behavior was 10:09 on March 5, 2019, and the analysis of the students with abnormal behavior requires a continuous video frame of 1 minute, then the current abnormal behavior student analysis should be performed at 10:10 on March 5, 2019, then the target time period is 10:09-10:10 on March 5, 2019. Obtaining continuous video frames can more accurately analyze whether the students have abnormal behavior.

[0060] Step S320, determining students with abnormal behavior based on the abnormal action probability set of each student corresponding to the continuous video frames and the concentration probability set of each student.

[0061] Among them, the abnormal behavior students are determined according to the abnormal action probability and concentration probability of each student in multiple continuous classroom video frames in the target time period. The continuous multiple times can be the abnormal action probability and concentration probability of each student corresponding to all classroom video frames in the target time period. The students with abnormal behavior cannot be accurately judged from the abnormal action probability and concentration probability in a single classroom video frame, which is prone to errors. Therefore, the abnormal behavior students are obtained by continuous multiple comprehensive judgments.

[0062] In the above abnormal behavior determination method, the server obtains the classroom video collected by the camera device, and the camera device shoots the classroom scene, and then inputs each classroom video frame of the obtained classroom video into the human motion recognition model for human motion analysis to determine the abnormal motion probability of each student, so that the students with abnormal motion can be identified, and each classroom video frame of the classroom video is input into the concentration recognition model to determine the concentration probability of each student, so that the students with low concentration can be identified, and the classroom video frames, the abnormal motion probability of each student in the classroom video frames, and the concentration probability of each student are stored in the target database accordingly. When an abnormal behavior determination instruction carrying a target time period is received, multiple classroom video frames corresponding to the target time period are determined from the target database to obtain continuous video frames, and the abnormal behavior students are determined based on the abnormal motion probability set of each student corresponding to the continuous video frames and the concentration probability set of each student, and the students with abnormal behavior are judged whether each student has abnormal behavior through the student's motion and concentration, thereby improving the accuracy of identifying abnormal behavior of each student.

[0063] In one embodiment, before step S220, the method further includes sending a video shooting instruction to a corresponding camera device through a student assessment instruction.

[0064] Among them, the course schedule can be entered into the server, and the server can automatically trigger the student assessment instruction according to the class time in the course schedule, send a video shooting instruction to the corresponding camera device, and make the camera device shoot the classroom video. For example, if there is a class in Classroom 1 from 9:00 to 10:00 on October 5, 2018, the server can trigger the student assessment instruction before 9:00, send a video shooting instruction to the camera device, and make the camera device shoot the classroom video. It can also send a stop video shooting instruction to the camera device after 10:00 to turn off the camera device.

[0065] Connect the camera to the server. The connection method can be wireless or wired, such as WIFI, USB. The camera is installed on one side of the projection screen in each classroom. The camera can automatically rotate 180° horizontally and 90° vertically. It can also adjust the angle according to the acquired pictures. For example, the server acquires the pictures taken by the camera, analyzes the pictures, determines the adjustment angle of the camera, and sends an angle adjustment instruction to the camera according to the adjustment angle, so that the camera is adjusted to the appropriate shooting angle. The camera can transmit the collected video to the server in real time. The opening or closing time can be automatically determined according to the class time, saving resources, reducing user participation, and improving work efficiency.

[0066] In one embodiment, after the step of determining the abnormal behavior students based on the abnormal action probability set of each student corresponding to the continuous video frames and the concentration probability set of each student, the method further includes:

[0067] Obtain the student information and person-in-charge information of students with abnormal behavior; send the corresponding classroom abnormal data to the corresponding terminal based on the information of each student and person-in-charge.

[0068] Among them, the student information and person-in-charge information corresponding to the student with abnormal behavior are obtained. The student information can be a mobile phone number, WeChat number, email number, etc., which can enable the student terminal to receive reminder messages. The person-in-charge information can be a mobile phone number, WeChat number, email number, etc., which can enable the person-in-charge terminal to receive information about the student's situation. Classroom abnormality data can be the student's abnormal movement probability and concentration probability, and can also include corresponding video frames. It can also be the student's behavior data judged by behavior based on the student's abnormal movement probability and concentration probability. The student's behavior data is used as classroom abnormality data, such as: based on the student's abnormal movement probability and concentration probability, it is judged that the student is sleeping in class, and the content in the classroom abnormality data is the content that reminds him to sleep in class, etc. Students can be reminded to attend classes seriously in time.

[0069] In one embodiment, the step of determining students with abnormal behavior based on the abnormal action probability set of each student corresponding to the continuous video frames and the concentration probability set of each student includes:

[0070] Based on the abnormal action probability set of each student and the concentration probability set of each student corresponding to continuous video frames; the abnormal action probability set of each student and the concentration probability set of each student are analyzed based on preset thresholds to determine the number of classroom abnormalities of each student exceeding the preset threshold within the target time period; students whose classroom abnormalities exceed the preset value are determined as students with abnormal behavior.

[0071] Among them, the action abnormality probability set is the probability of each action abnormality of the students corresponding to the action abnormality probability set in the continuous video frames, and the concentration probability set is the probability of each concentration of the students corresponding to the concentration probability set in the continuous video frames. The number of classroom abnormalities refers to the number of times the action abnormality probability exceeds the preset threshold plus the number of times the concentration probability exceeds the preset threshold. The preset time period can be determined according to the actual situation. For example, within 2 minutes, the same abnormal action can be determined as an abnormal action. The preset threshold can be determined according to the accuracy of the action abnormality probability and the concentration probability. The preset threshold of the action abnormality probability and the preset threshold of the concentration probability can be different or the same. For example, if the action abnormality probability is 0%-100%, the students with the action abnormality probability of 0%-50% can be determined as having no action abnormality, and the students with the action abnormality probability of 50%-100% can be determined as having action abnormality, then 50% is set as the preset threshold. The preset value can be determined according to the number of classroom video frames in the preset time period, such as: there are 30 consecutive classroom video frames in the preset time period. Based on the majority principle, the preset value can be set to more than 15, and can also be adjusted according to other situations. Improve the accuracy of judging students with abnormal behavior.

[0072] In one embodiment, the step of inputting each classroom video frame of the classroom video into a human motion recognition model to perform human motion analysis and determine the probability of abnormal motion of each student includes:

[0073] Perform face recognition on each student in each classroom video frame of the classroom video to determine the label of each student in each classroom video frame; extract human features from each classroom video frame based on the label of each student in each classroom video frame to obtain the human features of each student corresponding to each classroom video frame; input the human features of each student corresponding to each classroom video frame into a human motion recognition model to perform human motion analysis to determine the probability of abnormal motion of each student in each classroom video frame.

[0074] Among them, the server inputs the classroom video frames in the received classroom video into the face recognition model. The face recognition model compares the facial features in the classroom video frames with the facial feature information recorded by the face recognition model according to the facial features in the classroom video frames, and determines the labels of each student in the classroom video frames. The student labels can be the names of the students, or numbers, etc. The main purpose is to determine the identities of each student. Based on the labels of each student in each classroom video frame, the human body features of each classroom video frame are extracted to obtain the human body features of each student corresponding to each classroom video frame. The human body shape features of the students can be extracted according to the shape of the human body of the students in the classroom video frames. For example, a person is composed of four limbs, a head, and a body. An arm has two upper and lower sections, one section can be regarded as a straight line, and two hands have four straight lines. A leg also has two upper and lower sections, one section can be regarded as a straight line, and two legs have four straight lines. The body can be regarded as one straight line. The head can be regarded as one ball. The four straight lines of the two hands, the four straight lines of the two legs, the one straight line of the body and the ball are connected to form a human body feature. The human features of each student corresponding to each classroom video frame are input into the human motion recognition model to perform human motion analysis, and the abnormal motion probability of each student in each classroom video frame is determined.

[0075] In one embodiment, the step of inputting each classroom video frame of the classroom video into the concentration recognition model to determine the concentration probability of each student includes:

[0076] The preset areas in each classroom video frame of the classroom video are divided into regions to obtain a set of face images corresponding to each classroom video frame; the face image set corresponding to each classroom video frame is input into a concentration recognition model to perform concentration analysis, and the concentration probability of each student in each classroom video frame is determined.

[0077] Among them, the preset area is determined according to which areas in the classroom video frame are the seats of each student, and the preset area is determined according to the body area of ​​the student. The corresponding face image set in each classroom video frame is input into the concentration recognition model for concentration analysis to determine the concentration probability of each student in each classroom video frame. The concentration recognition model is obtained by training based on a large number of classroom video frames that have been marked with the concentration of students. The concentration recognition model can judge the concentration probability of each student according to the different positions of each student in the classroom video frame and the corresponding position of each face image.

[0078] In one embodiment, the steps of dividing the preset face regions in each classroom video frame of the classroom video to obtain the corresponding face image set in each classroom video frame include:

[0079] Perform regional division on preset areas in each classroom video frame of the classroom video to obtain corresponding regional images in each classroom video frame; perform color analysis on the corresponding regional images in each classroom video frame to determine the distribution of each human body area in each classroom video frame; perform facial image position analysis based on the distribution of each human body area in each classroom video frame to obtain a set of facial images corresponding to each classroom video frame.

[0080] Among them, the regional image refers to the regional image of the preset area determined according to the body area of ​​the student, and is the regional image of the corresponding area intercepted according to the preset area. The regional image may also contain images of other students, walls, and desks. By performing color analysis on the corresponding regional images in each classroom video frame, the distribution of the human body area in the regional image is analyzed, the position of the face in the human body area is determined, and the face image in the human body area is intercepted to obtain the face image set corresponding to each classroom video frame. The center position of the facial features and the facial boundary of the face can be determined in the human body area, the rectangular boundary of the face area can be delineated, and the face image can be extracted according to the corresponding rectangular boundary of the face area to obtain the face image set of each classroom video frame.

[0081] In one embodiment, the method also includes: obtaining all video frames stored in the target database during the training process, and evaluating the students based on the abnormal action probability set of each student and the concentration probability set of each student corresponding to all video frames; and obtaining the evaluation score of each student based on the abnormal action probability set of each student and the concentration probability set of each student.

[0082] Among them, by setting a basic score for each student, each student is given corresponding points for addition or subtraction based on the set of abnormal action probabilities of each student corresponding to all video frames of each student, and the set of concentration probability of each student. For example, a basic score of 90 points is set for each student, and the basic score of student A is 90 points. According to the name of student A, all abnormal action probabilities and concentration probabilities of student A are obtained, and the number of classroom abnormalities exceeding the preset threshold is judged. For example, if student A has 10 classroom abnormalities exceeding the preset threshold, 1 point is deducted each time. The deduction standard for student A is 10. There are 80 points left, and the average concentration probability of student A is 68. According to the principle that the average concentration probability reaches 80, points are added. If the average concentration probability is below 80, points are subtracted accordingly. 80-90 add 5 points, 90-100 add 10 points, 80-60 subtract 10 points, 60-40 subtract 20 points, 40-20 subtract 30 points, 20-0 subtract 40 points, then student A's final score is 70 points. There is no fixed order for adding or subtracting points for each student according to the number of abnormal classroom times and the average concentration probability of each student. It can effectively improve the control and training quality of training, and also reduce the cost of offline control work of teaching assistants.

[0083] It should be understood that although Figure 2 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0084] In one embodiment, Figure 3 As shown, an abnormal behavior determination device is provided, including: a classroom video frame acquisition module 310, an abnormal action probability determination module 320, a concentration probability determination module 330, a storage module 340, a continuous video frame acquisition module 350 and an abnormal behavior student determination module 360, wherein:

[0085] The classroom video frame acquisition module 310 is used to acquire the classroom video captured by the camera device.

[0086] The abnormal motion probability determination module 320 is used to input each classroom video frame of the classroom video into the human motion recognition model to perform human motion analysis, and determine the abnormal motion probability of each student in each classroom video frame.

[0087] The concentration probability determination module 330 is used to input each classroom video frame of the classroom video into the concentration recognition model to determine the concentration probability of each student in each classroom video frame.

[0088] The storage module 340 is used to store the classroom video frames, the abnormal movement probability of each student in the classroom video frames, and the concentration probability of each student in the target database.

[0089] The continuous video frame acquisition module 350 is used to determine multiple classroom video frames corresponding to the target time period from the target database to obtain continuous video frames when receiving an abnormal behavior determination instruction carrying a target time period.

[0090] The classroom behavior student determination module 360 ​​is used to determine students with abnormal behavior based on the abnormal action probability set of each student corresponding to the continuous video frames and the concentration probability set of each student.

[0091] In one embodiment, the abnormal behavior determination device also includes: an information acquisition module, which is used to obtain student information and person-in-charge information of students with abnormal behavior; and a classroom abnormal data sending module, which is used to send corresponding classroom abnormal data to the corresponding terminal according to the information of each student and person-in-charge.

[0092] In one embodiment, the abnormal behavior student determination module 360 ​​is also used for: analyzing the action abnormality probability set of each student and the concentration probability set of each student corresponding to continuous video frames based on preset thresholds, and determining the number of classroom abnormalities of each student exceeding the preset threshold within the target time period; and determining students whose classroom abnormality times exceed the preset value as abnormal behavior students.

[0093] In one embodiment, the abnormal action probability determination module 320 is also used to: perform face recognition on each student in each classroom video frame of the classroom video to determine the label of each student in each classroom video frame; extract human features from each classroom video frame based on the label of each student in each classroom video frame to obtain the human features of each student corresponding to each classroom video frame; input the human features of each student corresponding to each classroom video frame into the human action recognition model to perform human action analysis to determine the abnormal action probability of each student in each classroom video frame.

[0094] In one embodiment, the concentration probability determination module 330 includes: a region division unit, which is used to divide the preset regions in each classroom video frame of the classroom video to obtain a set of face images corresponding to each classroom video frame; a concentration probability determination unit, which is used to input the face image set corresponding to each classroom video frame into a concentration recognition model to perform concentration analysis, and determine the concentration probability of each student in each classroom video frame.

[0095] In one embodiment, the area division unit is also used to: divide the preset areas in each classroom video frame of the classroom video into regions to obtain corresponding area images in each classroom video frame; perform color analysis on the corresponding area images in each classroom video frame to determine the distribution of each human body area in each classroom video frame; perform facial image position analysis based on the distribution of each human body area in each classroom video frame to obtain a set of facial images corresponding to each classroom video frame.

[0096] In one embodiment, the abnormal behavior determination device also includes: an assessment score acquisition module, which is used to obtain all video frames stored in the target database during the training process, and evaluate the students based on the action abnormality probability set of each student corresponding to all video frames, and the concentration probability set of each student, to obtain the assessment score of each student.

[0097] For the specific definition of the abnormal behavior determination device, please refer to the definition of the abnormal behavior determination method above, which will not be repeated here. Each module in the above abnormal behavior determination device can be implemented in whole or in part by software, hardware and a combination thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0098] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store classroom abnormality data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an abnormal behavior determination method is implemented.

[0099] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0100] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0101] Acquire classroom videos captured by a camera device; input each classroom video frame of the classroom video into a human motion recognition model for human motion analysis, and determine the abnormal motion probability of each student in each classroom video frame; input each classroom video frame of the classroom video into a concentration recognition model, and determine the concentration probability of each student in each classroom video frame; store the classroom video frames, the abnormal motion probability of each student in the classroom video frames, and the concentration probability of each student in a target database; when an abnormal behavior determination instruction carrying a target time period is received, multiple classroom video frames corresponding to the target time period are determined from the target database to obtain continuous video frames; based on the abnormal motion probability set of each student corresponding to the continuous video frames and the concentration probability set of each student, determine the students with abnormal behavior.

[0102] In one embodiment, when the processor executes the computer program, the following steps are also implemented: obtaining the student information and person-in-charge information of the students with abnormal behavior; and sending the corresponding classroom abnormality data to the corresponding terminal according to the information of each student and person-in-charge.

[0103] In one embodiment, when the processor executes the computer program, the following steps are also implemented: based on the action abnormality probability set of each student and the concentration probability set of each student corresponding to the continuous video frames; analyzing the action abnormality probability set of each student and the concentration probability set of each student based on preset thresholds, and determining the number of classroom abnormalities of each student exceeding the preset threshold within the target time period; and determining students whose classroom abnormality times exceed the preset value as students with abnormal behavior.

[0104] In one embodiment, the processor also implements the following steps when executing the computer program: performing face recognition on each student in each classroom video frame of the classroom video to determine the label of each student in each classroom video frame; extracting human features from each classroom video frame based on the label of each student in each classroom video frame to obtain the human features of each student corresponding to each classroom video frame; inputting the human features of each student corresponding to each classroom video frame into a human motion recognition model to perform human motion analysis to determine the probability of abnormal motion of each student in each classroom video frame.

[0105] In one embodiment, when the processor executes the computer program, the following steps are also implemented: dividing the preset areas in each classroom video frame of the classroom video to obtain a set of face images corresponding to each classroom video frame; inputting the corresponding face image set in each classroom video frame into the concentration recognition model to perform concentration analysis, and determine the concentration probability of each student in each classroom video frame.

[0106] In one embodiment, when the processor executes the computer program, it also implements the following steps: performing regional division on preset areas in each classroom video frame of the classroom video to obtain corresponding regional images in each classroom video frame; performing color analysis on the corresponding regional images in each classroom video frame to determine the distribution of each human body area in each classroom video frame; performing facial image position analysis based on the distribution of each human body area in each classroom video frame to obtain a set of facial images corresponding to each classroom video frame.

[0107] In one embodiment, when the processor executes the computer program, the following steps are also implemented: obtaining all video frames stored in the target database during the training process, and evaluating the students based on the abnormal action probability set of each student and the concentration probability set of each student corresponding to all video frames; and evaluating the students based on the abnormal action probability set of each student and the concentration probability set of each student to obtain the evaluation score of each student.

[0108] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0109] Acquire classroom videos captured by a camera device; input each classroom video frame of the classroom video into a human motion recognition model for human motion analysis, and determine the abnormal motion probability of each student in each classroom video frame; input each classroom video frame of the classroom video into a concentration recognition model, and determine the concentration probability of each student in each classroom video frame; store the classroom video frames, the abnormal motion probability of each student in the classroom video frames, and the concentration probability of each student in a target database; when an abnormal behavior determination instruction carrying a target time period is received, multiple classroom video frames corresponding to the target time period are determined from the target database to obtain continuous video frames; based on the abnormal motion probability set of each student corresponding to the continuous video frames and the concentration probability set of each student, determine the students with abnormal behavior.

[0110] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: obtaining the student information and person-in-charge information of the students with abnormal behavior; and sending the corresponding classroom abnormality data to the corresponding terminal according to the information of each student and person-in-charge.

[0111] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: based on the action abnormality probability set of each student and the concentration probability set of each student corresponding to the continuous video frames; analyzing the action abnormality probability set of each student and the concentration probability set of each student based on preset thresholds, and determining the number of classroom abnormalities of each student exceeding the preset threshold within the target time period; and determining students whose classroom abnormality times exceed the preset value as students with abnormal behavior.

[0112] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: performing face recognition on each student in each classroom video frame of the classroom video to determine the label of each student in each classroom video frame; extracting human features from each classroom video frame based on the label of each student in each classroom video frame to obtain the human features of each student corresponding to each classroom video frame; inputting the human features of each student corresponding to each classroom video frame into a human motion recognition model to perform human motion analysis to determine the probability of abnormal motion of each student in each classroom video frame.

[0113] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: dividing the preset areas in each classroom video frame of the classroom video to obtain a set of face images corresponding to each classroom video frame; inputting the corresponding face image set in each classroom video frame into a concentration recognition model to perform concentration analysis, and determine the concentration probability of each student in each classroom video frame.

[0114] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: performing regional division on preset areas in each classroom video frame of the classroom video to obtain corresponding regional images in each classroom video frame; performing color analysis on the corresponding regional images in each classroom video frame to determine the distribution of each human body area in each classroom video frame; performing facial image position analysis based on the distribution of each human body area in each classroom video frame to obtain a set of facial images corresponding to each classroom video frame.

[0115] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: obtaining all video frames stored in the target database during the training process, and evaluating the students based on the abnormal action probability set of each student and the concentration probability set of each student corresponding to all video frames; evaluating the students based on the abnormal action probability set of each student and the concentration probability set of each student to obtain the evaluation score of each student.

[0116] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0117] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0118] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for determining abnormal behavior, the method comprising: According to the class time in the entered course schedule, a student assessment instruction is triggered, and a video shooting instruction is sent to the corresponding camera device through the student assessment instruction, so that the camera device shoots the classroom video; Acquire a picture taken by the camera device, analyze the picture, determine an adjustment angle of the camera device, send an angle adjustment instruction to the camera device according to the adjustment angle, and adjust the shooting angle of the camera device; Obtain classroom videos captured by video equipment; Inputting each classroom video frame of the classroom video into a human action recognition model to perform human action analysis, and determining the abnormal probability of each student's action in each classroom video frame; Performing regional division on the preset areas in each of the classroom video frames of the classroom video, obtaining a set of face images corresponding to each of the classroom video frames, inputting the set of face images corresponding to each of the classroom video frames into a concentration recognition model for concentration analysis, and determining the concentration probability of each student in each of the classroom video frames, wherein the concentration recognition model determines the concentration probability of each student according to the position of each face image in the face image set in the preset area; The classroom video frame, the probability of abnormal movements of each student in the classroom video frame, and the probability of concentration of each student are correspondingly stored in a target database; When receiving an abnormal behavior determination instruction carrying a target time period, a plurality of classroom video frames corresponding to the target time period are determined from the target database to obtain continuous video frames; Based on a set of abnormal motion probabilities of each of the students corresponding to the continuous video frames and a set of concentration probabilities of each of the students; Analyze the action abnormality probability set and the concentration probability set of each student based on a preset threshold, and determine the number of classroom abnormalities of each student exceeding the preset threshold within the target time period; Students whose abnormal number of classroom behaviors exceeds a preset value are determined as students with abnormal behavior.

2. The method according to claim 1, characterized in that After determining the student whose abnormal number of classroom behaviors exceeds a preset value as a student with abnormal behavior, the method further includes: Obtain the student information and person-in-charge information of the student with abnormal behavior; According to the information of each student and the information of the person in charge, the corresponding classroom abnormality data is sent to the corresponding terminal.

3. The method according to claim 1, characterized in that The step of inputting each classroom video frame of the classroom video into a human action recognition model to perform human action analysis and determining the abnormal probability of each student's action in each classroom video frame comprises: Performing face recognition on each student in each classroom video frame of the classroom video to determine a label for each student in each classroom video frame; Extracting human features from each classroom video frame based on each student label in each classroom video frame to obtain human features of each student corresponding to each classroom video frame; The human features of each student corresponding to each classroom video frame are input into a human motion recognition model to perform human motion analysis, and the probability of abnormal motion of each student in each classroom video frame is determined.

4. The method according to claim 1, characterized in that: The step of dividing the preset area in each classroom video frame of the classroom video to obtain a set of face images corresponding to each classroom video frame includes: Divide the preset area in each classroom video frame of the classroom video into regions to obtain corresponding regional images in each classroom video frame; Performing color analysis on the corresponding regional images in each classroom video frame to determine the distribution of each human body region in each classroom video frame; The face image position analysis is performed according to the distribution of each of the human body regions in each of the classroom video frames to obtain the face image set corresponding to each of the classroom video frames.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: Acquire all video frames stored in the target database during the training process, and obtain a set of abnormal action probabilities of each trainee corresponding to all video frames, and a set of concentration probabilities of each trainee; The students are evaluated based on the abnormal action probability set of each student and the concentration probability set of each student to obtain an evaluation score for each student.

6. An abnormal behavior determination device, characterized in that: The device comprises: The classroom video frame acquisition module is used to obtain the classroom video captured by the camera device; An abnormal motion probability determination module is used to input each classroom video frame of the classroom video into a human motion recognition model to perform human motion analysis, and determine the abnormal motion probability of each student in each classroom video frame; A concentration probability determination module is used to divide the preset areas in each of the classroom video frames of the classroom video, obtain a set of face images corresponding to each of the classroom video frames, input the set of face images corresponding to each of the classroom video frames into a concentration recognition model for concentration analysis, and determine the concentration probability of each student in each of the classroom video frames. The concentration recognition model determines the concentration probability of each student according to the position of each face image in the face image set in the preset area. A storage module, used for storing the classroom video frame, the abnormal motion probability of each student in the classroom video frame, and the concentration probability of each student in the classroom video frame in a corresponding manner in a target database; A continuous video frame acquisition module is used to determine a plurality of classroom video frames corresponding to the target time period from the target database when receiving an abnormal behavior determination instruction carrying the target time period, and obtain continuous video frames; A classroom behavior student determination module is used to analyze the action abnormality probability set and the concentration probability set of each student based on a preset threshold value based on the action abnormality probability set and the concentration probability set of each student corresponding to the continuous video frames, determine the number of classroom abnormalities of each student exceeding the preset threshold value within the target time period, and determine the student whose classroom abnormality number exceeds the preset value as an abnormal behavior student; The device is also used to: trigger a student assessment instruction according to the class time in the entered course schedule, send a video shooting instruction to the corresponding camera device through the student assessment instruction, so that the camera device can shoot classroom video, obtain the picture taken by the camera device, analyze the picture, determine the adjustment angle of the camera device, send an angle adjustment instruction to the camera device according to the adjustment angle, and adjust the shooting angle of the camera device.

7. The device according to claim 6, characterized in that The device also includes an assessment score determination module, which is used to obtain all video frames stored in the target database during the training process, and evaluate the students according to the action abnormality probability set of each student and the concentration probability set of each student corresponding to all the video frames to obtain the assessment score of each student.

8. The device according to claim 6, characterized in that The device also includes an information acquisition module for acquiring student information and person-in-charge information of students with abnormal behavior; and a classroom abnormal data sending module for sending corresponding classroom abnormal data to corresponding terminals according to the information of each student and person-in-charge.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images