A method, system, and memory for attention monitoring based on successive video frames

Abstract

The present application relates to the field of concentration monitoring, in particular to a concentration monitoring method, system and memory based on continuous video frames. When calculating the concentration of the monitoring object through the concentration scoring model, first, the monitoring video needs to be obtained, the video frame of the monitoring video is analyzed, and the duration t of each abnormal behavior of each monitoring object is obtained (i,n) and the corresponding total monitoring time T (i,n) ; then the video frame analysis data t (i,n) and T (i,n) of the monitoring object are substituted into the concentration scoring model to obtain the concentration score of each monitoring object. The weight of the abnormal behavior in the present application is set by an expert, and the abnormal behavior recognition can be obtained through a machine learning recognition model. Through the innovative integration of expert evaluation and machine learning, the concentration is quickly and accurately evaluated.

Description

Technical Field

[0001] This invention relates to the field of attention monitoring, and more particularly to an attention monitoring method, system, and memory based on continuous video frames. Background Technology

[0002] Learning focus refers to a student's ability to concentrate and focus on learning. Classroom focus often affects a student's learning efficiency and may even impact their learning ability. Therefore, classroom focus is an important indicator for assessing a student's learning ability.

[0003] Traditional psychological assessment methods mostly measure classroom attention through scales or the completion of specific tasks. This approach is static and requires periodic changes or adjustments to the scale content.

[0004] With the development of artificial intelligence technology, the upgrading of facilities and equipment, and the accumulation of a large number of classroom videos, a solid foundation has been laid for video-based classroom attention monitoring. Currently, many studies have analyzed whether students are focused using facial recognition technology, eye-tracking technology, and brainwave data. However, the focus on factors indicating a lack of focus is not comprehensive enough, and the calculation methods are not scientific enough, resulting in existing attention recognition results often having large deviations and low reliability. Summary of the Invention

[0005] To address the shortcomings of existing video-based attention monitoring methods, which suffer from significant biases, this invention proposes a attention monitoring method based on continuous video frames. This method can combine various abnormal behaviors to assess the attention of the monitored object, resulting in highly reliable assessment results.

[0006] This invention proposes a focus monitoring method based on continuous video frames, comprising the following steps:

[0007] S1. Set up an abnormal behavior and attention scoring model. The attention scoring model is as follows:

[0008]

[0009] in, C n For monitoring objects n Attention score I The number of abnormal behaviors, i For ordinal numbers, 1≤i≤I ; a i Indicates the first i The weight of each abnormal behavior, t (i,n) Indicates the monitored object n Performance i The duration of each abnormal behavior,T (i,n) representing t (i,n) corresponding total monitoring duration;

[0010] S2, acquiring the monitoring video, performing video frame analysis on the monitoring video, and acquiring the duration of each abnormal behavior of each monitoring object t (i,n) and the corresponding total monitoring duration T (i,n) , N for a monitoring object set, ;

[0011] S3, performing video frame analysis on the monitoring object n to obtain video frame analysis data t (i,n) and T (i,n) substitute into the concentration score model to obtain the concentration score of each monitoring object.

[0012] Preferably, the camera cruise track contains K cruise points, the camera time of each cruise point is t0 , the monitoring period is T0= Q x K x t0 , and let K n represent a set of cruise points, when the camera is at K n any one of the cruise points, the monitoring object n is within the monitoring range of the camera;

[0013] For the monitoring object n , the calculation formula of t (i,n) and T (i,n) in a monitoring period is as follows:

[0014]

[0015] r (q,k,n,j,i) represent the number of continuous video frames of the monitoring object q in the k th cruise of the camera at the j th cruise point in the n th shooting, in the i th abnormal behavior; J (q,k,n,i) represent the number of times of the monitoring object q in the k th cruise of the camera at the n th cruise point in the i th abnormal behavior;m This indicates the number of consecutive video frames within a unit of time (1 second). Indicates to r (q,k,n,j,i) / m Round up;

[0016]

[0017] y (q,k,n,i) Represents a binary number, when the camera's... q During the second cruise k Monitoring targets were detected at each patrol point. n Performance i An abnormal behavior, then y (q,k,n,i) Take 1; otherwise, y (q,k,n,i) Take 0.

[0018] Preferably, the abnormal behavior includes abnormal head behavior, and the judgment criteria for abnormal head behavior are: the head attitude yaw angle is greater than or equal to a set yaw threshold, or the head attitude pitch angle is greater than or equal to a set pitch threshold; the head attitude yaw angle of the monitored object x and head posture pitch angle y The calculation formula is:

[0019] δ x = γ x - (θ h -180°) ; δ y = θ v + γ y

[0020] x =f h - δ x ; y =f v + δ y

[0021] δ x This indicates the horizontal error in the head posture angle. δ y This indicates the vertical error of the head posture rotation angle;

[0022] θ h horizontal angle of the center point of the camera shooting direction, θ v vertical angle of the center point of the camera shooting direction; γ x horizontal actual angle difference of the target point on the video frame m and the center point o ; γ y vertical actual angle difference of the target point m and the center point o ; the target point m is the center point of the target face on the video frame, and the target face is the face of the monitoring object;

[0023] f h head pose yaw angle initial value obtained by analyzing the face in the video frame through the head pose algorithm, f v head pose pitch angle initial value obtained by analyzing the face in the video frame through the head pose algorithm.

[0024] Preferably:

[0025] when x < w / 2, then: γ x =a x -β x

[0026] when x ≥ w / 2, then: γ x =a x +β x

[0027] γ y =a y +β y

[0028]

[0029]

[0030] w and h respectively represent the length and height of the circumscribed rectangle of the video frame, (x, y) and (w / 2, h / 2)respectively represent the target point m and the center point o coordinates on the video frame, a x represent the horizontal angle error of the target point m to the center point o a y represent the vertical angle error of the target point m to the center point o

[0031] β x represent the error compensation of the target point m(x, y) in the horizontal direction, β y represent the error compensation of the target point m(x, y) in the vertical direction.

[0032] k h represent the error value of the camera field of view angle and the video frame field of view angle in the horizontal direction; k v represent the error value of the camera field of view angle and the video frame field of view angle in the vertical direction; k dh represent the error component of the diagonal direction field of view angle of the video frame in the horizontal direction; k dv represent the error component of the diagonal direction field of view angle of the video frame in the vertical direction.

[0033] Preferably,

[0034]

[0035]

[0036] wherein, fov h represent the horizontal field of view angle of the video frame, fov v represent the vertical field of view angle of the video frame.

[0037] Preferably, the abnormal behaviors further include yawning, expression abnormality and sitting posture abnormality.

[0038] Preferably, the abnormal behaviors are analyzed by a corresponding recognition model; the input of the recognition model is the monitoring video, the recognition model is used to analyze the monitoring video into continuous video frame output, and label whether each video frame contains the corresponding abnormal behavior; the recognition model is a neural network model obtained by machine learning.

[0039] Preferably, the weight in the concentration score model is an artificial annotation value.​​

[0040] The application further provides a continuous video frame-based attention monitoring system and a memory, which provide a carrier for the continuous video frame-based attention monitoring method and facilitate popularization and application of the method.

[0041] A continuous video frame-based attention monitoring system comprises a storage module and a processor, the storage module stores a computer program, the processor is connected with the storage module, and the processor is used for executing the computer program to realize the continuous video frame-based attention monitoring method.

[0042] A memory, characterized by storing a computer program, wherein the computer program is executed to realize the continuous video frame-based attention monitoring method.

[0043] The application has the following advantages:

[0044] (1) The continuous video frame-based attention monitoring method provided by the application can evaluate the attention of a monitoring object in combination with various abnormal behaviors, the evaluation result has high reliability, the attention evaluation efficiency is improved, and a large amount of manual observation and data arrangement work is reduced.

[0045] (2) The weight of the abnormal behavior in the application is set by an expert, the abnormal behavior recognition can be obtained through a machine learning recognition model, and through the innovative fusion of expert evaluation and machine learning, the attention is quickly and accurately evaluated.

[0046] (3) The application can evaluate the attention of each monitoring object based on continuous video frames obtained by analyzing a monitoring video, and the sampling error caused by a traditional method and the experimental error caused by directly collecting data from a non-real environment are avoided.

[0047] (4) The abnormal behavior in the application can be flexibly set, and the attention scoring model is constructed based on a deduction principle (subtracting the score of the abnormal behavior). The attention monitoring method in the application is more suitable for attention evaluation in an actual classroom scene, and is also applicable to various different scenes, so the application range is wide.

[0048] (5) When the application is applied to classroom attention evaluation, the scoring result has high consistency with the teacher evaluation, and the reliability of the application applied to classroom attention calculation is effectively verified.

[0049] (6) In the present application, the center point of the target face on the video frame is taken as the target point, and the head posture turning angle error is calculated according to the relative position of the target point and the center point of the video frame, so as to realize the error calculation for each target face alone, and the error correction of each target face according to its position on the video frame, so as to accurately calculate the deflection angle of the target face in the camera monitoring picture, and obtain more accurate head posture recognition result when the head posture analysis is carried out, and ensure the accuracy of head abnormal behavior judgment.

[0050] (7) In the present application, the inconsistency between the camera field of view angle and the video frame field of view angle is fully considered, the position of the target point is first compensated for error, then the angle difference between the target point and the center point is calculated according to the compensated target point, and then the head posture turning angle is corrected, so as to ensure the accuracy of error calculation, and further ensure the accuracy of head posture recognition. BRIEF DESCRIPTION OF DRAWINGS

[0051] Figure 1 It is a schematic diagram of the camera cruising point shown in the embodiment;

[0052] Figure 2 It is a schematic diagram of the student position distribution in the embodiment;

[0053] Figure 3 It is a flow chart of a concentration monitoring method based on continuous video frames;

[0054] Figure 4 It is a head posture yaw angle x and a head posture pitch angle y flow chart;

[0055] Figure 5 It is a head posture algorithm flow chart. DETAILED DESCRIPTION

[0056] A concentration monitoring method based on continuous video frames

[0057] The concentration monitoring method based on continuous video frames proposed in the present embodiment calculates the concentration score of the monitored object through the concentration score model combined with the specified abnormal behavior. The higher the concentration score is, the higher the concentration of the monitored object is.

[0058] The concentration score model is:

[0059]

[0060] Among them, C n The concentration score of the monitored object n is, I the number of abnormal behaviors,i For ordinal numbers, 1≤i≤I ; a i Indicates the first i The weight of each abnormal behavior, t (i,n) Indicates the monitored object n Performance i The duration of each abnormal behavior, T (i,n) express t (i,n) The corresponding total monitoring duration. In this implementation, a i Manually labeled.

[0061]

[0062] r (q,k,n,j,i) Indicates the camera number q During the second cruise k At the first cruise point j The surveillance object was captured on camera. n In the first i The number of consecutive video frames of an abnormal behavior; Q represents the maximum number of cruises, which is the number of times the camera cycles within a monitoring period; one cycle of the camera refers to the trajectory of the camera passing through each cruise point exactly once. K n This represents the set of cruise points when the camera is in... K n At any patrol point, the monitored object n Within the camera's surveillance range;

[0063] J (q,k,n,i) Indicates the camera number q During the second cruise k The surveillance target was captured at each patrol point. n In the first i The number of times an abnormal behavior occurred; m This indicates the number of consecutive video frames within a unit of time (1 second). Indicates to r (q,k,n,j,i) / m Round up.

[0064]

[0065] y (q,k,n,i) Represents a binary number, when the camera's... q During the second cruise k Monitoring targets were detected at each patrol point.n performing the i y (q,k,n,i) 1; otherwise, y (q,k,n,i) 0.

[0066] Embodiment 1 t (i,n) and T (i,n) Example of the calculation process

[0067] Suppose that students are arranged in three rows in a classroom, and the cruising points 1-6 of the camera are as shown in Figure 1 The path of one cruise is "1-2-3-4-5-6" or "6-5-4-3-2-1". The camera stays at each cruising point for t0 , and one monitoring period contains Q cruises, and the camera takes m frames of images per second;

[0068] In this embodiment, K=6 Suppose that:

[0069] t0= 1 minute, Q = 2, m = 30, n = 2;

[0070] The monitoring period is T0= Q x K x t0= 12 minutes.

[0071] Suppose that the shooting areas corresponding to the cruising points 1, 2, 3, and 4 are as shown in the circles A1, A2, A3, and A4 in Figure 2 .

[0072] In one monitoring period, the camera shooting time at each cruising point is as follows:

[0073] Cruising point 1: 1st minute + 12th minute

[0074] Cruising point 2: 2nd minute + 11th minute

[0075] Cruising point 3: 3rd minute + 10th minute

[0076] Cruising point 4: 4th minute + 9th minute

[0077] Cruising point 5: 5th minute + 8th minute

[0078] Cruising point 6: 6th minute + 7th minute

[0079] Referring to Figure 2 ​, student A is located in position B1 which is only in the shooting area corresponding to cruise point 1, student B is located in position B12 which is in the shooting area corresponding to cruise point 1 and cruise point 2, and student C is located in position B234 which is in the shooting area corresponding to cruise point 2, cruise point 3 and cruise point 4. The yawning monitoring results of students A, B and C by the camera in the monitoring period are shown in Table 1.

[0080] Table 1: Yawning monitoring statistics of A, B and C

[0081]

[0082] In this embodiment, let r (q,k,n,j,哈欠) Let N represent the number of continuous video frames of the monitoring object yawning in the first shooting of the i-th cruise point in the i-th cruise in the monitoring period, and let N represent the number of continuous video frames of the monitoring object yawning in the first shooting of the i-th cruise point in the i-th cruise in the monitoring period. q k j n

[0083] From Table 1, it can be seen that:

[0084] K 甲 = {Cruise point 1};

[0085] r (q=1,k=1,甲,j=1,哈欠) = 10 frames; r (q=1,k=1,甲,j=2,哈欠) = 21 frames; r (q=1,k=1,甲,j=3,哈欠) = 6 frames;

[0086] r (q=2,k=1,甲,j=1,哈欠) = 10 frames; r (q=2,k=1,甲,j=2,哈欠) = 21 frames; r (q=2,k=1,甲,j=3,哈欠) = 6 frames;

[0087] K 乙 = {Cruise point 1; Cruise point 2};

[0088] r (q=1,k=1,乙,j=1,哈欠) = 21 frames; r (q=1,k=1,乙,j=2,哈欠) = 6 frames;

[0089] r ​​​​(q=1,k=2,乙,j=1,哈欠) = 25 frames; r (q=1,k=2,乙,j=2,哈欠) = 31 frames;

[0090] r (q=2,k=1,乙,j=1,哈欠) = 20 frames; r (q=2,k=2,乙,j=1,哈欠) = 6 frames;

[0091] K 丙 = {Cruise point 2; Cruise point 3; Cruise point 4}.

[0092] r (q=1,k=2,丙,j=1,哈欠) = 25 frames; r (q=1,k=2,丙,j=2,哈欠) = 31 frames;

[0093] r (q=1,k=3,丙,j=1,哈欠) = 11 frames; r (q=1,k=4,丙,j=1,哈欠) = 26 frames;

[0094] r (q=2,k=2,丙,j=1,哈欠) = 6 frames; r (q=2,k=3,丙,j=1,哈欠) = 31 frames; r (q=2,k=4,丙,j=1,哈欠) = 6 frames;

[0095] It can be seen that in the current monitoring period, for student A, there are:

[0096]

[0097]

[0098] For student B, there are:

[0099]

[0100] For student C, there are:

[0101]

[0102] Referring to Figure 3In the embodiment, when the concentration score model is used to calculate the concentration of the monitoring object, the monitoring video is first obtained, the video frame of the monitoring video is analyzed, and the duration of each abnormal behavior of each monitoring object is obtained t (i,n) and the corresponding total monitoring time T (i,n) The video frame analysis data of the monitoring object t (i,n) and T (i,n) is substituted into the concentration score model to obtain the concentration score of each monitoring object.

[0103] The abnormal behavior can be set as head abnormal behavior, yawning, expression abnormality, and sitting posture abnormality.

[0104] The abnormal behavior can be identified by a recognition model obtained by machine learning. For example, a neural network recognition model can be constructed to identify the input monitoring video as a continuous video frame output, and the recognition model is also used to label whether each video frame contains the corresponding abnormal behavior. In this way, the duration of the abnormal behavior corresponding to the monitoring video can be calculated according to the analysis of the recognition model. t (i,n) and T (i,n) .

[0105] The abnormal behavior can also be identified by setting the corresponding judgment condition. After the monitoring video is analyzed into continuous video frames, the judgment condition is used to judge whether the monitoring object on each video frame has the corresponding abnormal behavior.

[0106] A head abnormal behavior identification method

[0107] In the embodiment, the judgment condition of the head abnormal behavior is set. When the target face on the video frame meets the judgment condition, it is determined that the monitoring object on the video frame has head abnormal behavior. The target face is the face of the monitoring object.

[0108] The judgment condition of the abnormal behavior is that the head posture yawing angle is greater than or equal to the set yawing threshold, or the head posture pitch angle is greater than or equal to the set pitch threshold.

[0109] The existing head posture algorithm is used to analyze the shooting image to obtain the head posture rotation angle of the target face in the shooting image. The head posture rotation angle includes the head posture yawing angle f h and the head posture pitch angle f v . The existing head posture algorithm only considers the shooting image when analyzing the head posture rotation angle, without considering the camera distortion, so there is an error in the finally obtained head posture rotation angle.

[0110] In the embodiment, when calculating the yaw angle of head posture x and the pitch angle of head posture y , the influence of camera distortion on the video frame is considered, so as to calculate the horizontal error δ x and the vertical error δ y of the head posture rotation angle caused by camera distortion, and then combine the horizontal error δ x and the vertical error δ y of the head posture rotation angle to correct the head posture rotation angle obtained by the head posture algorithm, so as to ensure the accuracy of the head posture rotation angle recognition.

[0111] With reference to Figure 4 , the head posture correction method based on camera distortion compensation in the embodiment specifically includes the following steps SA1-SA3.

[0112] SA1, obtaining the horizontal field of view angle fov h , the vertical field of view angle fov v , the component of the diagonal direction field of view angle in the horizontal direction fov dh and the component of the diagonal direction field of view angle in the vertical direction fov dv of the video frame; obtaining the horizontal field of view angle FOV h and the vertical field of view angle FOV v of the camera providing the video frame; and obtaining the video frame from the monitoring video of the camera.

[0113] SA2, analyzing the video frame by the head posture algorithm to obtain the yaw angle of head posture f h and the pitch angle of head posture f v of the target face; obtaining the coordinates of the target point m and the center point o on the video frame, and calculating the horizontal error δ x and the vertical error δ y of the head posture rotation angle; the target point is the center point of the target face on the video frame, and specifically, the center point of the circumscribed rectangle or regular polygon of the target face on the video frame can be selected as the target point.

[0114] SA3, combining the horizontal error fh 、 f v 、 δ x and δ y calculating a corrected head pose rotation angle; the corrected head pose rotation angle comprises a corrected head pose yaw angle x and a corrected head pose pitch angle y .

[0115] x =f h - δ x (1-1)

[0116] y =f v + δ y (1-2)

[0117] x represents the corrected head pose yaw angle, i.e. the horizontal rotation angle; y represents the corrected head pose pitch angle, i.e. the vertical rotation angle; f h represents the head pose yaw angle obtained by analyzing the face in the video frame through the head pose algorithm, f v represents the head pose pitch angle obtained by analyzing the face in the video frame through the head pose algorithm.

[0118] head pose rotation angle horizontal error δ x and vertical error δ y The calculation formula is:

[0119] δ x = γ x - (θ h -180°) (2-1)

[0120] δ y = θ v + γ y (2-2)

[0121] θ h horizontal angle of the center point of the camera shooting direction, θ v vertical angle of the center point of the camera shooting direction; γ x horizontal actual angle difference of the target point on the video frame m and the center point o ; γ y vertical actual angle difference of the target point m and the center point o ; the target point m is the center point of the target face on the video frame.

[0122] When x < w / 2, then: γ x =a x -β x

[0123] When x ≥ w / 2, then: γ x =a x +β x

[0124] γ y =a y +β y

[0125] a x horizontal angle error of the target point m on the video frame to the center point o ; a y vertical angle error of the target point m on the video frame to the center point o ; β x error compensation of the target point m in the horizontal direction; β y error compensation of the target point m in the vertical direction.

[0126]

[0127]

[0128] w and h denote the length and height of the video frame's bounding rectangle, respectively, (x, y) and (w / 2, h / 2) denote the coordinates of the target point m and the center point o on the video frame; fov h denotes the horizontal field of view of the video frame, fov v denotes the vertical field of view of the video frame.

[0129]

[0130]

[0131] k h denotes the error value of the camera's field of view and the video frame's field of view in the horizontal direction; k v denotes the error value of the camera's field of view and the video frame's field of view in the vertical direction; k dh denotes the error component of the video frame's diagonal field of view in the horizontal direction; k dv denotes the error component of the video frame's diagonal field of view in the vertical direction.

[0132]

[0133] fov h denotes the horizontal field of view of the video frame, fov v denotes the vertical field of view of the video frame, fov dh denotes the component of the video frame's diagonal field of view in the horizontal direction, fov dv denotes the component of the video frame's diagonal field of view in the vertical direction; FOV h denotes the horizontal field of view of the camera, FOV v denotes the vertical field of view of the camera; FOV h and FOV v is a camera intrinsic parameter and can be read from the camera's specification.

[0134] Head pose algorithm

[0135] There are many head posture algorithms in the prior art, among which, neural network is commonly used to implement the head posture algorithm. In the embodiment, a head posture recognition model is obtained through autonomous learning of a machine model, so as to analyze a video frame through the head posture recognition model and obtain the head posture rotation angle of each face.

[0136] With reference to Figure 5 In the embodiment, the steps of implementing the head posture algorithm are as follows.

[0137] SB1, constructing a neural network and labeling samples; the samples are video frames, the labeling labels of the samples are the head posture rotation angles of each face in the video frames, the input of the neural network is the video frames, and the output of the neural network is the head posture rotation angles of each face in the video frames;

[0138] SB2, learning the labeled samples by the neural network to train network parameters, and taking the trained neural network as a head posture recognition model;

[0139] SB3, inputting a target video frame into the head posture recognition model to obtain the head posture rotation angles of each face in the target video frame output by the head posture recognition model, that is, the head posture yaw angle of the face f h and the head posture pitch angle of the face f v .

[0140] In the embodiment, the labeling labels of the samples are artificially labeled.

[0141] Specifically, a plurality of samples are constructed in SB1, and the samples are divided into training samples and test samples; in SB2, the neural network is iterated for multiple times, each time the neural network learns a specified number of training samples to update the network parameters, and then the head posture rotation angles of each face in a set number of test samples are calculated by the updated neural network as the model labels of the test samples, the model labeling accuracy of the neural network is calculated according to the difference between the model labels of the test samples and the labeling labels, and when the model labeling accuracy is lower than a set value, the neural network learns new training samples; in this way, through parameter iteration, until the model labeling accuracy of the neural network reaches the set value, the neural network is taken as the head posture recognition model.

[0142] Example 2: Concentration monitoring of a school classroom

[0143] In this embodiment, the concentration monitoring method based on continuous video frames provided by the application is used to monitor the concentration of students in a school classroom.

[0144] In this embodiment, the abnormal behaviors are set as head abnormal behavior, yawning, expression abnormality and sitting posture abnormality. Among them, the head abnormal behavior is judged by the head abnormal behavior recognition method described above; the yawning, expression abnormality and sitting posture abnormality are judged by the neural network-based recognition model of machine learning.

[0145] In this embodiment, the concentration levels are set by multiple subject teachers and class teachers, and each level is provided with a corresponding concentration score interval. In this embodiment, the concentration scores of each student given by multiple subject teachers and class teachers are taken as standard scores, the concentration level corresponding to the concentration score of the student obtained by the student through the application is taken as a test result, and the concentration level corresponding to the standard score of the student is taken as a true result.

[0146] In this embodiment, the number of students tested is 311 students belonging to 6 classes in total, and the effective test results obtained are 308 students. The effective test results are the test results consistent with the true results. In this embodiment, the Pearson correlation coefficient of the test results and the true results is 0.8267, and the test results are significantly correlated with the true results. It is proved by the Pearson correlation coefficient that the concentration evaluation results of the application have high consistency with the teacher evaluation.

[0147] The above is only a preferred embodiment of the application, and is not used to limit the application. Any modification, equivalent replacement and improvement made within the spirit and principle of the application shall be included in the protection scope of the application.

Claims

1. A method for monitoring concentration based on consecutive video frames, characterized in that, The method comprises the following steps: S1, setting an abnormal behavior and a concentration score model, the concentration score model is: in, C n For monitoring objects n Attention score I The number of abnormal behaviors, i For ordinal numbers, 1<=i<=I ; a i Indicates the first i The weight of each abnormal behavior, t (i,n) Indicates the monitored object n Performance i The duration of each abnormal behavior, T (i,n) express t (i,n) The corresponding total monitoring time; S2, acquire the monitoring video, perform video frame analysis on the monitoring video, and acquire the duration of each abnormal behavior of each monitoring object t (i,n) and the corresponding total monitoring duration T (i,n) , N for the monitoring object set, ; S3, parse the video frame data of the monitoring object n t (i,n) and T (i,n) substitute into the concentration score model to obtain the concentration score of each monitoring object;​ Let the camera cruise track contain K a plurality of cruise points, the camera time of each cruise point is t0 , the monitoring period is T0=Q×K ×t0 , let K n represent a cruise point set, when the camera is in K n any one of the cruise points, the monitoring object n is in the monitoring range of the camera; Q represents the maximum cruise number; For monitoring objects n its value in a monitoring period t (i,n) and T (i,n) The calculation formula is as follows: r (q,k,n,j,i) represent the number of times the camera takes a picture of the monitoring object in the first abnormal behavior in the first cruise point in the first cruise q k j n i J (q,k,n,i) represent the number of times the camera takes a picture of the monitoring object in the first abnormal behavior in the first cruise point in the first cruise q k n i m represent the number of continuous video frames in 1 second per unit time, represent the number of times the camera takes a picture of the monitoring object in the first abnormal behavior in the first cruise point in the first cruise r (q,k,n,j,i) / m round up;​​​​​​​​​ y (q,k,n,i) indicates a binary number, when the camera detects the monitoring object at the first q cruise point in the first k cruise, and shows the first n abnormal behavior, then i y (q,k,n,i) take 1; otherwise, y (q,k,n,i) take 0.​ 2. The continuous video frame-based concentration monitoring method of claim 1, wherein, The abnormal behavior includes head abnormal behavior, and a judgment condition of the head abnormal behavior is that a head posture yaw angle is greater than or equal to a set yaw threshold value or a head posture pitch angle is greater than or equal to a set pitch threshold value; the head posture yaw angle of the monitoring object x and the head posture pitch angle y are calculated by the following formula: delta x =gamma x -(theta h -180°) ； delta y =theta v +gamma y x =f h -delta x ； y =f v +delta y delta x a horizontal error representing a head pose rotation angle, delta y a vertical error representing a head pose rotation angle; theta h horizontal angle of the center point of the camera shooting direction, theta v vertical angle of the center point of the camera shooting direction; gamma x horizontal actual angle difference of the target point on the video frame m from the center point o , gamma y vertical actual angle difference of the target point m from the center point o ; Target point m The target point is a center point of a target face on a video frame, and the target face is a face of a monitoring object. f h represents a head pose yaw angle initial value obtained by analyzing a face in a video frame through a head pose algorithm, f v represents a head pose pitch angle initial value obtained by analyzing a face in a video frame through a head pose algorithm.

3. The method of claim 2, wherein: When x<w / 2, then: gamma x =a x -β x When x>=w / 2, then: gamma x =a x +β x gamma y =a y +β y w and h denote the length and height of the video frame's bounding rectangle, respectively, (x,y) and (w / 2,h / 2) denote the coordinates of the target point m and the center point o on the video frame, a x denote the horizontal angular error of the target point m from the center point o on the video frame, a y denote the vertical angular error of the target point m from the center point o on the video frame; β x representative target point m(x,y) error compensation in the horizontal direction, β y representative target point m(x,y) error compensation in the vertical direction; k h represents the error value of the camera field of view and the video frame field of view in the horizontal direction; k v represents the error value of the camera field of view and the video frame field of view in the vertical direction; k dh represents the error component of the video frame diagonal field of view in the horizontal direction; k dv represents the error component of the video frame diagonal field of view in the vertical direction.

4. The method of claim 3, wherein: wherein, fov h denotes a horizontal field of view angle of the video frame, fov v denotes a vertical field of view angle of the video frame; a x denotes a horizontal angle error of a target point m to a center point o of the video frame, a y denotes a vertical angle error of a target point m to a center point o of the video frame.

5. The continuous video frame-based concentration monitoring method of claim 1, wherein, The abnormal behavior further comprises yawning, expression abnormality and sitting posture abnormality.

6. The continuous video frame-based concentration monitoring method of claim 1, wherein, The abnormal behavior is analyzed by a corresponding recognition model, the input of the recognition model is a monitoring video, the recognition model is used to analyze the monitoring video into continuous video frames and mark whether each video frame contains the corresponding abnormal behavior, and the recognition model is a neural network model obtained by machine learning.

7. The continuous video frame-based concentration monitoring method of claim 1, wherein, The weight in the concentration score model is an artificial annotation value.

8. A concentration monitoring system based on consecutive video frames, characterized by, A computer program is stored in the storage module, and the processor is connected with the storage module, and the processor is used to execute the computer program to realize the method of claim 1-7.

9. A memory, comprising: A computer program is stored, and the computer program is executed to realize the method of claim 1-7.

Images