An intelligent video surveillance and analysis system based on face recognition

Abstract

The present invention belongs to the technical field of monitoring and analysis, and specifically discloses a smart video monitoring and analysis system based on face recognition, which comprises: a student video monitoring module, a student identity analysis module, a student attendance confirmation module, a student action analysis module, a student score confirmation module, a database and a student information feedback terminal; the present invention analyzes the identity information of each student and confirms the attendance information of each student according to the monitoring video of the student classroom monitoring time period, thereby ensuring the reliability of the subsequent analysis of the student's classroom performance, thereby reducing the difficulty of subsequent teaching management, and at the same time, through the monitoring video of the student movement monitoring time period, analyzing the action standard index of each student, and performing multi-dimensional student teaching effect evaluation from the student face image and the monitoring video, thereby ensuring the effectiveness of the student action standard analysis, thereby improving the timeliness of the discovery of students' potential action problems.

Description

Technical Field

[0001] The present invention belongs to the technical field of monitoring and analysis, and relates to an intelligent video monitoring and analysis system based on face recognition. Background Art

[0002] Video surveillance analysis refers to the extraction of valuable information by processing, identifying and parsing the content of surveillance videos, using related technologies such as image recognition and behavioral analysis. For example, in terms of student attendance, it is possible to accurately determine whether students are present or not based on the facial images of students captured in the video. In terms of sports scoring, it is able to analyze students' postures, the degree of standardization of movements, and the quality of completed movements during exercise, and then give corresponding scores, helping to achieve accurate student management and teaching quality assessment.

[0003] For example, the Chinese invention patent with publication number CN117201742A discloses a video surveillance system for smart classrooms, including: information collection module, information sorting module, information analysis module and adjustment and optimization module. Through different stage analysis, the stage performance values ​​before and after the learning reminder are obtained to intuitively reflect whether the student's classroom learning performance has improved after receiving the learning reminder. At the same time, digging out the performance coefficient corresponding to each classroom information of the student and the stage performance value of each stage can intuitively reflect the long-term tracking of students' performance in the classroom for teachers, and provide teachers with good teaching assistance, which helps to enhance teachers' teaching cognition and feedback ability, improve teaching effectiveness, and better meet students' personalized needs, so that teachers can provide different learning reminders for students with different classroom performances in a timely manner.

[0004] For example, the Chinese invention patent with publication number CN111800646A discloses a method, device, medium and electronic device for monitoring teaching effects, the method comprising: obtaining live video streams of students attending classes sent by multiple acquisition sources. Obtaining the video timestamp of each live video stream based on a preset extraction rule, and extracting the image of the student attending the class from the live video stream based on the video timestamp. Performing facial recognition on the image of the student attending the class to obtain the student's facial image. Inputting the student's facial image into the teaching effect analysis model to obtain the corresponding teaching effect type. The present disclosure obtains the teaching effect type of each student by analyzing the facial images of students within the same time threshold, and then obtains the teaching effect evaluation by counting the teaching effect types. This solves the problem of evaluating the teaching effect of teachers in remote live teaching, can assist live teachers in improving the teaching process, and at the same time help live schools select excellent teachers more objectively.

[0005] The above existing technologies have the following deficiencies: 1. Currently, the performance coefficient corresponding to each classroom information of the students and the stage performance value of each stage are analyzed only by collecting student facial images. Student attendance analysis is not performed based on student facial images, and the accuracy of student attendance information confirmation cannot be guaranteed, which leads to insufficient reliability of subsequent student classroom performance analysis, thereby increasing the difficulty of subsequent teaching management, and at the same time leading to reduced teaching efficiency and waste of teaching resources.

[0006] 2. Currently, the teaching effectiveness evaluation of students is only carried out based on a single facial image dimension, and the students' movement scores are not confirmed based on their movement standardization index. Therefore, the effectiveness of the teaching effectiveness evaluation of students cannot be guaranteed, and students' potential learning problems cannot be discovered. At the same time, the accuracy of the feedback on teaching effectiveness cannot be guaranteed. Summary of the invention

[0007] In view of this, in order to solve the problems raised in the above background technology, an intelligent video surveillance analysis system based on face recognition is proposed.

[0008] The purpose of the present invention can be achieved through the following technical solutions: The present invention provides an intelligent video monitoring and analysis system based on face recognition, including: a student video monitoring module, used to monitor students' monitoring videos.

[0009] The student identity analysis module is used to extract the surveillance videos of students during the classroom monitoring period based on the students' surveillance videos and analyze the identity information of each student.

[0010] The student attendance confirmation module is used to confirm the student's attendance information based on the identity information of each student.

[0011] The student movement analysis module is used to extract the surveillance videos of the students' movement monitoring time period based on the students' surveillance videos, and then analyze the movement standardization index of each student.

[0012] The student scoring confirmation module is used to confirm the action scoring coefficient of each student based on the action standard index of each student.

[0013] The database is used to store the student face image set, store the identity information of each student, and store the action standard index of each student in each historical movement.

[0014] The student information feedback terminal is used to provide corresponding feedback based on the student's attendance information and the action scoring coefficient of each student.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) The present invention analyzes the identity information of each student based on the surveillance video of the student classroom monitoring period, and then confirms the student's attendance information, thereby avoiding the current deficiency of not performing student attendance analysis based on student facial images, ensuring the accuracy of student attendance information confirmation, and then ensuring the reliability of subsequent student classroom performance analysis, thereby reducing the difficulty of subsequent teaching management, and at the same time achieving improved teaching efficiency and rational use of teaching resources.

[0016] (2) The present invention analyzes the movement standard index of each student based on the surveillance video of the student movement monitoring period, breaking the current deficiency of not confirming the student movement score based on the student movement standard index. It conducts multi-dimensional student teaching effect evaluation based on the student face image and surveillance video, thereby ensuring the effectiveness of the student movement standard analysis, thereby improving the timeliness of discovering students' potential movement problems and ensuring the accuracy of movement score feedback.

[0017] (3) The present invention reduces the interference in the confirmation of the identity information of students with complete face images by first processing the identity information confirmation of students with complete face images, and then confirming the identity information of students with occluded face images, thereby improving the accuracy of the confirmation of the identity information of students with occluded face images, thereby improving the positioning accuracy of the identity information of students with occluded face images, and at the same time improving the efficiency of student identity matching.

[0018] (4) The present invention improves the comprehensiveness of the student's action scoring by comprehensively confirming the student's action scoring coefficient based on the student's action standard index and action standard improvement index, thereby improving the pertinence of the student's action scoring and ensuring the effectiveness of the student's action scoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for describing the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0020] Figure 1 It is a schematic diagram of the connection of each module of the system of the present invention.

[0021] Figure 2 It is a connection diagram of the identity information confirmation step of the present invention.

[0022] Figure 3 It is a connection diagram of the action standard index analysis steps of the present invention. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] See also Figure 1 As shown, the present invention provides an intelligent video surveillance and analysis system based on face recognition, which includes: a student video monitoring module, a student identity analysis module, a student attendance confirmation module, a student action analysis module, a student scoring confirmation module, a database and a student information feedback terminal.

[0025] In the above, the student identity analysis module is connected to the student video monitoring module, the student attendance confirmation module and the database respectively, the student attendance confirmation module is connected to the database and the student information feedback terminal respectively, the student action analysis module is connected to the student video monitoring module and the student scoring confirmation module respectively, and the student scoring confirmation module is also connected to the database and the student information feedback terminal respectively.

[0026] The student video monitoring module is used to monitor the surveillance videos of students.

[0027] It should be added that the surveillance videos of students were captured by surveillance cameras installed in the student activity area.

[0028] The student identity analysis module is used to extract the surveillance videos of students during the classroom monitoring period based on the surveillance videos of the students and analyze the identity information of each student.

[0029] It should be added that the extraction process of the surveillance video of the student classroom monitoring time period is as follows: the class time of the first class of each class is obtained from the school's academic affairs system, and then the class time of the first class of the class is obtained, and the corresponding course schedule information of the class is extracted from the school's academic affairs system, and the class length information of the class is obtained therefrom. In the school's academic affairs system, the class length is generally standardized. For example, each class may be 40 minutes, 45 minutes or other fixed durations. The time range corresponding to each class of the class is determined by the above-obtained first class class time point and class length. For example, if the first class of a class starts at 8:00 and the class length is 45 minutes, the student surveillance video between 8:00 and 8:45 will be used as the student classroom monitoring time period surveillance video of the class.

[0030] See also Figure 2As shown, exemplarily, the analysis of the identity information of each student includes: F1, extracting the facial image of each student at each monitoring time point from the surveillance video of the student classroom monitoring time period, and counting the completeness of the facial image of each student at each monitoring time point.

[0031] Furthermore, the statistics of the completeness of the facial image of each student at each monitoring time point include: F1-1, extracting the eye area and the unobstructed eye area in the facial image of each student at each monitoring time point from the facial image of each student at each monitoring time point.

[0032] It should be added that the area of ​​the eye region is obtained by using advanced facial feature point detection technology to locate key feature points such as the inner canthus point, outer canthus point, eyebrow edge point, eyelid edge point, eye socket peripheral point and cheekbone near point from the face image, and then the area of ​​an irregular polygonal region is delineated based on this, and it is used as the area of ​​the eye region. For example, the upper boundary is usually determined by the curve formed by connecting the lower edge points of the eyebrows, the lower boundary is based on the lower edge point of the eyelid and extends downward for a certain distance, the left boundary and the right boundary start from the inner canthus point and extend outward along the eye socket peripheral points, cheekbone near points, etc., and then the area value of the region is calculated by the polygonal area calculation method, which is used as the area of ​​the eye region.

[0033] F1-2, the ratio of the unobstructed eye area to the eye area in the face image of a student at a certain monitoring time point is taken as the complete eye area ratio of the face image of the student at that monitoring time point, denoted as .

[0034] F1-3, according to The statistical method is similar to that of obtaining the nose area complete ratio, mouth area complete ratio, forehead area complete ratio and cheek area complete ratio of the student's face image at the monitoring time point, and they are recorded as , , and .

[0035] F1-4: Count the completeness of the student’s face image at the monitoring time point ,

[0036] , , , , and The weights of the eye area complete ratio, nose area complete ratio, mouth area complete ratio, forehead area complete ratio and cheek area complete ratio are set respectively. , , and then obtain the completeness of each student’s face image at each monitoring time point.

[0037] It should be added that the eyes are one of the most important features in face recognition. The eyes are highly unique, including eye shape, eye distance, pupil size and shape, etc. In face recognition algorithms, the features of the eyes play a vital role in distinguishing different individuals. Even if other facial areas are blocked or incomplete to a certain extent, as long as the eye area can be clearly identified, accurate identity recognition can still be performed to a certain extent. The nose is also a significant feature of the face. The nose shapes of different people vary greatly, such as the height of the nose bridge, the width of the nose wings, etc. The nose is in the center of the face, and its shape and structure have an important influence on the overall contour and features of the face. Although its importance is slightly lower than that of the eyes, it plays a greater role in assisting identity recognition. The features of the mouth area It also has certain value in face recognition. The shape, thickness of lips and arrangement of teeth have certain individual differences. However, compared with eyes and nose, the mouth will have greater morphological changes when the expression changes, which reduces its weight in static face recognition to a certain extent. The forehead area is relatively flat. The differences between individuals are mainly reflected in the height of the hairline and the width of the forehead. These features are not as distinguishable as eyes, nose and mouth in face recognition, so the weight is relatively low. The cheek area is mainly the soft tissue part of the face. The differences between individuals are relatively small. It is mainly the fatness of the face that affects the appearance of the cheek. The uniqueness and importance of the cheek area in face recognition are low, so the weight is the smallest, so it is set , in order to facilitate analysis, The specific value can be 0.3. The specific value can be 0.25. The specific value can be 0.2. The specific value can be 0.15. The specific value can be 0.1.

[0038] In a specific embodiment, it is assumed that the weight of the eye region completeness ratio is 0.3, the weight of the nose region completeness ratio is 0.25, the weight of the mouth region completeness ratio is 0.2, the weight of the forehead region completeness ratio is 0.15, and the weight of the cheek region completeness ratio is 0.1.

[0039] If the area of ​​the unobstructed eyes in a student's face image is , the eye area is , then the eye area complete ratio is .

[0040] If the area of ​​the unobstructed nose in the student's face image is , the nose area is , then the complete ratio of the nose area is .

[0041] If the area of ​​the unobstructed mouth in the student's face image is The mouth area is , then the mouth area complete ratio is .

[0042] If the unobstructed forehead area in the student's face image is , the forehead area is , then the forehead area complete ratio is .

[0043] If the unobstructed cheek area in the student's face image is The cheek area is , then the cheek area complete ratio is .

[0044] The completeness of the student's face image at the monitoring time point is

[0045] .

[0046] F2. Filter out the maximum value from the facial image completeness of each student at each monitoring time point as the facial image completeness of each student, and compare it with the facial image completeness of the set reference.

[0047] F3. Record each student whose facial image completeness is greater than or equal to the set reference facial image completeness as a student with a complete facial image, and then confirm the identity information of each student with a complete facial image.

[0048] Furthermore, the confirmation of the identity information of the complete students of each facial image includes: F3-1, extracting a set of stored student facial images from a database.

[0049] F3-2. Match the facial images of each student with a complete facial image with the student stored facial image set to obtain the matching degree between the facial images of each student with a complete facial image and each student stored facial image, and select the maximum matching degree from them, and then match the student stored facial image corresponding to the maximum matching degree with its corresponding complete facial image student to obtain the student stored facial image corresponding to each complete facial image student.

[0050] F3-3. Extract the stored identity information of each student from the database, and then extract the identity information of each student's stored facial image, so as to obtain the identity information of each student with a complete facial image based on the stored facial image of the student corresponding to each student with a complete facial image.

[0051] F4. Record each student whose face image integrity is less than the set reference face image integrity as a face image occluded student.

[0052] F5. The stored face images in the student stored face image set that have not been successfully matched are combined into a stored face image set of students to be matched. Then, according to the students blocked by each face image and the stored face image set of students to be matched, the identity information of each blocked student is obtained in the same way as the complete student identity information of each face image.

[0053] It should be added that the stored facial images that were not successfully matched refer to the stored facial images that failed to match the facial images in the process of matching the facial images of each complete student with the student stored facial image set, as the stored facial images that were not successfully matched.

[0054] The embodiment of the present invention reduces the interference in confirming the identity information of students with complete face images by first processing the identity information confirmation of students with complete face images, and then confirming the identity information of students with occluded face images, thereby improving the accuracy of confirming the identity information of students with occluded face images, thereby improving the positioning accuracy of the identity information of students with occluded face images, and at the same time improving the efficiency of student identity matching.

[0055] The student attendance confirmation module is used to confirm the student's attendance information based on the identity information of each student.

[0056] Exemplarily, the confirmation of the student's attendance information includes: extracting each student's facial image monitoring time point from the surveillance video of the student's classroom monitoring time period, and then extracting each student's first facial image monitoring time point therefrom, and comparing it with the set reference monitoring time point.

[0057] If the time point of the first facial image monitoring of a student is greater than the set reference monitoring time point, the student will be recorded as a late student. Otherwise, the student will be recorded as a student with qualified attendance, so as to obtain all late students and all qualified attendance students.

[0058] According to the identity information of each student with a complete face image and each student with an obstructed face image, the stored identity information of each remaining student is filtered out from the stored identity information of each student, and then the remaining students are regarded as absent students.

[0059] Each absent student, each late student and each qualified student is taken as the student's attendance information.

[0060] The embodiment of the present invention analyzes the identity information of each student based on the surveillance video of the student classroom monitoring time period, and then confirms the student's attendance information, thereby avoiding the deficiency of the current attendance analysis of students not being performed based on student facial images, ensuring the accuracy of the confirmation of student attendance information, and then ensuring the reliability of subsequent analysis of student classroom performance, thereby reducing the difficulty of subsequent teaching management, and at the same time achieving improved teaching efficiency and rational use of teaching resources.

[0061] The student movement analysis module is used to extract the monitoring videos of the students' movement monitoring time period based on the students' monitoring videos, and then analyze the movement standard index of each student.

[0062] It should be added that the process of extracting surveillance videos during the student movement monitoring period is as follows: taking eye exercises as an example, the school's schedule is obtained from the school's academic affairs system, and then the fixed time period for eye exercises is obtained from it. If the fixed time for eye exercises is from 10:00 to 10:10, then the student surveillance videos between 10:00 and 10:10 are extracted from the student surveillance videos as the student movement monitoring period surveillance videos.

[0063] See also Figure 3 As shown, exemplarily, the analysis of the movement standard index of each student includes: G1, extracting the position coordinates of each body joint point of each student in each movement from the monitoring video of the student movement monitoring time period, and then calculating the qualified index of each student's body joint point in each movement , For the student number, , is the action number, .

[0064] Furthermore, the statistical evaluation of the qualified index of each student's body joints in each action includes: G1-1, comparing the position coordinates of each student's body joints in each action with the set reference position coordinates through the Euclidean distance formula to obtain the position coordinate deviation of each student's body joints in each action, and comparing it with the set allowable position coordinate deviation.

[0065] G1-2. If the position coordinate deviation of a certain body joint point of a student in a certain action is less than or equal to the set allowable position coordinate deviation, the body joint point of the student in the action is recorded as a qualified body joint point.

[0066] G1-3. Count the number of body joints and body joints of the student in the action, and then use the ratio of the number of body joints in the position to the number of body joints as the body joints qualification index of the student in the action, and then obtain the body joints qualification index of each student in each action, recorded as .

[0067] G2. Extract the limb angles of each student in each action from the surveillance video during the student movement monitoring period, and then calculate the qualified index of the limb angles of each student in each action .

[0068] It should be added that the limb angle refers to the angle formed between the various limbs of the body (such as arms, fingers, head, neck, etc.) during human movement. Taking the standard limb angles of each movement in eye exercises as an example, rubbing the Tianying acupoint: press the spiral surface of the thumbs of both hands on the Tianying acupoint below the eyebrows, and the other fingers are naturally relaxed with the fingertips against the forehead. The standard limb angle is: the angle between the thumb and index finger is about 120-150 degrees, the angle between the wrist and forearm is close to 180 degrees, and the head and neck are basically kept vertical, that is, the angle between the head and neck is about 90 degrees. Squeeze and press the Jingming acupoint: use the thumb and index finger of one hand to gently pinch the Jingming acupoint on both sides of the bridge of the nose, and perform a squeezing and pressing action. The standard limb angle is: the angle between the thumb and index finger will change during the operation. It is about 30-40 degrees at the beginning of squeezing and pressing. As the squeezing and pressing action proceeds, the angle may decrease. To about 20-30 degrees, the angle between the arm and the body is approximately 45-60 degrees, press and rub the Sibai acupoint: press the spiral surface of the index fingers of both hands on the Sibai acupoint in the center of the cheek, and perform circular massage. The standard limb angle is: the angle between the index finger and the middle finger is about 30-40 degrees, the wrist will rotate slightly during massage, and the angle between the wrist and the palm will change periodically between 0-30 degrees, the head remains upright, the angle between the cheek and the shoulder is approximately 90 degrees, press the temple and scrape the eye socket: press the spiral surface of the thumbs of both hands on the temple, and scrape the eye socket with the inner side of the second section of the index finger, the standard limb angle is: the angle between the thumb and the index finger will change greatly when scraping the eye socket, the starting angle is about 90 degrees, with the scraping action of the index finger, the angle may first decrease to 45-60 degrees, and then return to about 90 degrees, the angle between the arm and the shoulder is approximately 60-90 degrees.

[0069] Furthermore, the statistical calculation of the qualified index of the limb angle of each student in each action includes: G2-1, comparing each limb angle of each student in each action with the permitted range of limb angle set as a reference.

[0070] G2-2. If a student's limb angle in a certain movement is not within the permitted range of the limb angle set as a reference, the limb angle will be recorded as an abnormal limb angle.

[0071] G2-3. Count the number of abnormal limb angles and limb angles of the student in the action, and record them as and .

[0072] G2-4. Calculate the student's limb angle pass index for the action , , and then the qualified index of the limb angle of each student in each action is obtained and recorded as .

[0073] G3. Extract the movement range of each student in each action from the surveillance video during the student movement monitoring period, and then calculate the movement range qualification index of each student in each action .

[0074] Furthermore, the statistical analysis of the qualified index of the movement range of each student in each movement includes: G3-1, comparing the movement range of each student in each movement with the set allowed fluctuation range of the movement range.

[0075] G3-2. If the movement amplitude of a student in a certain action is greater than the upper limit of the allowed fluctuation range of the movement amplitude, the ratio of the set upper limit of the allowed fluctuation range of the movement amplitude to the movement amplitude shall be used as the student's movement amplitude qualification index for the action.

[0076] G3-3. If the student's movement range in a certain action is within the set allowable fluctuation range of the movement range, 1 will be used as the student's movement range qualification index for the action.

[0077] G3-4. If the student's movement amplitude in a certain action is less than the lower limit of the allowed fluctuation range of the movement amplitude, the ratio of the movement amplitude to the set lower limit of the allowed fluctuation range of the movement amplitude is taken as the student's movement amplitude qualification index for the action, and then the movement amplitude qualification index of each student in each action is obtained and recorded as .

[0078] G4. Extract the duration of each student's action from the surveillance video of the student's movement monitoring period, and then obtain the qualified index of each student's action duration in each action in the same way as the statistical method of the qualified index of the action amplitude of each student in each action, and record it as .

[0079] G5. Import the qualified index of body joints, limb angles, movement range and duration of each student in each action into the action standard index analysis model to obtain the action standard index of each student in each action, and record it as .

[0080] It should be added that the analysis formula of the action norm index analysis model is:

[0081] , , , and are the weights of the set body joint qualification index, limb angle qualification index, movement range qualification index and movement duration qualification index, respectively.

[0082] , .

[0083] It should be added that the correct movement of joints is also crucial to the continuity and fluency of movements. Taking eye exercises as an example, the coordinated movement of body joints is the core element to achieve qualified eye exercises. A high qualified index of joints means that the basic quality of the movement is guaranteed, so it is given the highest weight in the movement standard index. Correct limb angles help to exert the strength of various parts of the body and ensure the efficiency and effectiveness of the movement. However, compared with the body joints, the limb angles further refine the accuracy of the movement on the basis of the correct joints, so its weight is slightly lower than the qualified index of body joints. The movement amplitude reflects the completeness of the movement and is an important aspect of movement standardization. The qualification of the movement amplitude depends to a certain extent on the correct movement of the joints and the reasonable control of the limb angles. If the joint position is wrong or the limb angle is inappropriate, the movement amplitude may not meet the requirements. The duration of the movement is mainly used to measure the accuracy of the movement in the time dimension. In some movements with rhythm requirements, the accuracy of the duration of the movement can enhance the rhythm and expressiveness of the movement. However, compared with the previous three indicators, the qualification of the duration of the movement has a relatively small impact on the basic form, accuracy and completeness of the movement, so it is set , in order to facilitate analysis, The specific value can be 0.4. The specific value can be 0.3. The specific value can be 0.2. The specific value can be 0.1.

[0084] G6. Calculate the average of the action standardization index of each student in each action, and use the calculation result as the action standardization index of each student, denoted as .

[0085] The embodiment of the present invention analyzes the movement standard index of each student based on the surveillance video of the student movement monitoring time period, breaking the current deficiency of not confirming the student movement score based on the student movement standard index, and conducts multi-dimensional student teaching effect evaluation from the student facial image and surveillance video, thereby ensuring the effectiveness of the student movement standard analysis, thereby improving the timeliness of the discovery of students' potential movement problems, and ensuring the accuracy of the feedback of the movement score.

[0086] The student scoring confirmation module is used to confirm the action scoring coefficient of each student according to the action standard index of each student.

[0087] Exemplarily, the step of determining the action scoring coefficient of each student includes: extracting the action standard index of each student in each historical exercise from the database, and then combining it with the action standard index of each student to form the action standard index of each student in each exercise, and recording it as , For the order of movement, .

[0088] The action standard index of each student's first exercise is extracted from the action standard index of each student in each historical exercise, and recorded as .

[0089] Calculate the movement standard progress index of each student , , For the Students in The movement standardization index of the sub-movement.

[0090] It should be added that It is extracted from the movement standard index of each student in each exercise.

[0091] Calculate the action score coefficient of each student , , and are the weights of the set action standard index and action standard improvement index, , .

[0092] It should be added that the action standardization index mainly measures whether the student's movements meet the requirements of the standard movements in terms of form and amplitude. For example, in eye exercises, the action standardization index can judge whether the student's limbs are stretched in place, the body posture is correct, the movement rhythm is accurate, etc. This is the most direct assessment of the student's current movement quality and represents the degree to which the student has actually mastered the movement. The action standardization progress index is used to measure the student's progress relative to his or her previous performance. It takes into account the different starting points of individual students and has a certain incentive effect on those students with poor foundations but who are working hard to improve. However, it is based on the comparison of the students themselves and cannot replace the action standardization index as the main basis for scoring. For example, a student's initial action standardization index is only 30%, and after a period of time it increases to 40%. The progress index is high, but the actual level of action standardization is still low, so it is set. , in order to facilitate analysis, The specific value can be 0.7. The specific value can be 0.3.

[0093] The embodiment of the present invention improves the comprehensiveness of the student's action scoring by comprehensively confirming the student's action scoring coefficient based on the student's action standard index and action standard improvement index, thereby improving the pertinence of the student's action scoring and ensuring the effectiveness of the student's action scoring.

[0094] The database is used to store a set of student face images, store identity information of each student, and store the action standard index of each student in each historical exercise.

[0095] The student information feedback terminal is used to provide corresponding feedback based on the student's attendance information and the action scoring coefficient of each student.

[0096] The above contents are merely examples and explanations of the concept of the present invention. Those skilled in the art may make various modifications or additions to the specific embodiments described or replace them in a similar manner. As long as they do not deviate from the concept of the invention or exceed the scope defined by the present invention, they shall all fall within the protection scope of the present invention.

Claims

1. An intelligent video surveillance and analysis system based on face recognition, characterized in that: The system includes: Student video monitoring module, monitoring students’ surveillance videos; The student identity analysis module extracts the surveillance videos of students during the classroom monitoring period from the surveillance videos of students and analyzes the identity information of each student; The student attendance confirmation module confirms the student's attendance information based on the identity information of each student; The student movement analysis module analyzes the qualified index of each student's body joints according to the position coordinates, limb angles, movement amplitude and movement duration of each student's monitoring video. , Limb Angle Qualification Index , Movement Range Qualification Index and action duration qualification index ,in For the student number, , is the action number, ;Will , , and Import the action standard index analysis model to calculate each action standard index , the action standard index of each action The mean value was calculated as the action standard index of each student ; The student scoring confirmation module compares the action standard index of each student in each historical movement with the The action standard index of each student in each exercise is constituted and recorded as , For the order of movement, ; and combined with the movement standard index of each student's first exercise Calculate each student's movement standard progress index , , For the Students in The action standard index of the exercise; statistics of each student's action score coefficient , , and are the weights of the set action standard index and action standard improvement index, , ; A database storing a set of student face images, storing identity information of each student, and storing a movement standard index of each student in each historical movement; The student information feedback terminal provides corresponding feedback based on the student's attendance information and each student's action scoring coefficient.

2. The intelligent video surveillance and analysis system based on face recognition according to claim 1 is characterized in that: The analysis of the identity information of each student includes: F1. Extract the facial images of each student at each monitoring time point from the surveillance video of the student classroom monitoring period, and count the completeness of the facial images of each student at each monitoring time point; F2. Filter out the maximum value from the facial image completeness of each student at each monitoring time point as the facial image completeness of each student, and compare it with the facial image completeness of the set reference; F3. Record each student whose face image integrity is greater than or equal to the set reference face image integrity as a student with a complete face image, and then confirm the identity information of each student with a complete face image; F4, recording each student whose face image integrity is less than the set reference face image integrity as a face image occluded student; F5. The stored face images in the student stored face image set that have not been successfully matched are combined into a stored face image set of students to be matched. Then, according to the students blocked by each face image and the stored face image set of students to be matched, the identity information of each blocked student is obtained in the same way as the complete student identity information of each face image.

3. The intelligent video surveillance and analysis system based on face recognition according to claim 2 is characterized in that: The counting of the completeness of the facial images of each student at each monitoring time point includes: Extracting the eye area and the unobstructed eye area in the face image of each student at each monitoring time point from the face image of each student at each monitoring time point; The ratio of the unobstructed eye area to the eye area in the face image of a student at a certain monitoring time point is taken as the complete eye area ratio of the face image of the student at that monitoring time point, denoted as ; according to The statistical method is similar to that of obtaining the nose area complete ratio, mouth area complete ratio, forehead area complete ratio and cheek area complete ratio of the student's face image at the monitoring time point, and they are recorded as , , and ; Count the completeness of the student's face image at the monitoring time point , , , , , and The weights of the eye area complete ratio, nose area complete ratio, mouth area complete ratio, forehead area complete ratio and cheek area complete ratio are set respectively. , , and then obtain the completeness of each student’s face image at each monitoring time point.

4. The intelligent video surveillance and analysis system based on face recognition according to claim 2 is characterized in that: The confirmation of the identity information of each student whose face image is complete includes: Extracting a student stored face image set from a database; Match the facial images of each student with a complete facial image with the student stored facial image set to obtain the matching degree between the facial images of each student with a complete facial image and each student stored facial image, and select the maximum matching degree from them, and then match the student stored facial image corresponding to the maximum matching degree with the corresponding student with a complete facial image to obtain the student stored facial image corresponding to each student with a complete facial image; The stored identity information of each student is extracted from the database, and then the identity information of each student's stored face image is extracted, so as to obtain the identity information of each student with a complete face image according to the stored face image of the student corresponding to each student with a complete face image.

5. The intelligent video surveillance and analysis system based on face recognition according to claim 4 is characterized in that: The confirmed student attendance information includes: Extracting the facial image monitoring time points of each student from the surveillance video during the classroom monitoring period, and then extracting the first facial image monitoring time point of each student, and comparing it with the set reference monitoring time point; If the time point of the first facial image monitoring of a student is greater than the set reference monitoring time point, the student will be recorded as a late student. Otherwise, the student will be recorded as a qualified student. In this way, the late students and qualified students are obtained. According to the identity information of each student with a complete face image and each student with an obstructed face image, the stored identity information of each remaining student is filtered out from the stored identity information of each student, and then the remaining students are regarded as absent students; Each absent student, each late student and each qualified student is taken as the student's attendance information.

6. The intelligent video surveillance and analysis system based on face recognition according to claim 1, characterized in that: The statistical evaluation of the passing index of each student's body joints in each action includes: The position coordinates of each body joint point of each student in each action are compared with the set reference position coordinates through the Euclidean distance formula to obtain the position coordinate deviation of each body joint point of each student in each action, and then the deviation is compared with the set allowable position coordinate deviation; If the position coordinate deviation of a certain body joint point of a student in a certain action is less than or equal to the set permissible position coordinate deviation, the body joint point of the student in the action is recorded as a position qualified body joint point; The number of the student's qualified body joints and body joints in the action are counted, and then the ratio of the number of qualified body joints in the position to the number of body joints is used as the qualified index of the student's body joints in the action, and then the qualified index of the body joints of each student in each action is obtained, which is recorded as .

7. The intelligent video surveillance and analysis system based on face recognition according to claim 1 is characterized in that: The statistical evaluation of the qualified index of the limb angles of each student in each action includes: Compare the limb angles of each student in each action with the permitted limb angle range of the set reference; If a student's limb angle in a certain action is not within the permitted range of the limb angle set as a reference, the limb angle will be recorded as an abnormal limb angle; Count the number of abnormal limb angles and limb angles of the student in this action and record them as and ; Calculate the student's limb angle pass index for this action , , and then the qualified index of the limb angle of each student in each action is obtained and recorded as .

8. The intelligent video surveillance and analysis system based on face recognition according to claim 1 is characterized in that: The statistical evaluation of the qualified index of the movement range of each student in each movement includes: Compare the range of motion of each student in each action with the set range of permissible fluctuation of the range of motion; If the movement range of a student in a certain action is greater than the upper limit of the allowed fluctuation range of the movement range set by the student, the ratio of the set upper limit of the allowed fluctuation range of the movement range to the movement range is used as the movement range qualification index of the student in the action; If the student's movement range in a certain action is within the set range of movement range allowance, 1 is used as the student's movement range pass index for the action; If the student's movement amplitude in a certain action is less than the set lower limit of the movement amplitude allowable fluctuation range, the ratio of the movement amplitude to the set lower limit of the movement amplitude allowable fluctuation range is taken as the student's movement amplitude qualification index in the action, and then the movement amplitude qualification index of each student in each action is obtained and recorded as .

Images