Student classroom learning state intelligent supervision system and method based on deep learning

Abstract

The invention relates to the technical field of student classroom management, and discloses a student classroom learning state intelligent supervision system and method based on deep learning, and the system comprises a student classroom seat management module, a student classroom attendance management module, and a student classroom learning state management module. Based on the current classroom seat student identity recognition information, in combination with a deep learning text search algorithm and student classroom seat regulation student identity information based on big data storage, accurate recognition feedback of the current student classroom seat seating number abnormal state is carried out, and intelligent visual monitoring feedback of the student classroom seat seating number abnormal state is realized; the student attendance state of the current student classroom is dynamically monitored based on the current student classroom seat feature image information in combination with an intelligent image search algorithm and the standard stored student classroom seat stipulated student facial feature image information, and the classroom learning effect is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of student classroom management, and specifically to a system and method for intelligently supervising student classroom learning status based on deep learning. Background Art

[0002] The student classroom learning status monitoring system uses cameras and artificial intelligence technology to monitor students' classroom behavior in real time. Through image recognition and behavior analysis, the student classroom learning status monitoring system evaluates students' facial expressions, whether they whisper, classroom participation and interaction quality, and provides corresponding feedback and suggestions; however, the existing student classroom learning status monitoring system cannot realize intelligent monitoring of student seats and attendance, nor can it visually monitor students' classroom learning status.

[0003] A Chinese invention patent application, publication number CN104732465A, dated June 24, 2015, discloses a method for monitoring a student's learning status. The method detects abnormal heart rates corresponding to a user based on preset course information and issues a warning to a connected mobile terminal. Users are associated with other users through preset class information, and heart rate data is associated with historical heart rate data through preset course information. This method allows teachers and parents to monitor students' learning status at all times, promptly identifying any resistance to certain courses or teachers, and, through communication and education, prevent students from developing biased learning. However, the aforementioned technical solution fails to intelligently monitor student seating and attendance in class, reducing the applicability and efficiency of monitoring students' learning status in class. Summary of the Invention

[0004] (1) Technical problems solved

[0005] In order to solve the problem that some student classroom learning status supervision systems cannot realize intelligent monitoring of student seats and attendance, and cannot visually monitor student classroom learning status, the above purpose is to accurately analyze the abnormal status of student classroom seating, intelligently count the abnormal status of student attendance, and visually display the abnormal learning status of students in class.

[0006] (2) Technical solution

[0007] The present invention is implemented through the following technical solution: a method for intelligently supervising students' classroom learning status based on deep learning, the method comprising the following steps:

[0008] S1, collect the characteristic image data of the current students' classroom seats;

[0009] S2, performing student identity information recognition processing on the current student's seat based on the current student's classroom seat feature image data and the student identity feature text data corresponding to the classroom student's facial image, to generate student identity recognition data for the current student's seat;

[0010] S3, performing an abnormal seating state analysis process for the current student class based on the student identity identification data of the current class seats and the student identity feature text data of the class seats, generating and outputting abnormal seating state analysis data for the current class seats;

[0011] S4, performing student attendance status recognition processing for the current student class based on the current student class seat feature image data and the student class seat prescribed student facial feature image data, to generate current class student attendance status recognition data;

[0012] S5, performing abnormal attendance status student identity information recognition processing for the current student class based on the current class student attendance status recognition data and the student identity feature text data of the student class seat specification, generating and outputting abnormal attendance status student identity recognition data for the current class;

[0013] S6. Performing an abnormal learning state recognition process on the current student in the classroom based on the current student's classroom seat feature image data and the abnormal learning state feature image data of the current student in the classroom to generate abnormal learning state recognition data of the current student in the classroom;

[0014] S7. Perform student identity information identification processing on the abnormal learning status of the current student in the classroom based on the abnormal learning status identification data of the current student in the classroom and the student identity feature text data of the student classroom seat regulations, generate and output the abnormal learning status student identity identification data of the current student in the classroom.

[0015] Preferably, the steps for collecting the characteristic image data of the current student's classroom seat are as follows:

[0016] S11, collect the real-time state feature image information of each student seat in the classroom through the cloud camera online, and generate the current student classroom seat feature image data set P = (p1, ..., p u ,…,p α ), u=1,2,3,…,α; where p u It represents the current student classroom seat feature image data corresponding to the u-th student classroom seat collected, α represents the maximum number of student classroom seats; the current student classroom seat feature image data represents the latest status feature image information of the student classroom seats.

[0017] Preferably, the student identity information of the current student's classroom seat is identified based on the characteristic image data of the current student's classroom seat and the student identity feature text data corresponding to the classroom student's facial image, and the operation steps for generating the student identity identification data of the current classroom seat are as follows:

[0018] S21, establish a text data set K = (k1, ..., k v ,…,k β ), v=1,2,3,…,β; where k v The student facial image corresponding to the vth student in the class corresponds to the student identity feature text data, and β represents the maximum number of students in the class; the student facial image corresponding to the student identity feature text data represents a combination of the facial image information and the student identity feature information of the class student; the student identity feature information includes the student's name, student ID number, grade information, and ID number;

[0019] S22, using the FLANN image search algorithm to search the current student classroom seat feature image data set P for the current student classroom seat feature image data set P. u The student identity feature text data set K corresponding to the student facial images in the classroom is ordered according to the student classroom seat number, and the student identity feature text data set k corresponding to the student facial images in the classroom is ordered according to the student classroom seat number. v Perform image feature matching and generate the student identity recognition data set P'=(p'1,…,p' u ,…,p' α ), where p' u The student identity data of the current classroom seat corresponding to the u-th student's classroom seat;

[0020] When p u With k v If the image feature matching is successful, it means that the student sitting in the current u-th classroom seat is the v-th classroom student, then the student identity recognition data p' of the current classroom seat is output. u The identity characteristics text information of the v-th class student;

[0021] When p u With k v If the image features are not matched successfully, it means that there is no student object sitting in the current u-th student classroom seat, then the student identity recognition data p' of the current classroom seat is output. u There are no student objects.

[0022] Preferably, the steps of performing abnormal seat status analysis processing of the current student class based on the current class seat student identity identification data and the student class seat specification student identity feature text data, generating and outputting the abnormal seat status analysis data of the current class seat are as follows:

[0023] S31. Establish student classroom seat regulations and student identity feature text data set E = (e1,…,e u ,…,e α ), where e u The student classroom seat specified student identity feature text data corresponding to the u-th student classroom seat, wherein the student classroom seat specified student identity feature text data represents the identity feature information of the student object determined by the student classroom seat according to school regulations;

[0024] S32, using the KD tree nearest neighbor search algorithm to find the current classroom seat student identity recognition data set P' of the current classroom seat student identity recognition data set P' u The student identity feature text data set E of the student classroom seat specification is ordered according to the student classroom seat number and the student identity feature text data set E of the student classroom seat specification. u Perform keyword matching of student identity information, and generate the current classroom seat abnormality analysis data set E'=(e'1,…,e' u ,…,e' α ), where e' u Abnormal status analysis data of the current classroom seating arrangement corresponding to the u-th student's classroom seat;

[0025] When p' u With e u If the student identity information keyword matching is successful, it means that the current class seat of the u-th student is seated in accordance with the school regulations, and the abnormal state analysis data of the current class seat is outputted. u The seating arrangement is normal;

[0026] When p' u With e u If the student identity information keyword is not matched successfully, it means that the current class seat of the u-th student is not seated in accordance with the school regulations, then the abnormal state analysis data of the current class seat is outputted. u The seating arrangement is abnormal;

[0027] S33, the current classroom seat arrangement abnormal state analysis data set E' is generated. u The information is output online in order through the student classroom multimedia display screen according to the student's seat number.

[0028] Preferably, the student attendance status recognition process of the current student class is performed based on the current student class seat feature image data and the student class seat prescribed student facial feature image data, and the operation steps of generating the current class student attendance status recognition data are as follows:

[0029] S41. Establish student classroom seats and student facial feature image data set F = (f1, ..., f u ,…,f α ), where f u The student classroom seat prescribed student facial feature image data corresponding to the u-th student classroom seat represents the facial feature image information of the student object determined by the student classroom seat according to school regulations;

[0030] S42, using SURF image search algorithm to search the current student classroom seat feature image data set P for the current student classroom seat feature image data p u The student facial feature image data set F of the student classroom seat regulations are ordered according to the student classroom seat numbers and the student facial feature image data set F of the student classroom seat regulations. u Perform image feature matching and generate the current class student attendance status recognition data set F'=(f1',…,f u ',…,f' α ), where f u 'Indicates the current class student attendance status identification data corresponding to the u-th student's classroom seat;

[0031] When p u With f u The image feature matching is successful, indicating that the current student u is seated in the classroom and the student object has normal attendance, then the current classroom student attendance status identification data p' is output. u For normal attendance;

[0032] When p u With f u If the image feature matching fails, it means that the student object at the current class seat u is absent, then the current class student attendance status identification data p' is output. u For absence.

[0033] Preferably, the steps of performing abnormal attendance status student identity information recognition processing for the current student class based on the current class student attendance status recognition data and the student identity feature text data of the student class seat regulations, and generating and outputting the abnormal attendance status student identity recognition data for the current class are as follows:

[0034] S51, using the Boyer-Moore search algorithm to search the current class student attendance status identification data set F' for the current class student attendance status identification data f u 'The student class seat provisions student identity feature text data set E in the student class seat provisions student identity feature text data e u Match the student's classroom seat number and search for the current class student attendance status identification data f u 'Specify the student identity feature text data e for the student class seat corresponding to the absent student class seat u , and generate the current class attendance abnormal state student identification data set E"=(e" u1 ,…,e” u2 ), 1≤u1≤u≤u2≤α, where e” u1 and e” u2 Respectively represent the student identity identification data of the current class attendance abnormal status corresponding to the class seats of the u1th and u2th students;

[0035] S52, the current class attendance abnormal state student identity identification data set E" is generated. u1 to e” u2 The information is output online in order through the student classroom multimedia display screen according to the student's seat number.

[0036] Preferably, the abnormal learning state identification processing of the current student class is performed based on the current student class seat feature image data and the abnormal learning state feature image data of the current student class, and the operation steps of generating the abnormal learning state identification data of the current student class are as follows:

[0037] S61. Establish a dataset of characteristic images of students’ abnormal learning status in the classroom G = (g1,…,g o ,…,g χ ), o=1,2,3,…,χ; where g o represents the abnormal learning state characteristic image data of the classroom students corresponding to the oth abnormal learning state type of the classroom students, and x represents the maximum number of abnormal learning state types of the classroom students; the abnormal learning state characteristic image data of the classroom students represents the standard abnormal learning state image information of the students set for different abnormal learning state types of the classroom students; the abnormal learning state types of the classroom students include fiddling with stationery, sleeping in class, looking out the window, biting fingers, walking around randomly, and fighting;

[0038] S62, the current student classroom seat feature image data p in the current student classroom seat feature image data set P uThe abnormal learning state feature image data set G of the students in the classroom are ordered according to the student seat numbers. o Perform image feature matching and generate the abnormal learning state recognition data set G'=(g'1,…,g' u ,…,g' α ), where g' u Represents the abnormal learning status identification data of the current class student corresponding to the u-th student's classroom seat;

[0039] When p u With g o If the image feature matching is successful, it means that the student sitting in the current u-th class seat has the o-th type of abnormal learning state of the class student, then the abnormal learning state identification data g' of the current class student is output. u is abnormal;

[0040] When p u With g o If the image features are not matched successfully, it means that the student sitting in the current u-th class seat does not have the abnormal learning state type of the class student, then the abnormal learning state identification data g' of the current class student is output. u It is normal.

[0041] Preferably, the steps of performing identification processing on the student identity information of the abnormal learning state of the current student class based on the abnormal learning state identification data of the current student class and the student identity feature text data of the student class seat regulations, generating and outputting the abnormal learning state student identity identification data of the current student class are as follows:

[0042] S71, using an iterative deepening search algorithm to identify the abnormal learning status of students in the current class. u The student class seat specification student identity feature text data set E and the student class seat specification student identity feature text data e u Match the student's classroom seat number and search for the abnormal learning status identification data g' of the current class student u The student identity feature text data e is specified for the student classroom seat corresponding to the abnormal student classroom seat u , and generate the current class attendance abnormal state student identification data set E"'=(e"' u1 ,…,e”' u2 ), 1≤u1≤u≤u2≤α, where e'' u1 and e"' u2Respectively represent the student identity identification data of the current class attendance abnormal status corresponding to the class seats of the u1th and u2th students;

[0043] S72, the current class attendance abnormal state student identity recognition data set E'' is generated. u1 to e"' u2 The information is output online in order through the student classroom multimedia display screen according to the student's seat number.

[0044] A deep learning-based intelligent supervision system for student classroom learning status, used to implement the deep learning-based intelligent supervision method for student classroom learning status, the system includes a student classroom seat management module, a student classroom attendance management module, and a student classroom learning status management module;

[0045] The student classroom seat management module includes a student classroom seat image acquisition unit, a classroom student facial image corresponding student identity information storage unit, a current classroom seat student object recognition unit, a student classroom seat prescribed student identity information storage unit, and a current classroom seat abnormal state analysis and feedback unit;

[0046] The student classroom seat image acquisition unit acquires the current student classroom seat feature image data through the cloud lens; the classroom student facial image corresponding student identity information storage unit is used to store the classroom student facial image corresponding student identity feature text data; the current classroom seat student object recognition unit performs student identity information recognition processing on the current student classroom seat based on the current student classroom seat feature image data and the classroom student facial image corresponding student identity feature text data, and generates the current classroom seat student identity recognition data; the student classroom seat specified student identity information storage unit is used to store the student classroom seat specified student identity feature text data; the current classroom seat seating abnormality state analysis and feedback unit performs seating abnormality state analysis processing on the current student classroom based on the current classroom seat student identity recognition data and the student classroom seat specified student identity feature text data, generates the current classroom seat seating abnormality state analysis data and outputs it through the student classroom multimedia display screen;

[0047] The student classroom attendance management module includes a student classroom seat specification student facial image storage unit, a current classroom attendance status recognition unit, and a current classroom attendance abnormal student object recognition feedback unit;

[0048] The student classroom seat specification student facial image storage unit is used to store the student classroom seat specification student facial feature image data; the current classroom attendance status recognition unit performs student attendance status recognition processing of the current student classroom based on the current student classroom seat feature image data and the student classroom seat specification student facial feature image data, and generates current classroom student attendance status recognition data; the current classroom attendance abnormal student object recognition feedback unit performs abnormal attendance status student identity information recognition processing of the current student classroom based on the current classroom student attendance status recognition data and the student classroom seat specification student identity feature text data, generates current classroom attendance abnormal status student identity recognition data and outputs it through the student classroom multimedia display screen;

[0049] The student classroom learning status management module includes a classroom student abnormal learning status image storage unit, a current classroom student abnormal learning status recognition unit, and a current classroom abnormal learning status student object recognition feedback unit;

[0050] The classroom student abnormal learning state image storage unit is used to store the classroom student abnormal learning state characteristic image data; the current classroom student abnormal learning state identification unit performs abnormal learning state identification processing of the current student classroom based on the current student classroom seat characteristic image data and the classroom student abnormal learning state characteristic image data, and generates current classroom student abnormal learning state identification data; the current classroom abnormal learning state student object identification feedback unit performs abnormal learning state student identity information identification processing of the current student classroom based on the current classroom student abnormal learning state identification data and the student identity characteristic text data of the student classroom seat specification, generates current classroom learning abnormal state student identity identification data and outputs it through the student classroom multimedia display screen.

[0051] (3) Beneficial effects

[0052] The present invention provides a system and method for intelligently monitoring students' classroom learning status based on deep learning. It has the following beneficial effects:

[0053] 1. Through the cloud lens, the characteristic image information of the current students' classroom seats is efficiently and accurately collected to provide reliable data support for the scientific monitoring of abnormal student classroom seating arrangements; the current student classroom seat student identity information is intelligently identified based on the characteristic image information of the current student classroom seats combined with the deep learning image search algorithm and the scientifically preset classroom student facial images corresponding to the student identity characteristic information, realizing dynamic detection of student classroom seat student identity information; based on the current classroom seat student identity identification information combined with the deep learning text search algorithm and the student classroom seat regulations based on big data storage, the current student classroom seat abnormal status is accurately identified and feedback is realized, realizing intelligent and visual monitoring and feedback of student classroom seat abnormalities, improving the effect of classroom learning, and improving the quality and efficiency of student classroom supervision.

[0054] 2. Dynamically monitor the current student attendance status in the classroom based on the characteristic image information of the current student's classroom seat combined with the intelligent image search algorithm and the standard stored student classroom seat regulations and student facial feature image information, and realize the independent and accurate identification of the student classroom attendance status; scientifically statistical feedback of the student identity information of the current student's abnormal attendance status in the classroom is carried out based on the current student attendance status identification information combined with the intelligent search algorithm and the student identity information of the student classroom seat regulations based on big data storage, so as to realize intuitive and efficient monitoring and feedback of student objects with abnormal classroom attendance status, improve the safety of classroom learning, and improve the applicability of student classroom supervision.

[0055] 3. Scientifically monitor the abnormal learning status of students in the current classroom based on the characteristic image information of the current students' classroom seats combined with deep learning image search algorithms and standard-set characteristic image information of the abnormal learning status of students in the classroom, and realize accurate monitoring of students' classroom learning status; intelligently monitor the identity information of students in the current classroom abnormal learning status based on the identification information of students' abnormal learning status combined with intelligent search algorithms and student identity information of students' classroom seats based on big data storage, and realize efficient identification and feedback of student identity information of students' abnormal classroom status, improve the functional diversity of student classroom supervision, and improve students' classroom learning discipline and learning atmosphere. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 This is a module diagram of the intelligent monitoring system for students' classroom learning status based on deep learning provided by the present invention;

[0057] Figure 2 This is a flowchart of the method for intelligently supervising students' classroom learning status based on deep learning provided by the present invention. DETAILED DESCRIPTION

[0058] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0059] The embodiments of the system and method for intelligently monitoring students’ classroom learning status based on deep learning are as follows:

[0060] Example 1:

[0061] See also Figure 1-Figure 2 , an intelligent supervision method for students' classroom learning status based on deep learning, the method includes the following steps:

[0062] S1, collect the characteristic image data of the current students' classroom seats;

[0063] S2. Perform student identity information recognition processing on the current student's seat based on the current student's classroom seat feature image data and the student identity feature text data corresponding to the classroom student's facial image to generate the current student seat student identity recognition data;

[0064] S3, based on the current classroom seat student identity recognition data and the student identity feature text data of the student classroom seat regulations, perform current classroom seat abnormality status analysis processing, generate current classroom seat abnormality status analysis data and output;

[0065] S4, performing student attendance status recognition processing for the current student class based on the current student class seat feature image data and the student class seat prescribed student facial feature image data, and generating current class student attendance status recognition data;

[0066] S5. Perform student identity information recognition processing on abnormal attendance status of the current student class based on the student attendance status recognition data and the student identity feature text data of the student class seat regulations, generate and output the abnormal attendance status student identity recognition data of the current class;

[0067] S6. Performing recognition processing of the abnormal learning state of the current student in the classroom based on the current student's classroom seat feature image data and the abnormal learning state feature image data of the current student in the classroom to generate abnormal learning state recognition data of the current student in the classroom;

[0068] S7. Perform student identity information recognition processing on the abnormal learning status of students in the current classroom based on the abnormal learning status recognition data of students in the current classroom and the student identity feature text data of the student classroom seat regulations, generate and output the abnormal learning status student identity recognition data of the current classroom.

[0069] For further information, see Figure 1-Figure 2 ,The steps for collecting the characteristic image data of the current student’s classroom seat are as follows:

[0070] S11, collect the real-time state feature image information of each student seat in the classroom through the cloud camera online, and generate the current student classroom seat feature image data set P = (p1, ..., p u ,…,p α ), u=1,2,3,…,α; where p u It represents the current student classroom seat feature image data corresponding to the u-th student classroom seat collected, α represents the maximum number of student classroom seats; the current student classroom seat feature image data represents the latest status feature image information of the student classroom seats.

[0071] The student identity information of the current student's classroom seat is recognized based on the characteristic image data of the current student's classroom seat and the student identity feature text data corresponding to the classroom student's facial image. The steps for generating the student identity recognition data of the current classroom seat are as follows:

[0072] S21, establish a text data set K = (k1, ..., k v ,…,k β ), v=1,2,3,…,β; where k v The student identity feature text data corresponding to the student facial image of the vth student in the class is represented by β, which represents the maximum number of students in the class. The student identity feature text data corresponding to the student facial image represents the combined information composed of the facial image information and the student identity feature information of the class students. The student identity feature information includes the student's name, student ID number, grade information, and ID number.

[0073] S22, using the FLANN image search algorithm to find the current student classroom seat feature image data set P of the current student classroom seat feature image data p u The text data set K of student identity features corresponding to the facial images of students in the classroom is ordered according to the students' classroom seat numbers, and the text data set k of student identity features corresponding to the facial images of students in the classroom is ordered according to the students' classroom seat numbers. v Perform image feature matching and generate the student identity recognition data set P'=(p'1,…,p' u ,…,p' α ),

[0074] where p' u The student identity data of the current classroom seat corresponding to the u-th student's classroom seat;

[0075] When p uWith k v If the image feature matching is successful, it means that the student sitting in the current class seat u is the v-th class student, then the student identity recognition data p' of the current class seat is output. u The identity characteristics text information of the v-th class student;

[0076] When p u With k v If the image features are not matched successfully, it means that there is no student object sitting in the current u-th student classroom seat, then the student identity recognition data p' of the current classroom seat is output. u There are no student objects.

[0077] The steps for analyzing and processing the abnormal state of the current student class seating arrangement based on the student identity recognition data of the current class seating arrangement and the student identity feature text data of the student class seating arrangement regulations are as follows:

[0078] S31. Establish student classroom seat regulations and student identity feature text data set E = (e1,…,e u ,…,e α ), where e u The student class seat specified student identity feature text data corresponding to the u-th student class seat, the student class seat specified student identity feature text data represents the identity feature information of the student object determined by the student class seat according to school regulations;

[0079] S32, using the KD tree nearest neighbor search algorithm to find the current classroom seat student identity identification data set P' of the current classroom seat student identity identification data set P' u The student identity feature text data set E is ordered according to the student classroom seat number and the student classroom seat regulations. The student identity feature text data set E is the student classroom seat regulations. u Perform keyword matching of student identity information, and generate the current classroom seat abnormality analysis data set E'=(e'1,…,e' u ,…,e' α ), where e' u Abnormal status analysis data of the current classroom seating arrangement corresponding to the u-th student's classroom seat;

[0080] When p' u With e u The student identity information keyword matching is successful, indicating that the current class seat of the u-th student is seated in accordance with the school regulations, and the abnormal state analysis data of the current class seat is output e' u The seating arrangement is normal;

[0081] When p' u With e u If the student identity information keyword is not matched successfully, it means that the current class seat of the u-th student is not seated according to the school regulations, then the abnormal state analysis data of the current class seat is output e' u The seating arrangement is abnormal;

[0082] S33, the generated current classroom seat abnormal state analysis data set E' of the current classroom seat abnormal state analysis data e' u The information is output online in order through the student classroom multimedia display screen according to the student's seat number.

[0083] Through the student classroom seat image acquisition unit, cloud lenses are used to efficiently and accurately collect the characteristic image information of the current student classroom seats, providing reliable data support for scientific monitoring of abnormal student classroom seating arrangements; the current classroom seat student object recognition unit intelligently identifies the student identity information of the current student classroom seat based on the current student classroom seat characteristic image information combined with deep learning image search algorithms and scientifically preset classroom student facial images corresponding to student identity feature information, thereby realizing dynamic detection of student classroom seat student identity information; the current classroom seat seating abnormality status analysis and feedback unit accurately identifies the abnormal status of the current student classroom seat based on the current classroom seat student identity recognition information combined with deep learning text search algorithms and student classroom seat regulations based on big data storage, thereby realizing intelligent and visual monitoring and feedback of student classroom seating abnormalities, improving classroom learning effects, and improving the quality and efficiency of student classroom supervision.

[0084] For further information, see Figure 1-Figure 2 Based on the current student classroom seat feature image data and the student classroom seat specification student facial feature image data, the student attendance status recognition processing of the current student classroom is performed, and the operation steps for generating the current class student attendance status recognition data are as follows:

[0085] S41. Establish student classroom seats and student facial feature image data set F = (f1, ..., f u ,…,f α ), where f u The student classroom seat prescribed student facial feature image data corresponding to the u-th student classroom seat represents the facial feature image information of the student object determined by the student classroom seat according to school regulations;

[0086] S42, using SURF image search algorithm to find the current student classroom seat feature image data set P of the current student classroom seat feature image data p uThe facial feature image data set F of students is ordered according to the student classroom seat number and the student classroom seat regulations. The facial feature image data set F of students in the classroom seat regulations u Perform image feature matching and generate the current class student attendance status recognition data set F'=(f1',…,f u ',…,f' α ), where f u 'Indicates the current class student attendance status identification data corresponding to the u-th student's classroom seat;

[0087] When p u With f u If the image feature matching is successful, it means that the current student u is seated in the classroom and the student object has normal attendance, then the current classroom student attendance status identification data p' is output. u For normal attendance;

[0088] When p u With f u If the image feature matching fails, it means that the student object at the current class seat u is absent, then the current class student attendance status recognition data p' is output. u For absence.

[0089] The steps for performing abnormal attendance status identification and student identity information processing for the current student class based on the current class student attendance status identification data and the student class seat specification student identity feature text data, and generating and outputting the abnormal attendance status student identity identification data for the current class are as follows:

[0090] S51, using the Boyer-Moore search algorithm to find the current class student attendance status identification data f in the current class student attendance status identification data set F' u 'Student class seat regulations student identity feature text data set E middle school student class seat regulations student identity feature text data e u Match the student's classroom seat number and search for the current class student attendance status identification data f u 'Specify the student identity feature text data e for the student class seat corresponding to the absent student class seat u , and generate the current class attendance abnormal state student identification data set E"=(e" u1 ,…,e” u2 ), 1≤u1≤u≤u2≤α, where e” u1 and e” u2 Respectively represent the student identity identification data of the current class attendance abnormal status corresponding to the class seats of the u1th and u2th students;

[0091] S52, the current class attendance abnormal state student identity identification data e" in the generated current class attendance abnormal state student identity identification data set E" u1 to e” u2 The information is output online in order through the student classroom multimedia display screen according to the student's seat number.

[0092] Through the current classroom attendance status recognition unit, the current student classroom attendance status is dynamically monitored based on the current student classroom seat feature image information combined with the intelligent image search algorithm and the standard stored student classroom seat regulations student facial feature image information, thereby realizing autonomous and accurate identification of the student classroom attendance status; the current classroom abnormal attendance student object recognition and feedback unit performs scientific statistical feedback on the student identity information of the current student classroom abnormal attendance status based on the current classroom student attendance status recognition information combined with the intelligent search algorithm and the student identity information of the student classroom seat regulations based on big data storage, thereby realizing intuitive and efficient monitoring and feedback of student objects in the abnormal student classroom attendance status, improving the safety of classroom learning, and improving the applicability of student classroom supervision.

[0093] For further information, see Figure 1-Figure 2 Based on the current student classroom seat feature image data and the classroom student abnormal learning state feature image data, the abnormal learning state of the current student classroom is identified and processed. The operation steps for generating the current classroom student abnormal learning state identification data are as follows:

[0094] S61. Establish a dataset of characteristic images of students’ abnormal learning status in the classroom G = (g1,…,g o ,…,g χ ), o=1,2,3,…,χ; where g o represents the abnormal learning state feature image data corresponding to the oth abnormal learning state type of the classroom student, and χ represents the maximum number of abnormal learning state types of the classroom student. The abnormal learning state feature image data of the classroom student represents the standard abnormal learning state image information of the classroom student set for different abnormal learning state types of the classroom student. The abnormal learning state types of the classroom student include fiddling with stationery, sleeping in class, looking out the window, biting fingers, walking around randomly, and fighting.

[0095] S62, the current student classroom seat feature image data set P of the current student classroom seat feature image data p u The abnormal learning state feature image data set G of the students in the classroom is sorted according to the students' classroom seat numbers and the abnormal learning state feature image data set G of the students in the classroom o Perform image feature matching and generate the abnormal learning state recognition data set G'=(g'1,…,g' u ,…,g' α), where g' u Represents the abnormal learning status identification data of the current class student corresponding to the u-th student's classroom seat;

[0096] When p u With g o If the image feature matching is successful, it means that the student sitting in the current class seat u has the oth type of abnormal learning state, then the abnormal learning state identification data g' of the current class student is output. u is abnormal;

[0097] When p u With g o If the image features are not matched successfully, it means that the student sitting in the current class seat u has not shown any abnormal learning status type of the class student, then the abnormal learning status identification data g' of the current class student is output. u It is normal.

[0098] The steps for performing identification processing of the student identity information of the abnormal learning state of the current student in the classroom based on the abnormal learning state identification data of the current student in the classroom and the student identity feature text data of the student seat regulations are as follows:

[0099] S71, using iterative deepening search algorithm to identify the abnormal learning state of the current class students in the abnormal learning state identification data set G' of the current class students' abnormal learning state identification data g' u Student Classroom Seat Regulations Student Identity Feature Text Data Set E Student Classroom Seat Regulations Student Identity Feature Text Data Set E u Match the student's classroom seat number and search for the abnormal learning status identification data g' of the current class students u Specify student identity feature text data e for the student classroom seat corresponding to the abnormal student classroom seat u , and generate the current class attendance abnormal state student identification data set E"'=(e"' u1 ,…,e”' u2 ), 1≤u1≤u≤u2≤α, where e'' u1 and e"' u2 Respectively represent the student identity identification data of the current class attendance abnormal status corresponding to the class seats of the u1th and u2th students;

[0100] S72, the current class attendance abnormal state student identity recognition data e'' in the generated current class attendance abnormal state student identity recognition data set E'' u1 to e"' u2 The information is output online in order through the student classroom multimedia display screen according to the student's seat number.

[0101] Through the current classroom student abnormal learning status recognition unit, the current student's abnormal learning status is scientifically monitored based on the current student's classroom seat feature image information combined with the deep learning image search algorithm and the standard setting of the classroom student's abnormal learning status feature image information, thereby realizing accurate monitoring of the student's classroom learning status; the current classroom abnormal learning status student object recognition feedback unit, based on the current classroom student abnormal learning status recognition information combined with the intelligent search algorithm and the student identity information of the student classroom seat regulations based on big data storage, realizes efficient identification and feedback of student identity information of student abnormal classroom status, improves the functional diversity of student classroom supervision, and improves student classroom learning discipline and learning atmosphere.

[0102] Example 2:

[0103] See also Figure 1-Figure 2 , a student classroom learning status intelligent supervision system based on deep learning, is used to implement a student classroom learning status intelligent supervision method based on deep learning. The system includes a student classroom seat management module, a student classroom attendance management module, and a student classroom learning status management module;

[0104] The student classroom seat management module includes a student classroom seat image acquisition unit, a student identity information storage unit corresponding to the classroom student facial image, a current classroom seat student object recognition unit, a student classroom seat prescribed student identity information storage unit, and a current classroom seat abnormal state analysis and feedback unit;

[0105] A student classroom seat image acquisition unit is used to collect the characteristic image data of the current student classroom seat through a cloud lens; a classroom student facial image corresponding student identity information storage unit is used to store the student identity feature text data corresponding to the classroom student facial image; a current classroom seat student object recognition unit is used to perform student identity information recognition processing on the current student classroom seat based on the current student classroom seat characteristic image data and the student identity feature text data corresponding to the classroom student facial image, and generate the current classroom seat student identity recognition data; a student classroom seat prescribed student identity information storage unit is used to store the student classroom seat prescribed student identity feature text data; a current classroom seat seating abnormality state analysis and feedback unit is used to perform current classroom seat seating abnormality state analysis processing based on the current classroom seat student identity recognition data and the student classroom seat prescribed student identity feature text data, generate current classroom seat seating abnormality state analysis data and output it through the student classroom multimedia display screen;

[0106] The student class attendance management module includes a student class seat regulation student face image storage unit, a current class attendance status recognition unit, and a current class attendance abnormal student object recognition feedback unit;

[0107] The student facial image storage unit of the student classroom seating regulations is used to store the facial feature image data of the student classroom seating regulations; the current class attendance status recognition unit performs student attendance status recognition processing of the current class based on the current student classroom seating feature image data and the student facial feature image data of the student classroom seating regulations, and generates current class student attendance status recognition data; the current class attendance abnormal student object recognition feedback unit performs abnormal attendance status student identity information recognition processing of the current class based on the current class student attendance status recognition data and the student identity feature text data of the student classroom seating regulations, generates current class attendance abnormal status student identity recognition data and outputs it through the student classroom multimedia display screen;

[0108] The student classroom learning status management module includes a classroom student abnormal learning status image storage unit, a current classroom student abnormal learning status recognition unit, and a current classroom abnormal learning status student object recognition feedback unit;

[0109] The classroom student abnormal learning state image storage unit is used to store the classroom student abnormal learning state characteristic image data; the current classroom student abnormal learning state identification unit performs abnormal learning state identification processing of the current student classroom based on the current student classroom seat characteristic image data and the classroom student abnormal learning state characteristic image data, and generates the current classroom student abnormal learning state identification data; the current classroom abnormal learning state student object identification feedback unit performs abnormal learning state student identity information identification processing of the current student classroom based on the current classroom student abnormal learning state identification data and the student identity characteristic text data of the student classroom seat regulations, generates the current classroom learning abnormal state student identity identification data and outputs it through the student classroom multimedia display screen.

[0110] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. An intelligent supervision method for students' classroom learning status based on deep learning, characterized by: The method comprises the following steps: S1, collect the characteristic image data of the current students' classroom seats; S2. Perform student identity information recognition processing on the current student's seat in the classroom to generate student identity recognition data for the current seat in the classroom; S3 performs an analysis of the abnormal state of the current student class seating arrangement, generates and outputs the abnormal state analysis data of the current class seating arrangement; S4, performing student attendance status recognition processing for the current student class, generating student attendance status recognition data for the current class; S5. Perform student identity information recognition processing on the abnormal attendance status of the current student class, generate and output the student identity recognition data of the abnormal attendance status of the current class; S6. Perform abnormal learning status identification processing on students in the current class to generate abnormal learning status identification data of students in the current class; S7. Perform identification processing on the student identity information of the abnormal learning state of the current student in the classroom, generate and output the student identity identification data of the abnormal learning state of the current classroom.

2. The method for intelligently monitoring students' classroom learning status based on deep learning according to claim 1 is characterized by: Said S1 comprises the following steps: S11, collect the real-time state feature image information of each student seat in the classroom through the cloud camera online, and generate the current student classroom seat feature image data set P = (p1, ..., p u ,…,p α ), u=1,2,3,...,α; where p u Represents the current student classroom seat feature image data corresponding to the u-th student classroom seat collected, and α represents the maximum number of student classroom seats.

3. The method for intelligently monitoring students' classroom learning status based on deep learning according to claim 2 is characterized by: The S2 comprises the following steps: S21, establish a text data set K = (k1, ..., k v ,…,k β ), v=1,2,3,…,β; where k v represents the student identity feature text data corresponding to the student facial image of the vth class student, and β represents the maximum number of students in the class; S22, using the FLANN image search algorithm to find the p in the P u The students' seats are numbered in order according to the K v Perform image feature matching and generate the student identity recognition data set P′=(p′1,…,p′ u ,…,p′ α ), where p′ u The student identity data of the current classroom seat corresponding to the u-th student's classroom seat; When p u With k v If the image feature matching is successful, the p′ is output. u The identity characteristics text information of the v-th class student; When p u With k v If the image features are not matched successfully, the p′ is output. u There are no student objects.

4. The method for intelligently monitoring students' classroom learning status based on deep learning according to claim 3 is characterized by: The S3 includes the following steps: S31. Establish student classroom seat regulations and student identity feature text data set E = (e1,…,e u ,…,e α ), where e u The student identity feature text data corresponding to the u-th student's classroom seat; S32, using the KD tree nearest neighbor search algorithm to find the p′ in the P′ u The students' seats are numbered in order according to the E u Perform keyword matching of student identity information, and generate the abnormal state analysis data set E′=(e′1,…,e′ u ,…,e′ α ), where e′ u Abnormal status analysis data of the current classroom seating arrangement corresponding to the u-th student's classroom seat; When p′ u With e u If the student identity information keyword matching is successful, the e′ is output. u The seating arrangement is normal; When p′ u With e u If the student identity information keyword fails to match successfully, the e′ is output. u The seating arrangement is abnormal; S33, the e' in the generated E' u The information is output online in order through the student classroom multimedia display screen according to the student's seat number.

5. The method for intelligently monitoring students' classroom learning status based on deep learning according to claim 4 is characterized by: The S4 comprises the following steps: S41. Establish student classroom seats and student facial feature image data set F = (f1, ..., f u ,…,f α ), where f u The facial feature image data of the student corresponding to the u-th student's classroom seat; S42, using SURF image search algorithm to find the p in P u The students are seated in the classroom in the order described in F u Perform image feature matching and generate the current class student attendance status recognition data set F′=(f′1,…,f′ u ,…,f′ α ), where f′ u Represents the current class student attendance status identification data corresponding to the u-th student's classroom seat; When p u With f u If the image feature matching is successful, the p′ is output. u For normal attendance; When p u With f u If the image feature matching fails, the p′ is output. u For absence.

6. The method for intelligently monitoring students' classroom learning status based on deep learning according to claim 5 is characterized by: The S5 comprises the following steps: S51, using the Boyer-Moore search algorithm to find the f′ in F′ u With the E in the e u Match the student's classroom seat number and search for the f' u The corresponding class seat for the absent student is u , and generate the current class attendance abnormal state student identification data set E″=(e″ u1 ,…,e″ u2 ), 1≤u1≤u≤u2≤α, where e″ u1 and e″ u2 Respectively represent the student identity identification data of the current class attendance abnormal status corresponding to the class seats of the u1th and u2th students; S52, the generated E″ u1 to e″ u2 The information is output online in order through the student classroom multimedia display screen according to the student's seat number.

7. The method for intelligently monitoring students' classroom learning status based on deep learning according to claim 6 is characterized by: The S6 comprises the following steps: S61. Establish a dataset of characteristic images of students’ abnormal learning status in the classroom G = (g1,…,g o ,…,g χ ), o=1,2,3,…,χ; where g o represents the characteristic image data of the abnormal learning state of the classroom students corresponding to the oth type of abnormal learning state of the classroom students, and χ represents the maximum number of abnormal learning state types of the classroom students; S62, the p in the P u The students' seats are numbered in order according to the G o Perform image feature matching and generate the abnormal learning state recognition data set G′=(g′1,…,g′ u ,…,g′ α ), where g′ u Represents the abnormal learning status identification data of the current class student corresponding to the u-th student's classroom seat; When p u With g o If the image feature matching is successful, the g′ is output. u is abnormal; When p u With g o If the image features are not matched successfully, the g′ is output. u It is normal.

8. The method for intelligently monitoring students' classroom learning status based on deep learning according to claim 7 is characterized by: The S7 comprises the following steps: S71, using an iterative deepening search algorithm to find the g′ in G′ u With the E in the e u Match the student's classroom seat number and search for the g' u The e corresponding to the abnormal student classroom seat u , and generate the current class attendance abnormal state student identification data set E″′=(e″′ u1 ,…,e″′ u2 ), 1≤u1≤u≤u2≤α, where e″′ u1 and e″′ u2 Respectively represent the student identity identification data of the current class attendance abnormal status corresponding to the class seats of the u1th and u2th students; S72, the generated E″′ u1 to e″′ u2 The information is output online in order through the student classroom multimedia display screen according to the student's seat number.

9. A system for intelligently monitoring students' classroom learning status based on deep learning, for implementing the method for intelligently monitoring students' classroom learning status based on deep learning according to any one of claims 1 to 8, characterized in that: The system includes a student classroom seat management module, a student classroom attendance management module, and a student classroom learning status management module.

Images