Teaching supervision method based on consistency analysis

Abstract

The invention belongs to the field of data analysis, particularly relates to a teaching supervision method based on consistency analysis, and aims to solve the problem that the efficiency of a teaching supervision system is low due to the fact that an existing teaching supervision method only analyzes specific behaviors of individuals. The method comprises the following steps: dividing to-be-analyzed data into different classroom scenes according to the distribution characteristics of voice segments; classroom voice data and classroom interaction voice data of a teacher are obtained in a voice recognition mode; selecting individual emotion index data in the same time period from the physiological index signals; performing short-time consistency analysis and long-time consistency analysis; and selecting a supervision object based on the short-time consistency analysis result and the long-time consistency analysis result. According to the invention, through utilization of long-term and short-term consistency analysis of student groups, real-time classroom student supervision groups and normalized personalized teaching supervision objects are screened in batches; and the accuracy of teaching supervision of the teacher end is improved by utilizing long-short time consistency analysis of the teacher end.

Description

Background Art

[0002] Accurate and timely classroom evaluation is an important part of teaching supervision. Most of the existing classroom evaluation methods use independent individual behavior analysis on the student side and independent teacher emotion recognition on the teacher side to conduct teaching evaluation. Since classroom teaching is a collective event, there is a high degree of consistency between individual students, and due to the relevance of teaching content, the behaviors and emotions of teachers and students are also highly correlated. Whether using an independent individual student behavior analysis model or ignoring the influence of students, participants in teaching events, on real-time teaching, and conducting independent teacher-side teaching evaluation, it will affect the accuracy of the evaluation, thereby causing the inefficiency of the teaching supervision system.

[0003] In order to solve the above problems, the present invention proposes a teaching supervision method based on consistency analysis. First, the present invention uses multimodal information to divide the classroom scenes; then, based on different classroom scenes, the consistency analysis data is extracted to conduct long- and short-term consistency analysis of students and teachers. By using the long- and short-term consistency analysis of the student group, the instant classroom student supervision group and the normalized personalized teaching supervision objects are batch screened out to improve the instant teaching efficiency of each classroom scene; by using the long- and short-term consistency analysis on the teacher side, long-term evaluation of teacher ethics and short-term reminders are conducted, and the accuracy of teaching supervision on the teacher side is further improved by eliminating the influence of uncontrollable student factors on unreasonable teacher evaluation in teacher evaluation. Summary of the invention

[0004] In order to solve the above-mentioned problem in the prior art, that is, the existing teaching supervision system ignores the factors of two-way influence on real-time teaching, resulting in insufficient evaluation accuracy, the present invention provides a teaching supervision method based on consistency analysis, the method comprising: step S1, collecting audio signals of multiple students during class time periods as data to be analyzed; and collecting physiological indicator signals; Step S2, based on the data to be analyzed, according to the distribution characteristics of the speech segments, the data to be analyzed is divided into different classroom scenes, and corresponding scene labels are added; Step S3, based on the data to be analyzed with scene tags added, the teacher's classroom voice data and classroom interaction voice data are obtained by voice recognition; based on the time corresponding to the teacher's classroom voice data and the time corresponding to the classroom interaction voice data, the individual emotion index data of the same period are selected from the physiological index signal; Step S4, performing short-term consistency analysis on the student end and the teacher end respectively through the teacher's classroom voice data and the classroom interactive voice data to obtain the short-term consistency analysis result; Step S5, based on the short-term consistency analysis result, performing long-term consistency analysis on the student end and the teacher end respectively to obtain a long-term consistency analysis result; Step S6, selecting a supervision object based on the short-term consistency analysis results and the long-term consistency analysis results.

[0005] Furthermore, the step S2 is specifically as follows: performing endpoint detection on the data to be analyzed, and dividing the audio data into speech segments and non-speech segments; Calculate the duration of each speech segment, the duration of the non-speech segment, and the average short-term amplitude of each speech segment ; in, represents the average short-term amplitude of each speech segment, Indicates The short-term amplitude of the frame audio, Indicates the frame length. Indicates The first frame of the audio signal Sample values, Indicates The starting frame number of the speech segment, Indicates The end frame signal of a speech segment, Indicates The number of frames contained in a speech segment; When the duration of classroom scene audio detection is greater than the time threshold, the average short-term amplitude of each speech segment is , calculate the sample mean square error ; Using the statistical characteristics of speech segments to identify classroom scenes: The first type of classroom scene is defined as the teacher lecturing and the students listening, the second type of classroom scene is defined as group teaching, the third type of classroom scene is defined as practice or testing, and the fourth type of classroom scene is self-study; If the speech segment duration is less than the duration threshold and the sample mean square error If it is less than the difference threshold, the current classroom scene is identified as one of the four types of classroom scenes; If the speech segment duration is less than the duration threshold and the sample mean square error If it is greater than or equal to the difference threshold, the current classroom scene is identified as a third-class classroom scene; If the current classroom scene identification is not both a Class 4 classroom scene and a Class 3 classroom scene, the pre-built teacher speech feature model is used to perform specific person recognition on the speech segment, obtain the specific person label of the speech segment, and simultaneously identify the unsuccessfully matched speech as the interactive speech of the corresponding specific person; if the total duration of each speech segment of the interactive speech of a specific person of the same class is greater than the specific person duration threshold, the current classroom scene identification is determined to be a Class 1 classroom scene; if it is less than or equal to the specific person duration threshold, the current classroom scene is determined to be a Class 2 classroom scene.

[0006] Furthermore, the teacher's classroom voice data and classroom interactive voice data are specifically: Perform specific person speech recognition on the data to be analyzed with added scene labels to obtain the teacher's complete classroom speech data within the current classroom evaluation analysis time window; The speech segment data outside the speech recognition result of a specific person is classified into the interactive speech corresponding to the specific person, and the complete classroom interactive speech data of the teacher within the current classroom evaluation and analysis time window is obtained; Loading physiological signals related to emotional fluctuations and capable of realizing emotion recognition recorded by personal wearable devices during the voice collection period as individual emotion indicator data; Based on complete classroom interaction voice data and individual emotion indicator data, teacher classroom voice data and classroom interaction voice data are obtained.

[0007] Furthermore, the step S4 is specifically: Based on the teacher's classroom voice data and classroom interactive voice data, a teacher's negative emotion set is constructed. , students' negative emotions , Student Safety Emotion Set ; For the data to be analyzed with the same scene labels, short-term consistency analysis is performed separately.

[0008] Furthermore, if the scene label is a class scene, the short-term consistency analysis includes: Teacher classroom voice data The emotion recognition is performed on each speech segment of the teacher's classroom speech data, and the emotion recognition results of the speech segments with the same emotion recognition results and adjacent in time are merged. The speech segments after emotion merging are regarded as the same emotion speech segments. The recognition results of the same emotion speech segments of the teacher's classroom speech data are recorded as ; Teacher classroom interactive voice data The emotion recognition is performed on each speech segment of the teacher's classroom interactive speech data, and the emotion recognition results of the speech segments with the same emotion recognition results and adjacent in time are merged. The speech segments after the emotion merger are taken as the same emotion speech segments. , Indicates the teacher number corresponding to the classroom voice [A1]; in, Represents the first Emotion recognition results of speech segments with the same emotion, Represents a set of emotion recognition result numbers, Represents the teacher's classroom interactive voice data Emotion recognition results of speech segments with the same emotion, Represents the numbered set of emotion recognition results of teacher classroom interactive speech data; Individual sentiment index data The physiological signal data in the data are divided into different individual data sets according to the device number of the signal source. Corresponding to the start and end time of each speech segment, the individual data set is divided into multiple sub-segments, and emotion recognition is performed on multiple segments of physiological signal data. The emotion recognition results of physiological signal data are: ; in, The first The individual Synchronous speech physiological signal emotion recognition results, Indicates the number of personal wearable devices; Construct interactive response emotion sequences based on the temporal nature of speech segments ; Specifically: Emotion recognition results of teacher classroom interactive speech data Find with Has the shortest time interval and the start collection time is later than The element of the end time is recorded as Interactive response emotions ; Calculate the first emotion sequence used to describe the emotional relevance of teacher-student interaction and a second emotion sequence used to describe the emotional relevance of students in the same class ; in, ; ; The valence-arousal arousal emotion model (VA) is used. The situation where the product of the valence components of two emotion recognition results is negative and the product of the arousal components of two emotion recognition results is negative is recorded as mutually exclusive; the situation where the product of the corresponding valence components of two emotion recognition results is negative is recorded as partially exclusive; the situation where the two emotion recognition results are different but the product of their valence components is positive and the product of their arousal components is positive is recorded as similar; The teacher-side short-term consistency analysis includes: teacher's short-term emotional control ; Construct short-term consistency analysis on the student side as a set of abnormal supervisors ; Construct the teacher-side short-term interaction degree ; in, Indicates a class scenario The degree of emotion following of each personal wearable device to the individual, ; represents the threshold of emotion consistency ratio; If Establishment of regular expression The value is 1, otherwise the expression value is 0; If Establishment of regular expression The value is 1, otherwise the expression value is 0; If , then obtain The corresponding device number classifies the student corresponding to the device number into the abnormal supervision personnel set; Indicates the number of elements contained in the abnormal supervisor set; It represents the number of normal abnormal persons, that is, if the ratio of persons in the set of abnormal supervisors who are classified as abnormal supervisors in a type of classroom scenario in which they appear is greater than the set threshold, they will be included in the calculation of the number of normal abnormal persons.

[0009] Furthermore, if the scene label is a second-class classroom scene, the short-term consistency analysis includes: Based on teacher classroom interactive voice data Obtain the recognition results of the same-emotion speech segments of the teacher's classroom interactive speech data ; For individual emotion index data Each physiological signal data is divided into different individual data sets according to the device number of the signal source. Corresponding to the start and end time of each speech segment, the physiological signal data is divided into multiple sub-segments, and emotion recognition is performed on the physiological signals of multiple sub-segments. The emotion recognition result of the physiological signal data is recorded as ; Get the second emotion sequence describing the emotional relevance of students in the same class ; Constructing short-term consistency analysis on the student side as an abnormal supervisory personnel set ; in, Indicates the second type of classroom scenario The degree of emotion following of each personal wearable device to the individual, , If , then obtain Corresponding to the device number, the student corresponding to the device number is classified into the abnormal supervision personnel set, wherein the device number is an individual number.

[0010] Furthermore, if the scene labels are three-class classroom scenes and four-class classroom scenes, the short-term consistency analysis includes: Individual sentiment index data The physiological signal data in the dataset are divided into different individual data sets according to the device number of the signal source, and the emotion recognition results are obtained as follows: ; in, express No. The individual Physiological emotion recognition results, Indicates the total number of emotions; Construct three types of classroom scenarios and analyze the short-term consistency of students as abnormal supervisors ; When constructing four types of classroom scenarios, the short-term consistency analysis of the student end is the abnormal supervisor set ; in, If , then obtain Corresponding device number, the student corresponding to the device number is classified into the abnormal supervision personnel set; If , then obtain Corresponding device number, the student corresponding to the device number is classified into the abnormal supervision personnel set; Express emotions separately duration, class duration constant; student safety emotion set The highest frequency emotions among all persons’ emotion recognition results are further expanded online to increase the safe emotion set of students in special contexts.

[0011] Furthermore, a long-term consistency analysis is conducted on the teacher side, specifically: Based on the short-term emotion control degree of each teacher, the long-term emotion control degree of each teacher is calculated as the ratio of each teacher's short-term emotion control degree to the length of their class; Based on the short-term interaction degree obtained by each teacher, the long-term interaction degree of each teacher is calculated as the median of the short-term interaction degree of each teacher.

[0012] Furthermore, a long-term consistency analysis is conducted on the student side, specifically: The frequency of each student in the data to be analyzed with each scenario label added is counted, and the frequency of being classified into the abnormal supervisor set is the ratio of the number of times they are classified into each abnormal supervisor set to the total number of their courses; the invalid listening rate of each student can be obtained in turn as the frequency of each student being classified into the abnormal supervisor set in the first and second classroom scenarios; the potential cheating rate of each student is the frequency of each student being classified into the abnormal supervisor set in the third classroom scenario; the self-study efficiency of each student is the frequency of each student being classified into the abnormal supervisor set in the fourth classroom scenario; Use the results of long- and short-time consistency analysis to supervise long- and short-time teaching.

[0013] Furthermore, the step S6 comprises: Specifically, short-term supervision involves taking the abnormal supervision personnel sets in three types of classroom scenarios as key inspection targets and conducting adaptive proctoring; A set of abnormal supervisors in a classroom scenario is used as the focus of classroom interaction to solve the problem that teachers cannot take into account the status of all students in large classrooms; Classify the teacher with the highest short-term emotional control in a class as the object of video verification of teacher behavior to solve the problem of timely discovery of inappropriate teaching behavior; Long-term supervision specifically includes: eliminating those who are at the top of the long-term emotional control system, and establishing a teacher screening system based on emotional control; eliminating those who are at the bottom of the long-term interaction system, and establishing a teacher recommendation system based on interactivity; promptly identifying those with abnormal behavior, and providing personalized targeted teaching.

[0014] Beneficial effects of the present invention: (1) The supervision method proposed in the present invention utilizes multimodal information to divide classroom scenes and extracts consistency analysis data based on different classroom scenes. It utilizes the long- and short-term consistency analysis in the student group to screen out the real-time classroom student supervision group and the normalized personalized teaching supervision object, thereby improving the real-time teaching efficiency of each classroom scene. It utilizes the long- and short-term consistency analysis on the teacher side to conduct long-term evaluation of teacher ethics and short-term reminders, and further improves the accuracy of teacher-side teaching supervision by eliminating the influence of uncontrollable student factors on unreasonable teacher evaluation in teacher evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings: Figure 1 It is a flowchart of a teaching supervision method based on consistency analysis in an embodiment of the present invention. DETAILED DESCRIPTION

[0016] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It is also necessary to explain that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.

[0017] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0018] In order to more clearly illustrate the teaching supervision method based on consistency analysis of the present invention, the following is combined with Figure 1 Each step in the embodiment of the present invention is described in detail.

[0019] A teaching supervision method based on consistency analysis according to the first embodiment of the present invention includes steps S1 to S6, each of which is described in detail as follows: Step S1, collecting audio signals of multiple students during class periods as data to be analyzed; and collecting physiological indicator signals; In this embodiment, the audio signals of multiple student course time periods are obtained by obtaining the union of the course schedules, constructing a classroom evaluation analysis time window, and determining whether the audio falls into the analysis window; when the course schedule cannot be obtained, the signal-to-noise ratio of the audio signal is calculated to determine, such as when the signal-to-noise ratio is lower than the set threshold and the current moment belongs to the classroom teaching time, the short-term classroom consistency analysis is started. The classroom teaching time is a non-typical classroom teaching time that does not belong to extreme midnight, early morning, lunch break, etc.

[0020] Step S2, based on the data to be analyzed, according to the distribution characteristics of the speech segments, the data to be analyzed is divided into different classroom scenes, and corresponding scene labels are added; In this embodiment, the step S2 specifically includes: performing endpoint detection on the data to be analyzed, and dividing the audio data into speech segments and non-speech segments; Calculate the duration of each speech segment, the duration of the non-speech segment, and the average short-term amplitude of each speech segment ; in, represents the average short-term amplitude of each speech segment, Indicates The short-term amplitude of the frame audio, Indicates the frame length. Indicates The first frame of the audio signal Sample values, Indicates The starting frame number of the speech segment, Indicates The end frame signal of a speech segment, Indicates The number of frames contained in a speech segment; When the duration of classroom scene audio detection is greater than the time threshold, the average short-term amplitude of each speech segment is , calculate the sample mean square error ; Using the statistical characteristics of speech segments to identify classroom scenes: The first type of classroom scene is defined as the teacher lecturing and the students listening, the second type of classroom scene is defined as group teaching, the third type of classroom scene is defined as practice or testing, and the fourth type of classroom scene is self-study; If the speech segment duration is less than the duration threshold and the sample mean square error If it is less than the difference threshold, the current classroom scene is identified as one of the four types of classroom scenes; If the speech segment duration is less than the duration threshold and the sample mean square error If it is greater than or equal to the difference threshold, the current classroom scene is identified as a third-class classroom scene; If the current classroom scene identification is not both a Class 4 classroom scene and a Class 3 classroom scene, the pre-built teacher speech feature model is used to perform specific person recognition on the speech segment, obtain the specific person label of the speech segment, and simultaneously identify the unsuccessfully matched speech as the interactive speech of the corresponding specific person; if the total duration of each speech segment of the interactive speech of a specific person of the same class is greater than the specific person duration threshold, the current classroom scene identification is determined to be a Class 1 classroom scene; if it is less than or equal to the specific person duration threshold, the current classroom scene is determined to be a Class 2 classroom scene.

[0021] Step S3, based on the data to be analyzed with scene tags added, the teacher's classroom voice data and classroom interaction voice data are obtained by voice recognition; based on the time corresponding to the teacher's classroom voice data and the time corresponding to the classroom interaction voice data, the individual emotion index data of the same period are selected from the physiological index signal; In this embodiment, the teacher's classroom voice data and classroom interaction voice data are specifically: Perform specific person speech recognition on the data to be analyzed with added scene labels to obtain the teacher's complete classroom speech data within the current classroom evaluation analysis time window; The speech segment data outside the speech recognition result of a specific person is classified into the interactive speech corresponding to the specific person, and the complete classroom interactive speech data of the teacher within the current classroom evaluation and analysis time window is obtained; Loading physiological signals related to emotional fluctuations and capable of realizing emotion recognition recorded by personal wearable devices during the voice collection period as individual emotion indicator data; Based on complete classroom interaction voice data and individual emotion indicator data, teacher classroom voice data and classroom interaction voice data are obtained.

[0022] Step S4, performing short-term consistency analysis on the student end and the teacher end respectively through the teacher's classroom voice data and the classroom interactive voice data to obtain the short-term consistency analysis result; In this embodiment, the step S4 is specifically as follows: Based on the teacher's classroom voice data and classroom interactive voice data, a teacher's negative emotion set is constructed. , students' negative emotions , Student Safety Emotion Set ; For the data to be analyzed with the same scene labels, short-term consistency analysis is performed separately.

[0023] In this embodiment, if the scene label is a class scene, the short-term consistency analysis includes: Teacher classroom voice data The emotion recognition is performed on each speech segment of the teacher's classroom speech data, and the emotion recognition results of the speech segments with the same emotion recognition results and adjacent in time are merged. The speech segments after emotion merging are regarded as the same emotion speech segments. The recognition results of the same emotion speech segments of the teacher's classroom speech data are recorded as ; Teacher classroom interactive voice data The emotion recognition is performed on each speech segment of the teacher's classroom interactive speech data, and the emotion recognition results of the speech segments with the same emotion recognition results and adjacent in time are merged. The speech segments after the emotion merger are taken as the same emotion speech segments. ; in, Represents the first Emotion recognition results of speech segments with the same emotion, Represents a set of emotion recognition result numbers, Represents the teacher's classroom interactive voice data Emotion recognition results of speech segments with the same emotion, Represents the numbered set of emotion recognition results of teacher classroom interactive speech data, Indicates the teacher number corresponding to the classroom voice; Individual sentiment index data The physiological signal data in the data are divided into different individual data sets according to the device number of the signal source. Corresponding to the start and end time of each speech segment, the individual data set is divided into multiple sub-segments, and emotion recognition is performed on multiple segments of physiological signal data. The emotion recognition results of physiological signal data are: ; in, The first The individual Synchronous speech physiological signal emotion recognition results, Indicates the number of personal wearable devices; Construct interactive response emotion sequences based on the temporal nature of speech segments ; Specifically: Emotion recognition results of teacher classroom interactive speech data Find with Has the shortest time interval and the start collection time is later than The element of the end time is recorded as Interactive response emotions ; Calculate the first emotion sequence used to describe the emotional relevance of teacher-student interaction and a second emotion sequence used to describe the emotional relevance of students in the same class ; in, ; ; The valence-arousal arousal emotion model (VA) is used. The situation where the product of the valence components of two emotion recognition results is negative and the product of the arousal components of two emotion recognition results is negative is recorded as mutually exclusive; the situation where the product of the corresponding valence components of two emotion recognition results is negative is recorded as partially exclusive; the situation where the two emotion recognition results are different but the product of their valence components is positive and the product of their arousal components is positive is recorded as similar; The teacher-side short-term consistency analysis includes: teacher's short-term emotional control ; Construct short-term consistency analysis on the student side as a set of abnormal supervisors ; Construct the teacher-side short-term interaction degree ; in, Indicates a class scenario The degree of emotion following of each personal wearable device to the individual, ; represents the threshold of emotion consistency ratio; If Establishment of regular expression The value is 1, otherwise the expression value is 0; If Establishment of regular expression The value is 1, otherwise the expression value is 0; If , then obtain The corresponding device number classifies the student corresponding to the device number into the abnormal supervision personnel set; Indicates the number of elements contained in the abnormal supervisor set; It represents the number of normal abnormal persons, that is, if the ratio of persons in the set of abnormal supervisors who are classified as abnormal supervisors in a type of classroom scenario in which they appear is greater than the set threshold, they will be included in the calculation of the number of normal abnormal persons.

[0024] In this embodiment, if the scene label is a second-class classroom scene, the short-term consistency analysis includes: Based on teacher classroom interactive voice data Obtain the recognition results of the same-emotion speech segments of the teacher's classroom interactive speech data ; For individual emotion index data Each physiological signal data is divided into different individual data sets according to the device number of the signal source. Corresponding to the start and end time of each speech segment, the physiological signal data is divided into multiple sub-segments, and emotion recognition is performed on the physiological signals of multiple sub-segments. The emotion recognition result of the physiological signal data is recorded as ; Get the second emotion sequence describing the emotional relevance of students in the same class ; Constructing short-term consistency analysis on the student side as an abnormal supervisory personnel set ; in, Indicates the second type of classroom scenario The degree of emotion following of each personal wearable device to the individual, , If , then obtain Corresponding to the device number, the student corresponding to the device number is classified into the abnormal supervision personnel set, wherein the device number is an individual number.

[0025] In this embodiment, if the scene labels are three-class classroom scenes and four-class classroom scenes, the short-term consistency analysis includes: Individual sentiment index data The physiological signal data in the dataset are divided into different individual data sets according to the device number of the signal source, and the emotion recognition results are obtained as follows: ; in, express No. The individual Physiological emotion recognition results, Indicates the total number of emotions; Construct three types of classroom scenarios and analyze the short-term consistency of students as abnormal supervisors ; When constructing four types of classroom scenarios, the short-term consistency analysis of the student end is the abnormal supervisor set ; in, If , then obtain Corresponding device number, the student corresponding to the device number is classified into the abnormal supervision personnel set; If , then obtain Corresponding device number, the student corresponding to the device number is classified into the abnormal supervision personnel set; Express emotions separately duration, class duration constant; student safety emotion set The highest frequency emotions among all persons’ emotion recognition results are further expanded online to increase the safe emotion set of students in special contexts.

[0026] Step S5, based on the short-term consistency analysis result, performing long-term consistency analysis on the student end and the teacher end respectively to obtain a long-term consistency analysis result; In this embodiment, a long-term consistency analysis is performed on the teacher side, specifically: Based on the short-term emotion control degree of each teacher, the long-term emotion control degree of each teacher is calculated as the ratio of each teacher's short-term emotion control degree to the length of their class; Based on the short-term interaction degree obtained by each teacher, the long-term interaction degree of each teacher is calculated as the median of the short-term interaction degree of each teacher.

[0027] In this embodiment, long-term consistency analysis is performed on the student end, specifically: The frequency of each student in the data to be analyzed with each scenario label added is counted, and the frequency of being classified into the abnormal supervisor set is the ratio of the number of times they are classified into each abnormal supervisor set to the total number of their courses; the invalid listening rate of each student can be obtained in turn as the frequency of each student being classified into the abnormal supervisor set in the first and second classroom scenarios; the potential cheating rate of each student is the frequency of each student being classified into the abnormal supervisor set in the third classroom scenario; the self-study efficiency of each student is the frequency of each student being classified into the abnormal supervisor set in the fourth classroom scenario; Use the results of long- and short-time consistency analysis to supervise long- and short-time teaching.

[0028] Step S6, selecting a supervision object based on the short-term consistency analysis results and the long-term consistency analysis results.

[0029] In this embodiment, step S6 includes: Specifically, short-term supervision involves taking the abnormal supervision personnel sets in three types of classroom scenarios as key inspection targets and conducting adaptive proctoring; A set of abnormal supervisors in a classroom scenario is used as the focus of classroom interaction to solve the problem that teachers cannot take into account the status of all students in large classrooms; Classify the teacher with the highest short-term emotional control in a class as the object of video verification of teacher behavior to solve the problem of timely discovery of inappropriate teaching behavior; Long-term supervision specifically includes: eliminating those who are at the top of the long-term emotional control system, and establishing a teacher screening system based on emotional control; eliminating those who are at the bottom of the long-term interaction system, and establishing a teacher recommendation system based on interactivity; promptly identifying those with abnormal behavior, and providing personalized targeted teaching.

[0030] Although the various steps in the above embodiment are described in the above-mentioned order, those skilled in the art can understand that in order to achieve the effect of this embodiment, different steps do not have to be executed in such an order. They can be executed simultaneously (in parallel) or in a reverse order. These simple changes are within the scope of protection of the present invention.

[0031] An electronic device according to a third embodiment of the present invention includes: at least one processor; and a memory communicatively connected to at least one of the processors; wherein, The memory stores instructions that can be executed by the processor, and the instructions are used to be executed by the processor to implement the above-mentioned teaching supervision method based on consistency analysis.

[0032] A fourth embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to be executed by the computer to implement the above-mentioned teaching supervision method based on consistency analysis.

[0033] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process and related instructions of the storage device and processing device described above can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.

[0034] Those skilled in the art should be able to appreciate that the modules and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software, or a combination of the two, and the programs corresponding to the software modules and method steps can be placed in random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the technical field. In order to clearly illustrate the interchangeability of electronic hardware and software, the composition and steps of each example have been generally described in the above description according to the function. Whether these functions are performed in electronic hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0035] The terms "first", "second", etc. are used to distinguish similar objects rather than to describe or indicate a particular order or sequence.

[0036] The term "comprise" or any other similar term is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus / device that includes a list of elements includes not only those elements but also other elements not expressly listed, or also includes elements inherent to such process, method, article, or apparatus / device.

[0037] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

[0038] In addition, I think this paragraph can be omitted because it does not perform any function. All the data that needs to be obtained has been realized in the above statement of this paragraph in right 3. Please consider whether it needs to be retained.

Claims

1. A teaching supervision method based on consistency analysis, characterized in that: The method comprises: Step S1, collecting audio signals of multiple students during class periods as data to be analyzed; and collecting physiological indicator signals; Step S2, based on the data to be analyzed, according to the distribution characteristics of the speech segments, the data to be analyzed is divided into different classroom scenes, and corresponding scene labels are added; Step S3, based on the data to be analyzed with scene tags added, the teacher's classroom voice data and classroom interaction voice data are obtained by voice recognition; based on the time corresponding to the teacher's classroom voice data and the time corresponding to the classroom interaction voice data, the individual emotion index data of the same period are selected from the physiological index signal; Step S4, performing short-term consistency analysis on the student end and the teacher end respectively through the teacher's classroom voice data and the classroom interactive voice data to obtain the short-term consistency analysis result; Step S5, based on the short-term consistency analysis result, performing long-term consistency analysis on the student end and the teacher end respectively to obtain a long-term consistency analysis result; Step S6, selecting a supervision object based on the short-term consistency analysis results and the long-term consistency analysis results.

2. According to claim 1, a teaching supervision method based on consistency analysis is characterized in that: The step S2 specifically includes: performing endpoint detection on the data to be analyzed and dividing the audio data into speech segments and non-speech segments; Calculate the duration of each speech segment, the duration of the non-speech segment, and the average short-term amplitude of each speech segment ; in, represents the average short-term amplitude of each speech segment, Indicates The short-term amplitude of the frame audio, Indicates the frame length. Indicates The first frame of the audio signal Sample values, Indicates The starting frame number of a speech segment, Indicates The end frame signal of a speech segment, Indicates The number of frames contained in a speech segment; When the duration of classroom scene audio detection is greater than the time threshold, the average short-term amplitude of each speech segment is , calculate the sample mean square error ; Using the statistical characteristics of speech segments to identify classroom scenes: The first type of classroom scene is defined as the teacher lecturing and the students listening, the second type of classroom scene is defined as group teaching, the third type of classroom scene is defined as practice or testing, and the fourth type of classroom scene is self-study; If the speech segment duration is less than the duration threshold and the sample mean square error If it is less than the difference threshold, the current classroom scene is identified as one of the four types of classroom scenes; If the speech segment duration is less than the duration threshold and the sample mean square error If it is greater than or equal to the difference threshold, the current classroom scene is identified as a third-class classroom scene; If the current classroom scene identification is not both a Class 4 classroom scene and a Class 3 classroom scene, the pre-built teacher speech feature model is used to perform specific person recognition on the speech segment, obtain the specific person label of the speech segment, and simultaneously identify the unsuccessfully matched speech as the interactive speech of the corresponding specific person; if the total duration of each speech segment of the interactive speech of a specific person of the same class is greater than the specific person duration threshold, the current classroom scene identification is determined to be a Class 1 classroom scene; if it is less than or equal to the specific person duration threshold, the current classroom scene is determined to be a Class 2 classroom scene.

3. The teaching supervision method based on consistency analysis according to claim 1 is characterized in that: The teacher classroom voice data and classroom interactive voice data are specifically: Perform specific person speech recognition on the data to be analyzed with added scene labels to obtain the teacher's complete classroom speech data within the current classroom evaluation analysis time window; The speech segment data outside the speech recognition result of a specific person is classified into the interactive speech corresponding to the specific person, and the complete classroom interactive speech data of the teacher within the current classroom evaluation and analysis time window is obtained; Loading physiological signals related to emotional fluctuations and capable of realizing emotion recognition recorded by personal wearable devices during the voice collection period as individual emotion indicator data; Based on complete classroom interaction voice data and individual emotion indicator data, teacher classroom voice data and classroom interaction voice data are obtained.

4. A teaching supervision method based on consistency analysis according to claim 3, characterized in that: The step S4 is specifically: Based on the teacher's classroom voice data and classroom interactive voice data, a teacher's negative emotion set is constructed. , students' negative emotions , Student Safety Emotion Set ; For the data to be analyzed with the same scene labels, short-term consistency analysis is performed separately.

5. A teaching supervision method based on consistency analysis according to claim 4, characterized in that: If the scene label is a class scene, the short-term consistency analysis includes: Teacher classroom voice data The emotion recognition is performed on each speech segment of the teacher's classroom speech data, and the emotion recognition results of the speech segments with the same emotion recognition results and adjacent in time are merged. The speech segments after emotion merging are regarded as the same emotion speech segments. The recognition results of the same emotion speech segments of the teacher's classroom speech data are recorded as ; Teacher classroom interactive voice data The emotion recognition is performed on each speech segment of the teacher's classroom interactive speech data, and the emotion recognition results of the speech segments with the same emotion recognition results and adjacent in time are merged. The speech segments after the emotion merger are taken as the same emotion speech segments. ; in, Represents the first Emotion recognition results of speech segments with the same emotion, Represents a set of emotion recognition result numbers, Represents the teacher's classroom interactive voice data Emotion recognition results of speech segments with the same emotion, Represents the numbered set of emotion recognition results of teacher classroom interactive speech data, Indicates the teacher number corresponding to the classroom voice; Individual sentiment indicator data The physiological signal data in the data are divided into different individual data sets according to the device number of the signal source. Corresponding to the start and end time of each speech segment, the individual data set is divided into multiple sub-segments, and emotion recognition is performed on multiple segments of physiological signal data. The emotion recognition results of physiological signal data are: ; in, The first The individual Synchronous speech physiological signal emotion recognition results, Indicates the number of personal wearable devices; Construct interactive response emotion sequences based on the temporal nature of speech segments ; Specifically: Emotion recognition results of teacher classroom interactive speech data Find with Has the shortest time interval and the start collection time is later than The element of the end time is recorded as Interactive response emotions ; Calculate the first emotion sequence used to describe the emotional relevance of teacher-student interaction and a second emotion sequence used to describe the emotional relevance of students in the same class ; in, ; ； The valence-arousal arousal emotion model (VA) is used. The situation where the product of the valence components of two emotion recognition results is negative and the product of the arousal components of two emotion recognition results is negative is recorded as mutually exclusive; the situation where the product of the corresponding valence components of two emotion recognition results is negative is recorded as partially exclusive; the situation where the two emotion recognition results are different but the product of their valence components is positive and the product of their arousal components is positive is recorded as similar; The teacher's short-term consistency analysis includes: teacher's short-term emotional control ; Construct short-term consistency analysis on the student side as a set of abnormal supervisors ; Construct the teacher-side short-term interaction degree ; in, Indicates a class scenario The degree of emotion following of each personal wearable device to the individual, ; represents the threshold of emotion consistency ratio; If Establishment of regular expression The value is 1, otherwise the expression value is 0; If Establishment of regular expression The value is 1, otherwise the expression value is 0; If , then obtain The corresponding device number classifies the student corresponding to the device number into the abnormal supervision personnel set; Indicates the number of elements contained in the abnormal supervisor set; It represents the number of normal abnormal persons, that is, if the ratio of persons in the set of abnormal supervisors who are classified as abnormal supervisors in a type of classroom scenario in which they appear is greater than the set threshold, they will be included in the calculation of the number of normal abnormal persons.

6. The teaching supervision method based on consistency analysis according to claim 5 is characterized in that: If the scene label is a second-class classroom scene, the short-term consistency analysis includes: Based on teacher classroom interactive voice data Obtain the recognition results of the same-emotion speech segments of the teacher's classroom interactive speech data ; For individual emotion index data Each physiological signal data is divided into different individual data sets according to the device number of the signal source. Corresponding to the start and end time of each speech segment, the physiological signal data is divided into multiple sub-segments, and emotion recognition is performed on the physiological signals of multiple sub-segments. The emotion recognition result of the physiological signal data is recorded as ; Get the second emotion sequence describing the emotional relevance of students in the same class ; Constructing short-term consistency analysis on the student side as an abnormal supervisory personnel set ; in, Indicates the second type of classroom scenario The degree of emotion following of each personal wearable device to the individual, , If , then obtain Corresponding to the device number, the student corresponding to the device number is classified into the abnormal supervision personnel set, wherein the device number is an individual number.

7. A teaching supervision method based on consistency analysis according to claim 6, characterized in that: If the scene labels are three-class classroom scenes and four-class classroom scenes, the short-term consistency analysis includes: Individual sentiment index data The physiological signal data in the dataset are divided into different individual data sets according to the device number of the signal source, and the emotion recognition results are obtained as follows: ; in, express No. The individual Physiological emotion recognition results, Indicates the total number of emotions; Construct three types of classroom scenarios and analyze the short-term consistency of students as abnormal supervisors ; When constructing four types of classroom scenarios, the short-term consistency analysis of the student end is the abnormal supervisor set ; in, If , then obtain Corresponding device number, the student corresponding to the device number is classified into the abnormal supervision personnel set; If , then obtain Corresponding device number, the student corresponding to the device number is classified into the abnormal supervision personnel set; Express emotions separately duration, class duration constant; student safety emotion set The highest frequency emotions among all persons’ emotion recognition results are further expanded online to increase the safe emotion set of students in special contexts.

8. The teaching supervision method based on consistency analysis according to claim 7 is characterized in that: Long-term consistency analysis is conducted on the teacher side, specifically: Based on the short-term emotion control degree of each teacher, the long-term emotion control degree of each teacher is calculated as the ratio of each teacher's short-term emotion control degree to the length of their class; Based on the short-term interaction degree obtained by each teacher, the long-term interaction degree of each teacher is calculated as the median of the short-term interaction degree of each teacher.

9. A teaching supervision method based on consistency analysis according to claim 8, characterized in that: Long-term consistency analysis is conducted on the student side, specifically: The frequency of each student in the data to be analyzed with each scenario label added is counted, and the frequency of being classified into the abnormal supervisor set is the ratio of the number of times they are classified into each abnormal supervisor set to the total number of their courses; the invalid listening rate of each student can be obtained in turn as the frequency of each student being classified into the abnormal supervisor set in the first and second classroom scenarios; the potential cheating rate of each student is the frequency of each student being classified into the abnormal supervisor set in the third classroom scenario; the self-study efficiency of each student is the frequency of each student being classified into the abnormal supervisor set in the fourth classroom scenario; Use the results of long- and short-time consistency analysis to supervise long- and short-time teaching.

10. A teaching supervision method based on consistency analysis according to claim 9, characterized in that: The step S6 comprises: Specifically, short-term supervision involves taking the abnormal supervision personnel sets in three types of classroom scenarios as key inspection targets and conducting adaptive proctoring; A set of abnormal supervisors in a classroom scenario is used as the focus of classroom interaction to solve the problem that teachers cannot take into account the status of all students in large classrooms; Classify the teacher with the highest short-term emotional control in a class as the object of video verification of teacher behavior to solve the problem of timely discovery of inappropriate teaching behavior; Long-term supervision specifically includes: eliminating those who are at the top of the long-term emotional control system, and establishing a teacher screening system based on emotional control; eliminating those who are at the bottom of the long-term interaction system, and establishing a teacher recommendation system based on interactivity; promptly identifying those with abnormal behavior, and providing personalized targeted teaching.

Images