Classroom analysis information presentation method, computer equipment and storage medium

Abstract

The embodiment of the invention relates to the technical field of teaching management, in particular to a classroom analysis information presentation method, which comprises the following steps: acquiring classroom observation data; determining a prompt word obtained by an interactive interface of the electronic equipment; generating classroom analysis information through a preset large language model according to classroom observation data and the cue word; and displaying the classroom analysis information on the interactive interface. Compared with the prior art, the presentation method provided by the embodiment of the invention can generate the corresponding classroom analysis information according to the cue word input by the user, so that the user can obtain the classroom analysis information of different dimensions by inputting the cue words of different dimensions, thereby improving the interactivity between the classroom analysis process and the user, and improving the user experience. The user can better understand the content of the classroom analysis information, and the classroom teaching effect is improved.

Description

Technical Field

[0001] Embodiments of the present application relate to the technical field of teaching management, and in particular, to a method for presenting classroom analysis information, a computer device, and a storage medium. Background Art

[0002] Classroom analysis is an important means of measuring the teaching level of a school. Reasonable and accurate classroom analysis helps improve students' academic performance and thinking ability, and improves teachers' teaching effectiveness. Traditional classroom analysis mainly takes the form of on-site classroom observation, which is inefficient and difficult to reflect the normal state of daily teaching, resulting in poor analysis results.

[0003] In order to solve the above problems, the current classroom analysis system mainly collects video data during classroom teaching and combines computer vision and automatic speech recognition technology for quantitative analysis to analyze whether the quality of classroom teaching has achieved the expected results. However, this method has poor interactivity with users, and the analysis information provided is simply presented to users in the form of data or charts, which makes it difficult for users to accurately understand the content of the analysis information, thus affecting the effect of classroom analysis. Summary of the invention

[0004] One purpose of an embodiment of the present application is to provide a method for presenting classroom analysis information to solve the technical problem that the current classroom analysis process has poor interactivity with users, which affects the effect of classroom analysis.

[0005] In a first aspect, an embodiment of the present application provides a method for presenting classroom analysis information, which is applied to an electronic device, comprising:

[0006] Obtain classroom observation data;

[0007] Determining a prompt word obtained by an interactive interface of the electronic device;

[0008] Generate classroom analysis information through a preset large language model according to the classroom observation data and the prompt words;

[0009] The classroom analysis information is presented on the interactive interface.

[0010] In conjunction with the first aspect, in a possible implementation manner, the interactive interface is configured with a text input box, and determining the prompt word obtained by the interactive interface of the electronic device includes:

[0011] Obtaining a portion of text obtained from the text input box;

[0012] Traversing one or more candidate words corresponding to the part of the text in a preset prompt word library;

[0013] When an operation of selecting a candidate word is detected, the selected candidate word is determined as a prompt word.

[0014] In combination with the first aspect, in a possible implementation, generating classroom analysis information by using a preset large language model according to the classroom observation data and the prompt words includes:

[0015] Generate a classroom analysis report based on the classroom observation data through a preset large language model;

[0016] Classroom analysis information is generated according to the classroom analysis report and the prompt words.

[0017] In combination with the first aspect, in a possible implementation, the classroom observation data includes classroom dynamic data and teacher static data, and generating a classroom analysis report based on the classroom observation data through a preset large language model includes:

[0018] Performing structural processing on the classroom dynamic data and the teacher static data respectively to obtain dynamic structured data and static structured data;

[0019] The dynamic structured data, the static structured data and the prompt words are input into the large language model so that the large language model outputs the classroom analysis report.

[0020] In combination with the first aspect, in a possible implementation manner, the classroom dynamic data includes audio data collected in the classroom, the dynamic structured data includes audio structured data, and the structural processing of the classroom dynamic data to obtain the dynamic structured data includes:

[0021] Converting the audio data into conversation text data;

[0022] The dialogue text data is structured to obtain the audio structured data.

[0023] In combination with the first aspect, in a possible implementation manner, converting the audio data into dialogue text data includes:

[0024] Preprocessing the audio data to obtain audio feature information;

[0025] Inputting the audio feature information into a preset voiceprint model for voiceprint recognition to obtain the identity information of the interlocutor;

[0026] Inputting the audio feature information and identity information into a preset acoustic model to perform phoneme prediction to obtain phoneme information;

[0027] The phoneme information is input into a preset language model for text conversion to obtain the dialogue text data.

[0028] In combination with the first aspect, in a possible implementation, the classroom dynamic data includes video data collected in the classroom, the dynamic structured data includes video structured data, and the structural processing of the classroom dynamic data to obtain the dynamic structured data includes:

[0029] Extracting features from the video data to obtain video feature information;

[0030] The video feature information is structured to obtain the video structured data.

[0031] In combination with the first aspect, in a possible implementation manner, the video feature information includes character information, emotion information, and action information, and the extracting features from the video data to obtain the video feature information includes:

[0032] Inputting the video data into a preset character recognition model to perform human posture recognition to determine the character information;

[0033] Inputting the video data into a preset expression recognition model to perform facial expression recognition to determine the emotion information;

[0034] The video data is input into a preset target detection model for action detection to determine the action information.

[0035] In combination with the first aspect, in a possible implementation manner, the performing structured processing on the video feature information to obtain the video structured data includes:

[0036] Slice the video features according to a preset slice duration to obtain multiple video clips;

[0037] Determining video feature information corresponding to each of the video clips;

[0038] Each of the video feature information is structured to obtain video structured data corresponding to each of the video clips.

[0039] In a second aspect, the present application also provides a device for presenting classroom analysis information, including:

[0040] Data acquisition module, used to obtain classroom observation data;

[0041] A data determination module, used to determine a prompt word obtained by the interactive interface of the electronic device;

[0042] An information generation module, used to generate classroom analysis information through a preset large language model according to the classroom observation data and the prompt words;

[0043] An interactive presentation module is used to present the classroom analysis information on the interactive interface.

[0044] In a third aspect, an embodiment of the present application further proposes a computer device, comprising a memory and a processor, wherein the memory is connected to the processor, and the processor is used to execute one or more computer programs stored in the memory, and when the processor executes the one or more computer programs, the computer device implements the method as described above.

[0045] In a fourth aspect, an embodiment of the present application further proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the method as described above.

[0046] The embodiments of the present application can achieve the following technical effects:

[0047] Based on the method proposed in the embodiment of the present application, after obtaining the classroom observation data, the prompt words input by the user through the electronic device are first determined, and based on the classroom observation data and the prompt words, the preset large language model is used to generate classroom analysis information, and finally the classroom analysis information is presented in the interactive interface of the electronic device. Compared with the prior art, the presentation method proposed in the embodiment of the present application can generate corresponding classroom analysis information according to the prompt words input by the user, so that the user can obtain classroom analysis information of different dimensions according to the prompt words of different dimensions, thereby improving the interactivity between classroom analysis and users, helping users to better understand the content of classroom analysis information and improve the effect of classroom teaching. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0049] Figure 1 A schematic diagram of a classroom analysis system provided in an embodiment of the present application;

[0050] Figure 2 A flowchart of a method for presenting classroom analysis information provided in an embodiment of the present application;

[0051] Figure 3 A schematic diagram of the structure of a device for presenting classroom analysis information provided in an embodiment of the present application;

[0052] Figure 4A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0054] It should be noted that, if there is no conflict, the various features in the embodiments of the present application can be combined with each other, all within the scope of protection of the present application. In addition, although the functional module division is performed in the device schematic diagram and the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in a sequence different from the module division in the device or the flow chart. Furthermore, the words "first", "second", "third", etc. used in this application do not limit the data and execution order, but only distinguish the same items or similar items with basically the same functions and effects.

[0055] In order to facilitate the explanation of this application, the application scenarios involved in the embodiments of this application are first introduced. Figure 1 , Figure 1 The figure shows a classroom analysis system 10, including a collection device 11, a server 12 and a terminal device 13. The server 12 is respectively connected to the collection device 11 and the terminal device 13 in communication, so as to receive the classroom observation data collected by the collection device 11 and analyze and process them. The results of the analysis and processing are sent to the terminal device 13 for presentation. The communication connection can be in the form of wired communication connection and wireless communication connection. The wired communication connection includes various communication connections that use tangible media such as metal wires and optical fibers to transmit information; the wireless communication connection includes 5G communication, 4G communication, 3G communication, 2G communication, CDMA, Zig-Bee, Bluetooth, wireless broadband (Wi-Fi), ultra-wideband (UWB) and near field communication (NFC), CDMA2000, GSM, Infrared (IR), ISM, RFID, UMTS / 3GPPw / HSDPA, WiMAXWi-Fi or ZigBee, etc.

[0056] As a feasible implementation, the collection device 11 is set in the classroom and is equipped with a communication module and a sensor component. The communication module is used to communicate with the server 12, and the sensor component is used to collect and record classroom observation data, mainly including audio and video data generated by teachers and students during classroom teaching. For example, the sensor component may include a microphone and / or a camera, the microphone is used to collect audio data during teaching, and the camera is used to collect video data during teaching.

[0057] In some embodiments, the server 10 stores program instructions for analyzing and processing classroom observation data. The program instructions are stored in a storage medium of the server 10. After obtaining the classroom observation data collected by the collection device 11, the server 10 calls and executes the program instructions in its storage medium to generate corresponding classroom analysis information. The classroom analysis information is sent to the terminal device 13. The terminal device 13 is provided with a display device, which presents the classroom analysis information sent by the server in a form that can be understood by the user, such as a data table, a bar chart or a text report, so that the user can understand the classroom teaching situation through the terminal device 13.

[0058] based on Figure 1 The classroom analysis system shown in the present application embodiment proposes a method for presenting classroom analysis information, which can be applied to Figure 1 Refer to the server or terminal device shown. Figure 2 ,like Figure 2 As shown, the methods of presenting classroom analysis information include:

[0059] Step S10, obtaining classroom observation data;

[0060] Among them, classroom observation data refers to relevant data that can be used to analyze the teaching effect of teachers. By obtaining and analyzing classroom observation data, it is possible to determine whether the classroom teaching has achieved the expected goals and whether there are areas for improvement, so as to make reasonable suggestions for the teacher's subsequent classroom teaching. It is understandable that based on the above Figure 1 Taking the classroom analysis system shown as an example, the classroom observation data is recorded in real time on the server 12 through the collection device 11, and the server 12 stores it in a built-in storage medium or in the cloud for subsequent analysis and processing.

[0061] In some embodiments, classroom observation data includes audio and video data generated by teachers and students during classroom teaching and recorded in real time by acquisition devices in the classroom. Subsequent analysis and processing of this audio and video data can be used to understand the actual situation in the classroom, which is conducive to teachers making more targeted teaching adjustments.

[0062] In some embodiments, classroom observation data may also include the teaching objectives set by the teacher before teaching, which can be achieved through this classroom teaching, etc. Combined with such data, the audio and video data in the classroom can be analyzed more accurately, which is conducive to determining the performance of a specific teacher in a specific course.

[0063] Step S20, determining a prompt word obtained from the interactive interface of the electronic device;

[0064] The electronic device referred to in this embodiment is a device with data processing functions and certain communication capabilities, which may include computers, servers, smart phones, tablet computers or other user terminals, etc. It is understandable that the electronic device is provided with a collection device for collecting user input information, such as a touch screen, keyboard, handwriting board, gesture collection device, etc. The user inputs the interaction information into the interaction interface through the collection device, so that the electronic device determines the prompt word input by the user according to the interaction information.

[0065] The prompt word refers to a keyword or phrase used to guide the language model to understand the question and generate a corresponding answer. The language model generates the answer expected by the user based on the prompt word input by the user. For example, in this embodiment, the prompt word input by the user can be an inquiry sentence, such as asking whether the expected teaching goal has been achieved in this classroom teaching, so that the electronic device returns an answer text based on the inquiry sentence.

[0066] Step S30, generating classroom analysis information through a preset large language model according to classroom observation data and prompt words;

[0067] Among them, the big language model refers to the neural network model used to understand and generate natural language text, and the classroom analysis information refers to the answer text obtained after understanding the prompt words through the big language model and analyzing and processing the classroom observation data.

[0068] In some embodiments, the large language model is obtained by training the initial model using a training data set. During the training process, the initial model learns specific grammar, vocabulary and word meanings through historical data sets, so that it can understand the analysis dimensions represented by the prompt words and generate answer text according to the analysis dimensions.

[0069] In some embodiments, the initial model is a model that can process sequential text data and extract text contextual relationships, and its specific form includes a recurrent neural network or its variants, such as a long short-term memory network or a gated recurrent unit, etc. The training data set is obtained by preprocessing the history classroom observation data, and the historical data set is input into the initial model. The initial model performs supervised learning based on the training data set. During the training process, the initial model predicts the next paragraph of text based on the input contextual information, and adjusts its own parameters based on the difference between the predicted result and the actual next paragraph of text. After multiple cycles of iteration, the final large language model is obtained.

[0070] Step S40: presenting the classroom analysis information on the interactive interface.

[0071] It can be understood that in this step, the classroom analysis information can be presented in a variety of forms, such as a data table, a bar chart, a text report, etc., that is, the results obtained for the same prompt word input by the user can be presented in a variety of easy-to-understand forms to enhance readability.

[0072] Furthermore, in the above embodiment, the interactive interface is configured with a text input box, and the step of determining the prompt word obtained by the interactive interface of the electronic device includes: obtaining part of the text obtained by the text input box; traversing one or more candidate words corresponding to the part of the text in a preset prompt word library; and when an operation of selecting a candidate word is detected, determining the selected candidate word as the prompt word.

[0073] Among them, the text input box is an input area configured on the interactive interface, which is used to display the entered text and the current editing position. For example, when the user enters text information in the text input box through an electronic device, the input event of the text input box will be triggered. Then, the text information entered by the user will be compared with the various candidate words in the preset prompt word library through the program interface corresponding to the text box, and the prompt word entered by the user will be determined.

[0074] In some embodiments, the electronic device confirms the prompt word that the user intends to input by comparing the similarity between the text input by the user and the candidate words. It is easy to understand that the text input by the user is sometimes difficult to completely correspond to the candidate words. Therefore, after the user enters the text, the candidate word closest to the user input text will be determined in the prompt word library. The candidate word is used to represent the prompt word that the user actually needs to input.

[0075] Furthermore, generating classroom analysis information through a preset large language model based on classroom observation data and prompt words includes: generating a classroom analysis report through a preset large language model based on the classroom observation data; generating classroom analysis information based on the classroom analysis report and the prompt words.

[0076] In this embodiment, the classroom analysis report refers to the analysis results of the classroom teaching obtained based on the classroom observation data, which includes multiple analysis results obtained after analyzing and processing the classroom observation data from various dimensions. For example, the dimensions of the large language model's analysis and processing of classroom observation data include the implementation of the new curriculum standards, teacher-student interaction analysis, and teacher question analysis. The classroom analysis report includes the analysis results based on the implementation of the new curriculum standards, teacher-student interaction analysis, and teacher question analysis. It is easy to understand that different users have different focuses on classroom analysis. Therefore, for a specific user, determining its focus based on the prompt words it inputs can make the classroom analysis information more readable and the user experience better.

[0077] In some embodiments, the classroom observation data includes classroom dynamic data and teacher static data. Based on the classroom observation data, a classroom analysis report is generated through a preset large language model, including: structuring the classroom dynamic data and the teacher static data respectively to obtain dynamic structured data and static structured data; inputting the dynamic structured data, static structured data and prompt words into the large language model, so that the large language model outputs the classroom analysis report.

[0078] Among them, classroom dynamic data refers to data collected during classroom teaching, teacher static data refers to data that can be determined before classroom teaching, and structured processing refers to organizing a large amount of classroom observation data into a data form that can be processed by the large language model. For example, classroom dynamic data can include audio and video data in the classroom collected by cameras, microphones and other equipment installed in the classroom during classroom teaching, which can reflect the actual situation in this classroom teaching process. Teacher static data can include teacher information, such as the teacher's subject section information (such as grade, subject, etc.), teaching experience, professional background, educational philosophy and classroom management style, etc., which helps the large language model to more accurately analyze the teacher's performance in actual classroom teaching.

[0079] In some embodiments, classroom dynamic data includes audio data collected in the classroom, and correspondingly, dynamic structured data includes audio structured data. Structuring the classroom dynamic data to obtain dynamic structured data includes: converting the audio data into dialogue text data; and structuring the dialogue text data to obtain the audio structured data.

[0080] Among them, audio data can be converted into textual dialogue data through technologies such as ASR (Automatic Speech Recognition). It is easy to understand that the large language model can understand the semantics of various dialogues in classroom teaching through the input dialogue text data, analyze and process the semantics, and combine it with other data to obtain a classroom analysis report.

[0081] In some embodiments, converting audio data into conversation text data includes: preprocessing the audio data to obtain audio feature information; inputting the audio feature information into a preset voiceprint model for voiceprint recognition to obtain identity information of the interlocutor; inputting the audio feature information and identity information into a preset acoustic model for phoneme prediction to obtain phoneme information; inputting the phoneme information into a preset language model for text conversion to obtain conversation text data.

[0082] Among them, the preprocessing stage is mainly used to convert the original audio signal in the audio data into a form more suitable for machine processing. This is achieved by extracting audio feature information from the original audio signal, such as Mel-Frequency Cepstral Coefficients (MFCC) features and Linear Predictive Cepstral Coefficients (LPCC) features. These features are used to characterize the key attributes of the original audio signal, such as pitch, timbre or tone.

[0083] The specific form of the voiceprint model can be any model that can identify an individual through audio feature information. Its basic principle is to determine the identity information of the interlocutor by analyzing and comparing the input audio feature information. Specifically, the interlocutors in this embodiment include teachers and students in a predetermined classroom teaching. The voiceprint model used for voiceprint recognition can be a GMM-UBM (Gaussian Mixture Model-Universal Background Model) model, an i-vector model, or other similar deep neural network models.

[0084] The specific form of the acoustic model can be any model that can perform phoneme analysis, such as HMM (Hidden Markov Model) or similar deep neural network models. The acoustic model in this embodiment is used to learn the relationship between the audio feature information and phonemes of each interlocutor, so as to predict the actual phoneme information based on the input audio feature information and identity information.

[0085] The specific form of the language model can be any model that can generate text based on the phonemes output by the acoustic model, such as an n-gram model, a recurrent neural network (RNN) model, a Transformer model, etc. The language model in this embodiment predicts the most likely word sequence by learning preset statistical properties of word sequences, and converts the phoneme information into dialogue text data.

[0086] In some embodiments, classroom dynamic data includes video data collected in the classroom, and accordingly, dynamic structured data includes video structured data. Structuring the classroom dynamic data to obtain dynamic structured data includes: extracting features from the video data to obtain video feature information; and structuring the video feature information to obtain video structured data.

[0087] The video feature information refers to valuable information obtained through feature extraction, specifically information related to different analysis dimensions.

[0088] In some embodiments, video feature information includes character information, emotion information and action information, and feature extraction is performed on video data to obtain video feature information, including: inputting the video data into a preset character recognition model for human posture recognition to determine character information; inputting the video data into a preset expression recognition model for emotion recognition to determine emotion information; inputting the video data into a preset target detection model for action detection to determine action information.

[0089] Among them, character information refers to the postures and movements of each character in the original video data, such as the movement trajectory of a teacher in the classroom; emotional information refers to the emotional state determined by the facial expressions of each character in the original video data; action information refers to the key actions of each character in the original video data, such as the action of a student raising his hand to answer a teacher.

[0090] The specific form of the person recognition model can be any model that can detect and estimate human posture, such as OpenPose and other similar deep learning models. Its basic principle is to recognize the posture and movement of the human body by identifying and tracking the key points of the human body (such as head, torso, arms, legs, etc.) and the connection relationship between the human body.

[0091] The specific form of the expression recognition model can be any model that can perform emotion recognition, such as EmotionNet and other similar deep learning models. Its basic principle is to infer the emotional state of each character by identifying the facial expressions of each character in the original video data.

[0092] The specific form of the target detection model can be any model that can perform key target detection, such as YOLO, SSD and other similar deep learning models. The basic principle is to train with a preset data set to identify key objects in the original video data and generate the bounding box coordinates of the key objects to achieve the effect of key target detection.

[0093] In some embodiments, performing structured processing on the video feature information to obtain video structured data includes: performing slicing processing on the video feature according to a preset slicing duration to obtain multiple video segments; determining video feature information corresponding to each of the video segments; and performing structured processing on each of the video feature information to obtain video structured data corresponding to each of the video segments.

[0094] Among them, the preset slice duration can be 3 to 5 seconds. According to the slice duration, the original video data can be cut into multiple video clips. The structured processing refers to a data object including the video feature information corresponding to the video clip, and its specific form can be a data structure that can represent the video feature information. Specifically, the structured object can be a JSON object, which includes the video feature information of a video clip. For example, the video feature information of a video clip can include the following fields: frame_id (frame number), timestamp (timestamp), people (list of people), objects (list of objects), wherein the people field includes the information of each person in the video clip, such as id (character number), pose (position key point coordinates), emotion (emotional state), etc.; the objects field includes the information of each behavior object, such as id (object number), type (object type), bbox (bounding box coordinates), etc.

[0095] In some embodiments, the large language model first generates a classroom analysis report based on classroom observation data, and then determines classroom analysis information in the classroom analysis report based on the prompt words input by the user and presents it to the user, that is, the classroom analysis report is generated by the large language model based on the dimensions represented by all prompt words in the preset prompt word library. Therefore, if the corresponding classroom analysis information cannot be determined in the classroom analysis report based on the prompt words input by the user, the prompt words and classroom observation data input by the user can be re-input into the large language model, so that the large language model outputs the classroom analysis information corresponding to the prompt words based on the input prompt words and classroom observation data, and adds the prompt words to the preset prompt word library to improve the diversity of the prompt word library.

[0096] It should be noted that, in each of the above-mentioned embodiments, there is not necessarily a certain order between the above-mentioned steps. A person skilled in the art can understand, based on the description of the embodiments of the present application, that in different embodiments, the above-mentioned steps may have different execution orders, that is, they may be executed in parallel, may be executed interchangeably, and so on.

[0097] As another aspect of the embodiment of the present application, the embodiment of the present application provides a device for presenting classroom analysis information. The device for presenting classroom analysis information can be a software module, which includes several instructions stored in a memory, and the processor can access the memory and call the instructions for execution to complete the method for presenting classroom analysis information described in the above-mentioned various embodiments.

[0098] In some embodiments, the presentation device of classroom analysis information can also be constructed by hardware devices. For example, the presentation device of classroom analysis information can be constructed by one or more chips, and each chip can work in coordination with each other to complete the presentation method of classroom analysis information described in each of the above embodiments. For another example, the presentation device of classroom analysis information can also be constructed by various logic devices, such as general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), single-chip microcomputers, ARM (Acorn RISC Machines) or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination of these components.

[0099] Specifically, see Figure 3 , Figure 3 This is a structural diagram of a device for presenting classroom analysis information proposed in an embodiment of the present application, such as Figure 3 As shown, the classroom analysis information presentation device 30 includes:

[0100] Data acquisition module 301, used to acquire classroom observation data;

[0101] A data determination module 302, used to determine a prompt word obtained from the interactive interface of the electronic device;

[0102] An information generation module 303 is used to generate classroom analysis information through a preset large language model according to the classroom observation data and the prompt words;

[0103] The interactive presentation module 304 is used to present the classroom analysis information on the interactive interface.

[0104] In a possible implementation, the interactive interface is configured with a text input box, and the data module determination module 302, when used to determine the prompt word obtained by the interactive interface of the electronic device, is specifically used to:

[0105] Obtaining a portion of text obtained from the text input box;

[0106] Traversing one or more candidate words corresponding to the part of the text in a preset prompt word library;

[0107] When an operation of selecting a candidate word is detected, the selected candidate word is determined as a prompt word.

[0108] In a possible implementation, the information generation module 303, when used to generate classroom analysis information according to the classroom observation data and prompt words through a preset large language model, is specifically used to:

[0109] Generate a classroom analysis report based on the classroom observation data through a preset large language model;

[0110] Classroom analysis information is generated according to the classroom analysis report and the prompt words.

[0111] In a possible implementation, the classroom observation data includes classroom dynamic data and teacher static data. When the information generation module 303 is used to generate a classroom analysis report according to the classroom observation data through a preset large language model, it is specifically used to:

[0112] Performing structural processing on the classroom dynamic data and the teacher static data respectively to obtain dynamic structured data and static structured data;

[0113] The dynamic structured data, the static structured data and the prompt words are input into the large language model so that the large language model outputs the classroom analysis report.

[0114] In a possible implementation, the classroom dynamic data includes audio data collected in the classroom, the dynamic structured data includes audio structured data, and the information generation module 303, when used to perform structured processing on the classroom dynamic data to obtain dynamic structured data, is specifically used to:

[0115] Converting the audio data into conversation text data;

[0116] The dialogue text data is structured to obtain the audio structured data.

[0117] In a possible implementation, the information generating module 303, when used to convert the audio data into the dialogue text data, is specifically used to:

[0118] Preprocessing the audio data to obtain audio feature information;

[0119] Inputting the audio feature information into a preset voiceprint model for voiceprint recognition to obtain the identity information of the interlocutor;

[0120] Inputting the audio feature information and identity information into a preset acoustic model to perform phoneme prediction to obtain phoneme information;

[0121] The phoneme information is input into a preset language model for text conversion to obtain the dialogue text data.

[0122] In a possible implementation, the classroom dynamic data includes video data collected in the classroom, the dynamic structured data includes video structured data, and the information generation module 303, when used to perform structured processing on the classroom dynamic data to obtain dynamic structured data, is specifically used to:

[0123] Extracting features from the video data to obtain video feature information;

[0124] The video feature information is structured to obtain the video structured data.

[0125] In a possible implementation, the video feature information includes character information, emotion information, and action information. The information generation module 303, when used to extract features from the video data to obtain the video feature information, is specifically used to:

[0126] Inputting the video data into a preset character recognition model to perform human posture recognition to determine the character information;

[0127] Inputting the video data into a preset expression recognition model to perform facial expression recognition to determine the emotion information;

[0128] The video data is input into a preset target detection model for action detection to determine the action information.

[0129] In a possible implementation, the information generating module 303, when used to perform structured processing on the video feature information to obtain the video structured data, is specifically used to:

[0130] Slicing the video data according to a preset slicing duration to obtain multiple video clips;

[0131] Determine the key information corresponding to each of the video clips to obtain the behavior object.

[0132] It should be noted that the above-mentioned classroom analysis information presentation device can execute the classroom analysis information presentation method provided in the implementation mode of this application, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not fully described in the implementation mode of the classroom analysis information presentation device, please refer to the classroom analysis information presentation method provided in the implementation mode of this application.

[0133] See also Figure 4 , Figure 44 is a schematic diagram of a computer device provided in an embodiment of the present application. The computer device includes one or more processors 41 and a memory 42. The memory 42 is connected to the one or more processors 41, for example, connected to the processor 41 via a bus.

[0134] The processor 41 is configured to support the computer device to perform the corresponding functions in the method in the above method embodiment. The processor 41 can be a central processing unit 41 (CPU), a network processor 41 (NP), a hardware chip or any combination thereof. The above hardware chip can be an application specific integrated circuit (ASIC), a programmable logic device (PLD) or a combination thereof. The above PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0135] The memory 42 is used to store program codes, etc. The memory 42 may include a volatile memory 42 (VM), such as a random access memory 42 (RAM); the memory 42 may also include a non-volatile memory 42 (NVM), such as a read-only memory 42 (ROM), a flash memory 42, a hard disk drive (HDD) or a solid-state drive (SSD); the memory 42 may also include a combination of the above-mentioned types of memory 42.

[0136] The memory 42 may be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions / modules corresponding to the method for presenting classroom analysis information in the embodiment of the present application. The processor 41 executes the various functional applications and data processing of the method for presenting classroom analysis information and the device for presenting classroom analysis information by running the non-volatile software programs, instructions and modules stored in the memory 42, that is, realizes the functions of the various modules or units of the method for presenting classroom analysis information and the device for presenting classroom analysis information provided in the above method embodiment.

[0137] The memory 42 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function. The data storage area may store data created according to the use of the presentation device of the classroom analysis information, etc. In some embodiments, the memory 42 may optionally include a memory 42 remotely arranged relative to the processor 41, and these remote memories 42 may be connected to the presentation device of the classroom analysis information via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0138] The one or more modules are stored in the memory 42, and when executed by the one or more processors 41, they execute the presentation device of classroom analysis information in any of the above-mentioned method embodiments, for example, execute the method steps described in the above-mentioned method embodiments, and realize the functions of the modules described in the above-mentioned device embodiments.

[0139] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a computer, the computer executes the method described in the above embodiment.

[0140] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0141] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A method for presenting classroom analysis information, characterized in that: Used in electronic equipment, including: Obtain classroom observation data; Determining a prompt word obtained by an interactive interface of the electronic device; Generate classroom analysis information through a preset large language model according to the classroom observation data and the prompt words; The classroom analysis information is presented on the interactive interface.

2. The method according to claim 1, characterized in that The interactive interface is configured with a text input box, and the determining of the prompt word obtained by the interactive interface of the electronic device includes: Obtaining a portion of text obtained from the text input box; Traversing one or more candidate words corresponding to the part of the text in a preset prompt word library; When an operation of selecting a candidate word is detected, the selected candidate word is determined as a prompt word.

3. The method according to claim 1, characterized in that: Generating classroom analysis information based on the classroom observation data and prompt words through a preset large language model includes: Generate a classroom analysis report based on the classroom observation data through a preset large language model; Classroom analysis information is generated according to the classroom analysis report and the prompt words.

4. The method according to claim 3, characterized in that The classroom observation data includes classroom dynamic data and teacher static data. The classroom analysis report is generated based on the classroom observation data through a preset large language model, including: Performing structural processing on the classroom dynamic data and the teacher static data respectively to obtain dynamic structured data and static structured data; The dynamic structured data, the static structured data and the prompt words are input into the large language model so that the large language model outputs the classroom analysis report.

5. The method according to claim 4, characterized in that The classroom dynamic data includes audio data collected in the classroom, the dynamic structured data includes audio structured data, and the structured processing of the classroom dynamic data to obtain the dynamic structured data includes: Converting the audio data into conversation text data; The dialogue text data is structured to obtain the audio structured data.

6. The method according to claim 5, characterized in that The step of converting the audio data into dialogue text data comprises: Preprocessing the audio data to obtain audio feature information; Inputting the audio feature information into a preset voiceprint model for voiceprint recognition to obtain the identity information of the interlocutor; Inputting the audio feature information and identity information into a preset acoustic model to perform phoneme prediction to obtain phoneme information; The phoneme information is input into a preset language model for text conversion to obtain the dialogue text data.

7. The method according to claim 3, characterized in that The classroom dynamic data includes video data collected in the classroom, the dynamic structured data includes video structured data, and the structured processing of the classroom dynamic data to obtain the dynamic structured data includes: Extracting features from the video data to obtain video feature information; The video structured data is generated according to the video data and the video feature information.

8. The method according to claim 7, characterized in that The video feature information includes character information, emotion information and action information. The feature extraction of the video data to obtain the video feature information includes: Inputting the video data into a preset character recognition model to perform human posture recognition to determine the character information; Inputting the video data into a preset expression recognition model for emotion recognition to determine the emotion information; The video data is input into a preset target detection model for action detection to determine the action information.

9. The method according to claim 7, characterized in that: The step of generating the video structured data according to the video data and the video feature information includes: Slice the video features according to a preset slice duration to obtain multiple video clips; Determining video feature information corresponding to each of the video clips; Each of the video feature information is structured to obtain video structured data corresponding to each of the video clips.

10. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory is connected to the processor, and the processor is used to execute one or more computer programs stored in the memory. When the processor executes the one or more computer programs, the computer device implements the method according to any one of claims 1 to 9.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor is caused to perform the method according to any one of claims 1 to 9.

Images