Classroom wonderful moment identification method and device

Abstract

The invention provides a classroom wonderful moment identification method and device. The method comprises the following steps: acquiring classroom teaching data of a to-be-identified classroom; based on classroom teaching data, a classroom recognition model is applied to carry out wonderful moment recognition to obtain the wonderful moment starting and ending time and wonderful moment description information of a to-be-recognized classroom, and the defects that traditional manual judgment is prone to missing important information and poor in accuracy and objectivity, and a coding mode is poor in result interpretability and poor in accuracy and objectivity are overcome. The invention aims to overcome the defects of complex process, large workload and low efficiency in the prior art, introduces a large model technology to perform classroom wonderful moment recognition, not only gets rid of dependence on personal subjective experience and realizes more comprehensive and accurate recognition, but also can be suitable for various types of classroom teaching processes, gives accurate description of wonderful moments, and improves the teaching efficiency. Personalization and specificity can be highlighted, and data support is provided for subsequent teaching evaluation.

Description

Technical Field

[0001] The present invention relates to the field of educational informatization technology, and in particular to a method and device for identifying wonderful moments in a classroom. Background Art

[0002] With the acceleration of educational informatization, improving classroom teaching quality has become a focus of attention in the education field. Effectively improving classroom teaching ability is a core component of teacher professional development. Discovering and presenting the highlights of teachers' classroom teaching is crucial for demonstrating their teaching wisdom and optimizing teaching strategies.

[0003] Currently, there are two main approaches to identifying exciting moments in the classroom. The first is the traditional classroom observation and evaluation model, which relies on on-site observation and written records by the teacher. This method identifies exciting moments based on subjective experience and presents the results orally or in writing. However, this approach relies too much on manual observation and experience, making it difficult to fully cover classroom details, easily missing important information, and resulting in a lack of objectivity and consistency in the recognition results. The second approach uses audio and video coding technology to identify exciting moments through classroom recordings and videos. However, this approach focuses on quantitative analysis of classroom behavior and struggles to deeply capture and describe exciting moments in the classroom, especially in terms of understanding teaching wisdom and emotional communication. Furthermore, the coding workload is large, the overall efficiency is low, and the results are difficult to interpret, making it difficult to meet actual teaching needs. Summary of the Invention

[0004] The present invention provides a method and device for identifying wonderful moments in a classroom, which are used to solve the problems in the prior art that manual judgment is prone to omitting important information, and has low accuracy, objectivity and consistency, as well as poor interpretability of the results of the encoding method, and the process is cumbersome, the workload is large, and the efficiency is low. The method introduces large model technology to accurately identify wonderful moments in the classroom, which not only has comprehensive coverage, but is also more efficient, quick, and more applicable.

[0005] The present invention provides a method for identifying wonderful moments in a classroom, comprising: Obtain classroom teaching data of the class to be identified; Based on the classroom teaching data, a classroom recognition model is applied to identify wonderful moments, and the start and end times and description information of the wonderful moments of the classroom to be identified are obtained.

[0006] According to a method for identifying wonderful moments in a classroom provided by the present invention, the classroom teaching data includes classroom teaching audio; Based on the classroom teaching data, the classroom recognition model is applied to identify the wonderful moments, and the start and end time of the wonderful moments of the classroom to be identified and the description information of the wonderful moments are obtained, including: Perform role separation and voice transcription based on the classroom teaching audio to obtain a teacher-student dialogue text; Based on the teacher-student dialogue text, the classroom recognition model is applied to perform wonderful moment recognition to obtain the start and end time and description information of the wonderful moments of the classroom to be recognized.

[0007] According to a method for identifying wonderful moments in a classroom provided by the present invention, the classroom teaching data includes classroom teaching videos; Based on the classroom teaching data, the classroom recognition model is applied to identify the wonderful moments, and the start and end time of the wonderful moments of the classroom to be identified and the description information of the wonderful moments are obtained, including: Performing action recognition based on the classroom teaching video to obtain video action information; Based on the video action information, the classroom recognition model is applied to perform wonderful moment recognition to obtain the start and end time of the wonderful moments of the classroom to be recognized and the wonderful moment description information.

[0008] According to a method for identifying wonderful moments in a classroom provided by the present invention, the classroom teaching data includes classroom teaching audio and classroom teaching video, and the classroom teaching audio and the classroom teaching video are aligned; Based on the classroom teaching data, the classroom recognition model is applied to identify the wonderful moments, and the start and end time of the wonderful moments of the classroom to be identified and the description information of the wonderful moments are obtained, including: Perform role separation and voice transcription based on the classroom teaching audio to obtain a teacher-student dialogue text; Performing action recognition based on the classroom teaching video to obtain video action information; Based on the teacher-student dialogue text and the video action information, the classroom recognition model is applied to perform wonderful moment recognition to obtain the start and end time and description information of the wonderful moments of the classroom to be recognized.

[0009] According to a method for identifying wonderful moments in a classroom provided by the present invention, the wonderful moment description information includes wonderful moment classification and wonderful moment description; Based on the classroom teaching data, the classroom recognition model is applied to identify the wonderful moments, and the start and end time of the wonderful moments of the classroom to be identified and the description information of the wonderful moments are obtained, including: Generate recognition prompt text based on the classroom teaching data and the description of the wonderful moment type; The generated recognition prompt text is input into the classroom recognition model to obtain the start and end time of the wonderful moments of the classroom to be identified, the classification of the wonderful moments, and the description of the wonderful moments.

[0010] According to a method for identifying wonderful moments in a classroom provided by the present invention, the classroom teaching data includes classroom teaching audio; and generating identification prompt text based on the classroom teaching data and a description of the wonderful moment type includes: Performing emotion recognition based on the classroom teaching audio to obtain audio emotion information; Based on the classroom teaching data, the audio emotion information and the description of the wonderful moment type, an identification prompt text is generated.

[0011] According to a method for identifying wonderful moments in a classroom provided by the present invention, the classroom teaching data includes classroom teaching audio and classroom teaching video, and the classroom teaching audio and the classroom teaching video are aligned; the wonderful moment description information includes wonderful moment video clips, wonderful moment video key frames and wonderful moment dialogue text; Based on the classroom teaching data, the classroom recognition model is applied to identify the wonderful moments, and the start and end time of the wonderful moments of the classroom to be identified and the description information of the wonderful moments are obtained, including: Based on the classroom teaching data, the classroom recognition model is applied to identify the wonderful moments, and the start and end times of the wonderful moments of the classroom to be identified are obtained; Determining a wonderful moment video clip from the classroom teaching video based on the start and end time of the wonderful moment, and extracting key frames based on the wonderful moment video clip to obtain a wonderful moment video key frame; Based on the start and end time of the wonderful moment, the wonderful moment dialogue text is determined from the teacher-student dialogue text; the teacher-student dialogue text is determined based on the classroom teaching audio.

[0012] According to a method for identifying wonderful moments in a classroom provided by the present invention, the method includes applying a classroom recognition model to identify wonderful moments based on the classroom teaching data, obtaining the start and end times of the wonderful moments of the classroom to be identified and the description information of the wonderful moments, and then further comprising: Encoding the start and end times of the wonderful moments and the description information of the wonderful moments, and generating a teaching analysis report of the to-be-identified class based on the wonderful moment feature vector obtained by encoding; Based on the teaching analysis report, the teacher's teaching strategy for the classroom to be identified is generated.

[0013] According to a method for identifying wonderful moments in a classroom provided by the present invention, the description of the types of wonderful moments includes at least two of the descriptions of wonderful question-and-answer moments, wonderful situational moments, technology integration moments, and ideas collision moments.

[0014] The present invention also provides a device for identifying wonderful moments in a classroom, comprising: An acquisition unit, used for acquiring classroom teaching data of a classroom to be identified; The recognition unit is used to apply the classroom recognition model to identify the wonderful moments based on the classroom teaching data, and obtain the start and end time and description information of the wonderful moments of the classroom to be identified.

[0015] The method and device for identifying wonderful moments in a classroom provided by the present invention obtain classroom teaching data of a classroom to be identified; based on the classroom teaching data, a classroom recognition model is applied to identify wonderful moments, and the start and end time of the wonderful moments in the classroom to be identified and the description information of the wonderful moments are obtained, which overcomes the defects of traditional manual judgment that is easy to miss important information, poor accuracy and objectivity, and poor interpretability of the results of the encoding method, as well as the defects of cumbersome process, large workload and low efficiency. The large model technology is introduced to identify wonderful moments in the classroom, which not only gets rid of the dependence on personal subjective experience and realizes more comprehensive and accurate recognition, but also can be applied to various types of classroom teaching processes, and gives an accurate description of the wonderful moments, which can highlight their personalized characteristics and provide data support for subsequent teaching evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1 1 is a flow chart of the method for identifying wonderful moments in a classroom provided by the present invention; Figure 2 This is an example diagram of displaying the wonderful moment description information provided by the present invention; Figure 3 It is a structural diagram of the device for identifying wonderful moments in a classroom provided by the present invention; Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0018] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0019] In the field of education, especially in the area of teacher professional development, classroom teaching ability has always been a focus. The key to effectively improving classroom teaching ability lies in accurately identifying the wonderful moments in teachers' classroom teaching. These moments embody the teacher's teaching wisdom and professional qualities, and serve as an important basis for evaluating and improving teachers' teaching ability.

[0020] Currently, the identification of exciting moments in the classroom mainly relies on two traditional technical approaches. The first approach focuses on on-site observation and recording by teachers who observe and evaluate the class. This approach uses the subjective experience of the teachers to judge the exciting moments in the classroom and presents the recognition results through oral or handwritten records. However, this approach has significant flaws: on the one hand, manual observation cannot fully cover classroom details, which can easily lead to the omission of important information; on the other hand, it is highly dependent on the personal experience and subjective judgment of the teachers who observe and evaluate the class, making the recognition results lack objectivity and standardization, making it difficult to achieve large-scale promotion and application; in addition, the original recording methods (such as paper notes) are easily lost or damaged, which is not conducive to the long-term preservation and in-depth analysis of data.

[0021] The second approach is to use classroom video and audio recordings to identify exciting moments through audio and video coding technology, and present the analysis results in the form of quantitative statistics. However, this approach focuses primarily on the quantitative statistics of classroom behavior, making it difficult to accurately identify exciting moments in the classroom, especially in terms of understanding deeper dimensions such as teaching strategies, teacher-student interaction, and emotional communication. In addition, under this approach, all classes use the same coding system, which fails to highlight the personalized content of exciting moments in the classroom. Moreover, the results obtained through this method are poorly interpretable (only timestamps are provided), and the coding process is complicated and labor-intensive, resulting in low overall efficiency and difficulty meeting the needs of actual teaching evaluation and improvement.

[0022] In this regard, the present invention provides a method for identifying wonderful moments in the classroom, which aims to introduce artificial intelligence technology, identify wonderful moments in the classroom through a large model, and realize intelligent detection and accurate description of the teacher's teaching wisdom expressive events (wonderful moments) during the classroom teaching process. It gets rid of the dependence on personal subjective experience, is more comprehensive, accurate and objective, and can be applied to various types of classroom teaching processes, and can highlight its personalized characteristics.

[0023] Figure 1 : is a flow chart of the method for identifying wonderful moments in a classroom provided by the present invention, such as Figure 1 As shown, the method includes: Step 110, obtaining classroom teaching data of the class to be identified; Step 120 : Based on the classroom teaching data, a classroom recognition model is applied to identify wonderful moments, and the start and end times of the wonderful moments of the classroom to be identified and the description information of the wonderful moments are obtained.

[0024] Specifically, considering the current reliance on manual judgment, which is highly subjective, difficult to fully cover, and has poor accuracy and consistency, making it difficult to preserve, and the recognition method based on audio and video coding is limited to shallow behavioral quantification, which cannot accurately identify the wonderful moments in the classroom, the results obtained have poor interpretability, and the encoding process has a large workload, resulting in overall low efficiency, in an embodiment of the present invention, it is proposed to introduce large model technology to automatically analyze classroom teaching data through the powerful learning, understanding and natural language processing capabilities of the large model to identify the wonderful moments in the classroom, and accurately locate and describe them, so as to provide data support for subsequent teaching evaluation, which can meet the needs of actual teaching evaluation, and the recognition process is very objective, comprehensive, fast and accurate, and there will be no omission of important information, large workload, and inaccurate recognition problems. It can be applicable to various types of classroom teaching processes.

[0025] Specifically, before identifying exciting moments in a classroom, it is necessary to first determine the target for identification, i.e., the classroom to be identified, and the basis for identification, i.e., the classroom teaching data of the classroom to be identified. The classroom to be identified here can be of various types, such as classrooms in various educational fields, such as medicine, new media, and computers, or various subjects in traditional teaching, such as Chinese, mathematics, physics, chemistry, and biology, or classrooms at different educational stages, such as elementary school, junior high school, and high school. This embodiment of the present invention does not specifically limit this.

[0026] As for the classroom teaching data of the identified classroom, it can include audio, video, teaching notes, lesson plan records, etc. The audio and video can be obtained from the same data source. For example, the classroom teaching process of the identified classroom can be recorded using audio and video capture equipment, and the audio and video can be separated from the recorded audio and video data. The audio and video can also be obtained separately. For example, the audio and video of the classroom teaching process of the identified classroom can be collected by audio capture equipment (such as microphones, portable voice recorders, smartphones, etc.) and video capture equipment (such as cameras, webcams, etc.) installed in the teaching space corresponding to the identified classroom. In online teaching scenarios, audio recording, video recording, or both can be performed directly within the teaching software. Teaching notes can be records of the classroom teaching process of the identified classroom by teachers, students, etc., while lesson plan records are records of the content that teachers need to teach in the identified classroom before the class begins.

[0027] After obtaining the classroom teaching data of the class to be identified, in an embodiment of the present invention, the wonderful moments of the class can be identified based on this classroom teaching data, so as to accurately identify the wonderful moments in the classroom teaching process and describe them, thereby realizing the accurate detection and explainable presentation of the teacher's teaching wisdom expressive events (wonderful moments) in the classroom teaching process.

[0028] Specifically, considering the current shallow recognition problems existing in the encoding method, which cannot deeply understand and accurately capture the wonderful moments, resulting in inaccurate recognition results, low efficiency, and poor interpretability, as well as the excellent capabilities of large language models (hereinafter referred to as large models), in the embodiment of the present invention, it is proposed to introduce large model technology to automatically analyze and process classroom teaching data through the excellent natural language understanding, analysis and processing capabilities of the large model, so as to capture the wonderful moments in the classroom and provide time information and description information.

[0029] Here, specifically, a model for identifying wonderful moments can be constructed first, and this model can be called a classroom recognition model; that is, a classroom recognition model for identifying wonderful moments in the classroom to be identified can be constructed on the basis of a large model. Here, a large model refers to a natural language processing model with a large parameter scale, and the number of model parameters and / or the complexity of the model structure exceeds a preset threshold. The model processes large-scale text data during training and has the ability to understand and generate natural language. For example, a large model can include a Spark large model, etc. Then, based on the classroom teaching data, the classroom recognition model can be applied to identify wonderful moments in the classroom to be identified, and the start and end time of the wonderful moments and the description information of the wonderful moments of the classroom to be identified can be obtained.

[0030] It should be noted that the wonderful moments that need to be identified can be defined in advance and given to the classroom recognition model so that the classroom recognition model can accurately identify them as required. For example, wonderful moments can be set to include wonderful question-and-answer moments, wonderful situational moments, etc., and definitions of wonderful question-and-answer moments and wonderful situational moments can be given, such as "Wonderful question-and-answer moments refer to classroom teaching, in which teachers use follow-up questions to conduct guided and heuristic teaching, allowing students to find answers themselves. In such moments, the teacher usually asks questions as the beginning." "Wonderful situational moments refer to classroom teaching, in which teachers create wonderful situations based on the teaching content. In such moments, they can be life situations created by teachers in the classroom to explain new knowledge, story situations created by teachers when teaching related content, or problem situations created by teachers when applying knowledge." Or other moments can be set, such as technology integration moments, thought collision moments, etc., and defined.

[0031] Specifically, classroom teaching data can be used as input and input into the classroom recognition model so that the classroom recognition model can identify wonderful moments based on this. That is, the classroom recognition model can observe the teaching process of the classroom to be identified based on the classroom teaching data, and conduct in-depth analysis and interpretation to capture wonderful moments, such as wonderful question-and-answer moments, wonderful situational moments, etc., and determine the start and end time of the wonderful moments, and provide a description of the wonderful moments, that is, wonderful moment description information. The wonderful moment description information here can include the type of the identified wonderful moment (such as a wonderful question-and-answer moment, a wonderful situational moment, or other types of wonderful moments), the name of the wonderful moment (that is, the naming of the wonderful moment, which can be directly given by the model), the summary of the wonderful moment (that is, the description of the wonderful moment, which can be directly given by the model), etc.

[0032] Compared with the current behavior quantification based on audio and video encoding, the quantitative analysis results obtained often only provide timestamps but lack event descriptions of wonderful moments. In the embodiment of the present invention, wonderful moment recognition is performed based on a classroom recognition model, which can not only accurately identify the wonderful moments in the classroom teaching process, but also provide descriptive information, accurately describe the wonderful moments and their core content, thereby facilitating subsequent teaching evaluation and presenting the wonderful moments in the classroom better, more comprehensively and more accurately.

[0033] It is worth noting that before inputting the teacher-student dialogue text into the classroom recognition model for wonderful moment recognition, in order to improve the model performance, optimize its performance in the classroom wonderful moment recognition task, and ensure the accuracy, comprehensiveness and interpretability of the recognition results, in the embodiment of the present invention, a large amount of historical teaching data (more than 2,000 cases) in classroom teaching scenarios can also be applied, that is, classroom teaching data in historical time, also called sample teaching data, and the corresponding start and end time and description information of the wonderful moments, to pre-train the classroom recognition model.

[0034] That is, the initial model for the training process can be constructed based on the large model. This can be done directly using the large model as the initial model, or by building other networks and algorithms on top of it to construct a model specifically designed for identifying highlights during classroom teaching, known as the initial model. The initial model is then trained using a large amount of pre-collected historical teaching data from classroom scenarios to create a classroom recognition model. The resulting model not only inherits the powerful learning and comprehension capabilities of the original large model (large language model), but also possesses superior text processing capabilities. This allows it to quickly and accurately identify highlights during classroom teaching and provide corresponding descriptive information, thereby improving the comprehensiveness, accuracy, timeliness, interpretability, and universality of the recognition results.

[0035] In an embodiment of the present invention, wonderful moments are identified by using a classroom recognition model constructed on the basis of a large model, which can be analyzed and identified in deep dimensions, such as teaching strategies, teacher-student interactions, and emotional exchanges. Different from traditional shallow recognition, the embodiment of the present invention can understand cognitive interactions, innovative practices, etc. in teaching situations, and can perform more comprehensive recognition. The recognition results are also more accurate, reliable, and more interpretable. Moreover, when performing recognition, the model can capture the characteristics of different classrooms, and the descriptions given are more in line with the classroom itself, and are more closely and closely related to it, and more personalized.

[0036] The method for identifying wonderful moments in a classroom provided by the present invention obtains classroom teaching data of a classroom to be identified, and applies a classroom recognition model to identify wonderful moments based on the classroom teaching data, so as to obtain the start and end time of the wonderful moments in the classroom to be identified and description information of the wonderful moments. The method overcomes the defects of traditional manual judgment that important information is easily missed, accuracy and objectivity are poor, and the result of the encoding method is poorly interpretable, and the process is cumbersome, the workload is large, and the efficiency is low. The large model technology is introduced to identify wonderful moments in the classroom, which not only gets rid of the dependence on personal subjective experience and realizes more comprehensive and accurate recognition, but also can be applied to various types of classroom teaching processes, and gives an accurate description of the wonderful moments, which can highlight their personalized characteristics and provide data support for subsequent teaching evaluation.

[0037] Based on the above embodiment, the classroom teaching data includes classroom teaching audio; Step 120 includes: Based on the classroom teaching audio, role separation and voice transcription are performed to obtain the teacher-student dialogue text; Based on the teacher-student dialogue text, the classroom recognition model is applied to identify the wonderful moments, and the start and end time and description information of the wonderful moments in the classroom to be identified are obtained.

[0038] Specifically, the process of applying the classroom recognition model to identify wonderful moments based on classroom teaching data to obtain the start and end times and description information of the wonderful moments of the classroom to be identified specifically includes the following steps: When classroom teaching data includes audio, it can be processed before identifying highlights based on the audio to convert it into a clearer and more intuitive dialogue text, specifically a teacher-student dialogue text, that presents the teaching process and content. The classroom recognition model can then be applied to this dialogue text to identify highlights, obtaining the start and end times and descriptions of the highlights in the classroom.

[0039] In detail, here we can first separate the roles of the classroom teaching audio to separate different speakers and clarify their identities, such as whether it is a teacher or a student, or student 1, student 2, student 3, etc. Through this process, the audio clips of each speaker can be determined from the classroom teaching audio, and then each audio clip can be transcribed in chronological order, so that the content of the conversation between the teacher and the students, that is, the teacher-student conversation text, can be obtained.

[0040] Here, role separation and speech transcription of classroom teaching audio can be achieved through conventional voiceprint separation algorithms and speech transcription algorithms, such as algorithms based on artificial neural networks, algorithms based on deep learning, etc., or through locally deployed voiceprint separation engines, speech transcription engines, etc., and can also be implemented by calling general multimodal large models, artificial intelligence large models, etc. The embodiment of the present invention does not make specific limitations on this.

[0041] Furthermore, after obtaining the teacher-student dialogue text, in an embodiment of the present invention, wonderful moments can be identified based on this teacher-student dialogue text to accurately identify the wonderful moments in the classroom to be identified, and provide the time information and description information of the wonderful moments, that is, the start and end time of the wonderful moments and the description information of the wonderful moments.

[0042] Specifically, the teacher-student dialogue text can be used as input and input into the classroom recognition model so that the classroom recognition model can identify wonderful moments based on this. That is, the classroom recognition model can observe the teaching process of the classroom to be identified based on the teacher-student dialogue text, and conduct in-depth analysis and interpretation to capture wonderful moments, such as wonderful question-and-answer moments, wonderful situational moments, etc., and determine the start and end time of the wonderful moments, and refine and summarize the dialogue content in this period, so as to give a description of the wonderful moments, that is, the wonderful moment description information.

[0043] The wonderful moment description information here may include the type of the identified wonderful moment, the name of the wonderful moment, a summary of the wonderful moment, the dialogue text of the wonderful moment (i.e., the dialogue content corresponding to the start and end time of the wonderful moment in the teacher-student dialogue text), etc.

[0044] Based on the above embodiment, the classroom teaching data includes classroom teaching videos; Step 120 includes: Perform action recognition based on classroom teaching videos to obtain video action information; Based on the video action information, the classroom recognition model is applied to identify the wonderful moments, and the start and end time and description information of the wonderful moments of the classroom to be identified are obtained.

[0045] Specifically, the process of applying the classroom recognition model to identify wonderful moments based on classroom teaching data to obtain the start and end times and description information of the wonderful moments of the classroom to be identified specifically includes the following steps: When classroom teaching data includes classroom teaching videos, before identifying wonderful moments based on the classroom teaching videos, they can also be processed first to convert the classroom teaching videos into information that can more intuitively represent behavioral actions in the classroom teaching process, that is, video action information. Based on this video action information, the classroom recognition model is applied to identify wonderful moments to obtain the start and end time of the wonderful moments of the classroom to be identified and the wonderful moment description information.

[0046] In detail, the classroom teaching video can be processed first to extract information such as behaviors and actions at the image level that are helpful for identifying wonderful moments, that is, video action information. Considering that the time and resources required to directly process the complete classroom teaching video are extremely large, in an embodiment of the present invention, key frame extraction can be performed on the classroom teaching video first to extract key frames, and then action recognition can be performed based on the key frames to identify the actions and characteristics in each key frame, or the actions and characteristics of the time period corresponding to each key frame, such as question-and-answer discussion, immersive thinking, etc., so as to obtain video action information.

[0047] After that, the wonderful moments can be identified based on the video action information to accurately identify the wonderful moments in the classroom to be identified, and provide the time information and description information of the wonderful moments, that is, the start and end time of the wonderful moments and the description information of the wonderful moments.

[0048] Specifically, video action information can be used as input and fed into a classroom recognition model so that the model can identify highlights based on this information. That is, the model can observe the teaching process of the classroom to be identified based on the video action information, and conduct in-depth analysis and interpretation to capture highlights, such as highlights Q&A moments, technology integration moments, etc., and determine the start and end times of the highlights. The model then understands, refines, and summarizes the video action information during this time period, thereby providing a description of the highlights, i.e., the highlights moment description information. Here, the highlights moment description information may include the type of highlights identified, the name of the highlights moment, a summary of the highlights moment, a video clip of the highlights moment (the video clip corresponding to the start and end times of the highlights moment in the classroom teaching video), and a video keyframe of the highlights moment (the video keyframe corresponding to the start and end times of the highlights moment in the classroom teaching video).

[0049] Based on the above embodiment, the classroom teaching data includes classroom teaching audio and classroom teaching video, and the classroom teaching audio and classroom teaching video are aligned; Step 120 includes: Based on the classroom teaching audio, role separation and voice transcription are performed to obtain the teacher-student dialogue text; Perform action recognition based on classroom teaching videos to obtain video action information; Based on the teacher-student dialogue text and video action information, the classroom recognition model is applied to identify the wonderful moments, and the start and end time and description information of the wonderful moments of the classroom to be identified are obtained.

[0050] Specifically, the classroom recognition model is applied to identify the wonderful moments based on the classroom teaching data to obtain the start and end time of the wonderful moments of the classroom to be identified and the description information of the wonderful moments. Specifically, the process of applying the classroom recognition model to identify wonderful moments based on classroom teaching data to obtain the start and end times and description information of the wonderful moments of the classroom to be identified specifically includes the following steps: When classroom teaching data includes both audio and video, prior to identifying highlights based on the audio, it can be processed to convert the audio into a clearer and more intuitive dialogue text that presents the teaching process and content, i.e., teacher-student dialogue text. The video can also be converted into information that more intuitively represents the actions taken during the teaching process, i.e., video action information. Then, based on this teacher-student dialogue text and video action information, the classroom recognition model is applied to identify highlights, obtaining the start and end times and descriptions of the highlights in the classroom to be identified.

[0051] In detail, here we can first separate the roles of the classroom teaching audio to separate different speakers and clarify their identities, such as whether it is a teacher or a student, or student 1, student 2, student 3, etc. Through this process, the audio clips of each speaker can be determined from the classroom teaching audio, and then each audio clip can be transcribed in chronological order, so that the content of the conversation between the teacher and the students, that is, the teacher-student conversation text, can be obtained.

[0052] At the same time, the classroom teaching video can be processed to extract information such as behaviors and actions at the image level that is helpful for identifying wonderful moments, that is, video action information. Considering that the time and resources required to directly process the complete classroom teaching video are extremely large, in an embodiment of the present invention, key frame extraction can be first performed on the classroom teaching video to extract key frames, and then action recognition can be performed based on the key frames to identify the actions and characteristics in each key frame, or the actions and characteristics of the time period corresponding to each key frame, such as question-and-answer discussion, immersive thinking, etc., so as to obtain video action information.

[0053] It's important to note that classroom video and audio must be aligned, meaning they're synchronized in time. This can also be understood as audio / video synchronization. If the audio and video are recorded separately, they need to be aligned beforehand. However, if they're separated from the audio and video, their timelines are already aligned.

[0054] After that, the wonderful moments can be identified based on the obtained teacher-student dialogue text and video action information to accurately identify the wonderful moments in the classroom to be identified, and provide the time information and description information of the wonderful moments, that is, the start and end time of the wonderful moments and the description information of the wonderful moments.

[0055] Specifically, the teacher-student dialogue text and video action information can be used as input and input into the classroom recognition model, so that the classroom recognition model can identify wonderful moments based on this. That is, the classroom recognition model can observe the teaching process of the classroom to be identified based on the teacher-student dialogue text and video action information, and conduct in-depth analysis and interpretation to capture wonderful moments, such as wonderful question-and-answer moments, wonderful situational moments, technology integration moments, etc., and determine the start and end time of the wonderful moments, and refine and summarize the dialogue content and video action information in this period, so as to give a description of the wonderful moments, that is, the wonderful moment description information.

[0056] Here, the wonderful moment description information may include the type of the recognized wonderful moment, the name of the wonderful moment, a summary of the wonderful moment, a dialogue text of the wonderful moment, a video clip of the wonderful moment, a video key frame of the wonderful moment, etc.

[0057] Based on the above embodiment, the wonderful moment description information includes the wonderful moment classification and the wonderful moment description; step 120 includes: Generate recognition prompt text based on classroom teaching data and description of wonderful moment types; The generated recognition prompt text is input into the classroom recognition model to obtain the start and end time of the wonderful moments of the classroom to be identified, the classification of the wonderful moments, and the description of the wonderful moments.

[0058] Descriptions of wonderful moment types include at least two of the following: descriptions of wonderful question-and-answer moments, descriptions of wonderful situational moments, descriptions of technological integration moments, and descriptions of moments of collision of ideas.

[0059] Specifically, the process of applying the classroom recognition model to identify wonderful moments based on classroom teaching data to obtain the start and end times and description information of the wonderful moments of the classroom to be identified may include the following steps: After obtaining classroom teaching data, in an embodiment of the present invention, wonderful moments can be identified based on this data to obtain the wonderful moment start and end time and wonderful moment description information of the wonderful moments of the classroom to be identified. However, considering that it is necessary to make the model "understand" what kind of moments are wonderful moments during identification, that is, the model needs to understand the wonderful moments identified by the user, therefore, it is necessary to describe the pre-defined wonderful moments and give them as input to the model so that the model can understand. Based on this, in an embodiment of the present invention, before performing wonderful moment identification, it is also necessary to obtain the description of the pre-defined wonderful moments, that is, the description of the wonderful moment type, and on this basis, combined with classroom teaching data, the classroom recognition model can be applied to perform wonderful moment identification for the classroom to be identified.

[0060] Furthermore, considering that the model learns a large amount of sample data and language rules during training, it often has more stringent requirements for input information, which must conform to the language form and specifications that it can receive and understand, so that the model can understand and analyze and output the corresponding start and end time and wonderful moment description information. In an embodiment of the present invention, before applying the classroom recognition model to recognize wonderful moments, it is also necessary to convert the information input to the model into a form that the model can understand, that is, to normalize it and refine it into structured information, so as to obtain recognition prompt text for prompting the model to recognize wonderful moments.

[0061] Specifically, here the classroom teaching data and descriptions of wonderful moment types may be processed to extract the above information into structured text, thereby obtaining recognition prompt text. The recognition prompt text here is essentially a type of data that carries the above information, such as prompt, which is data that the model can receive and understand. After inputting it into the model, the model can automatically identify wonderful moments based on the input information. However, it is worth noting that when generating recognition prompt text, it can be generated based on the recognition prompt template that contains the description of the wonderful moment type, combined with classroom teaching data fine-tuning. At this time, the recognition prompt template used is a prompt template with good recognition effect obtained after training and testing with a large amount of data (more than 100 cases); or it can be directly based on classroom teaching data and descriptions of wonderful moment types to generate a brand new recognition prompt text.

[0062] Among them, the description of the wonderful moment type can include any two or more of the description of the wonderful question and answer moment, the description of the wonderful situation moment, the description of the technology integration moment, and the description of the collision of ideas moment, that is, it includes at least two of the description of the wonderful question and answer moment, the description of the wonderful situation moment, the description of the technology integration moment, and the description of the collision of ideas moment.

[0063] Here, the recognition prompt text may be in the following forms or other forms, which are not specifically limited in the embodiment of the present invention.

[0064] Example of recognizing hint text: "##Role You are an excellent teacher, familiar with educational theory, understand the key points of classroom observation, and can find the highlights from the classroom; ##background Every class has its moments of brilliance, reflecting the wisdom of the teacher and the learning of the students. These moments can be summarized into the following four categories: 1. Wonderful Q&A Moments: In classroom teaching, teachers use follow-up questions to guide and inspire students to "find the answers" on their own. In such moments, the teacher usually starts with a question; 2. Wonderful situational moments: During classroom teaching, teachers create wonderful situations based on the teaching content. These moments can be life situations created by teachers to explain new knowledge in class, story situations created by teachers when teaching related content, or problem situations created by teachers when applying knowledge. 3. Technology Integration Moments: In classroom teaching, teachers use intelligent technology to assist teaching, improve the efficiency and quality of classroom teaching, and promote effective teaching. In such moments, teachers may use physical teaching aids or multimedia tools; 4. Moments of collision of ideas: In classroom teaching, teachers raise questions involving higher-order thinking, such as heuristic questions, driving questions that stimulate students' exploration, and students think and engage in group discussions or classroom debates, or students ask questions to teachers, and teachers and students explore the problems from multiple perspectives. Such moments may include when teachers invite multiple students to answer the same question, and other students can hear different ways of thinking and answers, and may include conflicts and solutions of opinions in group discussions.

[0065] ##Task 1. Input a text of a teacher-student conversation in a classroom, which includes the speaker's role information, the start and end time of the conversation, and the content of the conversation itself; 2. Please find the moments of exciting question and answer, exciting situational moments, technical integration moments, and moments of ideological collision from the text; 3. Output a phrase of no more than 8 words as the title based on the content of these highlights; 4. Summarize the main points of these highlights from the aspects of teaching content, teaching methods, core viewpoints, etc. 5. Calculate the start and end times of these highlights from the start and end times of the speech; ##limit 1. Output all possible moments, that is, there may be multiple moments of the same category 2. If there is no moment of a certain type, do not return 3. Don’t return meaningless beginnings or summaries; just return the results directly. ## Output reference { Wonderful moments category: Wonderful Q&A moments Highlight Moment Name: Review and Introduction of Parabola Description of wonderful moments: The teacher combined the picture presentation with three consecutive questions, such as "Which function from junior high school do you think of?", "What is the graph of a quadratic function?" and "If you rotate a parabola by a certain angle, is it still a parabola?", to gradually inspire students to think about parabolas. This not only reviewed junior high school knowledge, but also raised the question of whether a rotated parabola is still a parabola, which triggered students' thinking.

[0066] Wonderful moments start and end time: (00:38:18~00:38:52) }".

[0067] After that, this recognition prompt text can be input into the classroom recognition model, so that the classroom recognition model can automatically capture the wonderful moments in the classroom to be identified based on the input information, and classify, refine and summarize them, and finally output the start and end time of the wonderful moments, as well as the description of the wonderful moments, that is, the wonderful moment description information. The wonderful moment description information here can include the type of wonderful moments, that is, the wonderful moment classification, and the summary of the wonderful moments, that is, the wonderful moment description.

[0068] Compared with traditional solutions, which focus on a single dimension for classroom recognition, especially the recognition of wonderful moments, that is, only detecting a single behavior, event, such as asking questions, answering questions, etc., which brings about problems of incomplete detection and relatively weak reliability of results, the embodiment of the present invention focuses on multiple dimensions and performs recognition from at least two levels: question quality, situation setting, thinking guidance, and technology integration. It can not only realize multi-faceted detection and ensure the comprehensive and accurate results, but also improve the reliability of the results through multi-dimensional recognition.

[0069] Based on the above embodiment, the classroom teaching data includes classroom teaching audio; Based on classroom teaching data and descriptions of the types of highlights, recognition prompt text is generated, including: Perform emotion recognition based on classroom teaching audio to obtain audio emotion information; Generate recognition prompt text based on classroom teaching data, audio emotional information and description of the types of wonderful moments.

[0070] Specifically, the process of generating the recognition prompt text based on the classroom teaching data and the description of the wonderful moment type may include: Considering that in the process of identifying wonderful moments, if the recognition is based solely on classroom teaching data (such as teacher-student dialogue text or video action information determined based on classroom teaching data), the recognition accuracy may be reduced to a certain extent compared with the combination of multimodal data. Therefore, in order to further improve the recognition accuracy, in an embodiment of the present invention, emotional information at the audio level can also be introduced to identify wonderful moments through an emotional auxiliary model represented by the audio, thereby optimizing the recognition accuracy.

[0071] Specifically, here we can first perform emotion recognition on the classroom teaching audio to identify the emotions contained in the audio (such as enthusiasm, passion, etc.). For example, we can use current emotion recognition algorithms, emotion recognition systems, etc. to perform emotion recognition on the classroom teaching audio, and identify emotional information through the modal particles, words and phrases expressing emotions, volume, speaking speed, etc. involved therein, thereby obtaining audio emotional information.

[0072] After that, the recognition prompt text for prompting the model to identify the wonderful moments can be generated based on the recognized audio emotional information, classroom teaching data and description of the types of wonderful moments. That is, on the basis of the description of the types of wonderful moments, the classroom teaching data and audio emotional information are combined for processing to extract the above information into structured text, thereby obtaining the recognition prompt text. It is worth noting that when generating the recognition prompt text, the recognition prompt text can be generated based on the recognition prompt template containing the description of the types of wonderful moments, combined with the classroom teaching data and audio emotional information for fine-tuning. At this time, the recognition prompt template used is a prompt template with good recognition effect obtained after training and testing with a large amount of data; or a brand new recognition prompt text can be generated directly based on the classroom teaching data, audio emotional information and description of the types of wonderful moments.

[0073] Based on the above embodiment, the classroom teaching data includes classroom teaching audio and classroom teaching video, and the classroom teaching audio and classroom teaching video are aligned; the wonderful moment description information includes wonderful moment video clips, wonderful moment video key frames and wonderful moment dialogue text; Step 120 includes: Based on classroom teaching data, the classroom recognition model is applied to identify the wonderful moments and obtain the start and end time of the wonderful moments in the classroom to be identified; Based on the start and end time of the wonderful moment, the wonderful moment video clips are determined from the classroom teaching video, and key frames are extracted based on the wonderful moment video clips to obtain the wonderful moment video key frames; Based on the start and end time of the wonderful moment, the wonderful moment dialogue text is determined from the teacher-student dialogue text; the teacher-student dialogue text is determined based on the classroom teaching audio.

[0074] Specifically, the process of applying the classroom recognition model to identify wonderful moments based on classroom teaching data to obtain the start and end times and description information of the wonderful moments of the classroom to be identified may include the following steps: Considering that the results obtained by the current traditional encoding method have poor interpretability, usually only timestamp information is provided, but the event process is not restored. Therefore, in order to improve the interpretability of the recognition results, in an embodiment of the present invention, the recognition results can be presented from multiple dimensions and multiple modalities, that is, the wonderful moment description information includes wonderful moment video clips, wonderful moment video key frames and wonderful moment dialogue text.

[0075] In detail, it can be based on classroom teaching data (such as classroom teaching audio, classroom teaching video, or classroom teaching audio and classroom teaching video), and apply the classroom recognition model to identify wonderful moments to capture the wonderful moments in the classroom to be identified, and locate them on the timeline, giving the start and end time of the wonderful moments; the specific recognition process has been described in detail above and will not be repeated here.

[0076] Next, based on the start and end times of the highlight moment provided by the model, the video clip corresponding to the start and end times of the highlight moment can be determined from the classroom teaching video corresponding to the classroom teaching audio, and this can be used as the highlight moment video clip. Here, the classroom teaching audio and classroom teaching video are aligned on the timeline. Therefore, based on the located start and end times of the highlight moment, the corresponding video clip can be directly determined from the timeline. This video clip can be considered to be the video clip corresponding to the highlight moment of the classroom to be identified, and is therefore called a highlight moment video clip.

[0077] After this, keyframe positioning can be performed based on the wonderful moment video clip to determine the video keyframe corresponding to the wonderful moment of the classroom to be identified, that is, the wonderful moment video keyframe. Here, specifically, keyframe extraction can be performed on the wonderful moment video clip to extract the most core video frame that can best reflect the wonderful moment as the wonderful moment video keyframe. Here, the wonderful moment video keyframe can be one frame or multiple frames. The specific number of frames can be determined by the duration of the wonderful moment video clip, can be set by itself, or can be determined by other means. This embodiment of the present invention does not specifically limit this.

[0078] Accordingly, when presenting the results of highlight moment recognition at the video level, the results can be presented simultaneously at the text level. That is, based on the start and end times of the highlight moment provided by the model, the conversation segment corresponding to this highlight moment can be determined from the teacher-student conversation text and used as the highlight moment conversation text. Since the classroom teaching audio and the teacher-student conversation text are synchronized on the timeline, the conversation text corresponding to this period can be directly determined from the teacher-student conversation text based on the start and end times of the highlight moment. This conversation text can be considered the conversation text corresponding to the highlight moment of the classroom to be identified, and therefore can be called the highlight moment conversation text. The teacher-student conversation text can be obtained based on classroom teaching recognition. The specific recognition process has been described in detail above and will not be repeated here.

[0079] After obtaining the above-mentioned wonderful moment description information, namely, the wonderful moment video clips, wonderful moment video key frames and wonderful moment dialogue texts, in an embodiment of the present invention, they can be structuredly presented. The specific presentation method can be to display the above information on the same display interface. Figure 2 This is an example diagram of displaying the wonderful moment description information provided by the present invention. Figure 2 As shown, the start and end time of the wonderful moment can be displayed on the unified display interface, such as "48'16''-49'5''", the wonderful moment classification, such as "the color contrast of Suzhou gardens", the wonderful moment classification, such as the wonderful situation, the wonderful moment description, such as "the teacher created a vivid situation by comparing the color characteristics of the Palace Museum and Suzhou gardens. Through the contrasting color descriptions, the students deeply realized the simplicity and natural beauty of Suzhou gardens", as well as the wonderful moment video key frames, the wonderful moment dialogue text ( Figure 2 ) and highlights video clips (available through Figure 2 button in the Play Moment).

[0080] In an embodiment of the present invention, the recognition results are presented from multiple modalities such as video, audio, and text, which can achieve in-depth restoration of wonderful moments. Through multi-angle and multi-modal information, a comprehensive presentation of the wonderful moments of the classroom to be identified can be achieved, which can greatly enhance the interpretability of the recognition results.

[0081] Based on the above embodiment, in step 120, based on the classroom teaching data, the classroom recognition model is applied to identify the wonderful moments, and the start and end times and description information of the wonderful moments of the classroom to be identified are obtained. Then, the following steps are further included: Encode the start and end time of the wonderful moment and the description information of the wonderful moment, and generate a teaching analysis report of the class to be identified based on the encoded wonderful moment feature vector; Based on the teaching analysis report, generate the teacher's teaching strategy for the classroom to be identified.

[0082] Specifically, after the above-mentioned wonderful moments are identified through the classroom recognition model, in order to enable the recognition results to be better applied to subsequent teaching evaluation, in an embodiment of the present invention, they can also be processed to generate an analysis report, that is, the start and end time of the wonderful moments and the wonderful moment description information (such as wonderful moment classification, wonderful moment name, wonderful moment description, wonderful moment video clip, wonderful moment dialogue text, etc.) can be encoded to encode them in the form of feature vectors, thereby obtaining a wonderful moment feature vector, and an interpretable teaching analysis report for the classroom to be identified can be generated based on this wonderful moment feature vector.

[0083] Afterward, the teacher's teaching strategy for the identified classroom can be generated based on this teaching analysis report. This teaching strategy can include suggestions for improvement, reflections on problems encountered during the teaching process, breakthroughs in teaching and research, and praise for outstanding moments. It should be noted that the generation of the teaching analysis report and teacher teaching strategy can be implemented through large models, as well as conventional algorithms and software.

[0084] In an embodiment of the present invention, by encoding information to generate a teaching analysis report, and generating a teacher teaching strategy based on the teaching analysis report, it is possible to provide command-level suggestions and sustainable development directions for teachers in the education field.

[0085] The following describes the device for identifying wonderful classroom moments provided by the present invention. The device for identifying wonderful classroom moments described below and the method for identifying wonderful classroom moments described above can refer to each other.

[0086] Figure 3 Schematic diagram of the structure of the device for identifying wonderful moments in class provided by the present invention. Figure 3 As shown, the device includes: An acquisition unit 310 is used to acquire classroom teaching data of the classroom to be identified; The identification unit 320 is configured to identify wonderful moments based on the classroom teaching data by applying a classroom identification model, and obtain the start and end time and description information of the wonderful moments of the classroom to be identified.

[0087] The device for identifying wonderful moments in a classroom provided by the present invention obtains classroom teaching audio of a classroom to be identified; based on the classroom teaching audio, a classroom recognition model is applied to identify wonderful moments, and the start and end time of the wonderful moments in the classroom to be identified and the description information of the wonderful moments are obtained. The device overcomes the defects of traditional manual judgment that important information is easily missed, accuracy and objectivity are poor, and the result of the encoding method is poorly interpretable, and the process is cumbersome, the workload is large, and the efficiency is low. The large model technology is introduced to identify wonderful moments in the classroom, which not only gets rid of the dependence on personal subjective experience and realizes more comprehensive and accurate recognition, but also can be applied to various types of classroom teaching processes, and gives an accurate description of the wonderful moments, which can highlight their personalized characteristics and provide data support for subsequent teaching evaluation.

[0088] Based on the above embodiment, the classroom teaching data includes classroom teaching audio; The identification unit 320 is used to: Perform role separation and voice transcription based on the classroom teaching audio to obtain a teacher-student dialogue text; Based on the teacher-student dialogue text, the classroom recognition model is applied to perform wonderful moment recognition to obtain the start and end time and description information of the wonderful moments of the classroom to be recognized.

[0089] Based on the above embodiment, the classroom teaching data includes classroom teaching videos; The identification unit 320 is used to: Performing action recognition based on the classroom teaching video to obtain video action information; Based on the video action information, the classroom recognition model is applied to perform wonderful moment recognition to obtain the start and end time and description information of the wonderful moments of the classroom to be recognized.

[0090] Based on the above embodiment, the classroom teaching data includes classroom teaching audio and classroom teaching video, and the classroom teaching audio and the classroom teaching video are aligned; The identification unit 320 is used to: Perform role separation and voice transcription based on the classroom teaching audio to obtain a teacher-student dialogue text; Performing action recognition based on the classroom teaching video to obtain video action information; Based on the teacher-student dialogue text and the video action information, the classroom recognition model is applied to perform wonderful moment recognition to obtain the start and end time and description information of the wonderful moments of the classroom to be recognized.

[0091] Based on the above embodiment, the wonderful moment description information includes wonderful moment classification and wonderful moment description; The identification unit 320 is used to: Generate recognition prompt text based on the classroom teaching data and the description of the wonderful moment type; The generated recognition prompt text is input into the classroom recognition model to obtain the start and end time of the wonderful moments of the classroom to be identified, the classification of the wonderful moments, and the description of the wonderful moments.

[0092] Based on the above embodiment, the classroom teaching data includes classroom teaching audio; The identification unit 320 is used to: Performing emotion recognition based on the classroom teaching audio to obtain audio emotion information; Based on the classroom teaching data, the audio emotion information and the description of the wonderful moment type, an identification prompt text is generated.

[0093] Based on the above embodiment, the classroom teaching data includes classroom teaching audio and classroom teaching video, and the classroom teaching audio and the classroom teaching video are aligned; the wonderful moment description information includes wonderful moment video clips, wonderful moment video key frames and wonderful moment dialogue text; The identification unit 320 is used to: Based on the classroom teaching data, the classroom recognition model is applied to identify the wonderful moments, and the start and end times of the wonderful moments of the classroom to be identified are obtained; Determining a wonderful moment video clip from the classroom teaching video based on the start and end time of the wonderful moment, and extracting key frames based on the wonderful moment video clip to obtain a wonderful moment video key frame; Based on the start and end time of the wonderful moment, the wonderful moment dialogue text is determined from the teacher-student dialogue text; the teacher-student dialogue text is determined based on the classroom teaching audio.

[0094] Based on the above embodiment, the apparatus further includes a report generating unit, configured to: Encoding the start and end times of the wonderful moments and the description information of the wonderful moments, and generating a teaching analysis report of the to-be-identified class based on the wonderful moment feature vector obtained by encoding; Based on the teaching analysis report, the teacher's teaching strategy for the classroom to be identified is generated.

[0095] Based on the above embodiment, the description of the wonderful moment type includes at least two of the following: description of a wonderful question-and-answer moment, description of a wonderful situation moment, description of a technology integration moment, and description of a collision of ideas moment.

[0096] Figure 4 An example of a physical structure diagram of an electronic device is shown below. Figure 4As shown, the electronic device may include: a processor 410, a communications interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communications interface 420, and the memory 430 communicate with each other via the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute a method for identifying wonderful moments in a classroom, which includes: obtaining classroom teaching data of a classroom to be identified; and applying a classroom recognition model to identify wonderful moments based on the classroom teaching data to obtain the start and end times and description information of the wonderful moments in the classroom to be identified.

[0097] Furthermore, the logic instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0098] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the classroom wonderful moment identification method provided by the above methods, and the method includes: obtaining classroom teaching data of the classroom to be identified; based on the classroom teaching data, applying a classroom recognition model to identify wonderful moments, and obtaining the start and end time of the wonderful moments of the classroom to be identified and the wonderful moment description information.

[0099] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it is implemented to execute the classroom wonderful moment identification method provided by the above-mentioned methods. The method includes: obtaining classroom teaching data of the classroom to be identified; based on the classroom teaching data, applying a classroom recognition model to identify wonderful moments, and obtaining the start and end time of the wonderful moments of the classroom to be identified and the wonderful moment description information.

[0100] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0101] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0102] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for identifying wonderful moments in a classroom, characterized in that: include: Obtain classroom teaching data of the class to be identified; Based on the classroom teaching data, a classroom recognition model is applied to identify wonderful moments, and the start and end times and description information of the wonderful moments of the classroom to be identified are obtained.

2. The method for identifying wonderful moments in a classroom according to claim 1, characterized in that: The classroom teaching data includes classroom teaching audio; Based on the classroom teaching data, the classroom recognition model is applied to identify the wonderful moments, and the start and end time of the wonderful moments of the classroom to be identified and the description information of the wonderful moments are obtained, including: Perform role separation and voice transcription based on the classroom teaching audio to obtain a teacher-student dialogue text; Based on the teacher-student dialogue text, the classroom recognition model is applied to perform wonderful moment recognition to obtain the start and end time and description information of the wonderful moments of the classroom to be recognized.

3. The method for identifying wonderful moments in a classroom according to claim 1, characterized in that: The classroom teaching data includes classroom teaching videos; Based on the classroom teaching data, the classroom recognition model is applied to identify the wonderful moments, and the start and end time of the wonderful moments of the classroom to be identified and the description information of the wonderful moments are obtained, including: Performing action recognition based on the classroom teaching video to obtain video action information; Based on the video action information, the classroom recognition model is applied to perform wonderful moment recognition to obtain the start and end time and description information of the wonderful moments of the classroom to be recognized.

4. The method for identifying wonderful moments in a classroom according to claim 1, characterized in that: The classroom teaching data includes classroom teaching audio and classroom teaching video, and the classroom teaching audio and the classroom teaching video are aligned; Based on the classroom teaching data, the classroom recognition model is applied to identify the wonderful moments, and the start and end time of the wonderful moments of the classroom to be identified and the description information of the wonderful moments are obtained, including: Perform role separation and voice transcription based on the classroom teaching audio to obtain a teacher-student dialogue text; Performing action recognition based on the classroom teaching video to obtain video action information; Based on the teacher-student dialogue text and the video action information, the classroom recognition model is applied to perform wonderful moment recognition to obtain the start and end time and description information of the wonderful moments of the classroom to be recognized.

5. The method for identifying wonderful moments in a classroom according to any one of claims 1 to 4, characterized in that: The wonderful moment description information includes wonderful moment classification and wonderful moment description; Based on the classroom teaching data, the classroom recognition model is applied to identify the wonderful moments, and the start and end time of the wonderful moments of the classroom to be identified and the description information of the wonderful moments are obtained, including: Generate recognition prompt text based on the classroom teaching data and the description of the wonderful moment type; The generated recognition prompt text is input into the classroom recognition model to obtain the start and end time of the wonderful moments of the classroom to be identified, the classification of the wonderful moments, and the description of the wonderful moments.

6. The method for identifying wonderful moments in a classroom according to claim 5, characterized in that: The classroom teaching data includes classroom teaching audio; the generating of recognition prompt text based on the classroom teaching data and the description of the wonderful moment type includes: Performing emotion recognition based on the classroom teaching audio to obtain audio emotion information; Based on the classroom teaching data, the audio emotion information and the description of the wonderful moment type, an identification prompt text is generated.

7. The method for identifying wonderful moments in a classroom according to any one of claims 1 to 4, characterized in that: The classroom teaching data includes classroom teaching audio and classroom teaching video, and the classroom teaching audio and the classroom teaching video are aligned; the wonderful moment description information includes wonderful moment video clips, wonderful moment video key frames and wonderful moment dialogue text; Based on the classroom teaching data, the classroom recognition model is applied to identify the wonderful moments, and the start and end time of the wonderful moments of the classroom to be identified and the description information of the wonderful moments are obtained, including: Based on the classroom teaching data, applying the classroom recognition model to identify wonderful moments, and obtaining the start and end time of the wonderful moments of the to-be-identified classroom; Determining a wonderful moment video clip from the classroom teaching video based on the start and end time of the wonderful moment, and extracting key frames based on the wonderful moment video clip to obtain a wonderful moment video key frame; Based on the start and end time of the wonderful moment, the wonderful moment dialogue text is determined from the teacher-student dialogue text; the teacher-student dialogue text is determined based on the classroom teaching audio.

8. The method for identifying wonderful moments in a classroom according to any one of claims 1 to 4, characterized in that: Based on the classroom teaching data, applying the classroom recognition model to identify the wonderful moments, obtaining the start and end time of the wonderful moments of the classroom to be identified and the wonderful moment description information, and then further comprising: Encoding the start and end times of the wonderful moments and the description information of the wonderful moments, and generating a teaching analysis report of the to-be-identified class based on the wonderful moment feature vector obtained by encoding; Based on the teaching analysis report, the teacher's teaching strategy for the classroom to be identified is generated.

9. The method for identifying wonderful moments in a classroom according to claim 5, characterized in that: The description of the wonderful moment type includes at least two of the following: description of a wonderful question-and-answer moment, description of a wonderful situation moment, description of a technology integration moment, and description of a collision of ideas moment.

10. A device for identifying wonderful moments in a classroom, characterized in that: include: An acquisition unit, used for acquiring classroom teaching data of a classroom to be identified; The recognition unit is used to apply the classroom recognition model to identify the wonderful moments based on the classroom teaching data, and obtain the start and end time and description information of the wonderful moments of the classroom to be identified.

Images