Teaching content evaluation method and device, electronic equipment and medium

Abstract

The invention discloses a teaching content evaluation method and apparatus, an electronic device and a medium. The method comprises the steps of obtaining a subtitle file and a teaching outline text corresponding to teaching audio data; performing first segmentation processing on the subtitle file to obtain a plurality of teaching fragments, and vectorizing each teaching fragment into a first vector; performing second segmentation processing on the teaching outline text to obtain a plurality of key knowledge point fragments, vectorizing each key knowledge point fragment into a second vector, and storing the second vector into a vector database; for each first vector, performing multi-path retrieval in a vector database to obtain a candidate knowledge point fragment set, and determining a target knowledge point fragment corresponding to the first vector from the candidate knowledge point fragment set; and inputting the target knowledge point fragment and the teaching fragment into a discrimination model, and outputting a matching result of the teaching fragment and the target knowledge point fragment. And the matching result can accurately evaluate the coincidence degree between the teaching of the teacher and the teaching outline.

Description

Technical Field

[0001] This application relates to the field of information technology, and in particular to a method, apparatus, electronic device, and medium for evaluating teaching content. Background Technology

[0002] In smart classrooms at universities, teachers typically develop teaching plans or syllabi at the beginning of the semester, including the key points and class schedule for each lesson. These plans are used to assess whether teachers are adhering to the syllabus and to evaluate the effectiveness of their instruction.

[0003] After each lesson, a video recording is generated. An audio-to-text algorithm is used to transcribe the teacher's lecture video into subtitles, and then a text summarization model is used to summarize the subtitle content. Next, the course's key knowledge points are extracted from the syllabus, and a text similarity calculation module is used to analyze the similarity between the generated text summary and the syllabus's key knowledge points. Finally, the similarity results between the syllabus and the lecture video are obtained.

[0004] Because the subtitles transcribed from teachers' lecture videos contain some information unrelated to the text summary, even if the summary is highly relevant to the knowledge points, the similarity result after vectorization may be low, leading to inaccurate evaluation results. Summary of the Invention

[0005] In view of the above problems, embodiments of the present invention are proposed to provide a teaching content assessment method, apparatus, electronic device and medium that overcomes or at least partially solves the above problems.

[0006] In a first aspect, embodiments of this application disclose a method for evaluating teaching content, the method comprising: Obtain the subtitle file and teaching syllabus text corresponding to the teaching audio data; The subtitle file is segmented into multiple teaching segments, and each teaching segment is vectorized into a first vector. The teaching syllabus text is processed into a second segmentation to obtain multiple key knowledge point segments. Each key knowledge point segment is then vectorized into a second vector and stored in a vector database. For each of the first vectors, a multi-way search is performed in the vector database to obtain a set of candidate knowledge point fragments, and the target knowledge point fragment corresponding to the first vector is determined from the set of candidate knowledge point fragments. The target knowledge point segment and the teaching segment are input into the discrimination model, and the matching result between the teaching segment and the target knowledge point segment is output.

[0007] Secondly, embodiments of this application disclose a teaching content evaluation device, the device comprising: The acquisition module is used to acquire the subtitle file and teaching outline text corresponding to the teaching audio data; The first processing module is used to perform a first segmentation process on the subtitle file to obtain multiple teaching segments, and to vectorize each teaching segment into a first vector. The second processing module is used to perform a second segmentation process on the teaching syllabus text to obtain multiple key knowledge point segments, and then vectorize each key knowledge point segment into a second vector and store it in a vector database. The retrieval module is used to perform multi-way retrieval in the vector database for each first vector to obtain a set of candidate knowledge point fragments, and to determine the target knowledge point fragment corresponding to the first vector from the set of candidate knowledge point fragments; The discrimination module is used to input the target knowledge point segment and the teaching segment into the discrimination model and output the matching result between the teaching segment and the target knowledge point segment.

[0008] Thirdly, embodiments of this application disclose an electronic device, including: a processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the method described in the first aspect.

[0009] Fourthly, embodiments of this application disclose a computer-readable storage medium that, when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, enables the electronic device to perform the method described in the first aspect.

[0010] This application discloses a method for evaluating teaching content, which involves acquiring subtitle files and syllabus text corresponding to teaching audio data; performing a first segmentation process on the subtitle files to obtain multiple teaching segments, and vectorizing each teaching segment into a first vector; performing a second segmentation process on the syllabus text to obtain multiple key knowledge point segments, and vectorizing each key knowledge point segment into a second vector, which is then stored in a vector database; for each first vector, performing a multi-path search in the vector database to obtain a set of candidate knowledge point segments, and determining a target knowledge point segment corresponding to the first vector from the set of candidate knowledge point segments; inputting the target knowledge point segment and the teaching segments into a discrimination model, and outputting the matching result between the teaching segments and the target knowledge point segments. This application's method segments subtitle fragments, making each segment semantically independent. For each segment, a corresponding target knowledge point fragment is identified, reducing interference from irrelevant information. Simultaneously, a multi-path retrieval approach improves the recall quality of relevant knowledge point fragments, thereby enhancing the accuracy of the identified target knowledge point fragments. When evaluating teaching segments, a discriminative model is further used to determine the matching results between the teaching segments and the target knowledge point fragments. This ensures that the final matching results can accurately assess the degree of alignment between the teacher's instruction and the teaching syllabus, improving evaluation accuracy. Attached Figure Description

[0011] Figure 1 This is a flowchart illustrating the steps of a teaching content evaluation method provided in an embodiment of this application; Figure 2 This is a flowchart illustrating the steps of another teaching content evaluation method provided in this application embodiment; Figure 3 This is a flowchart illustrating the steps of the third teaching content evaluation method provided in the embodiments of this application; Figure 4 This is a diagram illustrating an implementation of the third teaching content evaluation method provided in this application. Figure 5 This is a block diagram of a teaching content evaluation device provided in an embodiment of this application; Figure 6 This is a block diagram of an electronic device provided in an embodiment of this application; Figure 7 This is a schematic diagram of another electronic device provided in the embodiments of this application. Detailed Implementation

[0012] Exemplary embodiments of the present application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this application will be thorough and complete, and will fully convey the scope of the present application to those skilled in the art.

[0013] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and are not limited in number; for example, a first object can be one or more. Furthermore, the term "and / or" in the specification and claims is used to describe the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. In the embodiments of this application, the term "multiple" refers to two or more, and other quantifiers are similar.

[0014] The concepts involved in this application are explained below.

[0015] LLM (Large Language Model): A large language model.

[0016] RAG (Retrieval Augmented Generation): Retrieval Augmented Generation.

[0017] ASR (Automatic Speech Recognition): Automatic speech recognition.

[0018] TF-IDF (Term Frequency-Inverse Document Frequency): A commonly used weighting technique for information retrieval and data mining.

[0019] BM25: A classic algorithm in the field of information indexing used to calculate the similarity score between query terms and documents.

[0020] Prompt: A prompt word, which is a command or hint provided to the large model.

[0021] refer to Figure 1 , Figure 1 This application illustrates a method for evaluating teaching content, the method comprising: Step 101: Obtain the subtitle file and teaching outline text corresponding to the teaching audio data.

[0022] In this embodiment, the teaching audio data can be audio and video data recorded from the teacher's lecture. By recognizing the teaching audio data using speech recognition technology, a corresponding subtitle file can be obtained. The subtitle file can contain all the audio content of the lecture. The teaching syllabus text is a document outlining the knowledge points that the teacher needs to teach.

[0023] Step 102: Perform a first segmentation process on the subtitle file to obtain multiple teaching segments, and vectorize each teaching segment into a first vector.

[0024] In this embodiment, the original subtitle file is obtained directly based on speech recognition and may contain multiple knowledge points. To more accurately evaluate the teaching content, the subtitle file can be first segmented. This first segmentation can be performed using a context-based semantic model. For example, based on the coherence and transition points of the topic, related short sentences can be merged into multiple semantically complete teaching segments. Each teaching segment represents a relatively independent topic.

[0025] The teaching segments obtained after segmentation can be transformed into first vectors using a pre-trained text embedding model (such as PGE).

[0026] Step 103: Perform a second segmentation process on the teaching syllabus text to obtain multiple key knowledge point segments, and then vectorize each key knowledge point segment into a second vector and store it in the vector database.

[0027] In this embodiment, after obtaining the teaching syllabus, it is also necessary to process the syllabus to obtain the key knowledge points. Specifically, the teaching syllabus text can be segmented into key knowledge point segments, with each segment corresponding to a description of a knowledge point. Then, the text embedding model mentioned in step 102 can be used to convert the key knowledge point segments into second vectors. The second vectors are then stored in a vector database (such as Milvus). By converting the key knowledge point segments into second vectors, the human-readable teaching syllabus is converted into machine-queryable language and stored as a vector knowledge base for subsequent query matching. The first and second vectors can be converted using the same model to ensure they belong to the same semantic space, facilitating subsequent retrieval operations.

[0028] Step 104: For each of the first vectors, perform a multi-way search in the vector database to obtain a set of candidate knowledge point fragments, and determine the target knowledge point fragment corresponding to the first vector from the set of candidate knowledge point fragments.

[0029] In this embodiment, for each teaching segment corresponding to the first vector obtained in step 102, it can be used as a query request to retrieve data from the vector knowledge base constructed in step 103. To improve accuracy, a multi-path retrieval strategy can be employed. Multi-path retrieval involves using different focuses for retrieval; for example, matching can be based on keywords (such as the BM25 algorithm), or similarity can be calculated based on semantic vectors. Specifically, candidate knowledge point segments matching keywords and candidate knowledge point segments matching semantics are identified and merged to form a set of candidate knowledge point segments.

[0030] After obtaining the candidate knowledge point fragment set, the candidate knowledge point fragments in the set can be sorted to determine the most matching target knowledge point fragment. Specifically, a re-ranking model (such as PGEReranker) can be used to sort the candidate knowledge point fragments, calculate the relevance score of each candidate knowledge point fragment to the first vector, and finally select the target knowledge point fragment with the highest score as the syllabus knowledge point most relevant to that teaching fragment.

[0031] Step 105: Input the target knowledge point segment and the teaching segment into the discrimination model, and output the matching result between the teaching segment and the target knowledge point segment.

[0032] In this embodiment, after determining the target knowledge point segment and its corresponding teaching segment, the data is input into a discrimination model, which can be a large language model (Qwen). Preset instructions are used to instruct the model to determine the matching result between the teaching segment and the target knowledge point segment.

[0033] Specifically, the preset instructions can be commands used to guide the output of the discriminative model. For example, a preset instruction could be: "Given a document and several questions, you need to determine whether the document contains answers to the questions. Output 1 if it does, and 0 if it doesn't. Your answer can only be 0 or 1." Here, the document corresponds to a teaching segment, and the answers correspond to a target knowledge point segment. In this application, the discriminative model only determines a match when the teaching segment actually contains a substantial explanation of the syllabus knowledge points. This makes the evaluation results of teaching content obtained based on the discriminative model closer to reality, improving the accuracy of teaching content evaluation. It also solves the problem of result bias caused by using only text similarity calculation.

[0034] In summary, this application discloses a method for evaluating teaching content, which involves obtaining subtitle files and syllabus text corresponding to teaching audio data; performing a first segmentation process on the subtitle files to obtain multiple teaching segments, and vectorizing each teaching segment into a first vector; performing a second segmentation process on the syllabus text to obtain multiple key knowledge point segments, and vectorizing each key knowledge point segment into a second vector, which is then stored in a vector database; performing a multi-path search on each first vector in the vector database to obtain a set of candidate knowledge point segments, and determining the target knowledge point segment corresponding to the first vector from the set of candidate knowledge point segments; inputting the target knowledge point segment and the teaching segments into a discrimination model, and outputting the matching result between the teaching segments and the target knowledge point segments. This application's method segments subtitle fragments, making each segment semantically independent. For each segment, a corresponding target knowledge point fragment is identified, reducing interference from irrelevant information. Simultaneously, a multi-path retrieval approach improves the recall quality of relevant knowledge point fragments, thereby enhancing the accuracy of the identified target knowledge point fragments. When evaluating teaching segments, a discriminative model is further used to determine the matching results between the teaching segments and the target knowledge point fragments. This ensures that the final matching results can accurately assess the degree of alignment between the teacher's instruction and the teaching syllabus, improving evaluation accuracy.

[0035] refer to Figure 2 , Figure 2 This application provides another method for evaluating teaching content, the method comprising: Step 201: Obtain the subtitle file and teaching outline text corresponding to the teaching audio data; Step 202: Perform a first segmentation process on the subtitle file to obtain multiple teaching segments, and vectorize each teaching segment into a first vector.

[0036] In this embodiment, after processing the original subtitle file of the teaching audio data, the subtitle file is first preprocessed. Noise in the text is purified and information density is enhanced through key information extraction and stop word filtering. Key information extraction involves retaining technical terms and entities related to the core teaching concepts, while filtering out irrelevant parenthetical phrases, filler words, or stop words. Stop words include high-frequency but low-information-contribution function words such as "this," "that," "then," and "well." After preprocessing, the subtitle file is converted into standardized text with high technical semantic density. The preprocessed subtitle file is then input into a BERT-based Cross-Segment segmentation model for the first segmentation process. By analyzing the semantic association strength of sentences in a high-dimensional vector space, signals of topic transitions are identified. For example, when the explanation transitions from "CPU pipeline" to "memory hierarchy," the sequential subtitle sentences are automatically clustered and merged into independent paragraphs. Finally, the subtitle file is intelligently divided into n teaching segments, each corresponding to a main topic or technical point.

[0037] Optionally, step 202 includes: Sub-step 2021 involves preprocessing the subtitle file to obtain a preprocessed subtitle file; the preprocessing includes at least one of: filtering stop words, removing modal particles, and text standardization; Sub-step 2022 involves segmenting the preprocessed subtitle file using a context-based paragraph segmentation model to obtain multiple teaching segments.

[0038] In this embodiment of the application, for sub-steps 2021 and 2022, after obtaining the subtitle file, since the original subtitle file contains a lot of noise unrelated to the teaching content, directly using the original subtitle file for processing will affect the accuracy of the final result. Therefore, after obtaining the subtitle file, the subtitle file is preprocessed to remove the noise in the original subtitle file and improve the information density of the subtitle text.

[0039] The preprocessing includes at least one of filtering stop words, removing modal particles, and text standardization. For example, stop words are high-frequency but low-information function words and conjunctions such as "de", "le", "zai", "then", etc. Stop words contribute less in semantic analysis, so stop words can be removed to reduce the data dimension. Modal particles such as "um", "ah", "this", "that", etc. Modal particles appear relatively frequently in the colloquial lecture process, and modal particles do not contain knowledge point information, so they can be removed to highlight the core sentences in the subtitle file. Text standardization is to convert the text into a unified format, such as correcting error characters, case conversion, etc. Through the preprocessing process of the original subtitle file, the subtitle file with redundant information and noise is converted into a standard text with a high density of core sentences, providing a high-quality input text for subsequent processing and avoiding interference of the matching results by irrelevant words.

[0040] The context-based paragraph segmentation model is used to divide the preprocessed subtitle file into multiple teaching segments. Each teaching segment consists of several sentences, which are semantically coherent and express a relatively independent and complete teaching sub-viewpoint.

[0041] Specifically, the context-based paragraph segmentation model aggregates continuously semantically related sentences together by calculating the semantic similarity of sentences in the vector space, and at the same time identifies the boundary points where the topic in the teaching content changes, so as to achieve the segmentation of the subtitle file. Each teaching segment is a processing unit with complete semantics and context association, improving the accuracy of matching knowledge point segments for teaching segments subsequently.

[0042] Step 203: Perform a second segmentation process on the teaching syllabus text to obtain multiple key knowledge point segments. After each key knowledge point segment is vectorized into a second vector, it is stored in the vector database.

[0043] In the embodiment of the present application, the complete teaching syllabus content of the course formulated by the teacher is obtained. The teaching syllabus content includes the core knowledge points of the course, such as specific knowledge points like "instruction-level parallelism" and "pipeline hazard". After obtaining the original syllabus text, a fixed-length sliding window algorithm is used for segmentation: the text is divided with 64 characters as a basic unit, and an 8-character overlapping window is set between adjacent units for segmentation. Through the overlapping window, a single complete knowledge point is prevented from being cut into different segments. Subsequently, meaningless text segments with too short length or only containing numbers are removed. After the text slicing is completed, the PGE model is used to vectorize each key knowledge point segment to obtain a second vector, and the second vector is stored in the Milvus vector database.

[0044] Optionally, step 203 includes: Sub-step 2031: Divide the teaching outline text into segments of a preset fixed length and set up overlapping windows to generate the key knowledge point segments; Sub-step 2032: Input the text of the key knowledge point segments into the PGE vector embedding model, and use the text embedding vector output by the PGE vector embedding model as the second vector.

[0045] In this embodiment, for sub-steps 2031 and 2032, the obtained syllabus text is first preprocessed. The preprocessing process may involve dividing the syllabus text into basic units of a preset fixed length (e.g., 64 characters), and setting an overlap window of a specific length (e.g., 8 characters) between adjacent units. This overlap maintains the contextual coherence of knowledge point boundaries, preventing individual knowledge points from being segmented. The syllabus text is then divided into key knowledge point segments, which are vectorized and used as a processing unit. This segmentation process ensures the integrity of text paragraphs, and the overlap reduces semantic loss caused by the segmentation operation.

[0046] The generated key knowledge point segments are input into the PGE vector embedding model, which then obtains the corresponding text embedding vectors as the second vector. The PGE vector embedding model can map semantically similar knowledge points to adjacent positions in the vector space, transforming the key knowledge point segments into the second vector. The PGE vector embedding model used here can be the same model used to vectorize the teaching segments, so that the first and second vectors are in the same semantic space, thus completing the subsequent vector retrieval steps. Step 204: Using the first vector as the query vector, perform a first search and a second search to obtain a first candidate knowledge point fragment and a second candidate knowledge point fragment; the first search is based on keyword matching, and the second search is based on semantic vector similarity. Step 205: Merge the first candidate knowledge point fragment and the second candidate knowledge point fragment into a candidate knowledge point fragment set.

[0047] In this embodiment of the application, for steps 204 and 205, a multi-path retrieval is used to comprehensively recall the key knowledge point segments related to each teaching segment.

[0048] Specifically, the first vector corresponding to the teaching segment is used as a unified query vector for multi-path retrieval. The first retrieval is based on a keyword matching algorithm (such as BM25). This algorithm matches the query vector with words in the knowledge point segments based on statistical methods such as overlap and frequency. Teaching content typically contains proper nouns or technical terms; the keyword matching algorithm can retrieve content related to these keywords, identifying knowledge points directly related to the core concepts mentioned in the teaching content. The second retrieval can be based on semantic vector similarity calculation. It determines semantically similar content by calculating the distance between the query vector and the knowledge point vector in a high-dimensional semantic space. This allows the query to not only match words but also identify knowledge points with different expressions but related semantics. For example, if the teaching content mentions "CPU," the second retrieval can find "central processing unit" as the corresponding knowledge point. The two retrieval processes output first and second candidate knowledge point segments respectively, which are combined to form a set of candidate knowledge point segments. In this application, there can be multiple first and second candidate knowledge point segments; no limitation is imposed here. After the first and second searches, a candidate knowledge point fragment set is obtained. This set includes all knowledge point fragments that have keyword matching and semantic relevance to the teaching fragments. Multi-path retrieval makes the candidate knowledge point fragment set more comprehensive, solving the problem of knowledge point omissions in single-search methods and improving the accuracy of the target knowledge point fragments corresponding to the final determined teaching fragments. In other words, this application utilizes a hybrid multi-path recall method to recall the syllabus knowledge point retrieval results and uses a re-ranking model to optimize the retrieval results, effectively improving the recall rate of knowledge point retrieval and solving the problem of irrelevant recalled content.

[0049] Step 206: Sort the candidate knowledge point fragments in the candidate knowledge point fragment set using the re-sorting model to obtain the sorting result; Step 207: Determine the candidate knowledge point fragment with the highest score in the sorting result as the target knowledge point fragment.

[0050] In this embodiment of the application, for steps 206 and 207, after determining the candidate knowledge point fragment set, the target knowledge point fragment that best matches the teaching fragment is further determined from the candidate knowledge point fragment set. For example, a re-ranking model can be used to rank the candidate knowledge point fragments in the candidate knowledge point fragment set, and the target knowledge point fragment is determined based on the ranking result.

[0051] Specifically, the re-ranking model performs deep interaction attention calculations between the query vector and each candidate knowledge point segment to identify more subtle semantic relationships between them. Based on the calculation results, it obtains a similarity score between each first vector and the second vector corresponding to the candidate knowledge point segment. Finally, the ranking result is obtained based on the similarity score. From the above ranking results, the candidate knowledge point segment with the highest score is selected and determined as the target knowledge point segment corresponding to the teaching segment. The re-ranking model ensures that the target knowledge point segment corresponding to each teaching segment is the most relevant and best-matched knowledge point.

[0052] For example, the re-ranking model could be the PGE Reranker model, which jointly encodes the current teaching segment text with each of the recalled candidate knowledge point segments, allowing them to perform semantic interaction and attention calculations in the model's deep network to obtain more subtle semantic relationships. After the PGE Reranker scores each element in the candidate knowledge point set, a ranking result is obtained, and the highest-ranked candidate knowledge point segment is selected as the target knowledge point segment.

[0053] Step 208: Input the target knowledge point segment and the teaching segment into the discrimination model, and output the matching result between the teaching segment and the target knowledge point segment.

[0054] In this embodiment, after determining the target knowledge point segment and its corresponding teaching segment, the data is input into a discrimination model, which can be a large language model (Qwen). Preset instructions are used to instruct the model to determine the matching result between the teaching segment and the target knowledge point segment.

[0055] Specifically, the preset instructions can be commands used to guide the output of the discriminative model. For example, a preset instruction could be: "Given a document and several questions, you need to determine whether the document contains answers to the questions. Output 1 if it does, and 0 if it doesn't. Your answer can only be 0 or 1." Here, the document corresponds to a teaching segment, and the answers correspond to target knowledge point segments. In this application, the discriminative model only determines a match when the teaching segment actually contains a substantial explanation of the syllabus knowledge points. This makes the evaluation results of teaching content obtained based on the discriminative model closer to reality, improving the accuracy of teaching content evaluation.

[0056] Optionally, step 208 includes: Sub-step 2081: Guide the output of the discrimination model with preset prompt words, and output the matching result between the teaching segment and the target knowledge point segment; The preset prompt words are used to instruct the discrimination model to perform an answer existence judgment and restrict the output of the discrimination model to a first value or a second value; the matching result is a binary judgment result, wherein the first value indicates that the teaching segment contains the answer of the target knowledge point segment, and the second value indicates that the teaching segment does not contain the answer of the target knowledge point segment.

[0057] In this embodiment, the discriminant model is used to determine whether a teaching segment matches a target knowledge point segment. In a teaching scenario, determining whether the teaching content contains knowledge points from the syllabus is crucial. This includes not only mentioning the terminology of the syllabus but also providing explanations of those knowledge points. Therefore, the discriminant model needs to determine whether the teaching segment contains explanations of the target knowledge point segment, not just the keywords of that segment. To achieve this, this application guides the output of the discriminant model using preset prompts, enabling it to perform an answer existence check. Simultaneously, the output is restricted to either a first or second numerical value, thus providing a clear matching result between the teaching segment and the target knowledge point segment.

[0058] Furthermore, pre-set prompts, such as "Given a document and several questions, you need to determine whether the document contains answers to the questions. Output 1 if yes, and 0 if no. Your answer can only be 0 or 1," instruct the discriminative model to determine whether a given instructional segment (as a document) actually contains the substantive answer to its paired target knowledge point segment (as a question). This shifts the model's focus from "topic relevance" to "whether an answer is provided." Moreover, during output, the pre-set prompts, through mandatory instructions, limit the model's output to a first value (e.g., 1) and a second value (e.g., 0). This ensures that the model's matching result is a clear binary judgment: when the instructional segment contains explanations or descriptions of the target knowledge point, the output is the first value representing "yes"; conversely, if the instructional segment only mentions the concept without elaboration, or the content is unrelated to the knowledge point, the output is the second value representing "no." This clear output facilitates the subsequent generation of quantitative teaching content evaluation results. By designing and optimizing the LLM using the pre-set prompts, the model's ability to handle syllabus and subtitle matching and recognition is significantly improved.

[0059] Optionally, the method further includes: Step 209: Calculate the matching results of all the teaching segments and the corresponding target knowledge point segments; Step 210: Determine the degree of matching between the teaching content and the teaching syllabus based on the ratio of the number of first values ​​in the matching results to the total number of matching results.

[0060] In this embodiment, steps 209 and 210 involve statistically analyzing the matching results generated by the discriminative model for all teaching segments and their corresponding target knowledge point segments. The frequencies of the first value (e.g., 1) representing "contains the answer" and the second value (e.g., 0) representing "does not contain the answer" are counted. The number of the first value is divided by the total number of matching results (i.e., the total number of teaching segments) to obtain the proportion of syllabus knowledge points contained in the teaching content. This proportion represents the matching degree between the teaching content and the syllabus. For example, if 80% of the teaching segments are determined to contain the answer of their corresponding knowledge point, the final matching degree is 80%. The matching degree reflects the actual coverage of the knowledge points specified in the syllabus by the teacher's classroom instruction. It can be used for evaluating teaching quality, optimizing curriculum design, and providing a basis for decision-making in teaching management.

[0061] refer to Figure 3 , Figure 3 The flowchart illustrates the third teaching content evaluation method provided in this application embodiment. Key knowledge points taught by teachers are extracted from the teaching syllabus, knowledge point metadata is obtained, and the key knowledge points are vectorized using the PGE algorithm. The vectorized knowledge points are stored in the vector database Milvus. The subtitle file is preprocessed, including key information extraction and stop word filtering. The preprocessed subtitle data is input into an intelligent segmentation model. Assuming the subtitles are divided into n segments, each of the n segments is vectorized using the same vectorization model as the syllabus knowledge point vectorization model. These n vectorized segments are used as query vectors for vector retrieval, extracting similarity information from the vector database. The vector recall module uses a multi-path recall algorithm, such as BM25 for keyword recall and the PGE algorithm for vector recall. This hybrid multi-path recall effectively alleviates the low recall rate caused by using only vector recall. Then, the PGE Reranker model is used to re-rank the recalled knowledge point segments based on similarity. Take the segment with the highest similarity ranking from the outline knowledge points. Thus, n segments of subtitle data can be converted into n segments of outline knowledge points. Input the n segments of outline knowledge points, the prompt word "Prompt," and the n segments of subtitle data into an LLM (Local Level Manager). The LLM evaluates the matching degree between these n segments of outline knowledge points and the n segments of subtitle data. If they match, it returns 1; otherwise, it returns 0. Assuming i segments match, the matching degree is i / n * 100%. Return the final matching degree result.

[0062] refer to Figure 4 , Figure 4This document illustrates an implementation of a teaching content evaluation method provided by an embodiment of this application. Taking a teacher teaching a course on computer system architecture as a specific example, it assumes that the teacher's syllabus for the semester and a recorded video of a lesson have been obtained, and the recorded video data has been transcribed into subtitle data using an automatic speech recognition method. All content from the teacher's proposed syllabus for computer system architecture is obtained, including knowledge point information and lesson time information for each lesson. This information is segmented into 64-character segments with an overlap window of 8 characters, and short texts or texts containing only numbers are filtered out as they are meaningless. Each segment is vectorized using a PGE model and stored in a Milvus vector database. The subtitle file transcribed from the computer system architecture course video is obtained, and then the subtitle file undergoes data preprocessing, including key information extraction and stop word filtering. The preprocessed subtitle data is intelligently segmented using a BERT-based Cross-Segment segmentation model, resulting in n segments. These segments are then vectorized using a PGE model. The first vector corresponding to the vectorized computer system architecture teaching segment is used as the query vector for vector retrieval. The BM25 model is used for keyword recall, and the PGE model for vector recall, employing a hybrid multi-path recall method to obtain multiple results. The PGE Reranker model is used to re-rank the recalled results, and the highest-ranking computer system knowledge point segment is selected. For each teaching segment, the above steps are repeated to obtain multiple knowledge point segments. The n teaching segments and n knowledge point segments after intelligent segmentation of computer system architecture are input into the LLM (Learning Manager for Computer System Architecture). The preset prompt, after optimization, is: "Given a document and several questions, you need to determine if there are answers to the questions in the document. If there are, output 1; otherwise, output 0. Your answer can only be 0 or 1." The LLM uses the open-source Qwen-14b chat model, which meets the needs of local deployment and ensures data security. Each result output by the LLM is counted, and the final matching degree is calculated. Here, we assume a result of 80%, indicating that the teaching syllabus and subtitle content are relatively well-matched, and teachers generally follow the teaching syllabus in their lessons.

[0063] In one embodiment, if the subtitle file obtained from the converted teaching audio file contains the following content: "Okay, students, next we'll look at a very important concept called data hazard. Um... that is, in a pipeline, if the following instruction needs the calculation result of the preceding instruction, but the preceding instruction hasn't finished calculating, then the following instruction has to stop and wait, right? This situation is a data hazard.", and the key knowledge points in the syllabus include: "K1: Basic principles and stage division of instruction pipelines, K2: Types of pipeline hazards: structural hazard, data hazard, control hazard, K3: Conditions for the occurrence of data hazards and solutions, K4: Basic principles of dynamic branch prediction.", the clean data after preprocessing the subtitle file is: "Students, look at the important concept: data hazard. A data hazard occurs when a subsequent instruction needs the calculation result of a preceding instruction, the preceding instruction hasn't finished calculating, and the subsequent instruction stops and waits." Stop words and interjections such as "okay," "we," "um...", "that is to say," "if," "but," "isn't it," and "right?" are filtered out. Because this segment semantically fully defines "data hazard," it is divided into an independent teaching segment. The syllabus knowledge points have been pre-segmented and quantified, and stored in the Milvus database. After the teaching segments are vectorized, a multi-path retrieval is performed. The first retrieval (keyword BM25 recall): the high-frequency word "data adventure" in the teaching segments has the highest matching degree with K2 and K3, and K2 and K3 are recalled. The second retrieval (semantic vector recall): the similarity between the teaching segment and all syllabus knowledge point vectors is calculated. If the similarity of K3 > K2 > K1 > K4, K3 and K2 can be recalled, resulting in a set of candidate knowledge point segments, including knowledge point segments K2 and K3. The re-ranking model further sorts K2 and K3, and calculates the matching score between the teaching segment and K2 and K3 respectively. If K3 > K2, then the target knowledge point segment is determined to be K3. The document "Students, pay attention to the important concepts of data hazards, pipelines, subsequent instructions must use the results of previous instructions, and subsequent instructions may stop before the previous instructions are finished, etc., leading to data hazards," the question "Conditions for the occurrence of data hazards and solutions," and the preset prompt "Given a document and several questions, you need to determine whether the document contains answers to the questions. If it does, output 1; otherwise, output 0. Your answer can only be 0 or 1." are input into the discriminant model LLM. If the LLM analysis finds that the document includes content on the conditions for the occurrence of hazards and solutions, output "1," indicating that the teaching content matches the syllabus knowledge points. If the LLM analysis finds that the document describes the conditions for the occurrence of data hazards but does not mention any "solutions," output "0," indicating that the teaching content is missing.

[0064] This application applies RAG-based technology to the scenario of matching teaching syllabus knowledge points with subtitle files. Utilizing RAG technology and LLM (Low-Level Modeling) effectively improves the accuracy of subtitle and syllabus matching, ultimately providing a more reasonable evaluation result. A hybrid multi-path recall method is used to retrieve teaching syllabus knowledge point search results, and a re-ranking model is used to optimize the search results, effectively improving the recall rate of knowledge point retrieval and solving the problem of irrelevant retrieved content. When employing LLM technology, this application designs a preset prompt word "Prompt" to fine-tune the LLM, thereby significantly improving the model's ability to handle the matching and recognition of teaching syllabus and subtitles.

[0065] In summary, this application discloses a method for evaluating teaching content, which involves obtaining subtitle files and syllabus text corresponding to teaching audio data; performing a first segmentation process on the subtitle files to obtain multiple teaching segments, and vectorizing each teaching segment into a first vector; performing a second segmentation process on the syllabus text to obtain multiple key knowledge point segments, and vectorizing each key knowledge point segment into a second vector, which is then stored in a vector database; performing a multi-path search on each first vector in the vector database to obtain a set of candidate knowledge point segments, and determining the target knowledge point segment corresponding to the first vector from the set of candidate knowledge point segments; inputting the target knowledge point segment and the teaching segments into a discrimination model, and outputting the matching result between the teaching segments and the target knowledge point segments. This application's method segments subtitle fragments, making each segment semantically independent. For each segment, a corresponding target knowledge point fragment is identified, reducing interference from irrelevant information. Simultaneously, a multi-path retrieval approach improves the recall quality of relevant knowledge point fragments, thereby enhancing the accuracy of the identified target knowledge point fragments. When evaluating teaching segments, a discriminative model is further used to determine the matching results between the teaching segments and the target knowledge point fragments. This ensures that the final matching results can accurately assess the degree of alignment between the teacher's instruction and the teaching syllabus, improving evaluation accuracy.

[0066] refer to Figure 5 This application illustrates a teaching content assessment device 30 provided in an embodiment of the present application, the device comprising: Module 301 is used to acquire the subtitle file and teaching outline text corresponding to the teaching audio data; The first processing module 302 is used to perform a first segmentation process on the subtitle file to obtain multiple teaching segments, and to vectorize each teaching segment into a first vector. The second processing module 303 is used to perform a second segmentation process on the teaching syllabus text to obtain multiple key knowledge point segments, and to vectorize each key knowledge point segment into a second vector and store it in a vector database. The retrieval module 304 is used to perform multi-way retrieval in the vector database for each first vector to obtain a set of candidate knowledge point fragments, and to determine the target knowledge point fragment corresponding to the first vector from the set of candidate knowledge point fragments. The discrimination module 305 is used to input the target knowledge point segment and the teaching segment into the discrimination model and output the matching result between the teaching segment and the target knowledge point segment.

[0067] Optionally, the retrieval module includes: The first retrieval submodule is used to perform a first retrieval and a second retrieval using the first vector as the query vector, to obtain a first candidate knowledge point fragment and a second candidate knowledge point fragment; the first retrieval is based on keyword matching, and the second retrieval is based on semantic vector similarity. The second retrieval submodule is used to merge the first candidate knowledge point fragment and the second candidate knowledge point fragment into a candidate knowledge point fragment set.

[0068] Optionally, the retrieval module includes: The first sorting submodule is used to sort the candidate knowledge point fragments in the candidate knowledge point fragment set through the re-sorting model to obtain the sorting result; The second sorting submodule is used to determine the candidate knowledge point segment with the highest score in the sorting results as the target knowledge point segment.

[0069] Optionally, the first processing module includes: The first processing submodule is used to preprocess the subtitle file to obtain a preprocessed subtitle file; the preprocessing includes at least one of: filtering stop words, removing modal particles, and text standardization; The second processing submodule is used to segment the preprocessed subtitle file using a context-based paragraph segmentation model to obtain multiple teaching segments.

[0070] Optionally, the second processing module includes: The third processing submodule is used to divide the teaching outline text into segments of a preset fixed length and set up overlapping windows to generate the key knowledge point segments. The fourth processing submodule is used to input the text of the key knowledge point segments into the PGE vector embedding model, and use the text embedding vector output by the PGE vector embedding model as the second vector.

[0071] Optionally, the discrimination module includes: The prompting submodule is used to guide the output of the discrimination model through preset prompt words, and output the matching result between the teaching segment and the target knowledge point segment; The preset prompt words are used to instruct the discrimination model to perform an answer existence judgment and restrict the output of the discrimination model to a first value or a second value; the matching result is a binary judgment result, wherein the first value indicates that the teaching segment contains the answer of the target knowledge point segment, and the second value indicates that the teaching segment does not contain the answer of the target knowledge point segment.

[0072] Optionally, the device further includes: The statistics module is used to calculate the matching results between all the teaching segments and the corresponding target knowledge point segments; The determination module is used to determine the degree of matching between the teaching content and the teaching syllabus based on the ratio of the number of first values ​​in the matching results to the total number of matching results.

[0073] In summary, this application discloses a method for evaluating teaching content, which involves obtaining subtitle files and syllabus text corresponding to teaching audio data; performing a first segmentation process on the subtitle files to obtain multiple teaching segments, and vectorizing each teaching segment into a first vector; performing a second segmentation process on the syllabus text to obtain multiple key knowledge point segments, and vectorizing each key knowledge point segment into a second vector, which is then stored in a vector database; performing a multi-path search on each first vector in the vector database to obtain a set of candidate knowledge point segments, and determining the target knowledge point segment corresponding to the first vector from the set of candidate knowledge point segments; inputting the target knowledge point segment and the teaching segments into a discrimination model, and outputting the matching result between the teaching segments and the target knowledge point segments. This application's method segments subtitle fragments, making each segment semantically independent. For each segment, a corresponding target knowledge point fragment is identified, reducing interference from irrelevant information. Simultaneously, a multi-path retrieval approach improves the recall quality of relevant knowledge point fragments, thereby enhancing the accuracy of the identified target knowledge point fragments. When evaluating teaching segments, a discriminative model is further used to determine the matching results between the teaching segments and the target knowledge point fragments. This ensures that the final matching results can accurately assess the degree of alignment between the teacher's instruction and the teaching syllabus, improving evaluation accuracy.

[0074] Figure 6 This is a block diagram illustrating an electronic device 600 according to an exemplary embodiment. For example, the electronic device 600 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0075] Reference Figure 6The electronic device 600 may include one or more of the following components: a processing component 602, a memory 604, a power supply component 606, a multimedia component 608, an audio component 610, an input / output (I / O) interface 612, a sensor component 614, and a communication component 616.

[0076] Processing component 602 typically controls the overall operation of electronic device 600, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 602 may include one or more processors 620 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 602 may include one or more modules to facilitate interaction between processing component 602 and other components. For example, processing component 602 may include a multimedia module to facilitate interaction between multimedia component 608 and processing component 602.

[0077] Memory 604 is used to store various types of data to support the operation of electronic device 600. Examples of this data include instructions for any application or method operating on electronic device 600, contact data, phonebook data, messages, pictures, multimedia, etc. Memory 604 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0078] Power supply component 606 provides power to various components of electronic device 600. Power supply component 606 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 600.

[0079] Multimedia component 608 includes a screen that provides an output interface between the electronic device 600 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may not only sense the boundaries of touch or swipe actions but also detect the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 608 includes a front-facing camera and / or a rear-facing camera. When the electronic device 600 is in an operating mode, such as a shooting mode or a multimedia mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0080] Audio component 610 is used to output and / or input audio signals. For example, audio component 610 includes a microphone (MIC) used to receive external audio signals when electronic device 600 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 604 or transmitted via communication component 616. In some embodiments, audio component 610 also includes a speaker for outputting audio signals.

[0081] I / O interface 612 provides an interface between processing component 602 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0082] Sensor assembly 614 includes one or more sensors for providing state assessments of various aspects of electronic device 600. For example, sensor assembly 614 can detect the on / off state of electronic device 600, the relative positioning of components such as the display and keypad of electronic device 600, changes in position of electronic device 600 or a component of electronic device 600, the presence or absence of user contact with electronic device 600, orientation or acceleration / deceleration of electronic device 600, and temperature changes of electronic device 600. Sensor assembly 614 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 614 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 614 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0083] Communication component 616 facilitates wired or wireless communication between electronic device 600 and other devices. Electronic device 600 can access wireless networks based on communication standards, such as WiFi, carrier networks (such as 2G, 3G, 4G, or 5G), or combinations thereof. In one exemplary embodiment, communication component 616 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 616 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0084] In an exemplary embodiment, the electronic device 600 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to implement a teaching content evaluation method provided in the embodiments of this application.

[0085] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 604 including instructions, which can be executed by a processor 620 of an electronic device 600 to perform the above-described method. For example, the non-transitory storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0086] Figure 7 This is a block diagram illustrating an electronic device 700 according to an exemplary embodiment. For example, the electronic device 700 may be provided as a server. (Refer to...) Figure 7 The electronic device 700 includes a processing component 722, which further includes one or more processors, and memory resources represented by memory 732 for storing instructions, such as application programs, that can be executed by the processing component 722. The application programs stored in memory 732 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 722 is configured to execute instructions to perform a teaching content evaluation method provided in embodiments of this application.

[0087] Electronic device 700 may also include a power supply component 726 configured to perform power management of electronic device 700, a wired or wireless network interface 750 configured to connect electronic device 700 to a network, and an input / output (I / O) interface 758. Electronic device 700 may operate on an operating system stored in memory 732, such as Windows Server™, MacOSX™, Unix™, Linux™, FreeBSD™, or similar.

[0088] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the teaching content evaluation method.

[0089] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the application disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0090] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for evaluating teaching content, characterized in that, The method includes: Obtain the subtitle file and teaching syllabus text corresponding to the teaching audio data; The subtitle file is segmented into multiple teaching segments, and each teaching segment is vectorized into a first vector. The teaching syllabus text is processed into a second segmentation to obtain multiple key knowledge point segments. Each key knowledge point segment is then vectorized into a second vector and stored in a vector database. For each of the first vectors, a multi-way search is performed in the vector database to obtain a set of candidate knowledge point fragments, and the target knowledge point fragment corresponding to the first vector is determined from the set of candidate knowledge point fragments. The target knowledge point segment and the teaching segment are input into the discrimination model, and the matching result between the teaching segment and the target knowledge point segment is output.

2. The method according to claim 1, characterized in that, For each of the first vectors, a multi-path search is performed in the vector database to obtain a candidate knowledge point fragment set, including: Using the first vector as the query vector, a first retrieval and a second retrieval are performed to obtain a first candidate knowledge point fragment and a second candidate knowledge point fragment; the first retrieval is based on keyword matching, and the second retrieval is based on semantic vector similarity. The first candidate knowledge point fragment and the second candidate knowledge point fragment are merged into a candidate knowledge point fragment set.

3. The method according to claim 1, characterized in that, The step of determining the target knowledge point fragment corresponding to the first vector from the candidate knowledge point fragment set includes: The candidate knowledge point fragments in the candidate knowledge point fragment set are sorted by a re-ranking model to obtain the ranking result; the candidate knowledge point fragment with the highest score in the ranking result is determined as the target knowledge point fragment.

4. The method according to claim 1, characterized in that, The subtitle file is first segmented to obtain multiple teaching segments, including: The subtitle file is preprocessed to obtain a preprocessed subtitle file; the preprocessing includes at least one of: filtering stop words, removing modal particles, and text standardization; The preprocessed subtitle file is segmented using a context-based paragraph segmentation model to obtain multiple teaching segments.

5. The method according to claim 1, characterized in that, The second segmentation process of the teaching syllabus text yields multiple key knowledge point segments, and each key knowledge point segment is vectorized into a second vector, including: The teaching syllabus text is divided into segments of a preset fixed length, and overlapping windows are set to generate the key knowledge point segments. The text segments of the key knowledge points are input into the PGE vector embedding model, and the text embedding vector output by the PGE vector embedding model is used as the second vector.

6. The method according to claim 1, characterized in that, The step of inputting the target knowledge point segment and the teaching segment into the discrimination model and outputting the matching result between the teaching segment and the target knowledge point segment includes: The output of the discrimination model is guided by preset prompt words, and the matching result between the teaching segment and the target knowledge point segment is output. The preset prompt words are used to instruct the discrimination model to perform an answer existence judgment and restrict the output of the discrimination model to a first value or a second value; the matching result is a binary judgment result, wherein the first value indicates that the teaching segment contains the answer of the target knowledge point segment, and the second value indicates that the teaching segment does not contain the answer of the target knowledge point segment.

7. The method according to claim 6, characterized in that, The method further includes: Statistically analyze the matching results of all the teaching segments and the corresponding target knowledge point segments; The degree of matching between the teaching content and the teaching syllabus is determined based on the ratio of the number of first values ​​in the matching results to the total number of matching results.

8. A teaching content assessment device, characterized in that, The device includes: The acquisition module is used to acquire the subtitle file and teaching outline text corresponding to the teaching audio data; The first processing module is used to perform a first segmentation process on the subtitle file to obtain multiple teaching segments, and to vectorize each teaching segment into a first vector. The second processing module is used to perform a second segmentation process on the teaching syllabus text to obtain multiple key knowledge point segments, and then vectorize each key knowledge point segment into a second vector and store it in the vector database. The retrieval module is used to perform multi-way retrieval in the vector database for each first vector to obtain a set of candidate knowledge point fragments, and to determine the target knowledge point fragment corresponding to the first vector from the set of candidate knowledge point fragments; The discrimination module is used to input the target knowledge point segment and the teaching segment into the discrimination model and output the matching result between the teaching segment and the target knowledge point segment.

9. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is enabled to perform the method as described in any one of claims 1 to 7.

Images