A teaching interaction quality evaluation method and system based on a large language model

Abstract

The present application relates to the field of teaching interaction quality evaluation, and particularly relates to a teaching interaction quality evaluation method and system based on a large language model. The method comprises the following steps: audio transcription, converting classroom audio into original transcription text through speech activity detection, speaker classification, automatic speech recognition and punctuation recovery; transcription refinement, using a large language model combined with a pre-school education knowledge base to perform context-based text correction on the original transcription text to generate refined transcription text; quality evaluation step, based on a pre-school education quality evaluation scale, using a few-shot example guide and a thinking chain reasoning for each scoring point to determine whether there is a voice segment in the refined transcription text that meets the scoring point, performing binary scoring, and generating an interaction quality evaluation report containing each evaluation dimension compliance rate, teaching highlight analysis and phased teaching optimization suggestions. The present application significantly improves the evaluation efficiency. The present application is suitable for teaching interaction quality evaluation.

Description

Technical Field

[0001] This invention relates to the field of teaching interaction quality assessment, specifically to a teaching interaction quality assessment method and system based on a large language model. Background Technology

[0002] Currently, the assessment of preschool education quality mainly relies on professionally trained assessors using standardized scales for on-site observation and scoring. Commonly used assessment scales include SSTOW (Sustained Shared Thinking and Emotional Well-being), ECERS-E (Early Childhood Environment Rating Scale-Extension), and CLASS (Classroom Assessment Scoring System). These scales contain multiple assessment dimensions, each with several scoring items, and each scoring item contains multiple specific scoring points. Assessors need to determine whether each scoring point appeared during the observation process.

[0003] A complete manual assessment typically involves the following steps: First, assessors need to observe the classroom for 2 to 4 hours, recording the interactions between teachers and children throughout. Second, they must determine whether each scoring point meets the standards according to the scale. Finally, an assessment report containing the scoring results and improvement suggestions is written. The entire assessment process takes approximately 6 hours per classroom and requires professionally qualified assessors, resulting in high costs, low efficiency, and difficulty in achieving large-scale, high-frequency quality monitoring.

[0004] With the rapid development of automatic speech recognition and large language modeling technologies, automatic analysis and evaluation of audio-based classroom interactions have become possible. However, preschool education scenarios face the following unique challenges in terms of technological implementation:

[0005] First, there are challenges from the acoustic environment. Kindergarten classrooms are characterized by a complex and diverse range of noises, including the sounds of toys being operated, children moving around, and equipment colliding. Furthermore, there is overlap in the speech of multiple children and teachers, causing interference between their voice signals. Additionally, the pronunciation of children aged 3 to 6 is not yet fully developed, resulting in unclear articulation and non-standard pronunciation. Moreover, the distance between the recording equipment and the speaker is relatively large, leading to a decline in voice signal quality. These factors significantly impact the accuracy of speech recognition.

[0006] Second, there is the challenge of speech recognition errors. In the Chinese context, homophones are very common, meaning different Chinese characters have the same or similar pronunciations. Preschool education scenarios contain a large number of domain-specific terms, such as "morning talk," "district corner," "construction area," and "floating and sinking." These words are used less frequently in everyday language, and general-purpose automatic speech recognition systems easily misidentify them as common words with the same pronunciation but different meanings. Technical analysis shows that in speech transcription in preschool education scenarios, more than 50% of recognition errors stem from homophones or near-homophones.

[0007] Third, there is the challenge of assessment within long contexts. Educational quality assessment scales are complex, containing multiple dimensions and dozens of scoring points, each with detailed behavioral indicators. The full text of the scales typically reaches tens of thousands of words. Meanwhile, the transcribed text of a single 2-4 hour classroom interaction can also be tens of thousands of words long. Accurately locating evidence relevant to each scoring point within such a long context and making reliable and consistent assessments of achievement is a significant challenge for technological implementation.

[0008] While existing research has explored the use of automated methods to analyze classroom dialogues in K-12 education, these methods primarily target structured classroom settings for older students. Currently, there are no effective technological solutions for the automated assessment of interaction quality in preschool classrooms.

[0009] Therefore, there is an urgent need for an automated assessment method that can overcome the unique technological challenges of preschool education scenarios and achieve low-cost, high-efficiency, objective and consistent assessment of the quality of teacher-child interaction. Summary of the Invention

[0010] The purpose of this invention is to overcome the shortcomings of the prior art and provide a teaching interaction quality assessment method and system based on a large language model. It aims to solve the acoustic environment challenges, speech recognition errors, and long-context educational scale assessment problems in preschool education scenarios, and achieve high-accuracy and high-efficiency automated assessment of teacher-child interaction quality.

[0011] The present invention achieves the above objectives by adopting the following technical solution: Firstly, the present invention provides a method for evaluating the quality of teaching interaction based on a large language model, comprising:

[0012] S1. Audio transcription, generating the original transcribed text;

[0013] The method involves acquiring classroom audio data from a preschool education scenario. The classroom audio data is a mixed audio containing teacher and child voices. The method performs speech activity detection on the classroom audio data to obtain valid speech segments and filters out non-speech events. The valid speech segments are labeled as teacher voice, child voice, or other using a speaker classification model. Continuous speech segments from the same speaker are merged. An automatic speech recognition model is used to transcribe the merged teacher speech segments. The transcription results are then processed to restore punctuation and generate the original transcribed text.

[0014] S2. Transcription refinement and error correction based on large language models;

[0015] The original transcribed text and the knowledge base in the preschool education field are input into a large language model. Semantic analysis is performed through the large language model to identify recognition errors in the original transcribed text. The recognition errors are then corrected based on the preschool education context to generate a refined transcribed text. The knowledge base in the preschool education field includes an error lexicon and a list of common vocabulary in the field.

[0016] S3. Automatically score the refined transcribed text based on the preschool education quality assessment scale and generate an interactive quality assessment report;

[0017] Load the preschool education quality assessment scale, which contains multiple assessment dimensions, each assessment dimension contains several scoring items, and each scoring item contains multiple scoring points;

[0018] The refined transcribed text and the scoring criteria of the preschool education quality assessment scale are input into the large language model. For each scoring point, the large language model is guided by preset few-sample examples and thought chain reasoning prompts to search for whether there is a speech segment in the refined transcribed text that matches the scoring point. If it exists, the scoring point is scored as 1 point; if it does not exist, it is scored as 0 points. The achievement status of each scoring point is summarized. The large language model is used to summarize and analyze the achievement and non-achievement scoring points respectively, and generate an interactive quality assessment report that includes the achievement rate of each assessment dimension, teaching highlights analysis, and phased teaching optimization suggestions.

[0019] Furthermore, in step S1, the voice activity detection includes:

[0020] The classroom audio data is analyzed frame by frame using a speech activity detection model based on deep neural networks to identify audio regions containing human voice activity.

[0021] Set a minimum speech segment duration threshold, filter audio segments whose duration is less than the minimum speech segment duration threshold, and exclude them as noise or non-speech events;

[0022] Output the filtered valid audio segments and their corresponding timestamp information.

[0023] Furthermore, in step S1, the speaker classification model is a speech classification model fine-tuned with preschool education scenario data, used to distinguish between adult speech and children's speech based on the acoustic features of the speech.

[0024] The valid speech segments are input into the speaker classification model, which classifies each speech segment and outputs a classification label indicating whether the speech segment belongs to a teacher's speech, a child's speech, or another category.

[0025] Furthermore, the fine-tuning process includes:

[0026] Collect labeled audio data of preschool education scenarios, wherein each speech segment in the labeled audio data has been identified as teacher's speech, child's speech, or other;

[0027] Using a pre-trained speech classification model as the base model, the labeled audio data is used to fine-tune and train the base model.

[0028] By fine-tuning the speaker classification model, the acoustic feature differences between teacher and child speech in preschool education scenarios are learned, thereby improving classification accuracy.

[0029] Furthermore, in step S1, the continuous speech segments from the same speaker are merged, including: aligning the speaker classification result with the speech activity detection result, calculating the time overlap ratio between each classified speech segment and the detected speech segment, retaining speech segments with a time overlap ratio exceeding a preset overlap threshold, and excluding falsely detected speech segments with an overlap ratio that is too low; for adjacent speech segments from the same speaker and with a time interval less than a preset merging threshold, they are merged into a continuous speech segment, representing a complete speech by that speaker.

[0030] Furthermore, in step S1, when transcribing the teacher's speech segment using an automatic speech recognition model, an iterative hot word optimization strategy is employed, specifically including:

[0031] The teacher's speech segment was initially transcribed using a basic automatic speech recognition model to generate preliminary transcribed text.

[0032] Error analysis was performed on the preliminary transcribed text to identify recognition errors of specific vocabulary in the field of preschool education;

[0033] The correct words corresponding to the recognition errors are added to the hot word list, which is used to improve the recognition priority of specific words during the speech recognition process.

[0034] The teacher's speech segment was re-transcribed using an automatic speech recognition model that included an updated list of hot words;

[0035] Repeat the steps of error analysis, hot word addition, and re-transcription until no new vocabulary errors specific to the preschool education field are identified in the re-transcribed text, or the preset maximum number of iterations is reached.

[0036] Furthermore, in step S2, correcting recognition errors based on the preschool education context includes one or more of the following processes:

[0037] Homophone disambiguation: Identify word recognition errors caused by identical pronunciations, and replace incorrect words with words that have the same pronunciation but the correct meaning based on the context;

[0038] Near-homophone correction: Identify word recognition errors caused by similar pronunciations, and replace the incorrect words with words that are similar in pronunciation but correct in meaning according to the context;

[0039] Domain terminology standardization: Identify errors in the identification of preschool education-specific terms and correct them to standardized terminology.

[0040] Semantic coherence repair processing: Based on the context, identify semantic incoherence problems caused by missing words, extra words, or sentence breaks, and complete or correct them. The large language model is guided to perform the correction processing by constructing prompt words. The prompt words include: the original transcribed text, the background description of the preschool education scenario, common error examples in the error word library, and error correction instructions. The error correction instructions require the large language model to maintain the overall wording of the original text and only correct obvious recognition errors.

[0041] Furthermore, in step S2, the preschool education knowledge base is constructed in the following way:

[0042] Collect transcribed text samples from preschool education scenarios;

[0043] The transcribed text samples were reviewed by preschool education professionals and professional transcriptists to identify various recognition errors.

[0044] The correspondence between frequently occurring incorrect words and correct words is statistically analyzed to establish the aforementioned incorrect word database;

[0045] Analyze teaching activity records in preschool education scenarios, extract frequently used educational terms, activity names, and everyday expressions, and construct a common vocabulary list for the field.

[0046] Furthermore, in step S3, the criteria for judging each scoring point include:

[0047] Indicator attributes are used to identify whether the rating point is a positive indicator that is expected to occur or a negative indicator that is expected to be avoided.

[0048] The original description of the rating point, which describes the teacher behavior or interaction characteristics required for that rating point;

[0049] For each point awarded, list the circumstances under which the specific behavior or interaction characteristic described by the scoring point exists in the transcribed text.

[0050] For conditions that result in a score of 0, list situations in which the specific behavior or interaction feature described by the scoring point does not appear in the transcribed text.

[0051] Example of scoring 1 point: Provide a typical speech segment example that meets the criteria for scoring 1 point;

[0052] Example of a score of 0: Provide a typical speech segment example that meets the criteria for scoring 0.

[0053] Examples of few-sample usage for each rating point include:

[0054] Example of a score of 1: A transcribed text fragment that meets the criteria for scoring 1 point for this scoring point is marked as scoring 1 point.

[0055] Example fragments scoring 0 points: Transcribed text fragments that meet the criteria for scoring 0 points for this scoring point are marked as scoring 0 points.

[0056] The few-sample examples are pre-defined and kept fixed for each rating point to ensure rating consistency.

[0057] Furthermore, in step S3, the thought chain reasoning prompts guide the large language model to perform structured reasoning according to the following steps:

[0058] Evidence retrieval steps: Retrieve speech segments related to the current scoring point from the refined transcribed text;

[0059] Feature determination steps: Compare the retrieved speech segments with the scoring conditions of 1 point and 0 points for the scoring point to determine whether there are any speech segments that meet the scoring condition of 1 point.

[0060] Binary scoring steps: If a speech segment that meets the criteria for scoring 1 point exists, then that scoring point gets 1 point; otherwise, it gets 0 points.

[0061] Analysis and explanation steps: Output the judgment criteria and explain the specific reasons for determining that it meets the conditions for scoring 1 point or 0 points;

[0062] Improvement suggestion generation steps: Combine the indicator attributes of the scoring point with the binary scoring results to evaluate the achievement status. If it is determined that the standard is not met (i.e., a positive indicator gets 0 points, or a negative indicator gets 1 point), then based on the specific reasons mentioned above and the teacher's actual expression, generate targeted teaching improvement demonstration statements; if it is determined that the standard has been met, then mark it as no improvement is needed.

[0063] Furthermore, in step S3, for each scoring point, the scoring results output by the large language model include:

[0064] Binary score: The score for this scoring point is 0 or 1;

[0065] Supporting evidence: If the score is 1, the transcript must be cited verbatim; if the score is 0, the evidence is empty. Supporting evidence requires verbatim citation of the original dialogue, without paraphrasing or summarizing.

[0066] Analysis and Explanation: Explain whether the scoring point meets the eligibility criteria and the key judgment basis;

[0067] Achievement Status and Improvement Suggestions: The achievement status is determined by combining the indicator attributes and binary scores of the scoring point. Specifically, if the scoring point is a positive indicator with a score of 1, or a negative indicator with a score of 0, it is determined that the scoring point has been achieved and is marked as requiring no improvement. If the scoring point is a positive indicator with a score of 0, or a negative indicator with a score of 1, it is determined that the scoring point has not been achieved, and targeted teaching improvement suggestions are provided. The improvement suggestions should be specific and actionable, and improved demonstration language should be given based on the teacher's actual expression on the day.

[0068] The interaction quality assessment report includes:

[0069] The overall score section lists the number of score points and the pass rate for each evaluation dimension;

[0070] The teaching highlights section is grouped according to assessment dimensions. For each assessment dimension, the scoring indicators that have been judged to have met the standards are summarized and displayed. Each highlight indicator includes the indicator name, supporting evidence cited in the original text, and analysis explanation.

[0071] The teaching optimization suggestion section conducts a cross-dimensional comprehensive analysis based on all unmet scoring points, and generates improvement action suggestions in stages according to the time dimension. The time dimension includes immediate actions, short-term goals, and long-term goals. Each improvement action suggestion includes a description of the improvement behavior, the corresponding scale indicator number, specific improvement instructions, and a model statement based on the teacher's actual expression on that day.

[0072] Secondly, the present invention provides a teaching interaction quality assessment system based on a large language model, used to implement the teaching interaction quality assessment method based on a large language model described above. The quality assessment system includes:

[0073] The audio transcription module is used for audio transcription, generating the raw transcribed text.

[0074] Specifically, it is used to acquire classroom audio data in preschool education scenarios. The classroom audio data is a mixed audio containing teacher's voice and children's voice. The classroom audio data is subjected to voice activity detection to obtain effective voice segments and non-voice events are filtered out. The effective voice segments are labeled as teacher's voice, children's voice or other using a speaker classification model. Continuous voice segments from the same speaker are merged. The merged teacher's voice segments are transcribed using an automatic speech recognition model. The transcription results are then processed to restore punctuation and generate the original transcribed text.

[0075] The transcription refinement module is used to refine and correct transcriptions based on a large language model, generating refined transcribed text.

[0076] Specifically, it is used to input the original transcribed text and the knowledge base in the field of preschool education into a large language model, perform semantic analysis through the large language model to identify recognition errors in the original transcribed text, and correct the recognition errors based on the preschool education context to generate refined transcribed text. The knowledge base in the field of preschool education includes an error word library and a list of common words in the field.

[0077] The quality assessment module is used to automatically score the refined transcribed text based on the preschool education quality assessment scale and generate an interactive quality assessment report.

[0078] Specifically, it is used to load a preschool education quality assessment scale, which includes multiple assessment dimensions, each assessment dimension includes several scoring items, and each scoring item includes multiple scoring points.

[0079] The refined transcribed text and the scoring criteria of the preschool education quality assessment scale are input into the large language model. For each scoring point, the large language model is guided by preset few-sample examples and thought chain reasoning prompts to search for whether there is a speech segment in the refined transcribed text that matches the scoring point. If it exists, the scoring point is scored as 1 point; if it does not exist, it is scored as 0 points. The scores of each scoring point are summarized. The large language model is used to summarize and analyze the scoring points that meet the standards and those that do not, and generates an interactive quality assessment report that includes the pass rate of each assessment dimension, teaching highlights analysis, and phased teaching optimization suggestions.

[0080] The beneficial effects of this invention are as follows:

[0081] First, this invention specifically addresses the acoustic challenges of preschool education scenarios. By fine-tuning a speaker classification model using preschool education scenario data, it can accurately distinguish between teacher speech, children's speech, and other speech; through a segment alignment and merging strategy, it effectively handles the connection problem of speech segments; and through iterative hot word optimization, it significantly improves the recognition accuracy of specific terms in the field of preschool education.

[0082] Second, it effectively solves the problem of speech recognition errors. This invention constructs an error lexicon and a common vocabulary list for the field of preschool education, and combines this with the contextual understanding capabilities of a large language model to achieve end-to-end correction of various types of recognition errors, such as homophones, near-homophones, and domain-specific terms.

[0083] Third, it enables accurate evaluation of long contexts. This invention employs techniques such as score-based point processing, few-sample example guidance, and thought chain reasoning to enable large language models to accurately retrieve evidence and make reliable binary judgments in scale standards of tens of thousands of words and transcribed texts of tens of thousands of words.

[0084] Fourth, it significantly improves assessment efficiency. This invention reduces the assessment time for a single classroom from approximately 6 hours to less than 1 hour, achieving an efficiency improvement of more than 5 times, making large-scale, high-frequency, and continuous monitoring of preschool education quality possible.

[0085] Fifth, it provides interpretable evaluation results. The evaluation report output by this invention includes an evaluation overview, an analysis of teaching highlights, and phased teaching optimization suggestions. The teaching highlights are based on the original evidence of the achievement indicators, and the teaching optimization suggestions are generated in a hierarchical manner according to immediate actions, short-term goals, and long-term goals, which enhances the credibility and practicality of the evaluation results. Attached Figure Description

[0086] Figure 1 This is a flowchart of a teaching interaction quality assessment method based on a large language model provided by an embodiment of the present invention;

[0087] Figure 2 This is a flowchart of the audio transcription steps provided in the embodiments of the present invention;

[0088] Figure 3 This is a flowchart of the transcription refining steps provided in the embodiments of the present invention;

[0089] Figure 4 This is a flowchart of the quality assessment steps provided in the embodiments of the present invention;

[0090] Figure 5 This is a structural block diagram of a teaching interaction quality assessment system based on a large language model, provided by an embodiment of the present invention. Detailed Implementation

[0091] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0092] This invention provides a method for evaluating the quality of teaching interaction based on a large language model, such as... Figure 1 As shown, it includes audio transcription steps, transcription refining steps, and quality assessment steps.

[0093] The following sections will provide a detailed explanation of each step.

[0094] like Figure 2 As shown, the audio transcription step is used to convert audio data from preschool classrooms into raw transcribed text, providing a data foundation for subsequent transcription refinement and quality assessment.

[0095] First, classroom audio data from preschool education scenarios is acquired. This classroom audio data is a multi-speaker mixed audio recording containing both teacher and children's voices, sourced from actual recordings in kindergarten classrooms. The types of teaching activities covered by the classroom audio data include, but are not limited to, morning activities, group teaching, corner games, and outdoor activities. In one specific embodiment of this invention, the audio is collected using professional recording equipment with a sampling rate of 16kHz, mono format, and each audio segment is typically 2 to 4 hours long.

[0096] Then, speech activity detection is performed on the classroom audio data to obtain valid speech segments and filter out non-speech events.

[0097] Specifically, a speech activity detection model based on deep neural networks is used to analyze classroom audio data frame by frame to identify audio regions containing human voice activity. The speech activity detection model can be a VAD model based on the FSMN (Feedforward Sequential Memory Network) architecture, which can effectively identify regions of human voice activity in noisy environments.

[0098] Furthermore, a minimum speech segment duration threshold is set to filter audio segments with a duration less than this threshold. In one specific embodiment of the invention, the minimum speech segment duration threshold is set to 300 milliseconds; audio segments with a duration less than 300 milliseconds are identified as noise or non-speech events and excluded. The detection window is set to 30 milliseconds.

[0099] After speech activity detection and processing, the valid speech segments and their corresponding timestamp information are output for subsequent speaker classification processing.

[0100] Then, the valid speech segments are labeled as teacher speech, child speech, or other speech using a speaker classification model.

[0101] Specifically, the valid speech segments obtained from speech activity detection are input into the speaker classification model, which performs three-classification processing on each speech segment and outputs a classification label indicating whether the speech segment belongs to teacher speech, child speech, or other categories.

[0102] The speaker classification model is a speech classification model fine-tuned using data from preschool education scenarios. Because adults and children differ significantly in acoustic characteristics, including fundamental frequency range, formant distribution, and speech rate, and because classroom environments often contain speech segments that cannot be clearly categorized as either teacher or child (such as overlapping speech from multiple speakers, non-target voices, or other adult speech mixed in with the environment), fine-tuning allows the model to learn these feature differences, thereby accurately classifying speech segments as teacher speech, child speech, or other speech. After classification, teacher speech segments are extracted for subsequent transcription processing, while child speech segments and other speech segments are not included in the transcription process.

[0103] In one specific embodiment of the present invention, the fine-tuning process of the speaker classification model is as follows:

[0104] First, labeled audio data from preschool education scenarios is collected. Each speech segment in the labeled audio data has been manually identified as teacher's voice, child's voice, or other voices. The size of the labeled audio data can be determined according to actual needs.

[0105] Then, using a pre-trained speech classification model as the base model, the model was fine-tuned using labeled audio data. Through fine-tuning, the speaker classification model learned the acoustic feature differences between teacher and child speech in preschool education scenarios, improving classification accuracy. Finally, consecutive speech segments from the same speaker were merged.

[0106] Specifically, the speaker classification results are first aligned with the speech activity detection results. For each speech segment labeled by the speaker classification model, the temporal overlap ratio between it and the corresponding speech segment in the speech activity detection results is calculated.

[0107] Furthermore, a preset overlap threshold is set, retaining only speech segments with a temporal overlap ratio exceeding this threshold and excluding falsely detected speech segments with an excessively low overlap ratio. In one specific embodiment of the invention, the preset overlap threshold is set to 50%. This strategy can effectively reduce noise segments caused by classification errors or detection errors.

[0108] Furthermore, consecutive speech segments from the same speaker are merged. A preset merging threshold is set; adjacent speech segments from the same speaker with a time interval less than this threshold are merged into a single consecutive speech segment. In one specific embodiment of the invention, the preset merging threshold is set to 1 second. The merged speech segment represents a complete speech by the speaker, facilitating subsequent speech recognition and semantic analysis.

[0109] Then, an automatic speech recognition model was used to transcribe the merged teacher speech segments.

[0110] In the present invention, since classroom interaction is mainly led by teachers and the voice quality of teachers is relatively good, the focus is on transcribing the teacher voice segments. The automatic speech recognition model can adopt a speech recognition model based on the Transformer architecture.

[0111] To improve the recognition accuracy of specific vocabulary in the field of preschool education, the present invention adopts an iterative hot word optimization strategy, which specifically includes the following steps:

[0112] In the first step, use the basic automatic speech recognition model to conduct an initial transcription of the teacher voice segments to generate a preliminary transcription text.

[0113] In the second step, conduct an error analysis on the preliminary transcription text to identify the recognition errors of specific vocabulary in the field of preschool education. Common recognition errors include, but are not limited to: recognizing "morning talk" as "morning altar", "activity corner" as "area foot" or "curved corner", "floating and sinking" as "submission", "construction area" as "ditch construction area", etc.

[0114] In the third step, add the correct vocabulary corresponding to the recognition errors to the hot word list. The hot word list is a set of words that are given a higher recognition priority during the speech recognition process. Exemplarily, the hot word list may include common vocabulary in the field of preschool education such as "morning talk", "activity corner", "floating and sinking", "construction area", "reading area", "scientific inquiry", "independent play", "group teaching", etc.

[0115] In the fourth step, use the automatic speech recognition model containing the updated hot word list to conduct a re - transcription of the teacher voice segments to generate an improved transcription text.

[0116] In the fifth step, repeat the second step to the fourth step until no new recognition errors of specific vocabulary in the field of preschool education are identified in the re - transcribed text, or the preset upper limit of the iteration count is reached.

[0117] Finally, perform punctuation restoration processing on the transcription result to restore sentence boundaries and intonation cues.

[0118] Specifically, use a punctuation restoration model to process the unpunctuated transcription text. The punctuation restoration model predicts the positions and types of punctuation marks based on the semantic and syntactic structures of the text.

[0119] The punctuation restoration processing inserts punctuation marks such as full stops, question marks, commas, exclamation marks, etc. in the transcription text to restore sentence boundaries. At the same time, intonation cues can be inferred according to the types of punctuation marks, where a question mark marks an interrogative intonation and an exclamation mark marks an emphatic intonation.

[0120] Exemplarily, the effect of the punctuation restoration processing is as follows:

[0121] Input (transcribed text without punctuation): "Children, today we're going to learn about buoyancy and sinking. Look, what happens when you put this ball in the water?"

[0122] Output (text with punctuation restored): "Children, today we're going to learn about floating and sinking. Look, what happens when this ball is put in the water?"

[0123] After the above processing, the audio transcription step outputs the raw transcribed text. This raw transcribed text contains the teacher's speech and its timing information, but it may still contain various recognition errors, which need to be corrected through the transcription refinement step.

[0124] like Figure 3 As shown, the transcription refining step is used to correct and optimize the original transcribed text, eliminate various errors generated during speech recognition, and generate high-quality refined transcribed text.

[0125] First, a knowledge base for the field of preschool education is constructed. This knowledge base is a crucial supporting resource for the transcription and refinement process, and consists of two parts: an error dictionary and a list of common terms in the field.

[0126] The process of building the error dictionary is as follows:

[0127] The first step is to collect transcribed text samples from preschool education scenarios, including high-quality transcribed texts that have been manually proofread and the original ASR output text.

[0128] The second step involves preschool education professionals and professional transcriptists reviewing the transcribed text samples to identify various errors, including homophone errors, near-homophone errors, and domain terminology errors.

[0129] The third step is to statistically analyze the correspondence between frequently occurring incorrect words and correct words, and build an error word database. The error word database is stored in key-value pair format, where the key is the incorrect word and the value is the correct word.

[0130] For example, some entries in the error dictionary are shown in the table below:

[0131]

[0132] The process of constructing a domain-specific glossary is as follows:

[0133] Analyze teaching activity records, curriculum plans, evaluation reports and other documents in preschool education scenarios, extract frequently used educational terms, activity names and everyday expressions, and construct a vocabulary list of common terms in the field.

[0134] Exemplarily, the common vocabulary in the field includes but is not limited to: morning talk, corner activities, construction area, reading area, art area, scientific exploration area, role-playing area, independent play, group teaching, group activities, outdoor activities, transition links, daily life links, teacher-child interaction, peer interaction, etc.

[0135] Then, the original transcribed text and the knowledge base in the field of preschool education are input into the large language model. Through semantic analysis by the large language model, the recognition errors in the original transcribed text are identified and corrected based on the preschool education context.

[0136] In the present invention, correcting the recognition errors based on the preschool education context includes one or more of the following processes:

[0137] First, homophone disambiguation processing. Identify vocabulary recognition errors caused by the same pronunciation, and replace the incorrect vocabulary with a vocabulary that has the same pronunciation but correct semantics according to the context. For example, correct "Today we will learn the knowledge of submission" to "Today we will learn the knowledge of floating and sinking".

[0138] Second, near-homophone correction processing. Identify vocabulary recognition errors caused by similar pronunciations, and replace the incorrect vocabulary with a vocabulary that has a similar pronunciation but correct semantics according to the context. For example, correct "Now please ask the children to go to the corner of the area to play" to "Now please ask the children to go to the corner area to play".

[0139] Third, standardization processing of domain terms. Identify recognition errors of preschool education-specific terms and correct them to standard term expressions. For example, correct "Construction ditch area" to "Construction area".

[0140] Fourth, semantic coherence repair processing. Based on the context, identify semantic incoherence problems caused by missing words, extra words, or sentence-breaking errors, and complete or correct them. For example, correct "Kid, why do you like this toy" to "Kid, why do you like this toy".

[0141] In a specific embodiment of the present invention, the transcription refinement step guides the large language model to perform error correction tasks by constructing prompt words. The construction of the prompt words includes the following content:

[0142] Original transcribed text: The content of the transcribed text to be corrected.

[0143] Background description of the preschool education scenario: Explain that the text is from the recording of teacher-child interaction in the kindergarten classroom to help the model understand the context.

[0144] Common error examples in the error dictionary: Provide some对照示例 of incorrect and correct vocabulary to guide the model to identify similar errors.

[0145] Error correction instruction: Clearly require the large language model to maintain the overall wording of the original text and only correct obvious recognition errors, avoiding excessive modification of the original text expression.

[0146] Exemplarily, the prompt template is as follows:

[0147] You are a professional proofreader for preschool education transcription texts.

[0148] The following is a speech recognition transcription text of a kindergarten classroom, which may contain recognition errors.

[0149] The common error comparison table for preschool education is as follows:

[0150] "Morning altar" should be "Morning talk" (morning conversation activity)

[0151] "Submission" should be "Floating and sinking" (the phenomenon of objects floating and sinking in water)

[0152] "Area foot" should be "Area corner" (activity area corner)

[0153] "Building ditch area" should be "Construction area" (construction game area)

[0154] [More error comparisons...]

[0155] The common vocabulary in kindergartens is as follows:

[0156] Morning talk, area corner activities, construction area, reading area, scientific inquiry, independent play, group teaching...

[0157] The transcription text to be proofread:

[0158] [Original transcription text content]

[0159] Please carefully read the context, combine it with the preschool education scenario, identify and correct possible recognition errors. Note:

[0160] Only correct obvious recognition errors and keep the other wording of the original text unchanged

[0161] When correcting, consider the rationality of the preschool education context

[0162] Directly output the corrected text

[0163] Input the constructed prompt into the large language model. Based on the background information, error examples, and error correction instructions in the prompt, the large language model performs semantic analysis on the original transcription text, identifies potential recognition errors, and corrects them according to the context to output a refined transcription text.

[0164] In this invention, the large language model can adopt a model with strong language understanding ability, and this invention is not limited to a specific model product.

[0165] like Figure 4 As shown, the quality assessment step is used to automatically score the refined transcribed text based on the preschool education quality assessment scale and generate an interactive quality assessment report.

[0166] First, load the preschool education quality assessment scale. The preschool education quality assessment scale contains multiple assessment dimensions, each assessment dimension contains several scoring items, and each scoring item contains multiple specific scoring points.

[0167] This invention is applicable to various preschool education quality assessment scales, including but not limited to:

[0168] The SSTEW scale is used to assess the quality of interaction in the educational environment of children aged 2-6, with a particular focus on how teachers support children's cognitive development and emotional well-being.

[0169] The ECERS-E scale focuses on assessing extended curriculum content, including dimensions such as literacy, mathematics, science, and diversity.

[0170] The CLASS scale (Classroom Assessment Scoring System) focuses on assessing the quality of classroom interaction, including areas such as emotional support, classroom organization, and instructional support.

[0171] Each scoring point uses a binary criterion, which determines whether the teacher behavior or interaction characteristic described by the scoring point occurred during the observation process. If it occurred, the scoring point receives 1 point; if it did not occur, it receives 0 points.

[0172] In this invention, the criteria for judging each scoring point include the following:

[0173] Indicator attributes are used to identify whether the rating point is a positive indicator that is expected to occur or a negative indicator that is expected to be avoided.

[0174] The original description of the rating point, which describes the teacher behavior or interaction characteristics required for that rating point;

[0175] For each point awarded, list the circumstances under which the specific behavior or interaction characteristic described by the scoring point exists in the transcribed text.

[0176] For conditions that result in a score of 0, list situations in which the specific behavior or interaction feature described by the scoring point does not appear in the transcribed text.

[0177] Example of scoring 1 point: Provide a typical speech segment example that meets the criteria for scoring 1 point;

[0178] Example of a score of 0: Provide a typical audio segment that meets the criteria for a score of 0.

[0179] For example, taking a rating point under the "Encourage children to talk to others" dimension of the SSTEB scale as an example, the judgment criteria are as follows:

[0180] Scoring point 5.1 Indicator attribute: Positive indicator.

[0181] Original Description: Encourage children to talk to each other during activities or throughout the day. Teachers demonstrate and support this. 1-point criteria: At least one explicit encouragement for conversation between children, such as "You can tell Xiaoming..." or "Talk to the child next to you..."; Teachers demonstrate conversational styles between children, such as "You can say: 'Shall we play together?'"; Create opportunities for conversation between children, such as "Discuss it," or "Share with each other." 0-point criteria: No words encouraging conversation between children; Focus only on teacher-student dialogue, neglecting student-to-student interaction; Prevent conversation between children. 1-point examples: "Tell your friend your thoughts" (encouraging conversation between children); "You can ask her: 'What's your favorite color?'" (demonstrating conversation); "Xiaoming's idea is interesting, who wants to respond?" (creating opportunities for interaction). 0-point examples: "Draw your own picture, don't talk" (preventing conversation).

[0182] To further illustrate the automatic assessment capability of this invention for different attribute indicators, taking a negative rating point under the "Self-Regulation and Social Development" dimension of the SSTEB scale as an example, the judgment criteria are as follows:

[0183] Scoring point 1.1 Indicator attribute: Negative indicator.

[0184] Original Description: The teacher did not demonstrate acceptance of boundaries / rules / expectations, nor did they consistently use these boundaries / rules / expectations. 1-point criteria: At least one of the following exists in the transcribed text: the teacher's requirements for the same behavior are inconsistent in similar scenarios (e.g., initially allowing loud talking, then sternly demanding absolute silence); rule enforcement shows obvious arbitrariness or bias; in conflict or chaotic scenarios where rules are clearly needed, no boundaries or expectations are mentioned at all. 0-point criteria: The teacher explicitly mentions and consistently uses the same rules or requirements in the dialogue, and the requirements for the same behavior in similar scenarios remain consistent at different times (i.e., no inconsistencies or omissions of rules are detected). 1-point example: (Scenario 1) "Didn't I just say you could play freely?" (Shortly afterward) "Who told you to spill your toys all over the floor? You're not allowed to play anymore!" (Inconsistent requirements); (Scenario 2) When children are fighting and the scene is chaotic, the teacher only says "Oh, don't cry," without mentioning any rules about taking turns or sharing (no boundaries mentioned). Example of getting 0 points: "What did we agree on before we went to the slide? Yes, we have to line up and no pushing." (Then next to the slide) "Xiaoming, please follow our agreement and line up behind Xiaohong. We will take turns playing." (The rules are consistent and clear).

[0185] The scale data is stored in a structured format, which facilitates program reading and processing.

[0186] Then, the refined transcribed text and the scoring criteria of the preschool education quality assessment scale are input into the large language model, and a binary judgment is performed for each scoring point.

[0187] In one specific embodiment of the present invention, examples of using a small sample size for each rating point include:

[0188] Example fragment scoring 1 point: A transcribed text fragment that meets the criteria for scoring 1 point for this scoring point is marked as scoring 1 point. Example fragment scoring 0 points: A transcribed text fragment that meets the criteria for scoring 0 points for this scoring point is marked as scoring 0 points. The few sample examples are pre-defined and kept fixed for each scoring point to ensure consistency in scoring.

[0189] In one specific embodiment of the present invention, the thought chain reasoning prompt requires the large language model to perform structured reasoning according to the following steps:

[0190] The first step is evidence retrieval: Search the refined transcribed text for audio segments related to the current scoring point and identify dialogue content that may meet the criteria.

[0191] The second step is to determine whether the criteria are met. The retrieved audio segments are compared with the criteria for meeting and not meeting the criteria for that scoring point to determine whether there are any audio segments that meet the criteria.

[0192] The third step is binary scoring: if a speech segment meets the criteria, the scoring point is awarded 1 point; otherwise, it is awarded 0 points.

[0193] Step 4, Analysis and Explanation: Output the judgment criteria and explain the specific reasons for meeting or not meeting the criteria.

[0194] For example, the rating prompt word template is as follows:

[0195] Task Description

[0196] You are an early childhood education quality assessment specialist with SSTEB Advanced Certification. You will be analyzing a kindergarten.

[0197] Transcripts of teacher-student dialogues from a half-day teaching session. These texts were transcribed from audio recordings using ASR technology.

[0198] It mainly includes the teacher's remarks; children's responses may be more fragmented. Your task is based on the SSTOW quantity.

[0199] The scale is designed to score teacher performance. You must be as precise as a professional evaluator using a scale manual.

[0200] operate.

[0201] Specific scoring criteria

[0202] Assessment criteria: Encourage children to talk to each other.

[0203] Original description: Encourage children to talk to each other during activities or throughout the day. Teachers demonstrate and support this.

[0204] YES rating criteria:

[0205] Explicitly encourage dialogue between children at least once (e.g., “You can tell Xiaoming…”);

[0206] Teachers demonstrate how children should communicate with each other (e.g., "You can say: 'Shall we play together?'").

[0207] Create opportunities for children to talk to each other (such as "Let's discuss it" or "Share with each other").

[0208] NO. Criteria for a score of 0:

[0209] There was no wording encouraging children to talk to each other;

[0210] Focusing only on teacher-student dialogue while neglecting student-student interaction;

[0211] Stop children from talking to each other;

[0212] Typical example:

[0213] YES: "Share your thoughts with your friends" (encourage children to talk to each other);

[0214] YES: "You can ask her: 'What's your favorite color?'" (Demonstration of conversation);

[0215] NO: "Draw your own thing and don't talk" (stop talking);

[0216] Output format

[0217] Please output the scoring results in JSON format:

[0218] {

[0219] "5.1": {

[0220] "score": 0 or 1,

[0221] "evidence": ["dialogue excerpt 1", "dialogue excerpt 2"],

[0222] "analysis": "Analyze whether the indicator meets the scoring criteria and the key judgment basis",

[0223] "improvement": "If the score is 0, provide specific suggestions for improvement; if the score is 1, fill in 'No improvement needed'"

[0224] }

[0225] }

[0226] Special reminder:

[0227] Evidence must be a verbatim quote of the original dialogue; it cannot be paraphrased or summarized.

[0228] The analysis should be concise and clear, directly pointing out the specific conditions that are met or not met.

[0229] If a rating point has absolutely no relevant content in the conversation, it is given a score of 0 and the evidence array is empty.

[0230] The improvement should be specific and actionable, and it is best to provide an improved example based on the teacher's actual expression on the day.

[0231] Speculative language is prohibited;

[0232] It is strictly forbidden to directly insert fragments from typical examples into the evidence;

[0233] Below is the dialogue text to be evaluated.

[0234] Refined transcribed text content:

[0235] For each scoring point, the large language model outputs the following scoring results:

[0236] Binary score: The score for this scoring point is 0 or 1.

[0237] Supporting evidence: If the score is 1, the transcript must be cited verbatim and not modified; if the score is 0, the evidence is an empty array.

[0238] Analysis: Explain whether the scoring point meets the criteria and the key judgment basis. It should be concise and clear.

[0239] Improvement suggestions: If the score is 0, provide targeted teaching improvement suggestions, which should be specific and actionable, and preferably include improved sample statements based on the teacher's actual expression on the day; if the score is 1, mark it as no improvement is needed.

[0240] Repeat the above scoring process for all scoring points on the scale.

[0241] Finally, the scores for each rating point are summarized to generate an interaction quality assessment report.

[0242] The interaction quality assessment report includes the following:

[0243] The assessment overview section lists the number of achievement points, achievement rate, and overall indicator achievement rate for each assessment dimension, providing an overall overview of the quality of interaction.

[0244] The Teaching Highlights section summarizes and showcases the teacher's outstanding interactive behaviors during the day's teaching, based on all achieved scoring points. Each highlight indicator includes a title of the highlight behavior, the corresponding scale indicator number, supporting evidence cited in the original text, and an analysis explanation, helping teachers identify effective behaviors in their own teaching practice.

[0245] The growth suggestions section: Based on a comprehensive analysis of all unmet assessment points, targeted teaching optimization suggestions are generated in a tiered manner across three time dimensions: daily, weekly, and monthly. "Today's Action" refers to immediately actionable improvement strategies, focusing on small changes that can be tried immediately in today's teaching; "This Week's Goal" represents advanced strategies requiring some preparation and practice, suggesting teachers consciously practice them repeatedly throughout the week; and "This Month's Goal" is a more challenging long-term goal, requiring continuous practice and reflection to gradually internalize it into teaching habits. Each suggestion includes an improvement behavior title, the corresponding target indicator number, specific improvement instructions, and example wording tailored to the teaching scenario.

[0246] For example, the format of an interaction quality assessment report is as follows:

[0247] Kindergarten Teacher-Child Interaction Quality Assessment Report

[0248] I. Overview of the Assessment

[0249] Overall target achievement rate: 66% (62 / 94)

[0250] Trust and self-regulation: 18 out of 28 items met the standard (64.3%).

[0251] Language and Communication: 17 out of 22 items met the standard (77.3%);

[0252] Learning and Critical Thinking: 15 out of 29 items met the standard (51.7%).

[0253] Planning and evaluation: 12 / 15 items met the standards (80%).

[0254] II. Teaching Highlights

[0255] Six key behaviors: rule guidance, emotional support, and individual attention.

[0256] Highlight 1: Clear rule announcements and safety guidance (Indicators 1-5.1)

[0257] Supporting evidence: So, what should we pay attention to when playing on the Twin Towers Bridge? Safety first, no running, no pushing, right? Be friendly with other children. Okay, let me ask you, is this up or down? Here? This pavilion can only hold a maximum of two people at a time. So, if there's a child here, and you're walking here and find that the pavilion is already full (two people), what should you do? Stop and wait, right? And if the child in front of you is a little scared, what should you do? Well, cheer them on, right? Or stop and wait quietly.

[0258] Analysis and Explanation: Before the activity, the teacher systematically explained the rules and expectations to the children and guided them to think about "what should be done" through questioning. This not only prevented safety issues but also cultivated the children's awareness of rules and their ability to care for their peers, demonstrating high-quality rule guidance.

[0259] Highlight 2: Respectful and polite teacher-child interactions (Indicators 1-3.2)

[0260] Supporting evidence: "Thank you for helping me finish making it. Thank you so much, you're so fast! Okay, no problem. Could you please add it back for me? Hmm, thank you. Please go back to your seat. Okay, thank you to my little teachers."

[0261] Analysis and Explanation: The teacher frequently used polite phrases such as "thank you" and "okay?" to communicate with children in an attitude of equality and respect, setting a good example of social etiquette and creating a warm and respectful classroom atmosphere.

[0262] Highlight 3: Keen identification and support of individual needs (Indicators 1-5.3)

[0263] Supporting evidence: "You're not taking off your vest; you're dressed a bit too lightly. Missing your mom, are you? Then, is your mom alright? Would you like to sit next to me?"

[0264] Analysis shows that the teacher was able to pay attention to the special needs of individual children (being lightly dressed, emotional fluctuations) and make flexible adjustments in a personalized way (not removing the vest, allowing the child to sit next to the teacher), which reflects sensitivity and tolerance towards individual differences among children.

[0265] Highlight 4: Positive behavior guidance replaces prohibitive language (Indicators 1-5.4)

[0266] Supporting evidence: "When walking, please put the cap on your water bottle and look straight ahead. Hook your toes tight. Yes, the little friend reminded you, hook your toes tight. If you hook them tight, your toes will be fixed to the pillar by your whole body and you won't fall. You will slide down steadily."

[0267] Analysis and Explanation: By telling children "what they should do" rather than "what they shouldn't do" and explaining the reasons behind their behavior, teachers help children understand the meaning of rules and promote the development of their self-regulation abilities.

[0268] Highlight 5: Specific praise reinforces rule compliance (Indicators 1-7.1)

[0269] Supporting evidence: "Today I want to commend the block area and all the children who helped tidy up the block area. The block area built a very large structure today, so it used a lot of blocks, but we finished tidying it up very quickly. Why? Because of you, all the children in the block area, and the children who helped tidy up the block area. Because everyone cooperated and worked together, we quickly tidied up our block area."

[0270] Analysis and Explanation: The teacher specifically described the praiseworthy behaviors (cooperating in putting away toys, tidying up quickly) and clearly pointed out the connection between these behaviors and classroom routines, thus reinforcing the children's awareness of rules and cooperative behavior.

[0271] Highlight 6: Warm body language and emotional responses (Indicators 3-3.4)

[0272] Supporting evidence: "Sweetie, you're wearing your shoes on the wrong feet. Why don't you sit here and change them? Did you miss Mommy?...Then let me hug you again, okay?"

[0273] Analysis and Explanation: Teachers use affectionate terms and warm language to provide emotional support when children need it, demonstrating their attention to and responsiveness to children's emotional needs, which helps to establish a secure teacher-child attachment relationship.

[0274] III. Growth Suggestions

[0275] (a) Today's Actions (Immediate Results)

[0276] Based on your existing performance metrics, here are some action suggestions you can start trying today. These small changes can immediately improve the quality of your interactions!

[0277] Recommendation 1: Name children's feelings in emotional events (Target Indicators 1-5.2)

[0278] Current observations show that when children experience emotional fluctuations (such as missing their mother or experiencing conflict), teachers primarily offer comfort or solutions, rarely explicitly naming the child's emotion. Naming emotions can help children develop an emotional vocabulary and self-awareness. It is recommended that when a child exhibits obvious emotions, first describe your observed feelings in simple language before offering support.

[0279] Example script:

[0280] "You look very sad. Do you miss your mother?"

[0281] "You seem a little angry, is it because the building blocks were taken away?"

[0282] "I know you're very anxious right now, let's think of a solution together, okay?"

[0283] Recommendation 2: Create structured opportunities for peer interaction (Target Indicators 1-7.3)

[0284] While teachers encourage cooperation, there is a lack of planned and organized activities to systematically support children in practicing social skills such as sharing, taking turns, and listening. It is recommended that today's activity include a small task that requires children to take turns or cooperate (such as a ball game or a story-telling game), with clear demonstrations and guidance on social skills throughout the process.

[0285] Example script:

[0286] "Now it's Xiaoming's turn. Shall we wait for him to finish speaking?"

[0287] “You two can take turns. Xiao Hong can build one layer first, and then Xiao Ming can build one layer.”

[0288] "I see you are listening attentively to your friend, which will make your friend very happy."

[0289] (II) This Week's Goals (Mid-term Improvement)

[0290] This week, you can try the following advanced strategies. These require some preparation and practice, but can significantly improve the quality of interaction. It is recommended to consciously practice them 1-2 times a day.

[0291] Recommendation 1: Systematically observe and record children's social interactions (Target Indicators 3-5.1)

[0292] This week, you can begin consciously observing and recording how children express their feelings and needs during free play, as well as the social challenges they face. This will help you provide more targeted support for children's social development. It is recommended to select 2-3 children each day for 5 minutes of observation, recording how they express their needs, how they respond to their peers, etc.

[0293] Example script:

[0294] "Observation record: Xiaoming wanted a block from his classmate in the block area. He just grabbed it without saying anything."

[0295] "Observation record: After being rejected in the role-playing area, Xiao Hong walked away silently without expressing her feelings."

[0296] Based on observation, guide the conversation: "Xiaoming, you can try saying, 'I want to play this too, can I have it?'"

[0297] Recommendation 2: Guide the identification of others' feelings in conflict resolution (Target Indicator 3-7.3)

[0298] Currently, when mediating conflicts, teachers mainly guide children to "communicate well" or "cooperate," but rarely guide them to identify and respond to the other person's feelings. This week, when conflicts occur, teachers can first help children identify the other person's emotions before guiding communication. This can cultivate children's empathy and social problem-solving skills.

[0299] Example script:

[0300] "Look, he's very sad right now. Why do you think that is?"

[0301] "What was his expression like when you took his toy away?"

[0302] How can you make him feel better?

[0303] (III) This Month's Goals (Long-Term Improvement)

[0304] The following are more challenging long-term goals that require continuous practice and reflection. It is recommended to focus on one goal each week, gradually internalizing it as a teaching habit.

[0305] Recommendation 1: Establish a class conflict resolution procedure (Target Indicator 1-7.2)

[0306] The long-term goal is to establish a conflict resolution procedure that children can participate in (such as "state the problem - listen to the other party - think of solutions - choose a solution"), and to gradually teach children to resolve conflicts independently through visual support such as pictures and symbols. Starting this month, you can discuss with your children "what we can do when we have conflicts," create step-by-step diagrams, and guide them to use this procedure when conflicts occur.

[0307] Example script:

[0308] "Let's look at the steps to solve the problem: First, state your problem."

[0309] "Now we've moved on to the second step. Let's hear what the other side has to say."

[0310] "What methods have you come up with? Let's choose one and try it together."

[0311] [Other suggestions for improvement...]

[0312] Based on the above basic implementation method, the following optional technologies can be further added to improve the evaluation quality and reliability.

[0313] In one embodiment, during the scoring process of the quality assessment step, multiple rounds of scoring can be performed on key scoring points, and the mode of the scores can be taken to improve the stability of the scoring. Specifically, multiple rounds of scoring are performed on the same scoring point, with each round using slightly different prompts, and the final score is the mode of the scores from all rounds.

[0314] In one embodiment, a feedback mechanism is established to collect the results of manual review and continuously optimize the knowledge base and score examples with few samples in the preschool education field. When a human assessor discovers an automatic scoring error, the case is recorded, the cause of the error is analyzed, and the prompts are optimized, the few-sample examples are updated, and examples that meet and do not meet the criteria are supplemented accordingly.

[0315] like Figure 5 As shown, the present invention also provides a teaching interaction quality assessment system based on a large language model, including an audio transcription module, a transcription refinement module, a quality assessment module, and a knowledge base module.

[0316] The audio transcription module is used to acquire classroom audio data from preschool education scenarios and generate raw transcribed text. The audio transcription module includes the following units:

[0317] The speech activity detection unit is used to detect speech activity in classroom audio data. It identifies valid speech segments based on a deep neural network model and filters out non-speech events with a duration less than a preset threshold.

[0318] The speaker classification unit is used to label valid speech segments as teacher speech, child speech, or other speech using a speaker classification model. The speaker classification model is a speech classification model fine-tuned using data from preschool education scenarios.

[0319] The segment merging unit is used to align the speaker classification results with the speech activity detection results and to merge consecutive speech segments from the same speaker.

[0320] The speech recognition unit is used to transcribe the merged teacher speech segments using an automatic speech recognition model. The unit employs an iterative hot word optimization mechanism, maintaining a hot word list to improve the recognition accuracy of specific vocabulary in the preschool education field.

[0321] The punctuation recovery unit is used to perform punctuation recovery processing on the transcription results using the punctuation recovery model, restoring sentence boundaries and intonation markers to generate the original transcribed text.

[0322] The transcription refinement module performs context-based text correction and optimization on the original transcribed text to generate refined transcribed text. The transcription refinement module includes the following units:

[0323] The error identification unit is used to analyze the original transcribed text through a large language model and combine it with a knowledge base in the field of preschool education to identify homophone errors, near-homophone errors, domain terminology errors, and semantic incoherence problems.

[0324] The error correction unit is used to correct identified errors based on a knowledge base in the field of preschool education and contextual information, and output refined transcribed text.

[0325] The quality assessment module is used to automatically score refined transcribed texts based on a preschool education quality assessment scale and generate interactive quality assessment reports. The quality assessment module includes the following units:

[0326] The scale loading unit is used to load structured data from the preschool education quality assessment scale, including assessment dimensions, scoring items, scoring points, and their judgment criteria.

[0327] The evidence retrieval unit is used to search the refined transcribed text for each scoring point to determine whether there are any speech segments that meet the criteria for that scoring point.

[0328] The scoring unit is used to guide the large language model to perform binary scoring for each scoring point using few sample examples and thought chain reasoning prompts. If there is a speech segment that meets the criteria, the scoring point gets 1 point; otherwise, it gets 0 points.

[0329] The report generation unit is used to summarize the scores of each scoring point, calculate the pass rate of each dimension, and generate an interactive quality assessment report that includes the overall score, detailed scores, and teaching improvement suggestions.

[0330] The knowledge base module is used to store and manage a knowledge base for the field of preschool education. This preschool education knowledge base includes:

[0331] The error dictionary stores the mapping relationship between commonly identified incorrect words and correct words in preschool education scenarios, including homophone errors, near-homophone errors, and domain terminology errors.

[0332] A glossary of common terms in the field, storing frequently used educational terms, activity names, and everyday expressions in preschool education scenarios.

[0333] The knowledge base module is connected to the transcription and refining module, providing reference data for text correction. The knowledge base module supports dynamic updates, allowing for continuous supplementation and optimization of the lexicon content based on user feedback.

[0334] In this invention, both the transcription refinement module and the quality assessment module are implemented based on a large language model. Task prompt words are constructed through prompting engineering techniques to guide the large language model to complete text correction and quality scoring tasks.

[0335] Those skilled in the art should understand that:

[0336] First, the large language model in this invention can be the GPT series, Claude series, Qwen series, Llama series, or other models with similar language understanding and generation capabilities. This invention is not limited to specific model products.

[0337] Second, the automatic speech recognition model in this invention can be Paraformer, Whisper, WeNet, or other ASR systems with similar capabilities. This invention is not limited to specific speech recognition technologies.

[0338] Third, the preschool education quality assessment scale in this invention can be SSTEW, ECERS-E, CLASS or other preschool education quality assessment tools. The technical solution of this invention is universal and can be adapted to different assessment scales.

[0339] Fourth, the technical solution of this invention is applicable to multiple language environments. For different languages, only the training data of the speaker classification model, the content of the preschool education knowledge base, and the language version of the few sample examples need to be adjusted accordingly.

[0340] Fifth, the various steps and modules of this invention can be adjusted and optimized according to actual needs, such as adjusting processing parameters, adding or reducing sub-steps, replacing specific model implementations, etc. As long as they do not deviate from the core technical ideas of this invention, they should all be considered within the scope of protection of this invention.

[0341] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

Claims

1. A method for evaluating the quality of teaching interaction based on a large language model, characterized in that, include: S1. Audio transcription, generating the original transcribed text; The method involves acquiring classroom audio data from a preschool education scenario. The classroom audio data is a mixed audio containing teacher and child voices. The method performs speech activity detection on the classroom audio data to obtain valid speech segments and filters out non-speech events. The valid speech segments are labeled as teacher voice, child voice, or other voices using a speaker classification model. Continuous speech segments from the same speaker are merged. The merged teacher speech segments are transcribed using an automatic speech recognition model. The transcription results are then processed to restore punctuation and generate the original transcribed text. S2. Transcription refinement and error correction based on large language models; The original transcribed text and the knowledge base in the preschool education field are input into a large language model. Semantic analysis is performed through the large language model to identify recognition errors in the original transcribed text. The recognition errors are then corrected based on the preschool education context to generate a refined transcribed text. The knowledge base in the preschool education field includes an error lexicon and a list of common vocabulary in the field. S3. Automatically score the refined transcribed text based on the preschool education quality assessment scale and generate an interactive quality assessment report; Load the preschool education quality assessment scale, which contains multiple assessment dimensions, each assessment dimension contains several scoring items, and each scoring item contains multiple scoring points; The refined transcribed text and the scoring criteria of the preschool education quality assessment scale are input into the large language model. For each scoring point, the large language model is guided by preset few-sample examples and thought chain reasoning prompts to search for whether there is a speech segment in the refined transcribed text that matches the scoring point. If it exists, the scoring point is scored as 1 point; if it does not exist, it is scored as 0 points. The achievement status of each scoring point is summarized. The large language model is used to summarize and analyze the achievement and non-achievement scoring points respectively, and generate an interactive quality assessment report that includes the achievement rate of each assessment dimension, teaching highlights analysis, and teaching optimization suggestions generated in stages according to the time dimension.

2. The teaching interaction quality assessment method based on a large language model according to claim 1, characterized in that, In step S1, voice activity detection includes: The classroom audio data is analyzed frame by frame using a speech activity detection model based on deep neural networks to identify audio regions containing human voice activity. Set a minimum speech segment duration threshold, filter audio segments whose duration is less than the minimum speech segment duration threshold, and exclude them as noise or non-speech events; Output the filtered valid audio segments and their corresponding timestamp information.

3. The teaching interaction quality assessment method based on a large language model according to claim 1, characterized in that, In step S1, the speaker classification model is a speech classification model fine-tuned with preschool education scenario data, used to distinguish between adult speech and children's speech based on the acoustic features of speech. The effective speech segments are input into the speaker classification model, which classifies each speech segment and outputs a classification label indicating whether the speech segment belongs to a teacher's speech, a child's speech, or another speech. The data fine-tuning process includes: collecting labeled audio data of preschool education scenarios, wherein each speech segment in the labeled audio data has been identified as teacher speech, child speech, or other; using a pre-trained speech classification model as a base model, the labeled audio data is used to fine-tune and train the base model, so that the speaker classification model learns the acoustic feature differences between teacher speech and child speech in preschool education scenarios, thereby improving the classification accuracy. Merging consecutive speech segments from the same speaker includes: aligning the speaker classification result with the speech activity detection result; calculating the time overlap ratio between each classified speech segment and the detected speech segment; retaining speech segments with a time overlap ratio exceeding a preset overlap threshold; and excluding falsely detected speech segments with an overlap ratio that is too low. For adjacent speech segments from the same speaker with a time interval less than a preset merging threshold, they are merged into a single consecutive speech segment, representing a complete speech by that speaker.

4. The teaching interaction quality assessment method based on a large language model according to claim 1, characterized in that, In step S1, when transcribing the teacher's speech segment using an automatic speech recognition model, an iterative hot word optimization strategy is employed, specifically including: The teacher's speech segment was initially transcribed using a basic automatic speech recognition model to generate preliminary transcribed text. Error analysis was performed on the preliminary transcribed text to identify recognition errors of specific vocabulary in the field of preschool education; The correct words corresponding to the recognition errors are added to the hot word list, which is used to improve the recognition priority of specific words during the speech recognition process; The teacher's speech segment was re-transcribed using an automatic speech recognition model that included an updated list of hot words; Repeat the steps of error analysis, hot word addition, and re-transcription until no new vocabulary errors specific to the preschool education field are identified in the re-transcribed text, or the preset maximum number of iterations is reached.

5. The teaching interaction quality assessment method based on a large language model according to claim 1, characterized in that, In step S2, correcting recognition errors based on the preschool education context includes one or more of the following processes: Homophone disambiguation: Identify word recognition errors caused by identical pronunciations, and replace incorrect words with words that have the same pronunciation but the correct meaning based on the context; Near-homophone correction: Identify word recognition errors caused by similar pronunciations, and replace the incorrect words with words that are similar in pronunciation but correct in meaning according to the context; Domain terminology standardization: Identify errors in the identification of preschool education-specific terms and correct them to standardized terminology. Semantic coherence repair: Based on the context, identify semantic incoherence issues caused by missing words, extra words, or grammatical errors, and supplement or correct them. The large language model is guided to perform the correction process by constructing prompt words, which include: the original transcribed text, the background description of the preschool education scenario, common error examples in the error vocabulary, and error correction instructions. The error correction instructions require the large language model to maintain the overall wording of the original text and only correct obvious recognition errors.

6. The teaching interaction quality assessment method based on a large language model according to claim 1, characterized in that, In step S2, the preschool education knowledge base is constructed in the following way: Collect transcribed text samples from preschool education scenarios; The transcribed text samples were reviewed by preschool education professionals and professional transcriptists to identify various recognition errors. The correspondence between frequently occurring incorrect words and correct words is statistically analyzed to establish the aforementioned incorrect word database; Analyze teaching activity records in preschool education scenarios, extract frequently used educational terms, activity names, and everyday expressions, and construct a common vocabulary list for the field.

7. The teaching interaction quality assessment method based on a large language model according to claim 1, characterized in that, In step S3, the criteria for judging each scoring point include: Indicator attributes are used to identify whether the rating point is a positive indicator that is expected to occur or a negative indicator that is expected to be avoided. The original description of the rating point, which describes the teacher behavior or interaction characteristics required for that rating point; For each point awarded, list the circumstances under which the specific behavior or interaction characteristic described by the scoring point exists in the transcribed text. For conditions that result in a score of 0, list situations in which the specific behavior or interaction feature described by the scoring point does not appear in the transcribed text. Example of scoring 1 point: Provide a typical speech segment example that meets the criteria for scoring 1 point; Example of a score of 0: Provide a typical speech segment example that meets the criteria for scoring 0. Examples of few-sample usage for each rating point include: Example of a score of 1: A transcribed text fragment that meets the criteria for scoring 1 point for this scoring point is marked as scoring 1 point. Example fragments scoring 0 points: Transcribed text fragments that meet the criteria for scoring 0 points for this scoring point are marked as scoring 0 points. The few-sample examples are pre-defined and kept fixed for each rating point to ensure rating consistency.

8. The teaching interaction quality assessment method based on a large language model according to claim 1, characterized in that, In step S3, the thought chain reasoning prompt guides the large language model to perform structured reasoning according to the following steps: Evidence retrieval steps: Retrieve speech segments related to the current scoring point from the refined transcribed text; Feature determination step: Compare the retrieved speech segments with the scoring conditions of 1 point and 0 points for the scoring point to determine whether there are speech segments that meet the scoring condition of 1 point; Binary scoring step: If there are speech segments that meet the scoring condition of 1 point, the scoring point gets 1 point; otherwise, it gets 0 points. Analysis and explanation steps: Output the judgment criteria and explain the specific reasons for determining that it meets the conditions for scoring 1 point or 0 points; Improvement suggestion generation steps: Combine the indicator attributes of the scoring point with the binary scoring results to evaluate the achievement status. If it is determined that the standard is not met, that is, the positive indicator gets 0 points or the negative indicator gets 1 point, then based on the specific reasons mentioned above and the teacher's actual expression, generate targeted teaching improvement demonstration words; if it is determined that the standard has been met, then mark it as no improvement is needed.

9. The teaching interaction quality assessment method based on a large language model according to claim 1, characterized in that, In step S3, for each scoring point, the large language model outputs the following scoring results: Binary score: The score for this scoring point is 0 or 1; Supporting evidence: If the score is 1, the original text of the transcribed text is cited to support the judgment; if the score is 0, the evidence is empty. Supporting evidence requires verbatim citation of the original dialogue and cannot be rewritten or summarized. Analysis and Explanation: Explain whether the scoring point meets the eligibility criteria and the key judgment basis; Achievement Status and Improvement Suggestions: The achievement status is determined by combining the indicator attributes and binary scores of the scoring point. Specifically, if the scoring point is a positive indicator with a score of 1, or a negative indicator with a score of 0, it is determined that the scoring point has met the standard and is marked as requiring no improvement. If the scoring point is a positive indicator with a score of 0, or a negative indicator with a score of 1, it is determined that the scoring point has not met the standard, and targeted teaching improvement suggestions are provided. The requirements of the improvement suggestions are specific and actionable, and improved demonstration language is given based on the teacher's actual expression on the day. The interactive quality assessment report includes: an overall score section, listing the number and success rate of score points that meet the standards for each assessment dimension; a teaching highlights section, grouped by assessment dimension, summarizing and displaying the score indicators that have been judged to have met the standards for each assessment dimension, with each highlight indicator including the indicator name, supporting evidence cited in the original text, and analysis explanation; and a teaching optimization suggestion section, which conducts a cross-dimensional comprehensive analysis based on all score points that have not met the standards, and generates improvement action suggestions in stages according to the time dimension, including immediate actions, short-term goals, and long-term goals. Each improvement action suggestion includes a description of the improvement behavior, the corresponding scale indicator number, specific improvement instructions, and a model statement given based on the teacher's actual expression on that day.

10. A teaching interaction quality assessment system based on a large language model, used to implement the teaching interaction quality assessment method based on a large language model as described in any one of claims 1-9, characterized in that, The quality assessment system includes: The audio transcription module is used for audio transcription, generating the raw transcribed text. Specifically, it is used to acquire classroom audio data in preschool education scenarios. The classroom audio data is a mixed audio containing teacher's voice and children's voice. The classroom audio data is subjected to voice activity detection to obtain effective voice segments and non-voice events are filtered out. The effective voice segments are labeled as teacher's voice, children's voice or other voices by a speaker classification model. Continuous voice segments from the same speaker are merged. The merged teacher's voice segments are transcribed using an automatic speech recognition model. The transcription results are then processed to restore punctuation and generate the original transcribed text. The transcription refinement module is used to refine and correct transcriptions based on a large language model, generating refined transcribed text. The knowledge base module stores an error lexicon and a list of common domain terms for use by the transcription and refining module. Specifically, it is used to input the original transcribed text and the knowledge base in the field of preschool education into a large language model, perform semantic analysis through the large language model to identify recognition errors in the original transcribed text, and correct the recognition errors based on the preschool education context to generate refined transcribed text. The knowledge base in the field of preschool education includes an error word library and a list of common words in the field. The quality assessment module is used to automatically score the refined transcribed text based on the preschool education quality assessment scale and generate an interactive quality assessment report. Specifically, it is used to load a preschool education quality assessment scale, which includes multiple assessment dimensions, each assessment dimension includes several scoring items, and each scoring item includes multiple scoring points. The refined transcribed text and the scoring criteria of the preschool education quality assessment scale are input into the large language model. For each scoring point, the large language model is guided by preset few-sample examples and thought chain reasoning prompts to search for whether there is a speech segment in the refined transcribed text that matches the scoring point. If it exists, the scoring point is scored as 1 point; if it does not exist, it is scored as 0 points. The achievement status of each scoring point is summarized. The large language model is used to summarize and analyze the achievement and non-achievement scoring points respectively, and generate an interactive quality assessment report that includes the achievement rate of each assessment dimension, teaching highlights analysis, and phased teaching optimization suggestions.

Images