A teacher-child interaction analysis and growth report generation system for infant listening

Abstract

The application discloses a teacher-child interaction analysis and growth report generation system for infant listening, and relates to the technical field of voice processing and analysis, which comprises the following steps: collecting one-on-one listening records of a target infant, and generating an initial transcription text; dividing the initial transcription text into speech turns, and reconstructing structured teacher-child interaction data to obtain infant expression events and teacher listening behavior events; constructing a pre-infant expression window and an infant follow-up expression window, summarizing teacher support scores and corresponding characteristic value change amounts of teacher listening behavior events to obtain teacher support analysis results; extracting various types of expression characteristics corresponding to the infant expression events from the initial transcription text to obtain infant growth analysis results; and converting the teacher support analysis results and the infant growth analysis results into cumulative listening growth reports. The application keeps the analysis results associated with the original speech, provides verifiable data basis for the growth report, and improves the credibility and traceability of the report conclusion.

Description

Technical Field

[0001] This invention relates to the field of speech processing and analysis technology, and in particular to a system for analyzing teacher-child interaction and generating growth reports for young children's listening comprehension. Background Technology

[0002] With the development of digital preschool education and child development assessment technologies, kindergartens are gradually adopting audio, video, and electronic growth portfolios to record one-on-one listening sessions between teachers and children. These technologies have evolved from early manual observation and recording to utilizing automatic speech recognition to obtain dialogue texts. Combined with speaker recognition, text classification, keyword statistics, and natural language processing methods, these analyses examine the number of teacher and child speaking turns, the types of questions asked by teachers, and children's vocabulary, emotions, and social interaction expressions, providing data support for observing child development and evaluating teacher educational behaviors.

[0003] However, children's speech in one-on-one listening sessions is often characterized by short sentences, discontinuity, repetitive revisions, and cross-round supplementation. Conventional methods use single sentences, single rounds, or word frequencies as the unit of analysis, which can easily fragment the complete semantic expression. At the same time, it is difficult to determine whether changes in children's expression are related to specific teacher behaviors, resulting in a lack of objective basis for evaluating the effectiveness of support. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a teacher-child interaction analysis and growth report generation system for young children's listening to solve the problem of difficulty in determining whether changes in children's expressions are related to specific teacher behaviors.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: This invention provides a system for analyzing teacher-child interaction and generating growth reports for young children, which includes: The text transcription module collects one-on-one listening records of the target children, preprocesses the one-on-one listening records and performs automatic speech recognition to generate initial transcribed text; The expression reconstruction module performs speaker annotation, timestamp annotation, and speech turn segmentation on the initial transcribed text to generate structured teacher-child interaction data. It then reconstructs the structured teacher-child interaction data to obtain children's expression events and teachers' listening behavior events. The Support Results module constructs a pre-event child expression window and a subsequent child expression window for teacher listening behavior events. It calculates the change in feature values ​​in the pre-event and subsequent child expression windows, and performs a weighted summation of the feature value changes to obtain the teacher support score. It then summarizes the teacher support scores and corresponding feature value changes for teacher listening behavior events to obtain the teacher support analysis results. The growth indicator module extracts various expressive features corresponding to children's expressive events from the initial transcribed text, and summarizes them according to children's expressive events to obtain the results of children's growth analysis; The report generation module establishes a correlation between teacher listening behavior events and children's expression events, converts teacher support analysis results and children's growth analysis results into cumulative listening growth reports, and performs incremental updates based on one-on-one listening records.

[0007] As a preferred embodiment of the teacher-child interaction analysis and growth report generation system for listening to young children as described in this invention, the one-on-one listening record refers to the audio data of a conversation between a single teacher and a single child.

[0008] As a preferred embodiment of the teacher-child interaction analysis and growth report generation system for early childhood listening described in this invention, the specific steps for generating the initial transcribed text are as follows: Preprocess the one-on-one listening recordings to generate audio segments; The speech audio segments are framed and windowed, and the log-Mel spectrum features are calculated. The contextual acoustic features at each time position are extracted from the log-Mel spectrum features, and convolution operations are performed to output the character probability corresponding to each time position. Based on the character probability corresponding to each time position, line bundle search is performed on the characters corresponding to each time position to obtain the initial transcribed text.

[0009] As a preferred embodiment of the teacher-child interaction analysis and growth report generation system for early childhood listening described in this invention, the specific steps for generating structured teacher-child interaction data are as follows: Align the start and end timestamps of the initial transcribed text and the audio segment with the start and end timestamps of the audio segment according to the start and end positions of the audio segment in the one-on-one listening record; The speech audio segment is framed and its spectrum is transformed to obtain the voiceprint feature vector; By recording teachers' audio in a quiet environment and extracting the corresponding voiceprint feature vector, the pre-registered voiceprints of teachers are obtained, and a voiceprint matching threshold for teachers is set. Calculate the cosine similarity between the voiceprint feature vector of each speech audio segment and the teacher's pre-registered voiceprint. Speech audio segments with a cosine similarity not lower than the teacher's voiceprint matching threshold are labeled as teacher speech, and speech audio segments with a cosine similarity lower than the teacher's voiceprint matching threshold are labeled as children's speech. Using the speaker's switching position as the speech turn segmentation position, the initial transcribed text is segmented into speech turns to generate structured teacher-child interaction data with speech turns as the basic unit.

[0010] As a preferred embodiment of the teacher-child interaction analysis and growth report generation system for early childhood listening described in this invention, the specific steps for reconstructing structured teacher-child interaction data are as follows: The speakers labeled in the audio clips are used to divide the speech turns in the structured teacher-child interaction data into children's speech turns and teachers' speech turns; Based on the characteristics of children's natural dialogue, such as repetition, pauses, corrections, and additions, we set rules for reconstructing children's speech cycles and reconstructed the speech cycles to obtain children's expressive events. By examining the sentence structure, response space, and temporal relationship with adjacent children's speech rounds in the teacher's speech rounds, different types of teacher listening behavior events can be obtained.

[0011] As a preferred embodiment of the teacher-child interaction analysis and growth report generation system for children's listening in this invention, the aforementioned "pre-emptive children's expression window" refers to selecting a set of children's expression events before the occurrence of the teacher's listening behavior event, using the teacher's listening behavior event as the time anchor point, to represent the local baseline state of children's expression before the occurrence of the teacher's listening behavior event. The term "subsequent expression window of young children" refers to a set of children's expression events selected after the teacher's listening behavior event, using the teacher's listening behavior event as a time anchor point, to represent the immediate changes in children's expression after the teacher's listening behavior event.

[0012] As a preferred embodiment of the teacher-child interaction analysis and growth report generation system for children's listening comprehension described in this invention, the specific steps for calculating the changes in feature values ​​in the preceding child expression window and the subsequent child expression window are as follows: Extract various discourse features from the pre-early children's expression window and the subsequent children's expression window respectively; Calculate the average feature values ​​of various discourse features in the preceding children's expression window and the subsequent children's expression window, and then calculate the difference to obtain the change in feature values.

[0013] As a preferred embodiment of the teacher-child interaction analysis and growth report generation system for early childhood listening described in this invention, the specific steps for converting teacher support analysis results and child growth analysis results into a cumulative listening growth report are as follows. The results of the child development analysis of child expression events that are related to teacher support analysis results and teacher listening behavior events are summarized to obtain a single listening growth report. Based on the chronological order of the one-on-one listening records, summarize the individual listening growth reports to obtain the cumulative listening growth report; When a new listening record is generated, a corresponding single listening growth report is generated, and the cumulative listening growth report is incrementally updated.

[0014] As a preferred embodiment of the teacher-child interaction analysis and growth report generation system for early childhood listening described in this invention, the specific steps for classifying teacher speech turns are as follows: Determine whether the start time of the teacher's turn of speech occurs before the end time of the previous round of children's expression. If so, the teacher's listening behavior is an interruption; if not, it is a waiting behavior. If the teacher's turn of speech is a question and there is no limit to the space for answering, it is an open-ended question; if the space for answering is limited to a limited number of options, it is a closed-ended question. If the teacher's verbal turn is a declarative sentence, then the teacher's listening behavior event is a response.

[0015] As a preferred embodiment of the teacher-child interaction analysis and growth report generation system for children's listening oriented as described in this invention, the reconstruction rules for children's speech rounds include supplementary correction rules, teacher's brief encouragement interspersed rules, new topic splitting rules, topic similarity rules, and continuity rules, which are used to merge and split children's speech rounds.

[0016] The beneficial effects of this invention are as follows: By reconstructing children's discourse into expressive events with relatively complete semantics, it reduces semantic fragmentation caused by using single sentences or individual discourse turns as the unit of analysis, and reduces interference from fragmented expressions, repetitive content, and modified content on feature statistics, making statistical results such as expression length, event details, emotional expression, and proactive supplementation more accurate and stable. Simultaneously, by using teacher listening behavior events as time anchors, it compares changes in children's expressive characteristics before and after their occurrence, enabling a clear correlation between changes in children's expression and corresponding teacher listening behaviors. This reduces evaluation bias caused by separately statistically analyzing teacher behavior and children's expression, as well as erroneous attribution of changes in children's expression, thereby improving the objectivity, accuracy, and interpretability of teacher support effectiveness evaluation. Furthermore, the analysis results remain correlated with the original discourse, providing verifiable data for growth reports and improving the credibility and traceability of report conclusions. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A module diagram for a teacher-child interaction analysis and growth report generation system oriented towards listening to young children.

[0019] Figure 2 A flowchart for generating the initial transcribed text.

[0020] Figure 3 A flowchart for generating early childhood growth analysis and cumulative listening growth reports.

[0021] Figure 4 A flowchart for the generation and expression of structured teacher-child interaction data and event reconstruction. Detailed Implementation

[0022] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0023] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0024] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0025] Reference Figures 1-4 Tables 1 to 3, and an embodiment of the present invention, provide a system for analyzing teacher-child interaction and generating growth reports for young children's listening, comprising the following steps: The text transcription module collects one-on-one listening records of the target children, preprocesses the one-on-one listening records and performs automatic speech recognition to generate initial transcribed text.

[0026] Obtain one-on-one listening records of the target children in one-on-one listening scenarios, and establish the correlation between the one-on-one listening records and the target children's identity, teacher's identity, class information, collection time, and listening topic.

[0027] Among them, one-on-one listening records refer to at least one of the audio data in audio and video recordings of conversations between a single teacher and a single child.

[0028] Each one-on-one listening recording generates a unique record number, allowing the original audio, transcribed text, structured reconstruction, analysis, and report generation to all be traced back to the same one-on-one listening recording.

[0029] The audio data undergoes noise reduction, volume normalization, silence detection, Voice Activity Detection (VAD), and audio segmentation to obtain audio segments suitable for speech recognition. The specific steps are as follows: The audio data undergoes pulse code modulation (PCM) audio format conversion, decoding it into a standard audio sequence with a uniform sampling rate, quantization bit depth, and number of channels. Before acquiring the audio data, ambient audio of the teacher-child dialogue is collected (or audio from non-dialogue periods in the video is extracted), and the arithmetic mean of the amplitude spectrum of the corresponding frames of the ambient audio is calculated to obtain the noise amplitude spectrum. Bollinger subtraction is used to combine the denoised amplitude spectrum with the phase spectrum of the original audio frame, followed by an inverse fast Fourier transform. The time-domain audio is then recovered using an overlap-addition method to obtain the denoised audio sequence. The volume of each audio sample value in the denoised audio sequence is normalized to obtain the volume-normalized audio. The process involves: dividing the volume-normalized audio sequence into continuous detection frames; using a decibel full-scale amplitude threshold detection method to detect and label silent segments; dividing the volume-normalized audio sequence into continuous detection frames; inputting the WebRTC speech activity detection algorithm to determine whether the continuous detection frames are speech audio or non-speech audio, resulting in speech audio frames and non-speech audio frames; merging the continuous speech audio frames in chronological order to form initial speech activity intervals; extending the start and end times of each speech activity interval forward and backward, respectively; and segmenting the volume-normalized audio sequence according to the extended start and end times to obtain speech audio segments.

[0030] A trained Conformer encoder and Connectionist Temporal Classification (CTC) classifier decoder are used to perform Automatic Speech Recognition (ASR) on audio segments, generating initial transcribed text with timestamps. Specifically, each audio segment is framed and windowed, then arranged chronologically to obtain audio frames. The log-Melogram spectral features of each audio frame are calculated. These log-Melogram spectral features are input into the trained Conformer encoder for convolutional downsampling, multi-head self-attention, a feedforward network, and convolutional operations to extract contextual acoustic features (arranged by time step). A fully connected classification layer maps these contextual acoustic features to scores corresponding to each character and whitespace label in the character set. The Softmax function is used to normalize each score, obtaining the probability of the character at each time step. The connectionist temporal classifier decoder performs a beam search based on character probabilities, removing whitespace labels and merging consecutive repeating characters to obtain... Upon reaching the target character sequence, the target character sequence is connected with the character probabilities corresponding to each output time step for temporal classification and alignment. Dynamic planning is then performed on the probability of the character corresponding to each time step and the target character sequence to output the maximum probability alignment path of the target character sequence. The maximum probability alignment path records the target character or blank label corresponding to each output time step. For any target character in the target character sequence, the first output time step corresponding to the target character in the maximum probability alignment path is determined as the character start time position, and the last output time step consecutively corresponding to the target character is determined as the character end time position. The consecutive target characters are combined in character order to initially transcribe the text, and the start timestamp of the first target character is used as the start timestamp of the word or text segment, and the end timestamp of the last target character is used as the end timestamp of the word or text segment.

[0031] The expression reconstruction module performs speaker annotation, timestamp annotation, and speech turn segmentation on the initial transcribed text to generate structured teacher-child interaction data. It then reconstructs the structured teacher-child interaction data to obtain children's expression events and teachers' listening behavior events.

[0032] Align the timestamps of the initial transcribed text and audio segments, and perform speaker annotation, timestamp annotation, and turn segmentation to generate structured teacher-child interaction data. The specific steps are as follows: According to the start and end positions of the speech audio segments in the original audio, the speech audio segments and their corresponding initial transcribed texts are arranged in chronological order, and the association between each initial transcribed text and its corresponding speech audio segment is established. The start timestamp and end timestamp of the initial transcribed text are aligned according to the start and end times of the speech audio segments associated with the initial transcribed text.

[0033] The speech audio segment is framed and spectral transformed to extract acoustic features that characterize the vocal tract and articulation characteristics. The acoustic features are then input into a speaker embedding network (x-vector speaker embedding network or ECAPA-TDNN speaker embedding network) for feature encoding, statistical pooling, and vector normalization to obtain the voiceprint feature vector.

[0034] Calculate the cosine similarity between the voiceprint feature vector of each audio segment and the teacher's pre-registered voiceprint. Audio segments with a cosine similarity greater than the teacher's voiceprint matching threshold are labeled as teacher speech segments, while audio segments with a cosine similarity lower than the teacher's voiceprint matching threshold are labeled as children's speech segments.

[0035] Among them, the pre-registered voiceprint of the teacher refers to the audio of the teacher reading a specified text or speaking naturally in a quiet environment, and the voiceprint feature vector extracted; the voiceprint matching condition is that the voiceprint similarity is greater than the teacher's voiceprint matching threshold; for example, 0.75 is taken because the identification of teachers requires higher certainty.

[0036] The initial transcribed texts with completed speaker annotations are sorted according to the start timestamp of the initial transcribed text. The initial transcribed texts are then segmented into discourse turns based on the speaker switching position. Specifically, if the speaker type of the current text segment is different from that of the previous text segment, the time is discontinuous, or another speaker is inserted, the current text segment is divided into a new discourse turn. If the speaker type of the current text segment is the same as that of the previous text segment, the time is continuous, and no other speaker is inserted, the current text segment and the previous text segment are merged into the same discourse turn.

[0037] Based on the speaker's annotations, speech turns are divided into children's speech turns and teachers' speech turns.

[0038] Configure the speech round numbers sequentially according to the start timestamp of each speech round, and associate all speech rounds with the one-on-one listening record numbers; for any speech round, write the speech round number that is adjacent in time and located in front into the previous round number, and write the speech round number that is adjacent in time and located in the next round number into the next round number, thus establishing the round connection relationship in the continuous dialogue between teachers and children.

[0039] Structured teacher-child interaction data uses discourse turns as the basic unit. Each discourse turn includes at least: discourse turn number, one-on-one listening record number, speaker type, start time, end time, and corresponding initial transcription text, as shown in Table 1: Table 1: Example Table of Structured Teacher-Child Interaction Data Fields

[0040] Because young children's natural dialogues often involve short sentences, pauses, repetitions, corrections, and supplementary expressions, we do not use single sentences as the unit of growth analysis. Instead, we reconstruct one or more children's speech cycles in the structured teacher-child interaction data into children's expressive events. Children's expressive events refer to a set of children's speech cycles that form a relatively complete meaning around the same object, event, emotion, self-state, or social relationship.

[0041] Reconstruct the chronologically adjacent turns of children's speech according to the following rules: The children's and teachers' speech rounds were sorted according to timestamps. The children's speech rounds were segmented and tagged with parts of speech. Character words, object words, location words, action words, and emotion words were extracted from each children's speech round to form a set of thematic keywords for the corresponding children's speech rounds. Thematic keywords with the same or similar meanings were converted into unified keywords to form a synonym mapping table. For example, "mother" and "mommy" were converted into the same character keyword, "happy" and "joyful" were converted into the same emotion keyword, and "seaside" and "beach" were converted into the same location keyword.

[0042] Calculate the overlap ratio of thematic keywords in two adjacent children's speech rounds. The thematic keyword overlap ratio is the ratio of the number of identical unified keywords in the two thematic keyword sets to the total number of different unified keywords in the two thematic keyword sets.

[0043] Continuity rule: When there is no teacher speech cycle between two consecutive children's speech cycles, the time interval between cycles is not greater than the threshold for children's continuous expression time, and no new topic splitting condition is detected in the current children's speech cycle, the current children's speech cycle will be merged with the previous children's speech cycle.

[0044] Theme similarity rule: When the time interval between two children's speech rounds is not greater than the continuous expression time threshold, and the overlap of theme keywords is not less than the theme similarity threshold, the two children's speech rounds are merged.

[0045] Supplementary Correction Rules: Establish a correction marker word list and a supplementary marker word list. The correction marker word list includes words such as "no," "I misspoke," "incorrect," and "this is it," while the supplementary marker word list includes words such as "also," "then," "later," and "following up." For two adjacent children's speech rounds, match the latter speech round with the correction marker word list and the supplementary marker word list, and assign a correction marker or supplementary marker to the latter speech round. When the latter speech round contains a correction marker, extract the thematic keywords before and after the correction marker. If the thematic keyword after the correction marker belongs to the same keyword category as the thematic keyword in the former speech round, then merge the two speech rounds. When the latter speech round contains a supplementary marker, if the time interval between the two speech rounds is not greater than the continuous expression time threshold, and the two speech rounds have the same thematic keyword, or if the personal pronouns or demonstrative pronouns in the latter speech round point to the person or object in the former speech round, then merge the two speech rounds.

[0046] For example, if the previous child's utterance was "We're going to the park," and the next was "No, we're going to the beach," the "No, we're going to the beach..." correction phrase was detected. Since both "park" and "beach" were identified as location words by the location dictionary, the two utterances were merged and recorded as: Location words: park → beach. The merging is based on "correction marker + the corrected word and the correcting word belonging to the same keyword category," rather than simply assuming they are "related."

[0047] New theme segmentation rules: Children's speech cycles are divided into one or more segments based on punctuation, pauses, or topic transition words. Character words, object words, location words, action words, and emotion words are extracted from each segment. Character words acting as the subject of an action, action words acting as the main action, object words acting as the target of an action, location words indicating the location of an action, and directly stated emotion words are identified as the main theme keywords. Adjacent children's speech cycles or adjacent segments within the same cycle are compared. When a subsequent children's speech cycle or segment contains a main theme keyword not previously mentioned, and this main theme keyword cannot be mapped to a theme keyword in the preceding text through synonyms or pronouns, and the overlap ratio of theme keywords is below the theme similarity threshold, the starting position of the corresponding children's speech cycle or the starting position of the corresponding speech segment is identified as the segmentation position.

[0048] Rules for interspersing brief encouragement from teachers: A list of brief encouragement words for teachers is established, including "um," "oh," "and then," "continue," "anything else," and "what happened next." Text matching is performed on teacher speech turns located between two children's speech turns. When the text of a teacher's speech turn matches a word in the list of brief encouragement words, or when a teacher's speech turn consists only of words and interjections from the list and no new main theme words are extracted from the teacher's speech turn, the teacher's speech turn is marked as a brief encouragement statement. The time interval and the percentage of overlap of theme keywords between the two children's speech turns before and after the brief encouragement statement are calculated. When the time interval is not greater than the encouragement interspersing time threshold, and the percentage of overlap of theme keywords is not lower than the theme similarity threshold, or when the latter children's speech turn meets the supplementary correction rules, the two children's speech turns are merged.

[0049] Among them, the continuous expression time threshold for young children refers to the maximum allowable interval for judging whether two children's speech rounds are continuous in time when there is no teacher's speech round between two adjacent children's speech rounds. For example, it is set to 3 seconds, which is in line with the short pauses that children may make in natural expression due to organizing language, recalling events, or searching for words. The theme similarity threshold refers to the minimum proportion value for judging whether the theme keyword sets of two children's speech rounds have a sufficient degree of overlap. For example, it is set to 0.50, which means that the number of common theme keywords after normalization of the two children's speech rounds is not less than half of the total number of different theme keywords in the two speech rounds. This allows children to add new details while reducing the error merging caused by a single accidental repetition of a word. The encouragement interlude time threshold refers to the maximum allowable response time from the end of the teacher's short encouragement to the beginning of the next children's speech round when only a short encouragement from the teacher is interlude between two children's speech rounds. For example, it is set to 5 seconds. On the basis of the 3-second continuous expression time for children, children also need to understand the teacher's words, reorganize their language, and continue to respond. Therefore, a longer response time should be allowed.

[0050] Furthermore, when multiple rules for reconstructing children's discourse turns are simultaneously met, they are executed according to the following priority: supplementary correction rules > rules for interspersing brief encouragement from teachers > rules for splitting new themes > rules for thematic similarity > rules for continuity. The reasons are as follows: supplementary correction words are direct linguistic evidence and have the highest reliability; the appearance of new themes is a clear basis for splitting and should take precedence over simple temporal continuity; thematic similarity is better than time intervals in reflecting whether the content revolves around the same topic, and incorrect merging will distort the overall expression length, event details, and developmental indicators. Therefore, weak continuity evidence cannot cover strong splitting evidence.

[0051] As shown in Table 2: Table 2: Examples of Reconstructing Children's Turns of Speech

[0052] By reconstructing the order of children's speech, we can avoid the errors caused by directly calculating children's language indicators using fragmented short sentences, and make indicators such as expressive features have a more stable calculation basis.

[0053] By examining the sentence structure, response space, and temporal relationship with adjacent children's speech turns, teacher speech turns are categorized to determine the types of teacher listening behavior events. The specific steps are as follows. The test examines the sentence structure (questions, guidance, and responses, such as "What is it?", "Why?", and "Very good.") and the space for responses (closed and open responses) in the teacher's discourse.

[0054] The test examines whether the start time of the teacher's turn of speech occurs before the end time of the previous round of children's expressive events.

[0055] Based on sentence structure, response space, and time relationship, teacher discourse turns are divided into different types of teacher listening behavior events.

[0056] Furthermore, the timing behavior is determined independently of the content of the teacher's speech. The teacher's listening behavior event is divided into waiting behavior and interruption behavior. Therefore, it is first determined whether the start time of the teacher's speech round occurs before the end time of the previous round of children's expression event. If so, the teacher's listening behavior event is an interruption behavior; if not, it is a waiting behavior.

[0057] Based on the sentence structure and response space in the teacher's turn of speech, teacher listening behavior events are divided into open-ended questions, closed-ended questions, and responses. Specifically, if the teacher's turn of speech is an interrogative sentence and the response space is unrestricted, such as "why," "how," and "what happened next," requiring children to explain reasons, express feelings, or narrate events, it is an open-ended question. If the response space is limited to yes or no, quantity, color, name, or other limited options, it is a closed-ended question. Furthermore, if the teacher's turn of speech is a declarative sentence, then the teacher listening behavior event is a response, such as "Great job!"

[0058] Teacher listening behavior events include content-based behavior types and time-based behavior types; content-based behavior types include open-ended questions, closed-ended questions, and responses, while time-based behavior types include waiting behavior and interruption behavior. Each turn of a teacher's speech has one content-based behavior type and one time-based behavior type.

[0059] As shown in Table 3: Table 3: Rules for Identifying Types of Teacher Listening Behavior Events

[0060] The Support Results module constructs a pre-event child expression window and a subsequent child expression window for teacher listening behavior events. It calculates the change in feature values ​​in the pre-event and subsequent child expression windows, and performs a weighted summation of the feature value changes to obtain the teacher support score. Finally, it summarizes the teacher support scores and corresponding feature value changes for teacher listening behavior events to obtain the teacher support analysis results.

[0061] For each teacher listening behavior event, a pre-event child expression window and a post-event child expression window are established. The pre-event child expression window refers to a set of child expression events selected before the teacher listening behavior event occurs, using a teacher listening behavior event as the time anchor point, to record the local baseline state of the child's expression before the teacher listening behavior event occurs. The post-event child expression window refers to a set of child expression events selected after the teacher listening behavior event occurs, using a teacher listening behavior event as the time anchor point, to record the immediate changes in the child's expression after the teacher listening behavior occurs.

[0062] Each child's expressive event is assigned only to the child's subsequent expressive window of the most recent preceding teacher listening behavior event in its chronological order; it is not added repeatedly to the child's subsequent expressive windows of other teacher listening behavior events. The same child's expressive event can serve as the child's subsequent expressive data for the preceding teacher listening behavior event and as the baseline data for the following teacher listening behavior event. If there are no child expressive events matching the selection range before or after the current teacher listening behavior event, the corresponding window will be marked as empty, and the teacher listening behavior event will be marked as having insufficient expressive data.

[0063] Various discourse features, including expression length, event details, and active supplementation, are extracted from the preceding and subsequent children's expression windows. The average feature value of each discourse feature in the preceding and subsequent children's expression windows is calculated (the quotient of each feature value and the number of children's expression events contained in the corresponding window), and the difference is calculated (subtracting the preceding children's expression window from the subsequent children's expression window) to obtain the change in feature value (change in expression length, change in event details, and change in active supplementation). The change in feature value of each discourse feature is discretized to obtain the feature standard value of each discourse feature (if the change in feature value is positive, it means that the corresponding feature has increased, and is recorded as 1; if the change in feature value is 0, it means that the corresponding feature has not changed, and is recorded as 0; if the change in feature value is negative, it means that the corresponding feature has decreased, and is recorded as -1).

[0064] The length of expression is characterized by the number of effective words (words that, after Chinese word segmentation, can express characters, objects, actions, states, attributes, time, place, cause and effect, negation, emotions, social relationships, or grammatical relationships). Event details are characterized by the number of characters, time, place, actions, objects, causes, and results expressed by the child in the event; each different element is counted as one event detail. Active supplementation is characterized by the number of times a child provides new information related to the current topic but not directly requested by the teacher after fulfilling the teacher's question. For example, if the teacher asks, "What flavor did you eat?" and the child replies, "Strawberry flavor, I like strawberries the most," then "Strawberry flavor" is a direct answer, while "I like strawberries the most" is new preference information, and is counted as one active supplementation.

[0065] By statistically analyzing the increase and decrease of feature values ​​of various discourse features after different types of teacher listening behavior events, weights of feature standard quantities are set for different types of teacher listening behavior events (for example, under open-ended questions, the change in expression length has the greatest impact and is set to 0.5, followed by the change in event details and set to 0.3, and the change in proactive supplementation is set to 0.2). The feature standard quantities are then weighted and summed to obtain the teacher support score.

[0066] Different types of teacher listening behaviors have different support purposes. Therefore, key features are set for each type of teacher listening behavior event. For example, the key features of open-ended questions are expression length and event details. When the change in the feature value of at least one key feature corresponding to a teacher listening behavior event is positive and the teacher support score is greater than 0, the teacher support level is effective support. When the change in the feature value of the key feature corresponding to a teacher listening behavior event is not positive, but the teacher support score is greater than 0, the teacher support level is general support. When the teacher support score is not greater than 0, regardless of whether the change in the feature value of the key feature is positive, the teacher support level is ineffective support.

[0067] The teacher support analysis results are formed by summarizing the types of teacher listening behavior events, changes in various characteristic values, teacher support scores, and teacher support levels.

[0068] The growth indicator module extracts various expressive features corresponding to children's expressive events from the initial transcribed text, and summarizes them according to the children's expressive events to obtain the results of children's growth analysis.

[0069] Among them, the expressive features include language expression features, emotional expression features, self-expression features, and social interaction expression features. Language expression features include vocabulary richness, etc. Emotional expression features include the number of emotional words, the number of emotional categories, the number of explanations of emotional reasons, and the number of expressions of emotional transitions, etc. Self-expression features include the number of first-person expressions and the number of self-statements, etc. Social interaction expression features include the number of expressions to social objects and the number of interactive behaviors, etc.

[0070] The text content corresponding to the events expressed by the children was read one by one from the initial transcribed text to obtain the analysis text.

[0071] The analyzed text is segmented and effective words are selected. The effective words are deduplicated, and each different effective word is recorded as 1 and summed to obtain the number of different effective words. The number of effective words that are not deduplicated is taken as the total number of effective words, and the ratio of the number of different effective words to the total number of effective words is calculated to obtain the vocabulary richness.

[0072] To determine the frequency of emotion words, we retrieved emotion-related words from the analyzed text, including happy, scared, and sad. In the children's expressive events containing emotion words, we searched for causal connection structures such as "because" and "therefore" to obtain the frequency of emotional explanations. We detected transition words or words indicating changes in state, such as "but," "however," "later," "as a result," and "originally," comparing the emotion category and tendency before and after these transition words. When the emotion category changed, or when a shift occurred between a positive and negative emotion tendency, we recorded an expression of emotional transition. For example, we identified an expression of emotional transition from "I started to be a little scared, but then I wasn't scared anymore."

[0073] The number of times the first-person pronoun is used refers to the number of times the first-person pronouns such as "I" or "we" appear in the events expressed by the child; the number of times the self-state is expressed refers to the number of times the subject expresses a state in the first person, such as "I like...", "I can...", and "I hope".

[0074] The frequency of social object expressions refers to the number of times second-person pronouns, third-person pronouns, person names, and relational titles appear in a child's expressive event, such as "he," "Xiaoming," and "Mom." The frequency of interactive behavior expressions refers to the number of times a social object appears in a child's expressive event, and there are also expressions of joint activities, communication, help, conflict, cooperation, or other interactive behaviors related to that social object. For example, in the sentence "I went to the beach with my mom," "Mom" is a social object expression, and "went to the beach together" is an interactive expression with the social object.

[0075] Extract discourse evidence fragments corresponding to various expressive features from the analyzed text.

[0076] The analysis results of children's growth are obtained by summarizing the various expressive characteristics of children based on the events they express and the corresponding original discourse evidence fragments in the analyzed text.

[0077] The report generation module establishes a correlation between teacher listening behavior events and children's expression events, converts teacher support analysis results and children's growth analysis results into cumulative listening growth reports, and performs incremental updates based on one-on-one listening records.

[0078] Based on the children's subsequent expression windows, a correlation is established between teacher listening behavior events and children's expression events. Specifically, the children's subsequent expression window corresponding to each teacher listening behavior event is read, and the children's expression event number contained in the children's subsequent expression window is obtained. According to the children's expression event number, the corresponding children's expression event and the corresponding children's growth analysis result are found from the children's growth analysis results. Based on the children's expression event number contained in the children's subsequent expression window, the correlation relationship between teacher listening behavior events and corresponding children's expression events is obtained.

[0079] Based on the correlation between teacher listening behavior events and children's expressive events, the changes in various characteristic values, teacher support scores, and teacher support levels corresponding to various teacher listening behavior events in the teacher support analysis results, as well as the children's growth analysis results of children's expressive events that are related to teacher listening behavior events, are summarized to obtain a single listening growth report.

[0080] Based on the time sequence of the one-on-one listening records, single listening growth reports are compiled to obtain a cumulative listening growth report. As one-on-one listening records continue, new single listening growth reports are generated and appended to the end of the cumulative listening growth report sequence to form an incremental update of the cumulative listening growth report.

[0081] In summary, this invention achieves the following: Firstly, it reconstructs children's discourse into expressive events with relatively complete semantics, reducing semantic fragmentation caused by using single sentences or individual discourse turns as analysis units. Secondly, it minimizes interference from fragmented expressions, repetitive content, and corrections on feature statistics, making statistical results such as expression length, event details, emotional expression, and proactive supplementation more accurate and stable. Thirdly, it uses teacher listening behavior events as time anchors to compare changes in children's expressive characteristics before and after these events. This establishes a clear correlation between changes in children's expression and corresponding teacher listening behaviors, reducing evaluation bias caused by separate statistics of teacher behavior and children's expression, as well as erroneous attributions of changes in children's expression. This improves the objectivity, accuracy, and interpretability of teacher support effectiveness evaluation. Finally, the analysis results remain correlated with the original discourse, providing verifiable data for growth reports and enhancing the credibility and traceability of report conclusions.

[0082] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A system for analyzing teacher-child interaction and generating growth reports based on early childhood listening, characterized in that: include, The text transcription module collects one-on-one listening records of the target children, preprocesses the one-on-one listening records and performs automatic speech recognition to generate initial transcribed text; The expression reconstruction module performs speaker annotation, timestamp annotation, and speech turn segmentation on the initial transcribed text to generate structured teacher-child interaction data. It then reconstructs the structured teacher-child interaction data to obtain children's expression events and teachers' listening behavior events. The Support Results module constructs a pre-event child expression window and a subsequent child expression window for teacher listening behavior events. It calculates the change in feature values ​​in the pre-event and subsequent child expression windows, and performs a weighted summation of the feature value changes to obtain the teacher support score. It then summarizes the teacher support scores and corresponding feature value changes for teacher listening behavior events to obtain the teacher support analysis results. The growth indicator module extracts various expressive features corresponding to children's expressive events from the initial transcribed text, and summarizes them according to children's expressive events to obtain the results of children's growth analysis; The report generation module establishes a correlation between teacher listening behavior events and children's expression events, converts teacher support analysis results and children's growth analysis results into cumulative listening growth reports, and performs incremental updates based on one-on-one listening records.

2. The teacher-child interaction analysis and growth report generation system for early childhood listening as described in claim 1, characterized in that: The one-on-one listening record refers to the audio data of a conversation between a single teacher and a single child.

3. The teacher-child interaction analysis and growth report generation system for early childhood listening as described in claim 1, characterized in that: The specific steps for generating the initial transcribed text are as follows: Preprocess the one-on-one listening recordings to generate audio segments; The speech audio segments are framed and windowed, and the log-Mel spectrum features are calculated. The contextual acoustic features at each time position are extracted from the log-Mel spectrum features, and convolution operations are performed to output the character probability corresponding to each time position. Based on the character probability corresponding to each time position, line bundle search is performed on the characters corresponding to each time position to obtain the initial transcribed text.

4. The teacher-child interaction analysis and growth report generation system for early childhood listening as described in claim 1, characterized in that: The specific steps for generating structured teacher-child interaction data are as follows: Align the start and end timestamps of the initial transcribed text and the audio segment with the start and end timestamps of the audio segment according to the start and end positions of the audio segment in the one-on-one listening record; The speech audio segment is framed and its spectrum is transformed to obtain the voiceprint feature vector; By recording teachers' audio in a quiet environment and extracting the corresponding voiceprint feature vector, the pre-registered voiceprints of teachers are obtained, and a voiceprint matching threshold for teachers is set. Calculate the cosine similarity between the voiceprint feature vector of each speech audio segment and the teacher's pre-registered voiceprint. Speech audio segments with a cosine similarity not lower than the teacher's voiceprint matching threshold are labeled as teacher speech, and speech audio segments with a cosine similarity lower than the teacher's voiceprint matching threshold are labeled as children's speech. Using the speaker's switching position as the speech turn segmentation position, the initial transcribed text is segmented into speech turns to generate structured teacher-child interaction data with speech turns as the basic unit.

5. The teacher-child interaction analysis and growth report generation system for early childhood listening as described in claim 1, characterized in that: The specific steps for reconstructing the structured teacher-child interaction data are as follows. Based on the speakers marked in the audio segments, the speech turns in the structured teacher-child interaction data are divided into children's speech turns and teachers' speech turns; Based on the characteristics of children's natural dialogue, such as repetition, pauses, corrections, and additions, we set rules for reconstructing children's speech cycles and reconstructed the speech cycles to obtain children's expressive events. By examining the sentence structure, response space, and temporal relationship with adjacent children's speech turns in the teacher's speech turns, different types of teacher listening behavior events are obtained.

6. The teacher-child interaction analysis and growth report generation system for early childhood listening as described in claim 1, characterized in that: The aforementioned pre-early childhood expression window refers to a set of early childhood expression events selected before the occurrence of the teacher listening behavior event, using the teacher listening behavior event as the time anchor point, to represent the local baseline state of early childhood expression before the occurrence of the teacher listening behavior event; The term "subsequent expression window of young children" refers to a set of children's expression events selected after the teacher's listening behavior event, using the teacher's listening behavior event as a time anchor point, to represent the immediate changes in children's expression after the teacher's listening behavior event.

7. The teacher-child interaction analysis and growth report generation system for early childhood listening as described in claim 1, characterized in that: The specific steps for calculating the changes in feature values ​​in the preceding and subsequent child expression windows are as follows. Extract various discourse features from the pre-early children's expression window and the subsequent children's expression window respectively; Calculate the average feature values ​​of various discourse features in the preceding children's expression window and the subsequent children's expression window, and then calculate the difference to obtain the change in feature values.

8. The teacher-child interaction analysis and growth report generation system for early childhood listening as described in claim 1, characterized in that: The specific steps for converting teacher support analysis results and child development analysis results into a cumulative listening and growth report are as follows. The results of the child development analysis of child expression events that are related to teacher support analysis results and teacher listening behavior events are summarized to obtain a single listening growth report. Based on the chronological order of the one-on-one listening records, summarize the individual listening growth reports to obtain the cumulative listening growth report; When a new listening record is generated, a corresponding single listening growth report is generated, and the cumulative listening growth report is incrementally updated.

9. The teacher-child interaction analysis and growth report generation system for early childhood listening as described in claim 5, characterized in that: The specific steps for categorizing teacher turn-taking are as follows: Determine whether the start time of the teacher's turn of speech occurs before the end time of the previous round of children's expression. If so, the teacher's listening behavior is an interruption; if not, it is a waiting behavior. If the teacher's turn of speech is a question and there is no limit to the space for answering, it is an open-ended question; if the space for answering is limited to a limited number of options, it is a closed-ended question. If the teacher's verbal turn is a declarative sentence, then the teacher's listening behavior event is a response.

10. The teacher-child interaction analysis and growth report generation system for early childhood listening as described in claim 5, characterized in that: The rules for reconstructing children's speech turns include supplementary and corrective rules, rules for interspersing brief encouragement from teachers, rules for splitting new themes, rules for thematic similarity, and rules for continuity, which are used to merge and split children's speech turns.

Images