AI-based children language development evaluation method and system

Abstract

The invention discloses an AI-based children language development evaluation method and system, and belongs to the technical field of voice analysis and processing, and the method comprises the steps: obtaining a voice signal and situation data of a child, generating multi-mode perception data, carrying out the dynamic noise reduction processing, and generating a preprocessed voice signal; extracting multi-dimensional characteristic parameters from the preprocessed voice signals; based on the multi-dimensional feature parameters and a preset personal language development digital shadow, using a diagnosis model to generate a language development diagnosis result; and generating a real-time feedback instruction according to the language development diagnosis result. According to the comprehensive technical scheme, dynamic noise reduction is carried out by fusing situation information, personalized diagnosis is carried out based on personal language development digital shadows, and a real-time feedback instruction is generated, so that intelligent evaluation with accurate and dynamic language development and closed-loop intervention capability for children can be realized.

Description

Technical Field

[0001] This invention relates to the field of speech analysis and processing technology, and in particular to an AI-based method for assessing children's language development. Background Technology

[0002] Child language development assessment is a crucial component of early childhood health, education, and intervention. Its core lies in analyzing children's speech output to determine whether their language abilities are within the normal range for their age. With the development of artificial intelligence, the automated analysis of speech signals using computers has become a research hotspot in this field. This technology typically involves signal processing of collected children's speech, extracting acoustic features that reflect aspects such as clarity, fluency, and rhythm, and building models based on these features to assess language abilities or assist in language learning training.

[0003] Among related technologies, Chinese invention patent CN120356458A discloses a language learning assistance application system based on speech recognition, including: an intelligent speech acquisition and enhancement module, which realizes the acquisition and preprocessing of high-quality speech signals in complex environments; a multimodal feature extraction module, which dynamically extracts and fuses multi-dimensional learning feature information, and completes phoneme-level feature analysis and boundary segmentation, while also being responsible for feature normalization and optimization; a deep speech recognition module, which achieves high-precision speech recognition based on normalized features; an intelligent evaluation engine module, which realizes real-time, multi-dimensional evaluation of learners' language performance; a personalized learning management module, which integrates evaluation results to dynamically plan the optimal learning path for learners and manage their learning progress, while also providing learning effect prediction and intervention suggestions; and a learning interaction experience module, which is responsible for the visualization of learning feedback, multimodal human-computer interaction design, and user interface optimization.

[0004] However, the aforementioned existing technologies still have shortcomings when applied to the pathological diagnosis and long-term monitoring of early childhood language development. These systems are mostly designed as learning aids, heavily reliant on specific user-device interactions such as repeating and pronunciation tasks. They struggle to capture the spontaneous, unconscious natural language expressions of children aged 0-6, especially infants or children with communication difficulties, in their daily lives, resulting in limited ecological validity of the assessment results. While personalized learning path planning is achieved, it focuses on short-term learning effect feedback based on speech recognition accuracy, lacking a long-term longitudinal tracking mechanism based on children's individual historical data. This makes it difficult to accurately identify subtle developmental delays or pathological deviations by comparing individual historical developmental trajectories. Assessment indicators for language learning scenarios focus on pronunciation accuracy and learning progress, paying insufficient attention to early diagnostic indicators for complex language disorders involving cognitive and social dimensions, thus failing to meet the needs of clinical-level auxiliary diagnosis. Summary of the Invention

[0005] To address the aforementioned issues, this invention provides an AI-based method and system for assessing children's language development. It employs a comprehensive technical solution that integrates contextual information for dynamic noise reduction, performs personalized diagnosis based on individual language development digital shadows, and generates real-time feedback instructions. This enables precise, dynamic, and intelligent assessment of children's language development with closed-loop intervention capabilities.

[0006] The above objectives can be achieved through the following approach: An AI-based method for assessing children's language development includes acquiring children's speech signals and contextual data to generate multimodal perception data; performing dynamic noise reduction on the multimodal perception data to generate preprocessed speech signals; extracting multidimensional feature parameters from the preprocessed speech signals; generating language development diagnostic results using a diagnostic model based on the multidimensional feature parameters and a preset individual language development digital shadow; and generating real-time feedback instructions based on the language development diagnostic results.

[0007] Optionally, generating multimodal perception data includes: acquiring the child's voice signal through a preset high-sensitivity microphone array; acquiring the child's motion state data through a preset motion sensor to generate contextual data; identifying the child's current activity scene based on the voice signal and motion state data; and generating multimodal perception data according to the activity scene.

[0008] Optionally, generating the preprocessed speech signal includes: separating the speech signal and contextual data from the multimodal perception data; identifying the type of environmental noise based on the contextual data; selecting noise reduction algorithm parameters based on the type of environmental noise; and applying the noise reduction algorithm parameters to denoise the speech signal to generate the preprocessed speech signal.

[0009] Optionally, the step of extracting multi-dimensional feature parameters from the preprocessed speech signal includes: performing basic parameter analysis on the preprocessed speech signal to generate basic feature parameters; performing advanced parameter analysis on the preprocessed speech signal to generate advanced feature parameters; and combining the basic feature parameters and the advanced feature parameters to generate multi-dimensional feature parameters.

[0010] Optionally, generating language development diagnostic results using a diagnostic model based on the multi-dimensional feature parameters and the personal language development digital shadow includes: obtaining the personal language development digital shadow, which contains a historical development trajectory; comparing the multi-dimensional feature parameters with the historical development trajectory to calculate a feature deviation value; and classifying and grading the language development based on the feature deviation value using the diagnostic model to generate language development diagnostic results.

[0011] Optionally, the diagnostic model includes: constructing the diagnostic model using a deep learning architecture combined with a traditional acoustic model, wherein the deep learning architecture uses a long short-term memory network or the Transformer algorithm; the training dataset of the diagnostic model includes speech samples of normal children and children with language disorders, wherein the speech samples of children with language disorders cover stuttering, articulation disorders and hearing loss types; the diagnostic model is configured to receive the feature deviation value as input, output the classification result of children's language development, and output the severity level for the identified abnormal results.

[0012] Optionally, generating real-time feedback instructions includes: parsing the language development diagnostic results and identifying the target pronunciation error type; selecting a matching gamified intervention task from a preset gamified intervention task library based on the target pronunciation error type; and generating real-time feedback instructions containing the gamified intervention task.

[0013] Optionally, the method further includes: sending the real-time feedback instruction to the user terminal; performing the gamified intervention task on the user terminal for real-time pronunciation practice; and collecting the feedback speech signal during the real-time pronunciation practice as the subsequent speech signal acquisition.

[0014] Optionally, the method further includes: receiving the multi-dimensional feature parameters and language development diagnostic results; adjusting the development trajectory parameters of the personal language development digital shadow based on the language development diagnostic results; and updating the personal language development digital shadow using the adjusted development trajectory parameters.

[0015] Based on the same inventive concept, the present invention also provides an AI-based assessment system for children's language development, the system comprising: The multimodal perception acquisition module is used to acquire children's voice signals and contextual data to generate multimodal perception data; The dynamic noise reduction processing module is used to perform dynamic noise reduction processing on the multimodal sensing data to generate a preprocessed speech signal; The feature extraction module is used to extract multi-dimensional feature parameters from the preprocessed speech signal; The diagnostic model module is used to generate language development diagnostic results based on the multi-dimensional feature parameters and the preset personal language development digital shadow; The feedback generation module is used to generate real-time feedback instructions based on the language development diagnosis results.

[0016] Compared with the prior art, the present invention has the following advantages: 1. By constructing multimodal perception data, the system deeply integrates children's speech signals with contextual data such as their movement state and activity scenarios, enabling the system to understand the impact of the real-life environment on speech production and perform dynamic noise reduction accordingly. This improves the accuracy and robustness of speech signal acquisition and analysis in complex acoustic environments.

[0017] 2. The concept of a digital shadow of individual language development is introduced to establish a dynamically updated, personalized developmental baseline for each child. The assessment process compares a child's current language characteristics with their own historical trajectory, rather than with fixed age norms, achieving truly personalized diagnosis and enabling earlier and more accurate detection of subtle changes and potential risks in individual development.

[0018] 3. Through the intelligent linkage between the diagnostic model and the gamified intervention task library, abstract language assessment results are instantly transformed into concrete and engaging real-time feedback instructions. This seamless closed-loop design from diagnosis to intervention allows children to begin targeted practice the moment they receive assessment feedback, shortening the delay between problem identification and corrective training, and improving the efficiency of language intervention and children's willingness to participate.

[0019] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures pointed out in the description, claims and drawings. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating an AI-based method for assessing children's language development according to an embodiment of the present invention.

[0022] Figure 2 This is a three-dimensional surface schematic diagram of a scene classification and fusion model of an AI-based children's language development assessment method and system according to an embodiment of the present invention.

[0023] Figure 3 This is a schematic diagram of a statistical model for analyzing the digital shadow feature deviation of an AI-based method and system for assessing children's language development, according to an embodiment of the present invention.

[0024] Figure 4This is a schematic diagram of the digital shadow adaptive update trajectory trend of an AI-based children's language development assessment method and system according to an embodiment of the present invention.

[0025] Figure 5 This is a schematic diagram of the structure of an AI-based children's language development assessment system according to an embodiment of the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] Reference Figure 1 One embodiment of the present invention proposes an AI-based method for assessing children's language development. It employs a comprehensive technical solution that integrates contextual information for dynamic noise reduction, performs personalized diagnosis based on individual language development digital shadows, and generates real-time feedback instructions. This method enables accurate, dynamic, and intelligent assessment of children's language development with closed-loop intervention capabilities.

[0028] The method described in this embodiment specifically includes: acquiring the child's voice signals and contextual data, and generating multimodal perception data; The multimodal sensing data is subjected to dynamic noise reduction processing to generate a preprocessed speech signal; Extract multi-dimensional feature parameters from the preprocessed speech signal; Based on the multi-dimensional feature parameters and the preset personal language development digital shadow, a diagnostic model is used to generate language development diagnostic results. Based on the language development diagnosis results, generate real-time feedback instructions.

[0029] Specifically, by integrating children's speech with their contextual information, multimodal perceptual data containing environmental context is generated. This contextual information is then used to dynamically and adaptively denoise the speech signal, ensuring the purity of the analyzed data. A set of multi-dimensional feature parameters comprehensively reflecting acoustics, prosody, and intelligibility is extracted from the pure speech and compared with the child's unique digital shadow of language development, which records the child's historical development trajectory. The diagnostic model generates personalized language development diagnostic results with longitudinal reference value by analyzing the deviation between current characteristics and the individual's historical baseline. This abstract diagnostic result is instantly transformed into concrete, actionable real-time feedback instructions, completing a full technical loop from data collection and intelligent analysis to immediate intervention. This improves the accuracy, personalization, and intervention efficiency of children's language development assessment. By using contextual data for dynamic noise reduction, speech can be effectively collected and analyzed in real, non-ideal everyday environments, expanding the applicability of assessment scenarios and improving the reliability of assessment results. The introduction of the concept of a digital shadow of individual language development transforms assessment from a horizontal comparison with a general group of children to a longitudinal tracking of an individual's developmental trajectory. This allows for more sensitive detection of developmental delays or progress, providing personalized diagnoses. Furthermore, by generating real-time feedback instructions, assessment and intervention are seamlessly integrated, achieving a continuous support model of immediate assessment and practice. Compared to traditional periodic assessments and delayed interventions, this improves the efficiency of language correction and development, providing a dynamic, precise, and efficient intelligent support method for children's language development.

[0030] Optionally, the generation of multimodal sensing data includes: Children's voice signals are collected using a pre-set high-sensitivity microphone array; Contextual data is generated by collecting children's movement data through preset motion sensors; Based on voice signals and motion status data, identify the child's current activity scene; Based on the activity scenario, multimodal perception data is generated.

[0031] Specifically, a pre-set high-sensitivity microphone array is activated. This array, typically a ring or linear structure containing 4 to 8 microphone units, is deployed in the smart device within the child's primary activity area. The sensitivity of each microphone unit is configured within a range of -38dB to -3dB to ensure the capture of subtle speech details. The audio signal is digitized at a sampling rate of 16kHz or higher and 16-bit quantization precision. Beamforming techniques, such as delay-addition algorithms, are applied to dynamically focus the pickup on the child's sound source, initially suppressing environmental interference from other directions. A set of multi-channel raw speech signals is generated. To acquire contextual data, a pre-set motion sensor integrated into the wearable device or commonly used items by the child is activated. This is typically an inertial measurement unit (IMU) integrating an accelerometer and gyroscope. An IMU is an electronic device capable of measuring the three-axis acceleration and angular velocity of an object, used in engineering for attitude and motion tracking. Three-axis acceleration and angular velocity data are acquired at a frequency of 50Hz to 100Hz. The intensity of a child's movement is quantified by calculating the variance of the acceleration vector magnitude over a time window, such as 200 milliseconds, thus generating motion state data. Structured contextual data containing information such as movement intensity and frequency is generated. A lightweight classification model, such as a support vector machine or a small multilayer perceptron, is employed, pre-trained using scene-labeled data. The model's input features include speech activity detection results and signal-to-noise ratio extracted from the speech signal, and motion intensity indicators extracted from the motion state data. The model calculates membership scores for different scenes using a weighted fusion function S. ; Where S represents the confidence score for a specific activity scenario. Fa is the normalized audio feature value, such as the speech segment energy represented between 0 and 1. Fm is the normalized motion feature value, such as the acceleration variance represented between 0 and 1. Wa and Wm are the model weight coefficients, learned from the training data, and their sum is usually 1, used to adjust the importance of speech and motion information in scene judgment. Fa and Fm are normalized using Min-Max. , , Where Emin and Emax are the microphone's preset mute threshold (e.g., 30dB) and saturation threshold (e.g., 90dB); Mmin and Mmax are the IMU sensor's static noise floor (e.g., 0.01g). 2 And the threshold for strenuous exercise, such as 2.0g 2After calculating scores for multiple preset scenarios such as quiet reading and dynamic games, the scenario with the highest score is selected as the recognition result for the current activity scenario. All information is integrated to generate multimodal perception data in a unified format. This is a standardized data package whose internal data structure includes, but is not limited to, raw multi-channel speech signals from a high-sensitivity microphone array, contextual data containing motion intensity quantification values, and a label for the current activity scenario. Furthermore, the data package also includes a high-precision timestamp to ensure strict synchronization between speech and motion data. Figure 1 As shown, a 3D surface model illustrates the decision mechanism for multimodal data fusion. The X-axis represents audio features Fa, such as normalized energy; the Y-axis represents motion features Fm, such as normalized variance; and the Z-axis represents the calculated scene confidence score S. The colored surface shows that the scene score increases linearly with increasing audio and motion intensity. The red semi-transparent plane in the figure represents a preset classification threshold, such as S=0.65. When the calculated S value is above this plane, the current scene is determined to be the target scene, such as a dynamic game. This visualization model clearly explains the decision boundary of bimodal weighted fusion.

[0032] For example, in a children's play area equipped with smart monitoring devices, the process of generating multimodal data was verified through engineering methods. First, a deployed circular microphone array containing six units, with a sensitivity of -38dB, started recording at a 16kHz sampling rate. Simultaneously, a smart bracelet worn by the child, with an integrated IMU, collected data at a frequency of 50Hz, setting a time window of 200 milliseconds (containing 10 sampling points) to calculate motion intensity. At a certain time t, by calculating the variance of the acceleration vector modulus of the 10 sampling points, the normalized motion feature value Wm was obtained, which was 0.8. Simultaneously, the normalized speech energy feature value Fa extracted from the audio was 0.6. The system uses the formula... The scene score is calculated, with weighting coefficients preset based on the training data: Wa=0.3 for audio and Wm=0.7 for motion, emphasizing the recognition of game scenes through actions. Substituting the values, the calculation is: S=0.3*0.6+0.7*0.8=0.18+0.56=0.74. This score of 0.74 is higher than the preset binary classification threshold of 0.65, so the system classifies the current scene as a dynamic game and packages this label with the original speech and motion data to generate a multimodal perception data package with high-precision timestamps.

[0033] Optionally, generating the preprocessed speech signal includes: Separate speech signals and contextual data from the multimodal sensing data; Identify the type of environmental noise based on the contextual data; Select noise reduction algorithm parameters based on the type of ambient noise; The noise reduction algorithm parameters are applied to the speech signal to reduce noise and generate a preprocessed speech signal.

[0034] Specifically, the system receives a multimodal perception data packet, which is structured data containing timestamps, multi-channel speech signals, motion state data, and activity scene labels. By parsing this data packet, the original speech signal is extracted and sent to the signal processing pipeline; simultaneously, the contextual data containing the activity scene labels and motion state data is separated and sent to the environmental analysis module. A pre-set noise type recognition model is activated, which is essentially a decision tree or rule-based mapping engine. The system receives activity scene labels parsed from the contextual data as input, such as dynamic games or quiet reading. By querying a pre-set scene-noise mapping table, the activity scene is associated with a specific environmental noise type. For example, when the activity scene is a dynamic game, it is identified as a non-stationary impact noise type; when the activity scene is dining, it is identified as a mixed noise type of background voices and clinking tableware. Standardized environmental noise type labels are generated. A noise reduction algorithm parameter configuration library is maintained, storing the optimal parameter combinations for different environmental noise types. Upon receiving an environmental noise type label, the corresponding noise reduction algorithm parameters are automatically queried and loaded. For example, for steady-state background noise such as air conditioner noise, a set of parameters is selected that configures spectral subtraction with a high over-subtraction factor and a long noise estimation update period. For non-stationary impulsive noise, another set of parameters is selected, which may activate a wavelet threshold-based denoising module and set a specific threshold function. The denoising algorithm parameters are then generated. The separated original speech signal and the selected denoising algorithm parameters are fed into the denoising processor. The denoising processor performs a short-time Fourier transform on the speech signal, converting it to the frequency domain for processing. In the frequency domain, a gain function is applied to modulate each time-spectrum unit, a process that can be represented by the following equation: ; Here, Xest(k,t) represents the complex spectral component of the clean speech estimated after denoising at frequency index k and time frame index t. Y(k,t) represents the complex spectral component of the original noisy speech at the same position, obtained by performing frame windowing and Fourier transform on the input speech signal. G(k,t) is the real gain applied to this time-frequency unit, typically ranging from 0 to 1. The calculation method of this gain G is dynamically determined by the denoising algorithm parameters selected in the previous steps; for example, it may be a suppression factor calculated based on the signal-to-noise ratio estimation function, thereby achieving adaptive attenuation of noise components. After frequency domain processing, the signal is converted back to the time domain through inverse Fourier transform, and the final output signal is the high-quality preprocessed speech signal.

[0035] For example, the dynamic parameter adjustment and frequency domain gain calculation of the noise reduction module are verified. After receiving a data packet tagged as "dynamic game," the system associates it with a non-stationary impulsive noise type by looking up a table. Based on this type, the system loads a specific set of parameters, such as enabling wavelet thresholding noise reduction, and applies a gain function in the frequency domain. Suppose that at frame t, with frequency index k such as 500Hz, the complex spectral amplitude of the original noisy speech is... The value is 10.0. First, the signal-to-noise ratio at this time frequency point is estimated based on the previously selected parameters. If the calculated prior signal-to-noise ratio is... Given a value of 3.0, based on the Wiener filtering principle as a calculation logic for gain G, the gain G(k,t) can be calculated as follows: Substituting this gain into the formula: Xest(k,t)=0.75*10.0=7.5. This calculation process shows that 75% of the signal amplitude at this frequency point is preserved, effectively suppressing background noise, and finally outputting a high-purity pre-processed speech signal through inverse Fourier transform.

[0036] Optionally, the extraction of multi-dimensional feature parameters from the preprocessed speech signal includes: The preprocessed speech signal is subjected to basic parameter analysis to generate basic feature parameters; Advanced parameter analysis is performed on the preprocessed speech signal to generate advanced feature parameters; The basic feature parameters and advanced feature parameters are combined to generate multi-dimensional feature parameters.

[0037] Specifically, the preprocessed speech signal is received and segmented into continuous, partially overlapping short time frames, typically 20 to 30 milliseconds in length and 10 milliseconds in frame shift. For each short time frame, a set of fundamental feature parameters is calculated in parallel. These parameters mainly include Mel-frequency cepstral coefficients, a feature that simulates human auditory perception, obtained in engineering by Mel-scale filtering and discrete cosine transform of the signal spectrum, typically extracting 13 to 39 coefficients; the fundamental frequency, reflecting the frequency of vocal cord vibration; and energy, representing the loudness of the signal. These calculations constitute a matrix of fundamental feature parameters that varies over time. These parameters are usually calculated based on the fundamental feature parameters over a longer time scale, such as complete words or sentences. Advanced parameter analysis includes calculating speech rate, i.e., the number of syllables per unit time; calculating the proportion and average duration of silent segments to measure speech fluency; calculating fundamental frequency jitter and amplitude tremor, where fundamental frequency jitter refers to small changes in the amplitude between adjacent fundamental frequency periods, and amplitude tremor refers to small changes in the amplitude between adjacent periods; both are key engineering indicators for measuring sound quality stability; and analyzing the distribution of the first and second formants to assess the fullness and distinguishability of vowel pronunciation, where the formants are the resonant frequencies of the vocal tract. These calculation results collectively constitute advanced feature parameters characterizing the overall quality of a single vocal event. A feature combination operation is then performed to concatenate the generated feature parameters. This process can be represented as follows: ; Here, F represents the final generated multi-dimensional feature parameter vector. C represents the vector concatenation operation. Fb represents the sequence of basic feature parameters or their statistical values, consisting of Mel-frequency cepstral coefficients, fundamental frequency, etc. Fa represents the set of high-level feature parameters, consisting of speech rate, fundamental frequency jitter, amplitude tremor, etc. For example, the mean and variance of the 13-dimensional Mel-frequency cepstral coefficients of a sentence are concatenated with five high-level parameters: the average fundamental frequency, fundamental frequency jitter, amplitude tremor, and speech rate, to form a 32-dimensional feature vector. The multi-dimensional feature parameters are formatted and then sent to the diagnostic model module.

[0038] For example, feature extraction and fusion are performed on a 1-second preprocessed speech segment. First, in the basic parameter analysis stage, the signal is framed with a frame length of 25ms and a frame shift of 10ms, and the 13-dimensional MFCC coefficients and fundamental frequency F0 of each frame are extracted. Then, advanced parameter analysis is performed. Taking fundamental frequency jitter as an example, assuming that 5 consecutive fundamental frequency cycles with durations of [missing information] are extracted within the current word segment... According to the definition of Jitter, the sum of the absolute values ​​of the differences between adjacent periods is calculated as follows: |0.0100-0.0102| + |0.0102-0.0099| + |0.0099-0.0101| + |0.0101-0.0100| [citestart] = 0.0002 + 0.0003 + 0.0002 + 0.0001 = 0.0008s. The average absolute perturbation is 0.0008 / 4 = 0.0002s. Therefore, the relative Jitter value is... Finally, the feature concatenation operation F=C(Fb,Fa) is performed, where Fb is the mean vector of Mel-frequency cepstral coefficients of the entire speech segment, and Fa includes calculated jitter, amplitude tremors, speech rate, and other high-level features. The concatenated feature vector F has 18 dimensions and is passed as standard input to the diagnostic model.

[0039] Optionally, generating language development diagnostic results using a diagnostic model based on the multi-dimensional feature parameters and the individual's digital shadow of language development includes: Obtain the digital shadow of the individual's language development, which includes a historical development trajectory; The multi-dimensional feature parameters are compared with the historical development trajectory to calculate the feature deviation value; Based on the aforementioned feature deviation values, the diagnostic model is used for classification and grading to generate language development diagnostic results.

[0040] Specifically, based on the child's unique identifier, a query is initiated to the backend data warehouse to retrieve their exclusive personal language development digital profile. This personal language development digital profile is, in engineering terms, a structured data archive, with its core content being their historical development trajectory. This historical development trajectory is not a single curve, but rather a time-series statistical distribution model established for each key indicator among multi-dimensional feature parameters, such as the fundamental frequency mean, fundamental frequency jitter percentage, and speech rate. Specifically, it stores the mean and standard deviation of each language feature parameter for the child at different age stages in the past, for example, age segments on a monthly basis. A historical development trajectory containing the child's personal historical norm data is generated. Multi-dimensional feature parameters are received, and from the historical development trajectory, the expected values ​​and distribution data of the feature parameters that best match the child's current age are queried. Dimensional calculations are performed to generate feature deviation values. This calculation is performed using a standardized formula to eliminate the influence of different parameter units: ; Where D is the feature bias value of a single dimension, which is a dimensionless standardized score. Vc represents the measured value of a feature extracted from the currently acquired multi-dimensional feature parameters. Vt represents the expected mean of this feature for this child at this age, as found in the historical development trajectory. This represents the standard deviation of the feature recorded in the historical developmental trajectory within the corresponding age group, used to measure the dispersion of historical data. This calculation is repeated for all key indicators in the multi-dimensional feature parameters, generating a feature deviation value vector containing deviation scores across multiple dimensions. This feature deviation value vector is then input into the pre-trained diagnostic model. The diagnostic model performs classification and grading tasks based on the pattern of the input vector. The classification task aims to determine whether the child's current language development status is within the normal range or whether there is a risk of a specific type of impairment, such as abnormal articulation or fluency. If the classification result is abnormal, the grading task further assesses the severity of deviation from the historical trajectory, outputting severity levels such as mild, moderate, and severe. A structured language development diagnostic result is generated, containing qualitative classification conclusions and quantitative grading assessments. Figure 2 The graph shown is a normal distribution statistical chart, visually defining the numerical shadow of an individual's language development. The curve represents the child's historical baseline model constructed with the mean Vt and standard deviation σt, with the light blue shaded area covering the normal fluctuation range of ±2σ. The red dashed line and scatter plots indicate the current measured value Vc, such as an anomalous jitter value. The red arrows indicate the standardized distance between the measured value and the historical baseline, i.e., the eigenvalue bias D.

[0041] For example, a personalized diagnostic verification was performed on a 36-month-old child. First, the child's historical developmental trajectory in the digital shadow was obtained, and the expected mean of the child's fundamental frequency jitter (Vt) at 36 months of age was found to be 1.0%, with a standard deviation of [missing value]. Among them, the measured jitter value Vc=1.99% extracted in the previous embodiment is substituted into the standardization formula. The calculation is as follows: D = (1.99 - 1.0) / 0.2 = 0.99 / 0.2 = 4.95. This feature deviation value of 4.95 is a dimensionless fraction, indicating that the current vocal stability deviates from the individual's historical baseline by nearly 5 standard deviations. The deviation vector containing this value is input into the diagnostic model. Because the deviation value significantly exceeds the usual threshold of 3 standard deviations for a normal distribution, the model classifies it as abnormal in the classification task and sets the output as severe vocal cord instability in the grading task based on the deviation magnitude D > 3.0.

[0042] Optionally, the diagnostic model includes: The diagnostic model is constructed by combining a deep learning architecture with a traditional acoustic model, wherein the deep learning architecture uses a long short-term memory network or the Transformer algorithm. The training dataset for the diagnostic model contains speech samples from normal children and children with language disorders, with the speech samples from children with language disorders covering stuttering, articulation disorders, and hearing loss types. The diagnostic model is configured to receive the feature deviation value as input, output a classification result of the child's language development, and output a severity rating for the identified abnormal results.

[0043] Specifically, a deep learning architecture is employed, with a Long Short-Term Memory (LSTM) network chosen for its ability to effectively process and remember sequential information. The main body of the model consists of two to three stacked LSTM units, each containing 128 to 256 hidden nodes. Building upon this deep learning architecture, the analysis results of traditional acoustic models are integrated. This integration is reflected in the model's input, where the multi-dimensional feature parameters are generated by traditional acoustic analysis methods such as Mel-frequency cepstral coefficients and fundamental frequency extraction. The deep learning model is then responsible for performing deeper pattern recognition and semantic understanding on these traditional features. The training dataset for the diagnostic model consists of two main parts: one part comprises a large number of speech samples from normally developing children at different ages, and the other part comprises speech samples from children diagnosed with language disorders by professional speech pathologists. The latter meticulously covers different types of disorders, including stuttering characterized by repetition and interruption, articulation disorders characterized by pronunciation errors, and specific pronunciation patterns caused by hearing loss. Each speech sample has its multi-dimensional feature parameters pre-extracted and labeled with two tags: a classification tag (e.g., normal, stuttering, articulation disorder) and a severity tag (e.g., mild, moderate, severe) for the disorder sample. The model is configured to receive a feature bias vector as its sole input. The model's output layer is designed with two parallel branches: one branch is a softmax layer for outputting the classification results, with the number of neurons equal to the total number of categories plus one, outputting the probability of each category; the other branch is a softmax layer for outputting the severity levels, with the number of neurons equal to the number of severity levels. During training, the model is optimized using a composite loss function, as follows: ; Where L is the total loss calculated by the model in one iteration. Lc is the cross-entropy loss generated by the classification task, used to measure the accuracy of classification prediction. Ls is the cross-entropy loss generated by the severity grading task, used to measure the accuracy of grading prediction. α and β are hyperparameters, weight coefficients obtained through experiments during the model development phase, with values ​​between 0 and 1, used to balance the importance of classification and grading tasks during model training. In actual diagnosis, the model performs one forward propagation, selecting the item with the highest probability from each of the two output branches, combining them to form the final language development diagnosis result.

[0044] For example, during the model training phase, the calculation process of the composite loss function is verified. The constructed diagnostic model includes LSTM layers and dual-branch output layers for classification and grading, and the loss function is set as follows: Hyperparameters Prioritizing classification accuracy, β=0.3 is used to assist in grading accuracy. In one forward propagation, for a sample labeled as articulation disorder, moderate, the model calculates a classification cross-entropy loss Lc of 0.5 and a grading cross-entropy loss Ls of 0.2. Substituting these values ​​into the formula, the total loss is calculated as follows: The optimizer calculates the gradient and updates the network weights based on this total loss value of 0.41. This calculation ensures that the model can simultaneously consider both qualitative judgments of obstacle types and quantitative assessments of severity during the learning process.

[0045] Optionally, the real-time feedback generation instruction includes: Analyze the language development diagnostic results to identify the target pronunciation error type; Based on the target pronunciation error type, a matching gamified intervention task is selected from a preset gamified intervention task library; Generate real-time feedback instructions that include the gamified intervention task.

[0046] Specifically, the system receives language development diagnostic results. These results are data pairs containing classification conclusions and severity levels, such as "arthropathy - alveolar replacement - moderate". The parsing engine is then activated, its core being a rule mapping table. Based on the classification conclusions in the diagnostic results, such as arthropathy, it further refines the analysis to identify the target pronunciation error types. This identification may be based on finer-grained classification probabilities provided by the diagnostic model, or through backtracking analysis of multi-dimensional feature parameters that triggered the diagnosis, such as finding formant features related to alveolar sounds that deviate from the norm. One or more precise target pronunciation error type labels are generated, such as unclear sh pronunciation or l / n confusion. A pre-defined gamified intervention task library is maintained. This library is a structured database storing a large number of interactive games or practice tasks, each associated with one or more target pronunciation error types. For example, for unclear sh pronunciation, the library might include games like imitating a snake's hiss or blowing a feather; for l / n confusion, it might include tasks like a calf reciting tongue twisters. Each task includes attributes such as its invocation instructions, difficulty level, and incentive mechanism. Upon receiving a label indicating the type of pronunciation error, the system uses this label as a search index to retrieve information from a gamified intervention task library. The retrieval algorithm not only requires label matching but also incorporates severity ratings from language development diagnostic results to filter the initial difficulty of the task. For example, for moderate problems, a moderately difficult task is selected. The detailed information of the matched gamified intervention task, including the task ID, required resources such as animations and audio files, and interaction logic, is packaged into a standardized instruction data package. Real-time feedback instructions are generated. For example, an instruction might include: perform the task of blowing a small feather, play a guiding animation, play a demonstration audio of the target sound "sh," and activate the microphone to wait for the child to imitate. This real-time feedback instruction is then formatted into JSON or a similar data structure, ready to be sent to the user's terminal.

[0047] For example, specific intervention instructions are generated based on the diagnostic results. The received diagnostic results are "Classification: Articulation Disorder; Subtype: Absence of velar consonant / g / ; Grading: Moderate". The parsing engine first searches for relevant tasks in the gamified intervention task library based on the target pronunciation error type label "absence of velar consonant / g / ". The library returns two candidate options: "cooing pigeon" and "drinking water sound". Combining the "moderate" severity grading, high-difficulty continuous pronunciation tasks are excluded, and the moderately difficult imitation task "cooing pigeon" TaskID:G_002 is selected. Subsequently, the task ID, the corresponding animation resource file "pigeon.mp4", and the interaction logic: play the demonstration sound and wait 5 seconds for recording are encapsulated into a real-time feedback instruction in JSON format.

[0048] Optionally, the method further includes: Send the real-time feedback command to the user terminal; Perform the gamified intervention task on the user terminal for real-time pronunciation practice; Feedback speech signals collected during real-time pronunciation practice are used as subsequent speech signal acquisition.

[0049] Specifically, real-time feedback instructions are sent from the backend server to the designated user terminal via a secure network communication protocol, such as HTTPS. The user terminal can be a tablet, a smart toy, or any smart device integrated with a screen and microphone. This step is triggered by the successful generation of the real-time feedback instruction, which is essentially a standard client-server data push operation, ensuring low latency and high reliability in instruction delivery. When the application on the user terminal receives the real-time feedback instruction, its built-in task execution engine immediately parses the instruction. Based on the task ID in the instruction, the application loads and executes the corresponding gamified intervention task. For example, when executing a task called "blowing a feather," a feather animation is displayed on the screen, and a demonstrative "sh" sound is played, followed by prompts for the child to imitate. This process utilizes the terminal device's graphics rendering and audio playback capabilities to wrap pronunciation practice in a game, thereby increasing children's participation and compliance. During the execution of the gamified intervention task on the user terminal, when the child is prompted to perform real-time pronunciation practice, the application synchronously activates the device's microphone array to begin audio acquisition. The acquisition process uses the same high-fidelity parameter settings as the initial assessment, such as a 16kHz sampling rate and a 16-bit quantization depth, to ensure data consistency. The acquired audio data is the feedback speech signal. This feedback speech signal is labeled with a specific event type, such as intervention exercise, and is used as the raw speech signal for the next iteration, that is, as the subsequent speech signal acquisition, and is sent back to the starting point of the entire processing flow, namely the multimodal perception data generation step, thereby starting a new round of assessment-diagnosis-feedback loop.

[0050] For example, the complete closed-loop path from instruction issuance to data feedback is verified. A JSON instruction is sent to a child's tablet via HTTPS. After parsing the instruction, the terminal app plays a pigeon animation and demonstration sound on the screen, prompting the child to imitate. Simultaneously, the app uses a microphone array to record the child's imitation at a 16kHz sampling rate and 16-bit quantization precision. After recording, the audio is automatically tagged as a feedback voice signal by the app and sent back to the server along with an event tag for the intervention task. The server-side environmental analysis module recognizes this tag and directly sends it as new raw input to the feature extraction module, triggering a new round of evaluation, thus achieving a technical closed loop from diagnosis to intervention to verification.

[0051] Optionally, the method further includes: Receive the multi-dimensional feature parameters and language development diagnostic results; Based on the language development diagnosis results, the development trajectory parameters of the individual language development digital shadow are adjusted; The individual language development digital shadow is updated using the adjusted development trajectory parameters.

[0052] Specifically, after the assessment and diagnosis process is completed, the update module receives two sets of input data simultaneously: the multi-dimensional feature parameters from the feature extraction module, representing the objective quantitative result of the child's current language performance; and the language development diagnostic result from the diagnostic model module, representing the qualitative and graded judgment of this performance. These two sets of data are bound to a unified transaction ID to ensure data consistency and traceability. A weighted smoothing algorithm is used to adjust the historical development trajectory parameters in the individual's language development digital shadow. For example, the update process for the mean of a certain feature in the historical development trajectory can be represented by the following formula: ; Wherein, Pn represents the new developmental trajectory parameter value to be updated in the digital shadow. Po represents the old parameter value of this feature stored in the digital shadow before the update, i.e., the historical mean. Vc represents the measured value of this feature in this assessment, obtained from the received multi-dimensional feature parameters. α is an adaptive learning rate between 0 and 1, and its value is dynamically determined by the language development diagnostic results. If the diagnostic result is normal, a lower α value is set, for example, 0.05 to 0.1, indicating that the current developmental trajectory is stable and only requires fine-tuning; if the diagnostic result is abnormal, a higher α value is set according to the severity level, for example, 0.2 to 0.4, to accelerate the response speed of the developmental trajectory to new changes or corrective effects. All adjusted developmental trajectory parameters are obtained, including the mean, standard deviation, and other statistics of each feature, and a database write operation is initiated. This operation is executed in a transactional manner, writing the new parameter values ​​into the personal language development digital shadow file corresponding to the child's unique identifier. After the update is completed, the personal language development digital shadow containing the latest information will serve as a personalized reference baseline for the next language development assessment. Figure 3 The chart shown is a time series trend graph illustrating the evolution of the digital shadow baseline over time. The scatter plots represent the measured values ​​Vc collected in each assessment; these values ​​fluctuate due to environmental or conditional influences. The line represents the maintained digital shadow baseline Pn. Through an adaptive update algorithm, the baseline smoothly tracks the changing trends of children's actual abilities, such as the gradual improvement in jitter scores, while filtering out random noise from individual measurements. The purple arrows illustrate how each measurement influences the baseline for fine-tuning.

[0053] For example, after the closed-loop assessment, an adaptive algorithm is used to update the child's digital shadow. Assume that after intervention, the child's measured fundamental frequency jitter improves to Vc = 1.2%, while the old mean Po stored in the digital shadow is 1.0%. The adaptive learning rate is determined based on the nature of the diagnostic result. Since the diagnostic results still showed deviations but had improved, the following settings were adopted. The calculation is as follows: Pn = (1 - 0.1) * 1.0 + 0.1 * 1.2 = 0.9 + 0.12 = 1.02%. A new historical mean of 1.02% is calculated, and this value is then written to the child's personal profile via a database transaction. This process ensures that the baseline data dynamically follows changes in the child's actual abilities, allowing subsequent deviation calculations to be based on the updated 1.02%, thereby improving the accuracy and personalization of long-term assessments.

[0054] Reference Figure 5 Based on the same inventive concept, the present invention also provides an AI-based assessment system for children's language development, the system comprising: The multimodal perception acquisition module is used to acquire children's voice signals and contextual data to generate multimodal perception data; The dynamic noise reduction processing module is used to perform dynamic noise reduction processing on the multimodal sensing data to generate a preprocessed speech signal; The feature extraction module is used to extract multi-dimensional feature parameters from the preprocessed speech signal; The diagnostic model module is used to generate language development diagnostic results based on the multi-dimensional feature parameters and the preset personal language development digital shadow; The feedback generation module is used to generate real-time feedback instructions based on the language development diagnosis results.

[0055] It should be noted that the electrical connections between the various units described above do not necessarily represent direct or indirect connections. Any indirect connection method can be applied to the embodiments of the present invention as long as it achieves the purpose of the present invention. The above descriptions are merely exemplary embodiments of the present invention and should not be construed as limiting the scope of the present invention.

[0056] All equivalent changes and modifications made in accordance with the teachings of this invention are still within the scope of this invention. Those skilled in the art will readily conceive of other embodiments of this invention upon considering the specification and the disclosure of practical truth. This application is intended to cover any variations, uses, or adaptations of this invention that follow the general principles of this invention and include common knowledge or conventional techniques in the art not described herein.

Claims

1. An AI-based method for assessing children's language development, characterized in that, The method includes: Acquire children's speech signals and contextual data to generate multimodal perception data; The multimodal sensing data is subjected to dynamic noise reduction processing to generate a preprocessed speech signal; Extract multi-dimensional feature parameters from the preprocessed speech signal; Based on the multi-dimensional feature parameters and the preset personal language development digital shadow, a diagnostic model is used to generate language development diagnostic results. Based on the language development diagnosis results, generate real-time feedback instructions.

2. The AI-based method for assessing children's language development according to claim 1, characterized in that, The generated multimodal sensing data includes: Children's voice signals are collected using a pre-set high-sensitivity microphone array; Contextual data is generated by collecting children's movement data through preset motion sensors; Based on voice signals and motion status data, identify the child's current activity scene; Based on the activity scenario, multimodal perception data is generated.

3. The AI-based method for assessing children's language development according to claim 1, characterized in that, The generation of the preprocessed speech signal includes: Separate speech signals and contextual data from the multimodal sensing data; Identify the type of environmental noise based on the contextual data; Select noise reduction algorithm parameters based on the type of ambient noise; The noise reduction algorithm parameters are applied to the speech signal to reduce noise and generate a preprocessed speech signal.

4. The AI-based method for assessing children's language development according to claim 1, characterized in that, The extraction of multi-dimensional feature parameters from the preprocessed speech signal includes: The preprocessed speech signal is subjected to basic parameter analysis to generate basic feature parameters; Advanced parameter analysis is performed on the preprocessed speech signal to generate advanced feature parameters; The basic feature parameters and advanced feature parameters are combined to generate multi-dimensional feature parameters.

5. The AI-based method for assessing children's language development according to claim 1, characterized in that, The process of generating language development diagnostic results using a diagnostic model based on the multi-dimensional feature parameters and the individual's digital shadow of language development includes: Obtain the digital shadow of the individual's language development, which includes a historical development trajectory; The multi-dimensional feature parameters are compared with the historical development trajectory to calculate the feature deviation value; Based on the aforementioned feature deviation values, the diagnostic model is used for classification and grading to generate language development diagnostic results.

6. The AI-based method for assessing children's language development according to claim 5, characterized in that, The diagnostic model includes: The diagnostic model is constructed by combining a deep learning architecture with a traditional acoustic model, wherein the deep learning architecture uses a long short-term memory network or the Transformer algorithm. The training dataset for the diagnostic model contains speech samples from normal children and children with language disorders, with the speech samples from children with language disorders covering stuttering, articulation disorders, and hearing loss types. The diagnostic model is configured to receive the feature deviation value as input, output a classification result of the child's language development, and output a severity rating for the identified abnormal results.

7. The AI-based method for assessing children's language development according to claim 1, characterized in that, The real-time feedback generation command includes: Analyze the language development diagnostic results to identify the target pronunciation error type; Based on the target pronunciation error type, a matching gamified intervention task is selected from a preset gamified intervention task library; Generate real-time feedback instructions that include the gamified intervention task.

8. The AI-based method for assessing children's language development according to claim 7, characterized in that, The method further includes: Send the real-time feedback command to the user terminal; Perform the gamified intervention task on the user terminal for real-time pronunciation practice; Feedback speech signals collected during real-time pronunciation practice are used as subsequent speech signal acquisition.

9. The AI-based method for assessing children's language development according to claim 1, characterized in that, The method further includes: Receive the multi-dimensional feature parameters and language development diagnostic results; Based on the language development diagnosis results, the development trajectory parameters of the individual language development digital shadow are adjusted; The individual language development digital shadow is updated using the adjusted development trajectory parameters.

10. An AI-based children's language development assessment system, applied to an AI-based children's language development assessment method as described in any one of claims 1-9, characterized in that, The system includes: The multimodal perception acquisition module is used to acquire children's voice signals and contextual data to generate multimodal perception data; The dynamic noise reduction processing module is used to perform dynamic noise reduction processing on the multimodal sensing data to generate a preprocessed speech signal; The feature extraction module is used to extract multi-dimensional feature parameters from the preprocessed speech signal; The diagnostic model module is used to generate language development diagnostic results based on the multi-dimensional feature parameters and the preset personal language development digital shadow; The feedback generation module is used to generate real-time feedback instructions based on the language development diagnosis results.

Images