A language learning evaluation method based on a deep learning algorithm

Abstract

The application discloses a language learning evaluation method based on a deep learning algorithm, and relates to the technical field of language processing.The method comprises the following steps: receiving a voice signal and original text content; constructing a noisy voice sample through an adversarial noise generator; inputting a pre-trained voice recognition model; outputting voice recognition text and noise resistance confidence score; constructing a context semantic graph in a semantic analysis path; generating edge attention weight; calculating a semantic coherence index; activating a grammar analysis path to match a pre-defined grammar rule library to generate a grammar error label; dynamically generating feedback content based on a multi-dimensional evaluation matrix; and updating a pre-set corresponding threshold value and rule library weight according to user adoption of the feedback content.The application realizes collaborative quantitative analysis of pronunciation, semantics and grammar problems by constructing a multi-dimensional evaluation matrix of noise resistance confidence score, semantic coherence index and grammar error label.

Description

Technical Field

[0001] This invention relates to the field of language processing technology, and in particular to a language learning evaluation method based on deep learning algorithms. Background Technology

[0002] In recent years, the integration of speech recognition and natural language processing technologies has driven the development of intelligent language learning tools. Deep learning-based speech recognition models can now achieve high-precision speech-to-text conversion, while dependency parsing and semantic understanding technologies provide fundamental support for grammar correction and semantic evaluation. Adversarial training and multimodal data fusion methods have made significant progress in robust speech processing. Existing technologies typically employ independent modules to handle speech recognition, grammar detection, and semantic analysis, and generate feedback through static rule bases or pre-trained models.

[0003] The aforementioned technologies have limitations in terms of dynamic environment adaptability and multimodal collaborative optimization: traditional speech recognition models struggle to distinguish between pronunciation errors and environmental interference in complex noisy environments, leading to distorted evaluation results; grammar and semantic analysis modules typically operate independently, lacking joint modeling of the impact of pronunciation errors and failing to generate personalized feedback for language learners. Especially in non-native language learning scenarios, pronunciation errors are often intertwined with grammatical structure and semantic logic problems. Existing technologies, lacking cross-modal dynamic threshold adjustment and closed-loop optimization mechanisms, struggle to achieve accurate multi-dimensional language ability assessment. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a language learning assessment method based on deep learning algorithms to solve the problem of difficulty in coordinating and dynamically feeding back pronunciation deviations, grammatical errors, and semantic ambiguity in language learning assessment under complex environments.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] In a first aspect, the present invention provides a language learning evaluation method based on a deep learning algorithm, which includes receiving a speech signal and original text content, constructing a noisy speech sample through an adversarial perturbation generator, inputting it into a pre-trained speech recognition model, and outputting the speech recognition text and noise resistance confidence score.

[0008] The speech recognition text is aligned with the original text content. If the difference exceeds the dynamic difference threshold, the original text content is used to generate a pronunciation deviation heatmap.

[0009] Dependency parsing is performed on the original text content to calculate the dependency tree depth and semantic ambiguity index. If either index exceeds the preset threshold, the semantic analysis path is activated; otherwise, the syntax parsing path is activated.

[0010] In the semantic analysis path, a contextual semantic graph is constructed, edge attention weights are generated, and a semantic coherence index is calculated. In the active syntax parsing path, a predefined syntax rule base is matched to generate syntax error labels.

[0011] A multidimensional evaluation matrix is ​​generated by integrating noise resistance confidence score, semantic coherence index and grammatical error label;

[0012] Feedback content is dynamically generated based on a multidimensional evaluation matrix, and the preset thresholds and rule base weights are updated according to the user's adoption of the feedback content.

[0013] As a preferred embodiment of the language learning evaluation method based on deep learning algorithms described in this invention, the specific steps for outputting the speech recognition text and noise resistance confidence score are as follows:

[0014] The received speech signal is preprocessed to generate a preprocessed speech signal, and then time-frequency decomposition is performed to generate a time-frequency matrix.

[0015] Based on the extraction of speech content sensitivity features using the time-frequency matrix, a dynamic perturbation parameter space is constructed, and an adversarial perturbation signal is generated through a variational autoencoder.

[0016] The adversarial perturbation signal is superimposed on the preprocessed speech signal to generate noisy speech samples;

[0017] Input noisy speech samples into a pre-trained speech recognition model, and output the speech recognition text and the predicted probability distribution;

[0018] The information entropy is calculated based on the predicted probability distribution. The semantic similarity between the speech recognition text and the original text content is calculated, and the semantic similarity calculation result is output.

[0019] By fusing information entropy and semantic similarity calculation results, a noise-resistant confidence score is generated.

[0020] As a preferred embodiment of the language learning evaluation method based on deep learning algorithms described in this invention, the specific steps for generating the pronunciation deviation heatmap are as follows:

[0021] The speech recognition text and the original text content are converted into phoneme sequences respectively, and phoneme-level alignment path matching is performed by dynamic time warping algorithm;

[0022] The difference between the speech recognition text and the original text is calculated based on the alignment path matching results, and a dynamic difference threshold is generated by combining the signal-to-noise ratio of the original speech signal and the speech rate of the speech recognition text.

[0023] If the difference exceeds the dynamic difference threshold, the original text content is used as the reference text, and the position in the alignment path where the difference exceeds the limit is mapped to the time axis of the original speech signal to generate a timestamp.

[0024] Based on the timestamp and the temporal distribution of the adversarial disturbance signal, the disturbance intensity of the corresponding time interval is calculated as the thermodynamic value.

[0025] The heat values ​​are superimposed onto the time-frequency graph of the original speech signal to generate a pronunciation deviation heat map.

[0026] As a preferred embodiment of the language learning evaluation method based on deep learning algorithm described in this invention, the preset corresponding threshold includes a preset dependency tree depth threshold and a preset semantic ambiguity threshold.

[0027] As a preferred embodiment of the language learning evaluation method based on deep learning algorithms described in this invention, the steps of activating the semantic analysis path and otherwise activating the syntax parsing path are as follows:

[0028] Dynamic graph neural network-driven dependency parsing is performed on the original text content to generate a dynamic weighted dependency tree, and the maximum weighted path depth index of the dependency tree is calculated.

[0029] The semantics of words in the original text are encoded, and a global semantic ambiguity index is calculated.

[0030] If the maximum weighted path depth index exceeds the preset dependency tree depth threshold or the global semantic ambiguity index exceeds the preset semantic ambiguity threshold, the semantic analysis path is activated; otherwise, the syntax parsing path is activated.

[0031] As a preferred embodiment of the language learning evaluation method based on deep learning algorithms described in this invention, the specific steps for generating grammatical error labels are as follows:

[0032] A contextual semantic graph is constructed based on a dynamic weighted dependency tree. Edge attention weights are generated by fusing the edge weights of the dynamic weighted dependency tree with the semantic embedding vector.

[0033] Based on the heatmap values ​​of the time intervals marked in the side attention weight and pronunciation deviation heatmap, the semantic coherence index is calculated by time-domain integration;

[0034] If the semantic coherence index is lower than the preset coherence threshold, the activation decision of the semantic analysis path is adjusted by weighting.

[0035] Syntax error labels are generated by matching edge weights of a dynamically weighted dependency tree to syntactic rule patterns in a predefined syntactic rule base.

[0036] As a preferred embodiment of the language learning evaluation method based on deep learning algorithms described in this invention, the specific steps for generating the multidimensional evaluation matrix are as follows:

[0037] The noise resistance confidence score, semantic coherence index, and grammatical error labels are fused using multi-dimensional tensors to generate an initial multi-dimensional evaluation matrix.

[0038] Based on the thermal values ​​of the time interval marked in the pronunciation deviation heatmap, a dynamic nonlinear transformation is performed on the initial multidimensional evaluation matrix to generate an intermediate multidimensional evaluation matrix.

[0039] The intermediate multidimensional evaluation matrix is ​​normalized, and a multidimensional evaluation matrix is ​​generated by combining the statistical results of the distribution of grammatical errors in the user's historical learning data.

[0040] As a preferred embodiment of the language learning evaluation method based on deep learning algorithms described in this invention, the specific steps for updating the preset threshold and rule base weights are as follows:

[0041] Based on the correlation distribution of noise resistance confidence score, semantic coherence index and grammatical error label in the multidimensional evaluation matrix, personalized feedback content is generated for different language dimensions.

[0042] Based on users' real-time adoption behavior of personalized feedback, the feedback adoption rate is calculated and the statistical results of the distribution of grammatical errors in users' historical learning data are updated.

[0043] Based on the updated statistical results of grammatical error distribution and the dynamic heat value of the pronunciation deviation heatmap, the preset dependency tree depth threshold, semantic ambiguity threshold, and grammar rule base weights are dynamically adjusted.

[0044] In a second aspect, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program, wherein when the computer program is executed by the processor, it implements any step of the language learning evaluation method based on deep learning algorithms as described in the first aspect of the present invention.

[0045] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the language learning evaluation method based on deep learning algorithms as described in the first aspect of the present invention.

[0046] The beneficial effects of this invention are as follows: By constructing a multi-dimensional evaluation matrix comprising noise resistance confidence score, semantic coherence index, and grammatical error label, collaborative quantitative analysis of pronunciation, semantics, and grammar issues is achieved. The dynamic difference threshold and multimodal data fusion mechanism can accurately distinguish between environmental noise and pronunciation deviations, avoiding misjudgments based on a single dimension. The path activation strategy based on contextual semantic graphs and dependency tree depth indicators can adaptively select semantic or grammatical parsing modes, improving the parsing efficiency of complex language phenomena. It enhances the relevance and real-time nature of feedback content, improving the dynamic environmental adaptability and teaching effectiveness of language learning tools. Attached Figure Description

[0047] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a flowchart of a language learning evaluation method based on deep learning algorithms.

[0049] Figure 2 A flowchart for outputting speech recognition text and noise resistance confidence score.

[0050] Figure 3 A flowchart for generating a heatmap of pronunciation deviations.

[0051] Figure 4 A flowchart for activating the semantic analysis path or the syntax parsing path. Detailed Implementation

[0052] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0053] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0054] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0055] Reference Figures 1-4This is one embodiment of the present invention, which provides a language learning evaluation method based on a deep learning algorithm, including the following steps:

[0056] S1 receives the speech signal and the original text content, constructs noisy speech samples through an adversarial perturbation generator, inputs them into a pre-trained speech recognition model, and outputs the speech recognition text and noise resistance confidence score.

[0057] Furthermore, the received speech signal is preprocessed to generate a preprocessed speech signal, which is then decomposed into a time-frequency matrix.

[0058] Specifically, the received speech signal is sampled and normalized to eliminate silence segments and suppress background noise, resulting in a preprocessed speech signal. The preprocessed speech signal is then framed according to fixed-length time windows. A window function is applied to each frame of the preprocessed speech signal to reduce spectral leakage. A Fast Fourier Transform is performed on each windowed frame of the preprocessed speech signal to obtain spectral components. The spectral components of all frames are arranged in chronological order to form a time-frequency matrix. Rows of the time-frequency matrix correspond to frequency components, columns correspond to time windows, and matrix elements are the energy amplitude values ​​for the corresponding frequency and time window, thus generating the time-frequency matrix.

[0059] Based on the extraction of speech content sensitivity features using the time-frequency matrix, a dynamic perturbation parameter space is constructed, and an adversarial perturbation signal is generated through a variational autoencoder.

[0060] It should be noted that a labeled time-frequency matrix sample set is used as training data. Multiple convolutional and pooling layers are stacked in the convolutional neural network. The convolutional layers use three-dimensional convolutional kernels to extract local features in the time domain, frequency domain, and channel dimensions. The pooling layers use max pooling to reduce the resolution of the feature maps. A batch normalization layer and a ReLU activation function are added after each convolutional layer. The difference between the predicted spectral features and the labeled spectral features is calculated using the cross-entropy loss function. The convolutional kernel weight parameters are updated by backpropagation using the Adam optimization algorithm. During training, an early stopping method is used to terminate training when the validation set loss function does not decrease for three consecutive times. The convolutional neural network weight parameters with the highest accuracy on the validation set are retained for deep spectral feature extraction of the time-frequency matrix.

[0061] Specifically, the time-frequency matrix is ​​input into a pre-trained convolutional neural network to extract deep spectral features. These deep spectral features are then pooled along the time dimension and concatenated to form a speech content-sensitive feature vector. This speech content-sensitive feature vector is input into a fully connected layer to map it into a high-dimensional space. A normalized exponential function is used to constrain the probability distribution of the high-dimensional vector, forming a dynamic perturbation parameter space. In this dynamic perturbation parameter space, an adversarial perturbation signal is reconstructed using a variational autoencoder decoder, outputting the adversarial perturbation signal in the time domain.

[0062] The adversarial perturbation signal is superimposed on the preprocessed speech signal to generate noisy speech samples;

[0063] Specifically, the adversarial perturbation signal and the preprocessed speech signal are time-domain aligned to ensure a perfect time-axis match. The adversarial perturbation signal and the preprocessed speech signal are then added point-by-point in the time domain to obtain a mixed signal. The amplitude of the mixed signal is normalized so that the maximum amplitude does not exceed a preset maximum amplitude threshold. The output is a noisy speech sample that conforms to the characteristics of time-domain continuity and frequency-domain perturbation.

[0064] It should be noted that the preset maximum amplitude threshold is based on the dynamic range of the preprocessed speech signal and the upper limit of digital signal quantization; the example value is 0.9.

[0065] Input noisy speech samples into a pre-trained speech recognition model, and output the speech recognition text and the predicted probability distribution;

[0066] Specifically, noisy speech samples are divided into time windows with the same frame length and window function as the preprocessed speech signal to generate short-time spectral sequences. These short-time spectral sequences are then input into the spectral feature encoder of a pre-trained speech recognition model to convert them into high-dimensional feature vector sequences. The temporal classification network of the pre-trained speech recognition model performs multi-class probability prediction on the high-dimensional feature vector sequences to generate a prediction probability distribution matrix. The rows of the prediction probability distribution matrix correspond to time windows, the columns correspond to candidate words, and the element values ​​are the probability values ​​of different words in each time window. A greedy search or bundle search decoding algorithm is applied to the prediction probability distribution matrix to decode the word sequence with the highest comprehensive probability, which is then used as the speech recognition text.

[0067] It should be noted that short-time spectral sequences of labeled speech samples are used as input data. In the speech recognition model, encoder and decoder structures are stacked. The encoder consists of multiple convolutional layers and a bidirectional long short-term memory network to extract temporal spectral features. The decoder maps the hidden state sequence output by the encoder to a text sequence through a multi-head attention mechanism. The alignment loss between the predicted text sequence and the labeled text sequence is calculated using a connectionist temporal classification loss function. The network parameters are updated using the Adam optimization algorithm combined with a learning rate decay strategy. During training, an early stopping method is used to monitor the change in the validation set word error rate. Training is terminated when the validation set word error rate does not decrease for five consecutive rounds. The probability distribution output by the decoder is reordered using a beam search algorithm, and the speech recognition model parameters with the lowest validation set word error rate are retained for the short-time spectral sequence recognition task.

[0068] The information entropy is calculated based on the predicted probability distribution. The semantic similarity between the speech recognition text and the original text content is calculated, and the semantic similarity calculation result is output.

[0069] Specifically, the information entropy is calculated based on the predicted probability distribution, and the expression is:

[0070] ;

[0071] In the formula, Represents information entropy. Represents a time variable. Indicates the total duration. Represented as a time interval The entropy contributions at all times within the timeframe are summed. Represented as a Gaussian decay function over time. The weight, This indicates that the probability distributions of all candidate words are iterated and summed. Indicates the total number of candidate words. Indicates the index of candidate words. Represented as a speech recognition model in the time variable For the first The probability distribution of candidate words, This represents the signal-to-noise ratio adjustment factor. Indicates the time variable of noisy speech samples The signal-to-noise ratio, Represents the time variable Perform integration and differentiation. This represents the natural logarithm function.

[0072] It should be noted that, ,in The total number of candidate words. This indicates that the speech recognition model is completely determined. This indicates that the speech recognition model is completely uncertain.

[0073] The semantic similarity between the speech-recognized text and the original text is calculated using the following expression:

[0074] ;

[0075] In the formula, This represents the semantic similarity calculation result. This represents the aligned first line in the speech recognition text. Word vectors for each word pair position. This represents the aligned first line in the original text content. Word vectors for each word pair position. This represents the word pair weighting function. This represents summing the total number of word pairs after aligning all speech recognition text with the original text. Indicates the position index of word pairs. The term "total" indicates quantity or total number. The squared Euclidean distance between the word vectors of the speech recognition text and the word vectors of the original text content after nonlinear projection is expressed as follows: Represents a nonlinear projection function. This represents the aligned first line in the speech recognition text. The non-linear projection result of word vectors at each position. This represents the aligned first line in the original text content. The non-linear projection result of word vectors at each position. This represents the aligned first line in the speech recognition text. The squared Euclidean norm of the nonlinear projection of word vectors to positions of each word. This represents the aligned first line in the original text content. The squared Euclidean norm of the nonlinear projection of word vectors to positions of each word.

[0076] It should be noted that, ,in This indicates complete agreement (all words overlap after projection). This indicates a complete contradiction (projection directions are opposite).

[0077] By fusing information entropy and semantic similarity calculation results, a noise-resistant confidence score is generated.

[0078] ;

[0079] In the formula, This indicates the noise immunity confidence score. Indicates the weighting coefficient. Indicates the total number of candidate words. Represents the error function. This represents the mean of information entropy. It represents the standard deviation of information entropy.

[0080] It should be noted that, ,in This indicates complete credibility (low entropy + high similarity). This indicates that the information is completely unreliable.

[0081] The weighting coefficient is derived from the optimization of the contribution balance between information entropy and semantic similarity, with an example value of 0.7.

[0082] S2. Align the speech recognition text with the original text content based on the differences. If the difference exceeds the dynamic difference threshold, use the original text content and generate a pronunciation deviation heatmap.

[0083] Furthermore, the speech recognition text and the original text content are converted into phoneme sequences respectively, and phoneme-level alignment path matching is performed through a dynamic time warping algorithm;

[0084] Specifically, an International Phonetic Alphabet (IPA) conversion tool is used to map each word in the speech recognition text to its corresponding phoneme sequence. The same IPA conversion tool is used to map each word in the original text content to its corresponding phoneme sequence, ensuring consistency between the phoneme symbol table and the pronunciation rules. The edit distance matrix between the phoneme sequence of the speech recognition text and the phoneme sequence of the original text content is calculated as the alignment cost matrix. Dynamic programming is used to solve for the minimum cumulative cost path along the diagonal of the cost matrix, and the optimal path is backtracked to generate the phoneme-level alignment path.

[0085] The difference between the speech recognition text and the original text is calculated based on the alignment path matching results, and a dynamic difference threshold is generated by combining the signal-to-noise ratio of the original speech signal and the speech rate of the speech recognition text.

[0086] Specifically, the proportion of non-matching nodes between the phonemes of the speech recognition text and the phonemes of the original text content in the phoneme-level alignment path is used as the basic difference. The speech rate of the speech recognition text is calculated as the ratio of the total number of words in the speech recognition text to the total duration of the original speech signal. The signal-to-noise ratio of the original speech signal is compressed logarithmically to obtain a normalized signal-to-noise ratio parameter. The basic difference, the normalized signal-to-noise ratio parameter, and the speech rate parameter are input into a linear weighted formula to generate a dynamic difference threshold.

[0087] It should be noted that the dynamic difference threshold is set based on a linear weighted formula of the base difference, the normalized signal-to-noise ratio parameter, and the speech rate parameter. For example, the dynamic difference threshold is 0.35 when the base difference is 0.2, the normalized signal-to-noise ratio parameter is 0.5, and the speech rate parameter is 0.3.

[0088] If the difference exceeds the dynamic difference threshold, the original text content is used as the reference text, and the position in the alignment path where the difference exceeds the limit is mapped to the time axis of the original speech signal to generate a timestamp.

[0089] Specifically, the difference between each node in the phoneme-level alignment path between the speech recognition text and the original text content is compared with a dynamic difference threshold. All nodes whose differences exceed the dynamic difference threshold are marked. The start and end times of the original speech signal corresponding to the nodes with excessive differences are obtained according to the time correspondence of the phoneme-level alignment path. Continuous time intervals with excessive differences are merged into time segments, and a timestamp is generated for each time segment.

[0090] Based on the timestamp and the temporal distribution of the adversarial disturbance signal, the disturbance intensity of the corresponding time interval is calculated as the thermodynamic value.

[0091] It should be noted that the time intervals of difference exceeding the limit recorded by the timestamp are time-axis aligned with the intensity distribution of the adversarial perturbation signal in the time domain. The integral of the absolute value of the time-domain amplitude of the adversarial perturbation signal within the time interval from the start time to the end time corresponding to each timestamp is taken as the perturbation intensity of that time interval. The perturbation intensity and the time interval covered by the timestamp together constitute the basis for calculating the thermal value. The thermal value reflects the degree of energy accumulation of the adversarial perturbation signal in the time interval of difference exceeding the limit and is positively correlated with the degree of pronunciation deviation.

[0092] Specifically, based on the timestamp and the temporal distribution of the adversarial perturbation signal, the perturbation intensity for the corresponding time interval is calculated as the thermodynamic value, expressed as:

[0093] ;

[0094] In the formula, Indicates the heat value, Represented as time interval The internal heat value, Indicates the end time. Indicates the start time. Indicates from arrive Perform cumulative summation. Represented as the adversarial disturbance signal at the current time Energy density at that time Indicates the difference enhancement factor. Indicates the current time Dynamic differences over time Indicates the current time Accumulate points. This indicates the original speech signal. The signal energy within is accumulated globally. Represents the regularization constant. Indicates the end time to start time Time difference, This represents the global maximum sustained pronunciation duration of the text in the speech recognition.

[0095] The heat values ​​are superimposed onto the time-frequency graph of the original speech signal to generate a pronunciation deviation heat map.

[0096] Specifically, the time-frequency matrix of the original speech signal is normalized and converted into a grayscale image as the base layer. The heat values ​​corresponding to the timestamps are mapped to a heat value matrix of the same size as the base layer using a bilinear interpolation algorithm. The heat value matrix is ​​then input into a heat-color mapping function to generate a color coding matrix. In the color coding matrix, the intensity of the red channel is positively correlated with the heat value, while the intensity of the green and blue channels is negatively correlated with the heat value. The color coding matrix and the grayscale base layer are then pixel-level superimposed using a transparency blending formula. In the superimposed image, high-frequency energy areas and high heat value areas overlap and are displayed as red patches, while low-frequency energy areas and low heat value areas are displayed as a blue-green background, generating a pronunciation deviation heatmap.

[0097] It should be noted that if the difference does not exceed the dynamic difference threshold, speech recognition text is used to generate a pronunciation deviation heatmap.

[0098] S3. Perform dependency parsing on the original text content, calculate the dependency tree depth and semantic ambiguity index. If either index exceeds the preset threshold, activate the semantic analysis path; otherwise, activate the syntax parsing path.

[0099] The preset thresholds include a preset dependency tree depth threshold and a preset semantic ambiguity threshold.

[0100] It should be noted that the preset dependency tree depth threshold is based on the normal distribution statistics of the maximum weighted path depth index in the training corpus of the dynamically weighted dependency tree. The example value is 3.5, which is the mean of the maximum weighted path depth index in the training corpus plus twice the standard deviation. The preset semantic ambiguity threshold is based on the classification boundary between clear semantic samples and ambiguous semantic samples of the global semantic ambiguity index generated by the quantum encoder. The example value is 0.8, which is the maximum value of the global semantic ambiguity index of clear semantic samples.

[0101] Dynamic graph neural network-driven dependency parsing is performed on the original text content to generate a dynamic weighted dependency tree, and the maximum weighted path depth index of the dependency tree is calculated.

[0102] Specifically, each word in the original text is converted into a word vector. The word vectors and their positions in the sentence are concatenated to form a node feature vector. The dependency relationship types in the dependency syntax rule base are encoded as edge type vectors, forming a fully connected graph structure with words as nodes and dependency relationships as edges. The node feature vectors and edge type vectors are input to the graph attention layer for multi-hop message passing and feature aggregation. The dependency relationship strength between nodes is calculated using a multilayer perceptron as the edge weight. Based on the edge weights and the legal dependency relationship constraints in the dependency syntax rule base, a dynamic weighted dependency tree is generated using the maximum spanning tree algorithm.

[0103] The maximum weighted path depth index of the dependency tree is calculated using a dynamically weighted dependency tree, expressed as:

[0104] ;

[0105] In the formula, This represents the maximum weighted path depth metric. Indicates the maximum. This represents the complete syntactic relation chain from the root node to any leaf node in a dependency syntax tree. Indicates a path. Represents a set of paths. Indicates path Each dependency edge in Perform the accumulation operation. Indicates the parent node index. Indicates the child node index. Indicates the parent node and child nodes Dynamic edge weights, Indicates the parent node words Importance weight in the corpus Represents child nodes words Importance weight in the corpus Represents words.

[0106] The semantics of words in the original text are encoded, and a global semantic ambiguity index is calculated.

[0107] It should be noted that each word in the original text is converted into a corresponding word vector. The word vectors are then processed with context awareness to capture the semantic dependencies between words in the context of the sentence. The output hidden state vector is concatenated with the word vector to form a context-enhanced word vector. The context-enhanced word vector is then processed with layer normalization to eliminate feature scale differences. Finally, the normalized word vector is fused with the word part-of-speech tag embedding vector to generate a semantic encoding vector.

[0108] Specifically, the semantics of the words in the original text are encoded, and a global semantic ambiguity index is calculated, expressed as follows:

[0109] ;

[0110] In the formula, Represented as a global semantic ambiguity index, Represents the normalization coefficient. This indicates that the summation is performed by iterating through all words in the original text. This represents the total number of words in the original text. This represents the word index in the original text content. Represents a probability distribution. Indicates the first... The word in the first probability distribution on each ground state Indicates the ground state index. Represents the total number of ground states. This indicates that the summation is performed over all ground states. Indicates the first... The variance of the context embedding vector of each word. Indicates the first... The context embedding vector of each word. This represents the natural exponential function. This represents the temperature coefficient.

[0111] It should be noted that the temperature coefficient originates from the Boltzmann distribution in quantum statistical mechanics, for example, taking a value of... The normalization coefficient is derived from the total number of words in the original text. The reciprocal of the original text, for example, if the original text has 100 words, the value is 0.01.

[0112] If the maximum weighted path depth index exceeds the preset dependency tree depth threshold or the global semantic ambiguity index exceeds the preset semantic ambiguity threshold, the semantic analysis path is activated; otherwise, the syntax parsing path is activated.

[0113] Specifically, the maximum weighted path depth index is compared with a preset dependency tree depth threshold, and the global semantic ambiguity index is compared with a preset semantic ambiguity threshold. If either index exceeds the corresponding threshold, the semantic analysis path is activated to enter the context semantic graph construction and semantic coherence index calculation process. If neither index exceeds the corresponding threshold, the syntax parsing path is activated to enter the predefined syntax rule base matching and syntax error tag generation process. The activation states of the semantic analysis path and the syntax parsing path are mutually exclusive, and only one path's processing logic is executed.

[0114] S4. Construct a contextual semantic graph in the semantic analysis path, generate edge attention weights, calculate the semantic coherence index, and generate grammatical error labels by matching a predefined grammar rule base in the active syntax parsing path.

[0115] Furthermore, a contextual semantic graph is constructed based on a dynamically weighted dependency tree, and edge attention weights are generated by fusing the edge weights of the dynamically weighted dependency tree with the semantic embedding vector.

[0116] Specifically, the nodes of the dynamically weighted dependency tree correspond to the words in the original text content, and the edge connections of the dynamically weighted dependency tree are retained as the initial topology of the context semantic graph. The edge weights of the dynamically weighted dependency tree are concatenated with the semantic encoding vectors of the corresponding words to generate edge feature vectors. The edge feature vectors are then input into a pre-trained two-layer fully connected neural network to calculate the edge attention weights. The first layer of the two-layer fully connected neural network uses the ReLU activation function to achieve nonlinear transformation, and the second layer uses a normalized exponential function to probabilistically normalize the edge attention weights. The edge attention weights of the context semantic graph reflect the joint importance of grammatical structure and semantic association.

[0117] It should be noted that a set of labeled edge feature vector samples is used as input data. In the first layer of the two-layer fully connected neural network, a linear transformation and the LeakyReLU activation function are used to perform a non-linear mapping on the edge feature vectors. The second layer outputs scalar attention weight values ​​through a linear layer. The difference between the predicted edge attention weights and the labeled edge attention weights is calculated using the mean squared error loss function. The network weight parameters are updated by combining a gradient pruning strategy. During training, an early stopping method is used to monitor the change of the mean squared error of the validation set. Training is terminated when the mean squared error of the validation set does not decrease for ten consecutive rounds. The weight parameters of the two-layer fully connected neural network with the smallest mean squared error of the validation set are retained for the edge attention weight calculation task.

[0118] Based on the heatmap values ​​of the time intervals marked in the side attention weight and pronunciation deviation heatmap, the semantic coherence index is calculated by time-domain integration;

[0119] It should be noted that the edge attention weight is the weight parameter of each grammatical dependency relation edge in the dynamic weighted dependency tree in the context semantic graph. It is generated by fusing the edge weights of the dynamic weighted dependency tree with the semantic encoding vector of the corresponding word. The heat value of the time interval marked in the pronunciation deviation heat map is the integral of the absolute value of the temporal amplitude of the adversarial perturbation signal within the time interval of difference exceeding the limit. The heat value of the time interval and the edge attention weight are matched by time axis alignment to form a weighting factor. The weighting factor is used to calculate the joint influence weight of grammatical structure and pronunciation deviation. The joint influence weight is used as the temporal integral kernel function parameter in the semantic coherence index calculation formula.

[0120] Specifically, the semantic coherence index is calculated using time-domain integration, and the expression is as follows:

[0121] ;

[0122] In the formula, Indicates the semantic coherence index. Represented as the set of opposite edges Each dependency edge in Perform the accumulation operation. Denotes the set of edges. Indicates dependent edges At the center of the timeline, Indicates the center position. Indicates the time decay factor. This represents the total number of edges in the context semantic graph.

[0123] If the semantic coherence index is lower than the preset coherence threshold, the activation decision of the semantic analysis path is adjusted by weighting.

[0124] It should be noted that the preset coherence threshold is based on the orthogonal decomposition results of the statistical distribution of the semantic coherence index of a large-scale clear semantic sample and the heat value of the pronunciation deviation. The example value is 0.6, which is the lower limit of the semantic coherence index of the clear semantic sample within the 95% confidence interval.

[0125] Specifically, the deviation is calculated by subtracting the semantic coherence index from a preset coherence threshold. The deviation is then input into a Sigmoid function to generate a corrected weight coefficient. This corrected weight coefficient is then weighted and fused with the global semantic ambiguity index and pronunciation deviation heat value in the semantic analysis path. The edge weight distribution of the dynamically weighted dependency tree is adjusted using a probability normalization function. Based on the adjusted edge weights, the edge attention weights of the context semantic graph are recalculated. The corrected semantic coherence index is generated by performing a second time-domain integration using the updated edge attention weights and the time interval heat value of the pronunciation deviation heat value. If the corrected semantic coherence index is still lower than the preset coherence threshold, the activation weight coefficient of the semantic analysis path is reduced to the square root of the original value, and the upper limit of the activation probability of the syntax parsing path is limited. Finally, the activation decision result of the semantic analysis path is output.

[0126] Syntax error labels are generated by matching edge weights of a dynamically weighted dependency tree to syntactic rule patterns in a predefined syntactic rule base.

[0127] It should be noted that the predefined grammar rule base refers to a structured database that stores the mapping relationships between legal grammatical dependency relation types, legal intervals of dependency relation edge weights, and grammatical anomaly types. The legal interval of dependency relation edge weights is defined as a pre-selected quantile interval calculated by analyzing the statistical distribution of edge weights of the same type of dependency relation in the statistically normalized corpus. When the edge weight falls within this interval, it is determined to be a legal dependency relation edge; otherwise, it is marked as a grammatical anomaly edge.

[0128] Specifically, the edge weight of each dependency relation edge in the dynamic weighted dependency tree is compared with the valid range of dependency relation edge weights in the predefined syntax rule base. If the edge weight is lower than the minimum valid weight threshold or higher than the maximum valid weight threshold defined in the syntax rule base, it is determined to be a syntax error edge. According to the exception type mapping relationship in the syntax rule base, the dependency relation type corresponding to the exception edge is converted into a syntax error code. After classifying and statistically analyzing the syntax error codes according to the error type identifier, syntax error labels are generated.

[0129] S5. A multi-dimensional evaluation matrix is ​​generated by integrating noise resistance confidence score, semantic coherence index and grammatical error label.

[0130] Furthermore, the noise resistance confidence score, semantic coherence index, and grammatical error labels are fused using multi-dimensional tensors to generate an initial multi-dimensional evaluation matrix.

[0131] Specifically, the noise resistance confidence score is converted into a one-dimensional feature vector aligned with the semantic coherence index. The error type identifier and weight bias in the syntax error label set are encoded as sparse and dense vectors, respectively. The noise resistance confidence score feature vector, semantic coherence index feature vector, syntax error type sparse vector, and weight bias dense vector are concatenated along the newly added dimension to form a fourth-order tensor through tensor concatenation. The normalized fourth-order tensor is input into a pre-trained fully connected neural network for cross-dimensional feature mapping to generate a three-dimensional evaluation tensor, thus generating the initial multi-dimensional evaluation matrix.

[0132] It should be noted that a labeled normalized fourth-order tensor sample set is used as input data. A linear transformation and ReLU activation function are used between the input and hidden layers of the fully connected neural network to achieve cross-dimensional feature mapping of the fourth-order tensor. A linear transformation is used between the hidden and output layers to generate a three-dimensional evaluation tensor. The difference between the predicted and labeled three-dimensional evaluation tensors is calculated using the mean squared error loss function. The network weight parameters are updated using the Adam optimization algorithm combined with a learning rate scheduling strategy. During training, an early stopping method is used to monitor the change in the mean squared error of the validation set. Training is terminated when the mean squared error of the validation set does not decrease for eight consecutive rounds. The weight parameters of the fully connected neural network with the smallest mean squared error of the validation set are retained for the cross-dimensional feature mapping task from the normalized fourth-order tensor to the three-dimensional evaluation tensor.

[0133] Based on the thermal values ​​of the time interval marked in the pronunciation deviation heatmap, a dynamic nonlinear transformation is performed on the initial multidimensional evaluation matrix to generate an intermediate multidimensional evaluation matrix.

[0134] Specifically, the heat values ​​of each time interval in the pronunciation deviation heatmap are mapped to the time axis dimension of the initial multidimensional evaluation matrix using a bilinear interpolation algorithm, generating a time-weighted heat value distribution matrix. The heat value distribution matrix is ​​then subjected to a Hadamard product operation with the initial multidimensional evaluation matrix to obtain a weighted evaluation matrix. The weighted evaluation matrix is ​​input into the encoder part of the variational autoencoder to extract the latent space feature vector. The latent space feature vector is converted into a dynamic scaling factor and a shift factor using the Sigmoid function. The dynamic scaling factor is then multiplied element-wise with the initial multidimensional evaluation matrix. Finally, the shift factor is added element-wise to the product result to output an intermediate multidimensional evaluation matrix.

[0135] The intermediate multidimensional evaluation matrix is ​​normalized, and a multidimensional evaluation matrix is ​​generated by combining the statistical results of the distribution of grammatical errors in the user's historical learning data.

[0136] Specifically, the feature scale difference of each dimension of the intermediate multidimensional evaluation matrix is ​​eliminated, the statistical results of the distribution of syntax errors in the user's historical learning data are encoded into weight vectors according to error type identifiers and occurrence frequency, the weight vectors are subjected to Hadamard product operation with the normalized intermediate multidimensional evaluation matrix to obtain a weighted evaluation matrix, the weighted evaluation matrix is ​​input into a fully connected neural network for feature mapping to generate a latent matrix that integrates historical statistical features, and the row and column distribution of the latent matrix is ​​probabilistically normalized by a normalization exponential function to generate a multidimensional evaluation matrix.

[0137] S6. Dynamically generate feedback content based on a multi-dimensional evaluation matrix, and update the preset threshold and rule base weights according to the user's adoption of the feedback content.

[0138] Furthermore, based on the correlation distribution of noise resistance confidence score, semantic coherence index and grammatical error label in the multidimensional evaluation matrix, personalized feedback content is generated for different language dimensions.

[0139] Specifically, the time interval with the highest noise resistance confidence score weight is extracted from the multidimensional evaluation matrix as the pronunciation enhancement feedback interval, and the corresponding heat value of the time interval is combined with the pronunciation deviation heatmap to generate pronunciation accuracy improvement suggestions; the context paragraphs of the semantic coherence index are extracted as semantic coherence optimization paragraphs, and the edge attention weight distribution of the dynamic weighted dependency tree is combined to generate context logic enhancement suggestions; the error type with the highest frequency in the grammatical error label set is extracted as grammatical error correction priority label; the pronunciation enhancement feedback interval, semantic coherence optimization paragraphs, and grammatical error correction priority labels are mapped to the natural language feedback template to generate personalized feedback content.

[0140] It should be noted that the error type is defined as the set of syntax error codes corresponding to the syntax exception type mapping stored in the predefined syntax rule base.

[0141] Natural language feedback templates refer to a template library that stores three types of structured text: pronunciation improvement suggestions, semantic optimization prompts, and grammar corrections.

[0142] Based on users' real-time adoption behavior of personalized feedback, the feedback adoption rate is calculated and the statistical results of the distribution of grammatical errors in users' historical learning data are updated.

[0143] Specifically, the system records the number of times users clicked on the timestamps of pronunciation enhancement feedback intervals in personalized feedback content, the duration of repeated readings, the duration of viewing semantic coherence optimization paragraphs, and the number of confirmation operations for grammar correction priority tags. The number of each operation is compared with the total number of push notifications to calculate the feedback adoption rate. The feedback adoption rate is then weighted and updated with the grammar error distribution statistics in the user's historical learning data. The weight deviation of the corresponding error type identifiers in the grammar error tag set is adjusted, and the updated grammar error distribution statistics generate a new version of the user's historical learning data. The new version of the data is stored in the grammar error distribution statistics database, and the historical weights of the grammar error distribution statistics are dynamically adjusted with the feedback adoption rate and updated synchronously with the user's learning behavior data.

[0144] Based on the updated statistical results of grammatical error distribution and the dynamic heat value of the pronunciation deviation heatmap, the preset dependency tree depth threshold, semantic ambiguity threshold, and grammar rule base weights are dynamically adjusted.

[0145] Specifically, a sliding window is used to calculate the rate of change of grammatical error frequency and the integral of pronunciation heat value as adjustment factors. The preset dependency tree depth threshold is dynamically adjusted through linear interpolation. K-means clustering analysis is applied to the semantic ambiguity index, and the median of the cluster center distance is used to update the preset semantic ambiguity threshold. Combining error type and pronunciation heat value, Bayesian update is used to adjust the boundary of the weight interval of the grammar rule base. The minimum and maximum values ​​are inversely proportional to the error type and directly proportional to the heat value, respectively. The scaling factor is optimized through gradient descent to generate the updated weight parameters of the grammar rule base.

[0146] This embodiment also provides a computer device applicable to language learning evaluation methods based on deep learning algorithms, comprising: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the language learning evaluation method based on deep learning algorithms as proposed in the above embodiment.

[0147] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0148] This embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements the language learning evaluation method based on deep learning algorithms as proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0149] In summary, this invention achieves collaborative quantitative analysis of pronunciation, semantics, and grammar issues by constructing a multi-dimensional evaluation matrix comprising noise resistance confidence scores, semantic coherence indices, and grammatical error labels. Dynamic difference thresholds and multimodal data fusion mechanisms accurately distinguish between environmental noise and pronunciation deviations, avoiding misjudgments based on a single dimension. A path activation strategy based on contextual semantic graphs and dependency tree depth indicators adaptively selects semantic or grammatical parsing modes, improving the parsing efficiency of complex language phenomena. Enhanced relevance and real-time nature of feedback content improves the dynamic environmental adaptability and teaching effectiveness of language learning tools.

[0150] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A language learning evaluation method based on deep learning algorithms, characterized in that: include, It receives speech signals and original text content, constructs noisy speech samples through an adversarial perturbation generator, inputs them into a pre-trained speech recognition model, and outputs speech recognition text and noise resistance confidence score. The speech recognition text is aligned with the original text content based on differences. If the difference exceeds a dynamic difference threshold, the original text content is used to generate a pronunciation deviation heatmap. The specific steps are as follows. The speech recognition text and the original text content are converted into phoneme sequences respectively, and phoneme-level alignment path matching is performed by dynamic time warping algorithm; The difference between the speech recognition text and the original text is calculated based on the alignment path matching results, and a dynamic difference threshold is generated by combining the signal-to-noise ratio of the original speech signal and the speech rate of the speech recognition text. If the difference exceeds the dynamic difference threshold, the original text content is used as the reference text, and the position in the alignment path where the difference exceeds the limit is mapped to the time axis of the original speech signal to generate a timestamp. Based on the timestamp and the temporal distribution of the adversarial disturbance signal, the disturbance intensity of the corresponding time interval is calculated as the thermodynamic value. The heat values ​​are superimposed onto the time-frequency graph of the original speech signal to generate a pronunciation deviation heat map; Dependency parsing is performed on the original text content to calculate the dependency tree depth and semantic ambiguity index. If either index exceeds the preset threshold, the semantic analysis path is activated; otherwise, the syntax parsing path is activated. The semantic analysis path constructs a contextual semantic graph, generates edge attention weights, calculates a semantic coherence index, and generates grammatical error labels by matching a predefined grammar rule base in the activation grammar parsing path. The specific steps are as follows: A contextual semantic graph is constructed based on a dynamic weighted dependency tree. Edge attention weights are generated by fusing the edge weights of the dynamic weighted dependency tree with the semantic embedding vector. Based on the heatmap values ​​of the time intervals marked in the side attention weight and pronunciation deviation heatmap, the semantic coherence index is calculated by time-domain integration; If the semantic coherence index is lower than the preset coherence threshold, the activation decision of the semantic analysis path is adjusted by weighting. Based on the edge weight matching of the dynamic weighted dependency tree, a grammar rule pattern in a predefined grammar rule base is generated to produce grammar error labels; A multidimensional evaluation matrix is ​​generated by integrating noise resistance confidence score, semantic coherence index and grammatical error label; Feedback content is dynamically generated based on a multidimensional evaluation matrix, and the preset thresholds and rule base weights are updated according to the user's adoption of the feedback content.

2. The language learning evaluation method based on deep learning algorithms as described in claim 1, characterized in that: The specific steps for outputting the speech recognition text and noise resistance confidence score are as follows. The received speech signal is preprocessed to generate a preprocessed speech signal, and then time-frequency decomposition is performed to generate a time-frequency matrix. Based on the extraction of speech content sensitivity features using the time-frequency matrix, a dynamic perturbation parameter space is constructed, and an adversarial perturbation signal is generated through a variational autoencoder. The adversarial perturbation signal is superimposed on the preprocessed speech signal to generate noisy speech samples; Input noisy speech samples into a pre-trained speech recognition model, and output the speech recognition text and the predicted probability distribution; The information entropy is calculated based on the predicted probability distribution. The semantic similarity between the speech recognition text and the original text content is calculated, and the semantic similarity calculation result is output. By fusing information entropy and semantic similarity calculation results, a noise-resistant confidence score is generated.

3. The language learning evaluation method based on deep learning algorithms as described in claim 1, characterized in that: The preset corresponding thresholds include a preset dependency tree depth threshold and a preset semantic ambiguity threshold.

4. The language learning evaluation method based on deep learning algorithms as described in claim 3, characterized in that: The steps are as follows: Activate the semantic analysis path if necessary, otherwise activate the syntax parsing path. Dynamic graph neural network-driven dependency parsing is performed on the original text content to generate a dynamic weighted dependency tree, and the maximum weighted path depth index of the dependency tree is calculated. The semantics of words in the original text are encoded, and a global semantic ambiguity index is calculated. If the maximum weighted path depth index exceeds the preset dependency tree depth threshold or the global semantic ambiguity index exceeds the preset semantic ambiguity threshold, the semantic analysis path is activated; otherwise, the syntax parsing path is activated.

5. The language learning evaluation method based on deep learning algorithms as described in claim 1, characterized in that: The specific steps for generating the multidimensional evaluation matrix are as follows: The noise resistance confidence score, semantic coherence index, and grammatical error labels are fused using multi-dimensional tensors to generate an initial multi-dimensional evaluation matrix. Based on the thermal values ​​of the time interval marked in the pronunciation deviation heatmap, a dynamic nonlinear transformation is performed on the initial multidimensional evaluation matrix to generate an intermediate multidimensional evaluation matrix. The intermediate multidimensional evaluation matrix is ​​normalized, and a multidimensional evaluation matrix is ​​generated by combining the statistical results of the distribution of grammatical errors in the user's historical learning data.

6. The language learning evaluation method based on deep learning algorithms as described in claim 5, characterized in that: The specific steps for updating the preset threshold and rule base weights are as follows: Based on the correlation distribution of noise resistance confidence score, semantic coherence index and grammatical error label in the multidimensional evaluation matrix, personalized feedback content is generated for different language dimensions. Based on users' real-time adoption behavior of personalized feedback, the feedback adoption rate is calculated and the statistical results of the distribution of grammatical errors in users' historical learning data are updated. Based on the updated statistical results of grammatical error distribution and the dynamic heat value of the pronunciation deviation heatmap, the preset dependency tree depth threshold, semantic ambiguity threshold, and grammar rule base weights are dynamically adjusted.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the language learning evaluation method based on deep learning algorithms as described in any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the language learning evaluation method based on deep learning algorithms as described in any one of claims 1 to 6.

Images