A pronunciation optimization method and system based on speech visualization feedback

Abstract

The application discloses a pronunciation optimization method and system based on voice visualization feedback, and relates to the technical field of artificial intelligence assisted education. The application discloses a pronunciation optimization method and system based on voice visualization feedback, and relates to the technical field of artificial intelligence assisted education. The application discloses a pronunciation optimization method and system based on voice visualization feedback, and relates to the technical field of artificial intelligence assisted education. The application discloses a pronunciation optimization method and system based on voice visualization feedback, and relates to the technical field of artificial intelligence assisted education. The application discloses a pronunciation optimization method and system based on voice visualization feedback, and relates to the technical field of artificial intelligence assisted education. The application discloses a pronunciation optimization method and system based on voice visualization feedback, and relates to the technical field of artificial intelligence assisted education. The application discloses a pronunciation optimization method and system based on voice visualization feedback, and relates to the technical field of artificial intelligence assisted education. The application discloses a pronunciation optimization method and system based on voice visualization feedback, and relates to the technical field of artificial intelligence assisted education. The application discloses a pronunciation optimization method and system based on voice visualization feedback, and relates to the technical field of artificial intelligence assisted education. The application discloses a pronunciation optimization method and system based on voice visualization feedback, and relates to the technical field of artificial intelligence assisted education. The application discloses a pronunciation optimization method and system based on voice visualization feedback, and relates to the technical field of artificial intelligence assisted education. The application discloses a pronunciation optimization method and system based on voice visualization feedback, and relates to the technical field of artificial intelligence assisted education. The application discloses a pronunciation optimization method and system based on voice visualization feedback, and relates to the technical field of artificial intelligence assisted education. The application discloses a pronunciation optimization

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence-assisted education technology, specifically to a pronunciation optimization method and system based on voice visualization feedback. Background Technology

[0002] In language learning or speech rehabilitation training, traditional pronunciation optimization methods mainly rely on auditory guidance from professionals or learners' self-auditory feedback. These methods suffer from problems such as strong subjectivity in guidance, high costs, and low learning efficiency. Therefore, some software systems based on speech recognition or acoustic map comparison have emerged on the market.

[0003] However, these existing technological solutions generally suffer from a fundamental limitation: they all focus on judging the acoustic "results" of pronunciation, neglecting the analysis and guidance of the physiological "causes" of pronunciation. For example, the system might inform the user that the second formant frequency of a certain vowel is incorrect, but as a non-professional, the user has no idea how to adjust their tongue position, lip shape, and other articulatory organs to correct this second formant frequency. Furthermore, most existing technologies analyze isolated syllables and cannot handle the complex sound changes caused by coarticulation effects in natural speech, resulting in rigid learning outcomes that are difficult to apply to actual communication. Summary of the Invention

[0004] The purpose of this invention is to provide a pronunciation optimization method and system based on speech visualization feedback to solve the problems mentioned in the background art.

[0005] To address the aforementioned technical problems, this invention provides the following technical solution: a speech optimization system based on visual feedback, comprising a benchmark model construction module, an acoustic feature analysis module, an articulation path deviation analysis module, and a visual navigation feedback module; the benchmark model construction module, acoustic feature analysis module, articulation path deviation analysis module, and visual navigation feedback module are interconnected and communicate with each other; The baseline model construction module is used to build an offline database containing acoustic features of standard pronunciation and corresponding kinematic parameters of the articulatory organs; the acoustic feature analysis module is used to collect user pronunciation in real time and analyze its acoustic features; the articulation path deviation analysis module is used to reverse-calculate the user's actual articulatory organ movement path based on the user's real-time acoustic features, and compare it with the baseline model to obtain deviation correction instructions; the visual navigation feedback module is used to present the deviation correction instructions to the user in a dynamic and intuitive navigation format.

[0006] According to the above technical solution, the benchmark model construction module includes a standard phoneme gene library submodule and a co-articulation rule library submodule; the standard phoneme gene library submodule is used to store the standard articulation genes corresponding to a large number of syllables, and the standard articulation genes include target acoustic feature vectors and standard articulatory organ spatial position parameter vectors calibrated by the acoustic-articulation inverse model; the co-articulation rule library submodule is used to store the dynamic change rules of the standard articulation parameters of phonemes under different pre- and post-contextual contexts.

[0007] According to the above technical solution, the acoustic feature analysis module includes an audio acquisition submodule, an acoustic feature extraction submodule, and a personalized model probe submodule; the audio acquisition submodule is used to acquire user pronunciation in real time; the acoustic feature extraction submodule is used to extract its acoustic feature vector; the personalized model probe submodule is used to collect personalized acoustic data of users when distinguishing key phonemes through human-computer interaction tasks, and generate a personalized articulation mapping matrix for calibrating subsequent calculations.

[0008] According to the above technical solution, the articulation path deviation analysis module includes a baseline path query submodule, a real-time path reverse parsing submodule, and a deviation vector calculation submodule. The baseline path query submodule is used to call the co-pronunciation rule library and splice a baseline articulation path from the standard pronunciation gene library based on the user's speech flow to be practiced and its context. The real-time path reverse parsing submodule is used to reversely parse the user's real-time acoustic feature vector into the user's actual articulation path. The deviation vector calculation submodule is used to compare the user's actual articulation path with the baseline articulation path that has been personalized and calibrated, and calculate the articulation deviation vector used to guide correction.

[0009] According to the above technical solution, the visual navigation feedback module includes an oral cavity modeling submodule and a dynamic path navigation submodule; the oral cavity modeling submodule is used to construct a visual three-dimensional model of the oral cavity and articulatory organs; the dynamic path navigation submodule is used to translate the articulation deviation vector into dynamic visual instructions on the three-dimensional model, providing real-time navigation for the user's articulation organ adjustment.

[0010] According to the above technical solution, the pronunciation optimization method based on speech visualization feedback includes: Step S1: The user inputs the target speech stream to be learned, the system starts, and the baseline path query submodule generates a baseline articulation path from the standard phoneme gene library and the co-pronunciation rule library based on the phoneme context of the speech stream. Step S2: Activate the personalized model probe submodule, acquire personalized acoustic data of the user through human-computer interaction tasks, and generate a personalized articulation mapping matrix; Step S3: The user begins to speak, and the audio acquisition submodule and acoustic feature extraction submodule parse the user's acoustic feature vector sequence in real time; Step S4: The real-time path reverse parsing submodule reverse-parses the user's acoustic feature vector sequence into the user's actual articulation path; Step S5: The deviation vector calculation submodule uses the personalized articulation mapping matrix generated in step S2 to calibrate the reference articulation path generated in step S1, and then compares it with the actual articulation path obtained in step S4 to calculate the real-time articulation deviation vector sequence. Step S6: The visual navigation feedback module converts the articulation deviation vector sequence into dynamic navigation instructions on the three-dimensional virtual oral cavity model and presents them to the user.

[0011] According to the above technical solution, step S1 further includes: Step S11: Parse the target speech stream into a temporal phoneme sequence, wherein the temporal phoneme sequence includes the temporal domain division information of each phoneme; Step S12: Traverse each phoneme in the temporal phoneme sequence, and according to the phoneme and its context phonemes in the sequence, query the corresponding articulatory organ kinematic parameter sequence from the preset standard phoneme gene library; Step S13: Based on the temporal domain division information of the temporal phoneme sequence, the sequence of kinematic parameters of the articulatory organs obtained in step S12 is spliced ​​temporally to form an initial articulation path; Step S14: Apply a smoothing algorithm to the initial articulation path to generate a final reference articulation path that is continuous and smooth in the time domain.

[0012] According to the above technical solution, in step S12: The data entries stored inside the standard sound gene library include: a context phoneme key consisting of multiple phonemes, and data values ​​corresponding to the context phoneme key, which contain a set of acoustic feature sequences and a set of kinematic parameter sequences of the articulatory organs that are synchronized frame-to-frame with the context phoneme key; The context phoneme key specifically includes: the current phoneme, the preceding phoneme placed before the current phoneme, and the following phoneme placed after the current phoneme; The process of querying the standard gene library in step S12 adopts the longest context matching principle, that is, it prioritizes the use of the key containing the preceding phoneme, the current phoneme and the following phoneme for matching query. The sequence of kinematic parameters of the vocal organs consists of a set of three-dimensional parameter vectors arranged in time sequence. Each dimension of the three-dimensional parameter vector corresponds to at least one of the following three items: the anterior-posterior position of the tongue, the posterior-posterior position of the tongue, and the roundness of the lips.

[0013] According to the above technical solution, step S2 further includes: Step S21: Present the user with a set of words containing at least one pair that constitute a minimal pair in pronunciation, and collect the user's pronunciation audio when distinguishing the key distinguishing phonemes in the minimal pair in pronunciation; Step S22: Process the user's audio recording to extract a set of personalized acoustic feature vectors corresponding to the key distinguishing phonemes. ; Step S23: Obtain the standard articulation parameter vector corresponding to the key distinguishing phoneme from the standard phoneme gene library. ; Step S24: Solve using an optimization algorithm linear transformation matrix This serves as the personalized articulation mapping matrix.

[0014] According to the above technical solution, the specific steps for calculating the real-time articulation deviation vector sequence in step S5 include: Step S51: Convert the reference articulation path generated in step S1 The personalized articulation mapping matrix generated in step S2 Performing matrix multiplication yields a personalized, calibrated baseline articulation path, the calculation expression of which is: ; Step S52: Adjust the personalized calibration reference articulation path The actual articulation path of the user obtained in step S4 Perform real-time time-domain alignment; Step S53: Calculate the articulation deviation vector based on the subsequent temporal relationship. The calculation formula is as follows: ;in This is the time point aligned with time point t. Preset The diagonal perception weighted matrix.

[0015] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: The present invention reverse analyzes the actual vocal organ movement path of the user and compares it with the reference path, transforming the acoustic parameter differences abstracted from the prior art into specific three-dimensional physical space movement correction instructions. This fundamentally solves the problems of causal inversion and weak guidance in the prior art, and provides users with physiological movement guidance that can be directly imitated and executed. Attached Figure Description

[0016] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a schematic diagram of the module composition of a speech optimization system based on speech visualization feedback according to the present invention; Figure 2 This is a schematic diagram of a pronunciation optimization method based on speech visualization feedback according to the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Please see Figure 1 The present invention provides a technical solution: a pronunciation optimization system based on visual feedback, comprising a benchmark model construction module, an acoustic feature analysis module, an articulation path deviation analysis module, and a visual navigation feedback module; the benchmark model construction module, the acoustic feature analysis module, the articulation path deviation analysis module, and the visual navigation feedback module are interconnected. The baseline model building module is used to build an offline database containing the acoustic features of standard pronunciation and the corresponding kinematic parameters of the articulatory organs; the acoustic feature analysis module is used to collect user pronunciation in real time and analyze its acoustic features; the articulation path deviation analysis module is used to reverse-calculate the user's actual articulatory organ movement path based on the user's real-time acoustic features, and compare it with the baseline model to obtain deviation correction instructions; the visual navigation feedback module is used to present the deviation correction instructions to the user in a dynamic and intuitive navigation format.

[0019] The baseline model construction module includes a standard phoneme gene library submodule and a co-articulation rule library submodule. The standard phoneme gene library submodule is used to store the standard articulation genes corresponding to a large number of syllables. The standard articulation genes include the target acoustic feature vector and the standard articulation organ spatial position parameter vector calibrated by the acoustic-articulation inverse model. The co-articulation rule library submodule is used to store the dynamic changes of the standard articulation parameters of phonemes under different pre- and post-contextual contexts.

[0020] The acoustic feature analysis module includes an audio acquisition submodule, an acoustic feature extraction submodule, and a personalized model probe submodule. The audio acquisition submodule is used to acquire user pronunciation in real time. The acoustic feature extraction submodule is used to extract its acoustic feature vector. The personalized model probe submodule is used to collect personalized acoustic data of users when distinguishing key phonemes through human-computer interaction tasks, and generate a personalized articulation mapping matrix for calibrating subsequent calculations.

[0021] The articulation path deviation analysis module includes a baseline path query submodule, a real-time path reverse parsing submodule, and a deviation vector calculation submodule. The baseline path query submodule is used to call the co-pronunciation rule library and splice a baseline articulation path from the standard pronunciation gene library based on the user's speech flow to be practiced and its context. The real-time path reverse parsing submodule is used to reversely parse the user's real-time acoustic feature vector into the user's actual articulation path. The deviation vector calculation submodule is used to compare the user's actual articulation path with the baseline articulation path that has been personalized and calibrated, and calculate the articulation deviation vector used to guide the correction.

[0022] The visual navigation feedback module includes an oral cavity modeling submodule and a dynamic path navigation submodule. The oral cavity modeling submodule is used to construct a visual 3D model of the oral cavity and speech organs. The dynamic path navigation submodule is used to translate articulation deviation vectors into dynamic visual instructions on the 3D model, such as traction arrows and target area highlighting, and to provide real-time navigation for the user's speech organ adjustments.

[0023] Please see Figure 2 A pronunciation optimization method based on speech visualization feedback, the method comprising: Step S1: The user inputs the target speech stream to be learned, the system starts, and the baseline path query submodule generates a baseline articulation path from the standard phoneme gene library and the co-pronunciation rule library based on the phoneme context of the speech stream. Step S2: Activate the personalized model probe submodule, acquire personalized acoustic data of the user through human-computer interaction tasks, and generate a personalized articulation mapping matrix; Step S3: The user begins to speak, and the audio acquisition submodule and acoustic feature extraction submodule parse the user's acoustic feature vector sequence in real time. In this embodiment of the invention, the specific steps for parsing the user's acoustic feature vector sequence in real time include: dividing the user's real-time audio stream into audio frames with 25-millisecond frames and 10-millisecond frame shifts, and applying a Hamming window function to each audio frame; performing a fast Fourier transform on each windowed audio frame, and calculating multiple acoustic feature components, including Mel-frequency cepstral coefficients (MFCC), the first and second-order differences of MFCC, the fundamental frequency (F0), and the logarithmic energy, to combine them into a time-series, multi-dimensional acoustic feature vector sequence. Step S4: The real-time path reverse parsing submodule reversely parses the user's acoustic feature vector sequence into the user's actual articulation path; for example, this can be achieved by inputting the acoustic feature vector sequence into a pre-trained acoustic-articulation coupling model based on a Long Short-Term Memory (LSTM) network or a Gated Recurrent Unit (GRU); Step S5: The deviation vector calculation submodule uses the personalized articulation mapping matrix generated in step S2 to calibrate the reference articulation path generated in step S1, and then compares it with the actual articulation path obtained in step S4 to calculate the real-time articulation deviation vector sequence. Step S6: The visual navigation feedback module converts the articulation deviation vector sequence into dynamic navigation instructions on the three-dimensional virtual oral cavity model and presents them to the user; The specific steps in step S6 to convert the articulation deviation vector sequence into dynamic navigation instructions include: based on a three-dimensional virtual oral cavity model, according to the user's actual articulation path... The position and shape of the virtual vocal organs are driven in real time; starting from the current position of the virtual vocal organs, the articulation deviation vector is used. As a vector, a 3D traction arrow is generated as a dynamic navigation command; the length of the 3D traction arrow... and color Weighted magnitude of the articulation deviation vector Functional relationship, that is: This is to reflect the severity of the deviation through visual changes.

[0024] Step S1 further includes: Step S11: Parse the target speech stream into a temporal phoneme sequence, which includes the temporal domain division information of each phoneme; Step S12: Traverse each phoneme in the temporal phoneme sequence, and based on the phoneme and its context phonemes in the sequence, query the corresponding articulatory organ kinematic parameter sequence from the preset standard phoneme gene library; Step S13: Based on the temporal domain division information of the temporal phoneme sequence, the kinematic parameter sequence of the articulatory organs obtained in step S12 is spliced ​​temporally to form the initial articulation path; Step S14: Apply a smoothing algorithm to the initial articulation path to generate a final reference articulation path that is continuous and smooth in the time domain; wherein the smoothing algorithm is a cubic spline interpolation algorithm or a Kalman filter algorithm.

[0025] In step S12: The data entries stored inside the standard sound gene library include: a context phoneme key consisting of multiple phonemes, and data values ​​corresponding to the context phoneme key, which contain a set of acoustic feature sequences and a set of kinematic parameter sequences of the articulatory organs that are synchronized frame-to-frame with the context phoneme key; The context phoneme key specifically includes: the current phoneme, the preceding phoneme of the current phoneme, and the following phoneme of the current phoneme; In step S12, the process of querying the standard gene library adopts the longest context matching principle, that is, it prioritizes the use of keys containing the preceding phoneme, the current phoneme, and the following phoneme for matching queries. The sequence of kinematic parameters of the speech organs consists of a set of three-dimensional parameter vectors arranged in time sequence. Each dimension of the three-dimensional parameter vector corresponds to at least one of the following: the anteroposterior position of the tongue, the hyperextension of the tongue, and the roundness of the lips.

[0026] Step S2 further includes: Step S21: Present the user with a set of words containing at least one pair that constitutes a minimal pair in pronunciation, and collect the user's pronunciation audio when distinguishing key distinguishing phonemes in the minimal pair in pronunciation; thereby presenting the pronunciation probe task, which is presented in a gamified form of human-computer interaction in the embodiment of the invention. Step S22: Process the user's audio recordings to extract a set of personalized acoustic feature vectors corresponding to key discriminative phonemes. The process of extracting a user's personalized acoustic feature vector includes dynamically time-normalizing and aligning the user's pronunciation audio with a standard pronunciation audio; and in the aligned audio, locating the steady-state portion of key distinguishing phonemes and extracting their acoustic features to form a personalized acoustic feature vector. Step S23: Obtain the standard articulation parameter vectors corresponding to the key distinguishing phonemes from the standard phoneme gene library. ; Step S24: Solve using an optimization algorithm linear transformation matrix As a personalized articulation mapping matrix, the objective function of the optimization algorithm is to minimize the cost function J, which is defined as: in, It is an acoustic-articulation mapping model.

[0027] The specific steps for calculating the real-time articulation deviation vector sequence in step S5 include: Step S51: Convert the reference articulation path generated in step S1 The personalized articulation mapping matrix generated in step S2 Performing matrix multiplication yields a personalized, calibrated baseline articulation path, the calculation expression of which is: ; Step S52: Adjust the personalized calibration reference articulation path Compared with the actual articulation path of the user obtained in step S4 Perform real-time time-domain alignment; Step S53: Calculate the articulation deviation vector based on the subsequent temporal relationship. The calculation formula is as follows: ;in This is the time point aligned with time point t. Preset The diagonal perception weighted matrix is ​​used to penalize deviations in different articulation dimensions (such as tongue front and back, up and down, and lip rounding) with non-equal weights. Compared to existing technologies, this application goes beyond simply evaluating the acoustic "results" of pronunciation. Instead, it reverse-engineers the actual movement paths of the user's articulatory organs, directly addressing the physiological "causes" of pronunciation and providing users with precise and actionable physical guidance. To achieve accurate modeling of natural speech flow, this application generates a dynamic and continuous baseline articulation path by querying a co-pronunciation rule base and a standard phoneme gene base containing phoneme contextual relationships. This overcomes the co-pronunciation problem that existing technologies struggle to handle, ensuring the practicality of the learning outcomes. Furthermore, this application's unique personalized model probe mechanism calculates a unique articulation mapping matrix for each user through specific human-computer interaction tasks, precisely calibrating the baseline model and resolving the evaluation bias caused by traditional "one-size-fits-all" standards, achieving highly personalized guidance. Finally, through combined action, the complex articulation deviation vector is transformed into intuitive and efficient navigational visual feedback on a three-dimensional virtual oral cavity model, significantly reducing the user's cognitive load and learning cost, and achieving a significant improvement in pronunciation optimization efficiency.

[0028] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0029] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0030] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0031] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A pronunciation optimization system based on visual feedback of speech, characterized in that, The speech optimization system based on visual feedback includes a benchmark model construction module, an acoustic feature analysis module, an articulation path deviation analysis module, and a visual navigation feedback module; the benchmark model construction module, the acoustic feature analysis module, the articulation path deviation analysis module, and the visual navigation feedback module are interconnected. The baseline model construction module is used to build an offline database containing acoustic features of standard pronunciation and corresponding kinematic parameters of the articulatory organs; the acoustic feature analysis module is used to collect user pronunciation in real time and analyze its acoustic features; the articulation path deviation analysis module is used to reverse-calculate the user's actual articulatory organ movement path based on the user's real-time acoustic features, and compare it with the baseline model to obtain deviation correction instructions; the visual navigation feedback module is used to present the deviation correction instructions to the user in a dynamic and intuitive navigation format. The baseline model construction module includes a standard phoneme gene library submodule and a co-articulation rule library submodule. The standard phoneme gene library submodule is used to store the standard articulation genes corresponding to a large number of syllables. The standard articulation genes include target acoustic feature vectors and standard articulatory organ spatial position parameter vectors calibrated by the acoustic-articulation inverse model. The co-articulation rule library submodule is used to store the dynamic changes of the standard articulation parameters of phonemes under different pre- and post-contextual contexts. The acoustic feature analysis module includes an audio acquisition submodule, an acoustic feature extraction submodule, and a personalized model probe submodule. The audio acquisition submodule is used to acquire user pronunciation in real time. The acoustic feature extraction submodule is used to extract its acoustic feature vector. The personalized model probe submodule is used to collect personalized acoustic data of users when distinguishing key phonemes through human-computer interaction tasks, and generate a personalized articulation mapping matrix for calibrating subsequent calculations. The articulation path deviation analysis module includes a baseline path query submodule, a real-time path reverse parsing submodule, and a deviation vector calculation submodule. The baseline path query submodule is used to call the co-pronunciation rule library and splice a baseline articulation path from the standard pronunciation gene library based on the user's speech flow to be practiced and its context. The real-time path reverse parsing submodule is used to reversely parse the user's real-time acoustic feature vector into the user's actual articulation path. The deviation vector calculation submodule is used to compare the user's actual articulation path with the baseline articulation path that has been personalized and calibrated, and calculate the articulation deviation vector used to guide correction.

2. The pronunciation optimization system based on speech visualization feedback according to claim 1, characterized in that: The visual navigation feedback module includes an oral cavity modeling submodule and a dynamic path navigation submodule. The oral cavity modeling submodule is used to construct a visual three-dimensional model of the oral cavity and articulatory organs. The dynamic path navigation submodule is used to translate the articulation deviation vector into dynamic visual instructions on the three-dimensional model, providing real-time navigation for the user's articulation organ adjustment.

3. A pronunciation optimization method based on speech visualization feedback, characterized in that: The method includes: Step S1: The user inputs the target speech stream to be learned, the system starts, and the baseline path query submodule generates a baseline articulation path from the standard phoneme gene library and the co-pronunciation rule library based on the phoneme context of the speech stream. Step S2: Activate the personalized model probe submodule, acquire personalized acoustic data of the user through human-computer interaction tasks, and generate a personalized articulation mapping matrix; Step S3: The user begins to speak, and the audio acquisition submodule and acoustic feature extraction submodule parse the user's acoustic feature vector sequence in real time; Step S4: The real-time path reverse parsing submodule reverse-parses the user's acoustic feature vector sequence into the user's actual articulation path; Step S5: The deviation vector calculation submodule uses the personalized articulation mapping matrix generated in step S2 to calibrate the reference articulation path generated in step S1, and then compares it with the actual articulation path obtained in step S4 to calculate the real-time articulation deviation vector sequence. Step S6: The visual navigation feedback module converts the articulation deviation vector sequence into dynamic navigation instructions on the three-dimensional virtual oral cavity model and presents them to the user; Step S1 further includes: Step S11: Parse the target speech stream into a temporal phoneme sequence, wherein the temporal phoneme sequence includes the temporal domain division information of each phoneme; Step S12: Traverse each phoneme in the temporal phoneme sequence, and according to the phoneme and its context phonemes in the sequence, query the corresponding articulatory organ kinematic parameter sequence from the preset standard phoneme gene library; Step S13: Based on the temporal domain division information of the temporal phoneme sequence, the sequence of kinematic parameters of the articulatory organs obtained in step S12 is spliced ​​temporally to form an initial articulation path; Step S14: Apply a smoothing algorithm to the initial articulation path to generate a final reference articulation path that is continuous and smooth in the time domain.

4. The pronunciation optimization method based on speech visualization feedback according to claim 3, characterized in that: In step S12: The data entries stored inside the standard sound gene library include: a context phoneme key consisting of multiple phonemes, and data values ​​corresponding to the context phoneme key, which contain a set of acoustic feature sequences and a set of kinematic parameter sequences of the articulatory organs that are synchronized frame-to-frame with the context phoneme key; The context phoneme key specifically includes: the current phoneme, the preceding phoneme placed before the current phoneme, and the following phoneme placed after the current phoneme; The process of querying the standard gene library in step S12 adopts the longest context matching principle, that is, it prioritizes the use of the key containing the preceding phoneme, the current phoneme and the following phoneme for matching query. The sequence of kinematic parameters of the vocal organs consists of a set of three-dimensional parameter vectors arranged in time sequence. Each dimension of the three-dimensional parameter vector corresponds to at least one of the following three items: the anterior-posterior position of the tongue, the posterior-posterior position of the tongue, and the roundness of the lips.

5. The pronunciation optimization method based on speech visualization feedback according to claim 3, characterized in that: Step S2 further includes: Step S21: Present the user with a set of words containing at least one pair that constitute a minimal pair in pronunciation, and collect the user's pronunciation audio when distinguishing the key distinguishing phonemes in the minimal pair in pronunciation; Step S22: Process the user's audio recording to extract a set of personalized acoustic feature vectors corresponding to the key distinguishing phonemes. ; Step S23: Obtain the standard articulation parameter vector corresponding to the key distinguishing phoneme from the standard phoneme gene library. ; Step S24: Solve using an optimization algorithm linear transformation matrix This serves as the personalized articulation mapping matrix.

6. The pronunciation optimization method based on speech visualization feedback according to claim 5, characterized in that: The specific steps for calculating the real-time articulation deviation vector sequence in step S5 include: Step S51: Convert the reference articulation path generated in step S1 The personalized articulation mapping matrix generated in step S2 Performing matrix multiplication yields a personalized, calibrated baseline articulation path, the calculation expression of which is: ; Step S52: Adjust the personalized calibration reference articulation path The actual articulation path of the user obtained in step S4 Perform real-time time-domain alignment; Step S53: Calculate the articulation deviation vector based on the subsequent temporal relationship. The calculation formula is as follows: ;in This is the time point aligned with time point t. Preset The diagonal perception weighted matrix.

Images