A method, system and apparatus for assisting in training oral language

Abstract

The application belongs to the field of auxiliary learning, and particularly relates to a method, system and device for assisting in training oral language, tone, loudness and timbre data of pronunciation of each word or phrase in each sentence in a historical audio database and a standard audio database are collected; tone, loudness and timbre data of words or phrases in sentences in the standard sentence and the historical audio database are respectively counted; a voiceprint vector of tone, loudness range and timbre of words or phrases with correct pronunciation is acquired, and a three-dimensional pronunciation interval generation model is established; an identification range of correct pronunciation data is set, and a pronunciation identification margin is established by respectively based on the three-dimensional pronunciation interval generation model of the historical audio database and the standard audio database; when pronunciation of a learner falls into the set voiceprint vector of tone, loudness range and timbre, accurate identification of whether the pronunciation is correct can be realized in the case that the pronunciation is not quite standard, so that the accuracy and precision of pronunciation identification are effectively improved.

Description

Technical Field

[0001] This application belongs to the field of assisted learning, and specifically relates to a method, system and device for assisting in oral language training. Background Technology

[0002] With the widespread adoption of smart learning devices, self-directed oral English practice at home has become increasingly popular, supporting recitation training in multiple languages, including Chinese and English. However, home practice often lacks in-person teacher guidance, and pronunciation assessment and correction primarily rely on automated devices.

[0003] Current mainstream oral assessment schemes use professional standard audio as the sole benchmark for rigid comparison, ignoring individual differences caused by learners' vocal physiology. They judge minor pronunciation deviations within a reasonable range as errors, resulting in a low tolerance for error and potentially discouraging practice. Students often unconsciously repeat words during recitation; existing technology lacks audio preprocessing and cleaning mechanisms, and repeated pronunciations can cause misalignment of words and phrases, leading to misjudgments in sentence recognition.

[0004] Meanwhile, most devices use static pronunciation evaluation databases, which cannot be iteratively updated with effective samples generated by learners' practice. This makes it difficult to adapt to the pronunciation characteristics of learners of different ages, and the recognition accuracy cannot be continuously optimized with use. Some solutions rely on the cloud to complete acoustic calculations, requiring an internet connection to use. Uploading raw voice data to the cloud poses a risk of privacy leaks and can also cause network transmission latency and stuttering in interactive feedback.

[0005] Home practice environments are complex and easily disturbed by ambient noise. Existing equipment lacks noise reduction and audio adaptation strategies for nighttime practice, making it difficult for learners to pinpoint the root cause of pronunciation problems, leaving them with no choice but to blindly follow along. Some error correction pop-ups obscure the text being read, interrupting the reading rhythm and resulting in low training efficiency.

[0006] Therefore, there is an urgent need for a new oral communication training method to improve training efficiency and quality. Summary of the Invention

[0007] The purpose of this application is to provide a method, system, and apparatus for assisting in spoken language training, in order to solve the problem of difficulty in accurately judging the language being read aloud.

[0008] The technical solution of this application is: a method for assisting in spoken language training, comprising:

[0009] Collect historical audio data and corresponding standard audio data of each sentence from a certain number of learners, and establish historical audio database and standard audio database respectively; establish the association relationship between corresponding sentences in historical audio database and standard audio database;

[0010] Collect pitch, loudness, and timbre data for the pronunciation of each word or phrase in each sentence from historical audio databases and standard audio databases; statistically analyze the pitch, loudness, and timbre data of words or phrases in sentences from standard sentences and historical audio databases respectively; obtain pitch, loudness, and timbre data of correctly pronounced words or phrases, and establish a three-dimensional pronunciation interval generation model;

[0011] Based on the paragraph or sentence specified by the learner, the current audio data is collected and generated after the learner finishes pronouncing it; the paragraph includes several sentences; each word or phrase in the current audio data is identified through pronunciation, and the pitch, loudness, and timbre data of each word or phrase in the current audio data are obtained to form the current pronunciation sentence;

[0012] The pitch, loudness, and timbre data of each correctly pronounced word or phrase in the corresponding paragraph or sentence are retrieved from the three-dimensional pronunciation interval generation model. In the corresponding paragraph or sentence, each word or phrase in the correctly pronounced sentence and the current pronunciation sentence are compared by pitch, loudness, and timbre to obtain the incorrectly pronounced words or phrases in the current pronunciation sentence.

[0013] The system sends incorrectly pronounced words or phrases to the learner. After the learner reads the sentence aloud again, the system uses a three-dimensional pronunciation range generation model to identify the words and phrases again until all words or phrases in the sentence are pronounced correctly.

[0014] Preferably, once all the words or phrases in the corresponding sentence are pronounced correctly, the audio data of the current learner's correct pronunciation is collected, stored in the historical audio database, and the pitch, loudness, and timbre data of each word or phrase in each sentence in the historical audio database are re-statistically analyzed; the pitch, loudness, and timbre data of new correctly pronounced words or phrases are iteratively obtained, and the three-dimensional pronunciation interval generation model is updated.

[0015] Preferably, a tagging method is used to establish the association between the corresponding statements in the historical audio database and the standard audio database. First, a first tag is set for each standard statement in the standard audio database, and then a second tag is set for multiple corresponding statements in the historical audio database. The first tag and the second tag are then associated accordingly.

[0016] Preferably, the specific method for establishing a three-dimensional pronunciation range generation model is as follows: The pitch, loudness, and timbre data of each character or word in the statistical standard sentence are used as standard values; then, the pitch, loudness, and timbre data of each character or word in the historical audio database are statistically analyzed, and the timbre data is converted into corresponding voiceprint vectors. Then, the pitch, loudness, and voiceprint vectors of the corresponding characters or words are merged as standard values. Using the corresponding standard values ​​as a base, a set ratio of pitch, loudness, and voiceprint vectors is selected as the correct pronunciation range, resulting in the correct pitch, loudness, and voiceprint vector data of each character or word; the correct pronunciation sentences are then generated, and a three-dimensional pronunciation range generation model is established.

[0017] Preferably, the specific method for recognizing each character or word is as follows: assign an independent number to each character or word in each sentence, and then determine whether there are consecutive identical pronunciations by analyzing pitch, loudness, and voiceprint vector data. If so, determine again by analyzing the pronunciation whether the next pronunciation of the consecutive identical pronunciation is the correct pronunciation of the next character or word. If not, determine that the learner is repeating the same character, and then delete the repeated character or word.

[0018] Preferably, when comparing each character or word separately, first obtain the pitch, loudness, and correct pronunciation range of each character or word in the correct pronunciation sentence; then obtain the pitch, loudness, and voiceprint vector data of each character or word in the current pronunciation sentence, first determine whether the pitch and loudness of the corresponding character or word are within the correct pronunciation range of each character or word in the correct pronunciation sentence, if so, then match the output range of timbre through the voiceprint vector, if the match is successful, then the character or word is considered to be pronounced correctly.

[0019] Preferably, the output range of timbre is matched using voiceprint vectors, as follows:

[0020] For two audio segments A and B, extract the voiceprint vectors respectively:

[0021] , ;

[0022] in, For the encoder; perform L2 normalization on the voiceprint vector:

[0023] , ;

[0024] Calculate the cosine similarity of the normalized voiceprint vectors:

[0025] ;

[0026] The output range of s is ,when:

[0027] ;

[0028] To determine if they belong to the same timbre, among which The threshold value for the set timbre range.

[0029] Preferably, after collecting a specific number of learners' continuous pronunciation sentences, the frequency of pauses and the fluctuation range of pitch and loudness in all collected sentences are statistically analyzed. When the fluctuation range of pitch and loudness in a sentence exceeds a set range, and the frequency of pauses in a sentence exceeds a set number of times, the current oral training difficulty level is reduced.

[0030] The second technical solution of this application is: a system for assisting in spoken language training, comprising:

[0031] The audio database creation module is used to collect historical audio data and corresponding standard audio data of various sentences from a certain number of learners, and to create historical audio databases and standard audio databases respectively; and to establish the association relationship between corresponding sentences in the historical audio database and the standard audio database.

[0032] The three-dimensional pronunciation interval generation model building module is used to collect the pitch, loudness, and timbre data of each word or phrase in each sentence from the historical audio database and the standard audio database; to statistically analyze the pitch, loudness, and timbre data of words or phrases in the sentences in the standard sentences and the historical audio database respectively; to obtain the pitch, loudness, and timbre data of correctly pronounced words or phrases, and to build a three-dimensional pronunciation interval generation model.

[0033] The audio data acquisition module is used to acquire and generate the current audio data after the learner finishes pronouncing the paragraph or sentence specified by the learner. The paragraph includes several sentences. The module identifies each word or phrase in the current audio data through pronunciation, and obtains the pitch, loudness, and timbre data of each word or phrase in the current audio data to form the current pronunciation sentence.

[0034] The audio data comparison module is used to find the pitch, loudness, and timbre data of each word or phrase in the corresponding paragraph or sentence that is pronounced correctly from the three-dimensional pronunciation interval generation model; in the corresponding paragraph or sentence, the module compares each word or phrase in the correct pronunciation sentence with the current pronunciation sentence by pitch, loudness, and timbre to obtain the word or phrase that is pronounced incorrectly in the current pronunciation sentence;

[0035] The audio data repetition correction module is used to send incorrectly pronounced words or phrases to the learner. When the learner reads the text again, the three-dimensional pronunciation range generation model is used to identify the words or phrases again until all the words or phrases in the corresponding sentence are pronounced correctly.

[0036] Preferably, it also includes a three-dimensional pronunciation interval generation model update module, which is used to collect the audio data of the current learner's correct pronunciation after all the words or phrases in the corresponding sentence are pronounced correctly, store them in the historical audio database, and re-count the pitch, loudness, and timbre data of each word or phrase in each sentence in the historical audio database; iteratively obtain the pitch, loudness, and timbre data of new correctly pronounced words or phrases, and update the three-dimensional pronunciation interval generation model.

[0037] Preferably, in the audio database establishment module, a tagging method is used to establish the association between corresponding statements in the historical audio database and the standard audio database. First, a first tag is set for each standard statement in the standard audio database, and then a second tag is set for multiple corresponding statements in the historical audio database.

[0038] The third technical solution of this application is: a device for assisting in spoken language training, comprising an audio perception acquisition layer, an acoustic feature calculation and processing layer, a spoken language error correction and interactive decision-making layer, and a data iteration storage and synchronization layer;

[0039] The audio perception and acquisition layer is equipped with a sound pickup module and a multi-level audio noise reduction algorithm for audio data acquisition. The acoustic feature calculation and processing layer is configured with a local audio processing NPU, which can perform audio feature extraction, pronunciation range determination, and audio data comparison locally. The multi-level audio noise reduction algorithm is used to filter ambient noise during audio acquisition. The audio perception and acquisition layer is also configured with a night practice sub-mode, which reduces the sound pickup gain and lowers the output volume of the follow-up prompts.

[0040] Preferably, in the spoken language error correction interactive decision layer, in addition to outputting words or phrases with incorrect pronunciation, it also outputs detailed deviations in three dimensions: pitch, loudness, and timbre, and pushes targeted reading practice.

[0041] Preferably, the data iteration storage synchronization layer is used for local persistent storage of historical audio database, standard audio database and three-dimensional pronunciation range generation model; the data iteration storage synchronization layer has a built-in recognition database update module, and the entry of qualified audio into the database and the recalculation of pronunciation features are silently executed in the background during practice intervals.

[0042] The method, system, and apparatus for assisting spoken language training disclosed in this application generate models based on three-dimensional pronunciation intervals from historical audio databases and standard audio databases, respectively, thereby setting the recognition range of correct pronunciation data and establishing margin for pronunciation recognition. When the learner's pronunciation falls within the set pitch, loudness, and timbre data, accurate recognition of whether the pronunciation is correct can be achieved even when the pronunciation is not quite standard, thereby effectively improving the efficiency and quality of pronunciation recognition. Attached Figure Description

[0043] To more clearly illustrate the technical solutions provided in this application, the accompanying drawings will be briefly described below. Obviously, the drawings described below are merely some embodiments of this application.

[0044] Figure 1 This is a schematic diagram of the overall process of this application. Detailed Implementation

[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] Example 1: A method for assisting in spoken language training improves the accuracy of language recognition during recitation by setting the lip margin for pronunciation.

[0047] like Figure 1 As shown, the specific steps include the following:

[0048] Step S100: Collect historical audio data and corresponding standard audio data of each sentence from a certain number of learners, and establish a historical audio database and a standard audio database respectively; establish the association relationship between the corresponding sentences in the historical audio database and the standard audio database.

[0049] Learners include students and adults. Historical audio data consists of the pronunciation data of various sentences from different students or adults. For the same sentence, by collecting the pronunciation data from different students or adults, there are multiple sets of different sentences, forming different pronunciations.

[0050] The pronunciation data for the corresponding sentences in the historical audio data is also correct; only the pitch, loudness, and timbre of the same sentence differ. The standard audio data consists of pronunciation data from professionals.

[0051] Preferably, a tagging method is used to establish the association between corresponding statements in the historical audio database and the standard audio database. First, a first tag is set for each standard statement in the standard audio database, such as 1A0; then, a second tag is set for multiple corresponding statements in the historical audio database, such as 1A1, 1A2, 1A3, etc.; the first tag and the second tag are associated with the same statement. By tagging the statements in the historical audio database and the standard audio database, the associated statements can be quickly identified for subsequent processing.

[0052] Whenever a student or adult uses the tutoring device to study, new audio data is generated and saved to the historical audio database.

[0053] Step S200: Collect pitch, loudness, and timbre data of each word or phrase in each sentence from the historical audio database and the standard audio database; statistically analyze the pitch, loudness, and timbre data of words or phrases in the sentences from the standard sentences and the historical audio database respectively; obtain the pitch, loudness, and timbre data of correctly pronounced words or phrases, and establish a three-dimensional pronunciation interval generation model;

[0054] Specifically, the method for establishing a three-dimensional pronunciation range generation model is as follows: The pitch, loudness, and timbre data of each character or word in a statistical standard sentence are used as standard values. Then, the pitch, loudness, and timbre data of each character or word in a historical audio database are statistically analyzed. The timbre data is converted into corresponding voiceprint vectors. The pitch, loudness, and voiceprint vectors of the corresponding characters or words are then merged as standard values. Using these standard values ​​as a base, a set ratio of pitch, loudness, and voiceprint vectors is selected as the correct pronunciation range, resulting in the correct pitch, loudness, and voiceprint vector data for each character or word. The correct pronunciation sentences are then generated, and a three-dimensional pronunciation range generation model is established.

[0055] By using historical and standard audio databases to establish the correct range of pronunciation, the learner's pronunciation margin is increased, enabling them to pronounce correctly within a reasonable range and improving the accuracy of pronunciation.

[0056] Step S300: Based on the paragraph or sentence specified by the learner, after the learner finishes pronouncing, collect and generate the current audio data; the paragraph includes several sentences; identify each character or word in the current audio data through pronunciation, obtain the pitch, loudness and timbre data of each character or word in the current audio data, and form the current pronunciation sentence.

[0057] Because learners may repeat the same word or phrase when reciting, it may lead to misidentification, resulting in subsequent incorrect identification of the same word or phrase.

[0058] Therefore, in order to prevent pronunciation errors due to repeated recognition during the recognition process, a repeated pronunciation check method is set up. The specific method for recognizing each character or word is as follows:

[0059] Each word or phrase in a sentence is assigned a unique number. Then, by analyzing pitch, loudness, and timbre data, it is determined whether there are consecutive identical pronunciations, such as "at this time" or "very very". If so, the pronunciation is checked again to determine whether the next pronunciation of the consecutive identical pronunciation is the correct pronunciation of the next word or phrase. If not, it is determined that the learner has repeated the same word or phrase, and the repeated word or phrase is deleted.

[0060] Step S400: Find the pitch, loudness, and timbre data of each word or phrase that is pronounced correctly in the corresponding paragraph or sentence from the three-dimensional pronunciation interval generation model;

[0061] In the corresponding paragraph or sentence, each word or phrase in the correctly pronounced sentence and the currently pronounced sentence is compared by pitch, loudness, and timbre to identify the incorrectly pronounced words or phrases in the currently pronounced sentence.

[0062] The pitch, loudness, and timbre data of each character or word in the statistical standard sentence are used as standard values. Then, the pitch, loudness, and timbre data of each character or word in the historical audio database are statistically analyzed. The timbre data is converted into the corresponding voiceprint vector. Then, the pitch, loudness, and voiceprint vector of the corresponding character or word are merged as standard values. Using the corresponding standard values ​​as the base point, a set ratio of pitch, loudness, and voiceprint vector is selected as the correct pronunciation range to obtain the pitch, loudness, and voiceprint vector data of each correctly pronounced character or word. The correct pronunciation sentences are then generated, and a three-dimensional pronunciation range generation model is established.

[0063] Among them, the output range matching of timbre is performed using voiceprint vectors as follows:

[0064] For two audio segments A and B, extract the voiceprint vectors respectively:

[0065] , ;

[0066] in, For the encoder; perform L2 normalization on the voiceprint vector:

[0067] , ;

[0068] Calculate the cosine similarity of the normalized voiceprint vectors:

[0069] ;

[0070] The output range of s is ,when:

[0071] ;

[0072] To determine if they belong to the same timbre, among which The threshold value for the set timbre range.

[0073] Preferably, after collecting a specific number of learners' continuous pronunciation sentences, the frequency of pauses and the fluctuation range of pitch and loudness in all collected sentences are statistically analyzed. When the fluctuation range of pitch and loudness in a sentence exceeds the set range, and the frequency of pauses in a sentence exceeds the set number of times, it is determined that the learner is nervous or in poor condition, and the difficulty level of the current oral training is reduced to prevent ineffective training.

[0074] In step S500, words or phrases with incorrect pronunciation are sent to the learner. After the learner reads the text again, the three-dimensional pronunciation range generation model is used to identify the words or phrases again until all words or phrases in the corresponding sentence are pronounced correctly.

[0075] Step S600: After all the words or phrases in the corresponding sentence are pronounced correctly, collect the audio data of the current learner's correct pronunciation, store it in the historical audio database, and re-count the pitch, loudness, and timbre data of each word or phrase in each sentence in the historical audio database; iteratively obtain the pitch, loudness, and timbre data of new correctly pronounced words or phrases, and update the three-dimensional pronunciation interval generation model.

[0076] By continuously updating the three-dimensional pronunciation interval generation model, the recognition accuracy of the three-dimensional pronunciation interval generation model can be continuously improved, thereby ensuring the effectiveness of recognition.

[0077] In summary, this application generates a three-dimensional pronunciation range model based on historical audio databases and standard audio databases, thereby setting the recognition range of correct pronunciation data and establishing the margin for pronunciation recognition. When the learner's pronunciation falls within the set pitch, loudness, and timbre data, accurate recognition of whether the pronunciation is correct can be achieved even when the pronunciation is not quite standard, thereby effectively improving the precision and accuracy of pronunciation recognition.

[0078] Example 2: A system for assisting in spoken language training, comprising:

[0079] The audio database creation module is used to collect historical audio data and corresponding standard audio data of various sentences from a certain number of learners, and to create historical audio databases and standard audio databases respectively; and to establish the association relationship between corresponding sentences in the historical audio database and the standard audio database.

[0080] The three-dimensional pronunciation interval generation model building module is used to collect the pitch, loudness, and timbre data of each word or phrase in each sentence from the historical audio database and the standard audio database; to statistically analyze the pitch, loudness, and timbre data of words or phrases in the sentences in the standard sentences and the historical audio database respectively; to obtain the pitch, loudness, and timbre data of correctly pronounced words or phrases, and to build a three-dimensional pronunciation interval generation model.

[0081] The audio data acquisition module is used to acquire and generate the current audio data after the learner finishes pronouncing the paragraph or sentence specified by the learner. The paragraph includes several sentences. The module identifies each word or phrase in the current audio data through pronunciation, and obtains the pitch, loudness, and timbre data of each word or phrase in the current audio data to form the current pronunciation sentence.

[0082] The audio data comparison module is used to find the pitch, loudness, and timbre data of each word or phrase in the corresponding paragraph or sentence that is pronounced correctly from the three-dimensional pronunciation interval generation model; in the corresponding paragraph or sentence, the module compares each word or phrase in the correct pronunciation sentence with the current pronunciation sentence by pitch, loudness, and timbre to obtain the word or phrase that is pronounced incorrectly in the current pronunciation sentence;

[0083] The audio data repetition correction module is used to send incorrectly pronounced words or phrases to the learner. When the learner reads the text again, the three-dimensional pronunciation range generation model is used to identify the words or phrases again until all the words or phrases in the corresponding sentence are pronounced correctly.

[0084] Preferably, it also includes a three-dimensional pronunciation interval generation model update module, which is used to collect the audio data of the current learner's correct pronunciation after all the words or phrases in the corresponding sentence are pronounced correctly, store them in the historical audio database, and re-count the pitch, loudness, and timbre data of each word or phrase in each sentence in the historical audio database; iteratively obtain the pitch, loudness, and timbre data of new correctly pronounced words or phrases, and update the three-dimensional pronunciation interval generation model.

[0085] Preferably, in the audio database establishment module, a tagging method is used to establish the association between corresponding statements in the historical audio database and the standard audio database. First, a first tag is set for each standard statement in the standard audio database, and then a second tag is set for multiple corresponding statements in the historical audio database.

[0086] A third aspect of this application provides an apparatus for assisting in spoken language training, comprising an audio perception acquisition layer, an acoustic feature calculation and processing layer, a spoken language error correction and interactive decision-making layer, and a data iteration storage and synchronization layer.

[0087] The audio perception and acquisition layer is equipped with a sound pickup module and a multi-level audio noise reduction algorithm for audio data acquisition. The multi-level audio noise reduction algorithm is used to filter ambient noise during the audio acquisition process. The audio perception and acquisition layer is also equipped with a night practice sub-mode, which reduces the sound pickup gain and lowers the output volume of the follow-up prompts.

[0088] The acoustic feature calculation and processing layer is configured with a local audio processing NPU. The audio processing NPU can perform audio feature extraction, pronunciation range determination and audio data comparison locally without relying on cloud services, thus improving operating efficiency.

[0089] Preferably, in the oral pronunciation correction interactive decision layer, in addition to outputting words or phrases with incorrect pronunciation, it also outputs detailed deviations in three dimensions: pitch, loudness, and timbre, and pushes targeted follow-up reading exercises, forming a complete oral training process that can be implemented offline.

[0090] Preferably, the data iteration storage synchronization layer is used for local persistent storage of historical audio database, standard audio database and three-dimensional pronunciation interval generation model; the data iteration storage synchronization layer has a built-in recognition database update module, and qualified audio is entered into the database and pronunciation features are recalculated silently in the background during practice intervals, so as to accurately locate pronunciation defects, clarify pronunciation problems and avoid aimless repetitive reading.

[0091] Finally, it should be noted that the accompanying drawings of the embodiments disclosed in this invention only involve the structures involved in the embodiments disclosed in this invention. Other structures can refer to the general design. In the absence of conflict, the same embodiment and different embodiments of this invention can be combined with each other.

[0092] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for assisting in spoken language training, characterized in that, include: Collect historical audio data and corresponding standard audio data of each sentence from a certain number of learners, and establish historical audio database and standard audio database respectively; establish the association relationship between corresponding sentences in historical audio database and standard audio database; Collect pitch, loudness, and timbre data for the pronunciation of each word or phrase in each sentence from historical audio databases and standard audio databases; statistically analyze the pitch, loudness, and timbre data of words or phrases in sentences from standard sentences and historical audio databases respectively; obtain pitch, loudness, and timbre data of correctly pronounced words or phrases, and establish a three-dimensional pronunciation interval generation model; Based on the paragraph or sentence specified by the learner, the current audio data is collected and generated after the learner finishes pronouncing it; the paragraph includes several sentences; each word or phrase in the current audio data is identified through pronunciation, and the pitch, loudness, and timbre data of each word or phrase in the current audio data are obtained to form the current pronunciation sentence; The pitch, loudness, and timbre data of each correctly pronounced word or phrase in the corresponding paragraph or sentence are retrieved from the three-dimensional pronunciation interval generation model. In the corresponding paragraph or sentence, each word or phrase in the correctly pronounced sentence and the current pronunciation sentence are compared by pitch, loudness, and timbre to obtain the incorrectly pronounced words or phrases in the current pronunciation sentence. The system sends incorrectly pronounced words or phrases to the learner. After the learner reads the sentence aloud again, the system uses a three-dimensional pronunciation range generation model to identify the words and phrases again until all words or phrases in the sentence are pronounced correctly.

2. The method for assisting in spoken language training as described in claim 1, characterized in that: Once all the words or phrases in the corresponding sentence are pronounced correctly, the audio data of the current learner's correct pronunciation is collected and stored in the historical audio database. The pitch, loudness, and timbre data of each word or phrase in each sentence in the historical audio database are re-statistically analyzed. The pitch, loudness, and timbre data of new correctly pronounced words or phrases are iteratively obtained, and the three-dimensional pronunciation interval generation model is updated.

3. The method for assisting in spoken language training as described in claim 1, characterized in that: A tagging method is used to establish the association between corresponding statements in the historical audio database and the standard audio database. First, a first tag is set for each standard statement in the standard audio database, and then a second tag is set for multiple corresponding statements in the historical audio database. The first tag and the second tag are associated with the same statement.

4. The method for assisting in spoken language training as described in claim 1, characterized in that: The specific method for establishing a three-dimensional pronunciation range generation model is as follows: the pitch, loudness, and timbre data of each character or word in the statistical standard sentence are used as standard values; then the pitch, loudness, and timbre data of each character or word in the historical audio database are statistically analyzed, the timbre data is converted into the corresponding voiceprint vector, and then the pitch, loudness, and voiceprint vector of the corresponding character or word are merged as standard values. Using the corresponding standard value as the base point, the set ratio of pitch, loudness, and voiceprint vector is selected as the correct pronunciation range, and the pitch, loudness, and voiceprint vector data of each character or word with correct pronunciation are obtained. Organize and generate correctly pronounced sentences, and establish a three-dimensional pronunciation range generation model.

5. The method for assisting in spoken language training as described in claim 4, characterized in that: The specific method for recognizing each character or word is as follows: assign an independent number to each character or word in each sentence, and then determine whether there are consecutive identical pronunciations by analyzing pitch, loudness, and voiceprint vector data. If so, determine whether the next pronunciation of the consecutive identical pronunciation is the correct pronunciation of the next character or word. If not, it is determined that the learner has repeatedly read the same character, and the repeatedly read character or word is deleted.

6. The method for assisting in spoken language training as described in claim 4, characterized in that: When comparing each character or word separately, first obtain the pitch, loudness, and correct pronunciation range of each character or word in the correct pronunciation sentence; then obtain the pitch, loudness, and voiceprint vector data of each character or word in the current pronunciation sentence, first determine whether the pitch and loudness of the corresponding character or word are within the correct pronunciation range of each character or word in the correct pronunciation sentence, if so, then match the output range of timbre through the voiceprint vector, if the match is successful, then the character or word is considered to be pronounced correctly.

7. The method for assisting in spoken language training as described in claim 6, characterized in that: The output range of timbre is matched using voiceprint vectors, as follows: For two audio segments A and B, extract the voiceprint vectors respectively: ， ； in, For the encoder; perform L2 normalization on the voiceprint vector: ， ； Calculate the cosine similarity of the normalized voiceprint vectors: ； The output range of s is ,when: ； To determine if they belong to the same timbre, among which The threshold value for the set timbre range.

8. The method for assisting in spoken language training as described in claim 1, characterized in that: After collecting a specific number of learners' continuous pronunciation sentences, the frequency of pauses and the range of pitch and loudness fluctuations in all collected sentences are statistically analyzed. When the range of pitch and loudness fluctuations in a sentence exceeds the set range, and the frequency of pauses in the sentence exceeds the set number of times, the current oral training difficulty level is reduced.

9. A system for assisting in spoken language training, employing the method as described in any one of claims 1-8, characterized in that, include: The audio database creation module is used to collect historical audio data and corresponding standard audio data of various sentences from a certain number of learners, and to create historical audio databases and standard audio databases respectively; and to establish the association relationship between corresponding sentences in the historical audio database and the standard audio database. The three-dimensional pronunciation interval generation model building module is used to collect the pitch, loudness, and timbre data of each word or phrase in each sentence from the historical audio database and the standard audio database; to statistically analyze the pitch, loudness, and timbre data of words or phrases in the sentences in the standard sentences and the historical audio database respectively; to obtain the pitch, loudness, and timbre data of correctly pronounced words or phrases, and to build a three-dimensional pronunciation interval generation model. The audio data acquisition module is used to acquire and generate the current audio data after the learner finishes pronouncing the paragraph or sentence specified by the learner. The paragraph includes several sentences. The module identifies each word or phrase in the current audio data through pronunciation, and obtains the pitch, loudness, and timbre data of each word or phrase in the current audio data to form the current pronunciation sentence. The audio data comparison module is used to find the pitch, loudness, and timbre data of each word or phrase in the corresponding paragraph or sentence that is pronounced correctly from the three-dimensional pronunciation interval generation model; in the corresponding paragraph or sentence, the module compares each word or phrase in the correct pronunciation sentence with the current pronunciation sentence by pitch, loudness, and timbre to obtain the word or phrase that is pronounced incorrectly in the current pronunciation sentence; The audio data repetition correction module is used to send incorrectly pronounced words or phrases to the learner. When the learner reads the text again, the three-dimensional pronunciation range generation model is used to identify the words or phrases again until all the words or phrases in the corresponding sentence are pronounced correctly.

10. The system for assisting spoken language training as described in claim 9, characterized in that: It also includes a three-dimensional pronunciation interval generation model update module, which is used to collect the audio data of the current learner's correct pronunciation after all the words or phrases in the corresponding sentence are pronounced correctly, store them in the historical audio database, and re-count the pitch, loudness, and timbre data of each word or phrase in each sentence in the historical audio database; iteratively obtains the pitch, loudness, and timbre data of new correctly pronounced words or phrases, and updates the three-dimensional pronunciation interval generation model.

11. The system for assisting spoken language training as described in claim 9, characterized in that: In the audio database establishment module, a tagging method is used to establish the association between corresponding statements in the historical audio database and the standard audio database. First, a first tag is set for each standard statement in the standard audio database, and then a second tag is set for multiple corresponding statements in the historical audio database.

12. A device for assisting in spoken language training, employing the method as described in any one of claims 1-8, characterized in that: It includes an audio perception and acquisition layer, an acoustic feature calculation and processing layer, a spoken language error correction and interactive decision-making layer, and a data iteration, storage, and synchronization layer; The audio perception and acquisition layer is equipped with a sound pickup module and a multi-level audio noise reduction algorithm for audio data acquisition. The acoustic feature calculation and processing layer is configured with a local audio processing NPU, which can perform audio feature extraction, pronunciation range determination, and audio data comparison locally. The multi-level audio noise reduction algorithm is used to filter ambient noise during audio acquisition. The audio perception and acquisition layer is also configured with a night practice sub-mode, which reduces the sound pickup gain and lowers the output volume of the follow-up prompts.

13. The device for assisting in spoken language training as described in claim 12, characterized in that: In the spoken language error correction interactive decision layer, in addition to outputting words or phrases with incorrect pronunciation, it also outputs detailed deviations in three dimensions: pitch, loudness, and timbre, and pushes targeted reading practice.

14. The device for assisting in spoken language training as described in claim 12, characterized in that: The data iteration storage synchronization layer is used for local persistent storage of historical audio database, standard audio database and three-dimensional pronunciation range generation model; the data iteration storage synchronization layer has a built-in recognition database update module, and qualified audio is entered into the database and pronunciation features are recalculated silently in the background during practice intervals.

Images