A method and system for generating a multi-dimensional speech training program

Abstract

The application discloses a kind of multi-dimension speech training plan generation method and system.The method first collects patient age, gender and speech sample, obtains seven-dimensional evaluation parameters including sound pressure, amplitude perturbation, maximum vocalization duration, fundamental frequency perturbation, vowel space area, tone impairment and speech comprehension score.Subsequently, according to the preset logic, these parameters are sequentially determined based on the parameters, and dynamically combine different training modules such as loudness, breath, pitch, vowel, glide, tone and consonant into personalized speech training plan.The application solves the problem of existing technology training scheme solidification through multi-dimensional evaluation and dynamic module matching, significantly improves the individualization degree and rehabilitation effect of speech training.

Description

Technical Field

[0001] This invention relates to a method for generating a multi-dimensional speech training plan, and also to a system for implementing this method, belonging to the field of speech training technology. Background Technology

[0002] Speech rehabilitation training is a core means of improving speech function in patients with neurological disorders such as stroke, Parkinson's disease, and non-fluent aphasia. Its effectiveness is highly dependent on the personalization of the training program. In the prior art, as described in Chinese patent application No. 202510931372.4, although speech recognition can analyze patients' pronunciation problems and assess the degree of impairment, the generation of training programs still largely relies on static, pre-set single-dimensional scoring, lacking a systematic and comprehensive consideration of the patient's multidimensional speech physiological characteristics (such as sound pressure, fundamental frequency perturbation, vowel space area, etc.).

[0003] In recent years, academic research on the multidimensional features of speech prosody, pitch, and rhythm has deepened. For example, the study "Prosodic Features of Patients with Non-Fluent Aphasia" revealed significant differences in intonation among different tones and the clinical value of glissando training; the study "Construction and Intervention Research of Prosodic Assessment Indicators for Neurological Speech Disorders" further constructed an assessment system including intonation, rhythm, and glissando; and the study "Music Training Promotes the Interaction of Pitch and Rhythm Dimensions in Auditory Working Memory" confirmed the promoting effect of pitch practice on rhythm ability. These studies all indicate that effective rehabilitation training needs to take into account the synergistic improvement of multidimensional speech features.

[0004] However, most existing speech training systems fail to dynamically and quantitatively link these multi-dimensional assessment indicators with training plans. Their training content and intensity are often fixed, unable to accurately adapt to the specific impairments of users in different areas such as breathing, phonation, articulation, and rhythm. For example, Parkinson's disease patients often exhibit insufficient loudness and flat intonation, while stroke-induced aphasia patients often have disordered phonological programming. Existing systems struggle to automatically adjust the training focus and difficulty accordingly, resulting in insufficient training targeting and impacting rehabilitation efficiency and user experience.

[0005] Therefore, the current field of speech training technology urgently needs a method and system that can dynamically evaluate based on multi-dimensional speech features and automatically generate highly personalized training schemes accordingly, in order to solve the core problems of rigid training schemes, poor adaptability, and lack of precise personalized guidance in existing technologies. Summary of the Invention

[0006] The primary technical problem to be solved by this invention is to provide a method for generating a multi-dimensional speech training plan.

[0007] Another technical problem to be solved by the present invention is to provide a system for generating multi-dimensional speech training plans.

[0008] To achieve the above-mentioned technical objectives, the present invention adopts the following technical solution:

[0009] According to a first aspect of the present invention, a method for generating a multi-dimensional speech training plan is provided, comprising the following steps:

[0010] S1: Obtain the patient's age and gender information, and collect their voice samples for voice assessment to obtain assessment result parameters; wherein, the assessment result parameters include: sound pressure level, amplitude perturbation, maximum phonation duration, fundamental frequency perturbation, vowel space area, intonation impairment, and speech comprehension score;

[0011] S2: Determine the patient's sound pressure level;

[0012] S3: Determine the amplitude perturbation of the patient;

[0013] S4: Determine the patient's age;

[0014] If the patient's age is less than or equal to the age threshold, proceed to step S5; if the patient's age is greater than the age threshold, proceed to step S6.

[0015] S5: Determine the patient's maximum vocal duration;

[0016] S6: Determine the patient's gender;

[0017] If the gender is male, the maximum vocal duration for males is determined, training is performed based on the determination result, and then the process proceeds to step S7; if the gender is female, the maximum vocal duration for females is determined, training is performed based on the determination result, and then the process proceeds to step S7.

[0018] S7: Determine the patient's fundamental frequency perturbation;

[0019] S8: Further branching judgment based on the patient's age;

[0020] If the patient's age is less than or equal to the age threshold, proceed to step S9; if the patient's age is greater than the age threshold, proceed to step S10.

[0021] S9: Determine the area of ​​the patient's vowel space;

[0022] S10: Determine the patient's gender;

[0023] If the gender is male, the male vowel space area is determined, training is performed based on the determination result, and then the process proceeds to step S11; if the gender is female, the female vowel space area is determined, training is performed based on the determination result, and then the process proceeds to step S11.

[0024] S11: Assess the patient's intonation impairment;

[0025] If the patient has intonation impairment, add the glissando practice module and tone practice module to the speech training plan and end the speech training plan generation; if the patient does not have intonation impairment, proceed to step S12.

[0026] S12: Assess the patient's verbal comprehension score;

[0027] If the patient's speech comprehension score is greater than or equal to the passing threshold, the speech training plan generation ends; if the patient's speech comprehension score is less than the passing threshold, the consonant practice module is added to the speech training plan, and the speech training plan generation ends.

[0028] Preferably, in step S2, when determining the patient's sound pressure, if the sound pressure is less than the first sound pressure threshold, the loudness practice module is added to the speech training plan, and the speech training plan generation ends; if the sound pressure is greater than or equal to the first sound pressure threshold and less than the second sound pressure threshold, or if the sound pressure is greater than the third sound pressure threshold, the loudness practice module is added to the speech training plan, and the process proceeds to step S4; if the sound pressure is greater than or equal to the first sound pressure threshold and less than or equal to the third sound pressure threshold, the process proceeds to step S3.

[0029] Preferably, in step S3, when determining the patient's amplitude perturbation, if the patient's amplitude perturbation is greater than or equal to the amplitude perturbation threshold, the loudness practice module is added to the speech training plan and the process proceeds to step S4; if the patient's amplitude perturbation is less than the amplitude perturbation threshold, the process proceeds to step S4.

[0030] Preferably, in step S5, when determining the patient's maximum vocal duration, if the patient's maximum vocal duration is greater than the first maximum vocal duration threshold for children, then proceed to step S7; if the patient's maximum vocal duration is less than the second maximum vocal duration threshold for children, then add the breath training module and pitch training module to the speech training plan and end the speech training plan generation; if the patient's maximum vocal duration is greater than or equal to the second maximum vocal duration threshold for children and less than or equal to the first maximum vocal duration threshold for children, then add the breath training module to the speech training plan and proceed to step S7.

[0031] Preferably, in step S6, when determining the maximum vocal duration for a male, if the patient's maximum vocal duration is greater than the first maximum vocal duration threshold for a male, then proceed to step S7; if the patient's maximum vocal duration is less than the second maximum vocal duration threshold for a male, then add the breath training module and the pitch training module to the speech training plan, and end the speech training plan generation; if the patient's maximum vocal duration is greater than or equal to the second maximum vocal duration threshold for a male, and less than or equal to the first maximum vocal duration threshold for a male, then add the breath training module to the speech training plan, and proceed to step S7.

[0032] When determining the maximum vocal duration for a female, if the patient's maximum vocal duration is greater than the first maximum vocal duration threshold for a female, proceed to step S7; if the patient's maximum vocal duration is less than the second maximum vocal duration threshold for a female, add the breath training module and pitch training module to the speech training plan and end the speech training plan generation; if the patient's maximum vocal duration is greater than or equal to the second maximum vocal duration threshold for a female, and less than or equal to the first maximum vocal duration threshold for a female, add the breath training module to the speech training plan and proceed to step S7.

[0033] Preferably, in step S7, when determining the patient's fundamental frequency perturbation, if the patient's fundamental frequency perturbation is greater than the first fundamental frequency perturbation threshold, then the pitch practice module and singing practice module are added to the speech training plan, and the speech training plan generation ends; if the patient's fundamental frequency perturbation is less than the second fundamental frequency perturbation threshold, then proceed to step S8; if the patient's fundamental frequency perturbation is greater than or equal to the second fundamental frequency perturbation threshold and less than or equal to the first fundamental frequency perturbation threshold, then the pitch practice module is added to the speech training plan, and proceed to step S8.

[0034] Preferably, in step S9, when determining the vowel space area of ​​the patient, if the patient's vowel space area is greater than the threshold for the first vowel space area of ​​a child, then proceed to step S11; if the patient's vowel space area is less than the threshold for the second vowel space area of ​​a child, then add the vowel practice module to the speech training plan and end the speech training plan generation; if the patient's vowel space area is greater than or equal to the threshold for the second vowel space area of ​​a child, and less than or equal to the threshold for the first vowel space area of ​​a child, then add the vowel practice module to the speech training plan and proceed to step S11.

[0035] Preferably, in step S10, when determining the male vowel space area, if the patient's vowel space area is greater than the male first vowel space area threshold, then proceed to step S11; if the patient's vowel space area is less than the male second vowel space area threshold, then add the vowel practice module to the speech training plan and end the speech training plan generation; if the patient's vowel space area is greater than or equal to the male second vowel space area threshold and less than or equal to the male first vowel space area threshold, then add the vowel practice module to the speech training plan and proceed to step S11.

[0036] Preferably, when determining the area of ​​the female vowel space, if the patient's vowel space area is greater than the threshold for the area of ​​the first female vowel space, then proceed to step S11; if the patient's vowel space area is less than the threshold for the area of ​​the second female vowel space, then add the vowel practice module to the speech training plan and end the speech training plan generation; if the patient's vowel space area is greater than or equal to the threshold for the area of ​​the second female vowel space and less than or equal to the threshold for the area of ​​the first female vowel space, then add the vowel practice module to the speech training plan and proceed to step S11.

[0037] According to a second aspect of the present invention, a system for generating a multi-dimensional speech training plan is provided, comprising a processor and a memory; wherein the memory is coupled to the processor and is used to store a computer program, which, when executed by the processor, enables the processor to implement the above-described generation method.

[0038] Compared with existing technologies, this invention achieves precise quantitative analysis of patients' speech function by constructing a seven-dimensional assessment system encompassing sound pressure, amplitude perturbation, maximum phonation duration, fundamental frequency perturbation, vowel space area, intonation impairment, and speech comprehension score. Based on the assessment results and considering age and gender differences, this invention dynamically matches and combines training modules such as loudness, breath control, pitch, vowels, glissando, tone, and consonants through preset judgment logic to generate highly personalized speech training plans. This effectively solves the problems of rigid training programs and disconnection from the actual impairment characteristics of patients in existing technologies, significantly improving the pertinence, adaptability, and effectiveness of speech rehabilitation training. Attached Figure Description

[0039] Figure 1 A flowchart illustrating a method for generating a multi-dimensional speech training plan, as provided in the first embodiment of the present invention;

[0040] Figure 2 This is a schematic diagram of the loudness practice interface in the first embodiment of the present invention;

[0041] Figure 3 This is a schematic diagram of the interface for breath control exercises in the first embodiment of the present invention;

[0042] Figure 4 This is a schematic diagram of the interface for pitch practice in the first embodiment of the present invention;

[0043] Figure 5 This is a schematic diagram of the singing practice interface in the first embodiment of the present invention;

[0044] Figure 6 This is a schematic diagram of the vowel practice interface in the first embodiment of the present invention;

[0045] Figure 7 This is a schematic diagram of the interface for practicing glissando in the first embodiment of the present invention;

[0046] Figure 8 This is a schematic diagram of the tone practice interface in the first embodiment of the present invention;

[0047] Figure 9 This is a schematic diagram of the interface for consonant practice in the first embodiment of the present invention;

[0048] Figure 10 This is a schematic diagram of a multi-dimensional speech training plan generation system provided in the second embodiment of the present invention. Detailed Implementation

[0049] The technical content of the present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0050] First Embodiment

[0051] The technical concept in this invention is to construct a comprehensive multidimensional evaluation system that dynamically generates training schemes based on etiology-mechanism-function: etiology determines the breakpoint, the breakpoint determines the entry point, the entry point determines the dose, and the dose determines the training route, so as to achieve reaching the respective functional endpoints along the shortest path within the optimal plasticity window.

[0052] As is well known, different diseases cause different brain regions to be damaged, resulting in different windows of neuroplasticity. Therefore, different training stimulus parameters (intensity, frequency, feedback timing, etc.) are required. Using Parkinson's disease (PD), post-stroke aphasia (PSA), and functional dysarthria (FAD) as examples, the characteristics of the personalized training program that this invention can achieve are illustrated.

[0053] For a patient to speak a sentence well, the following steps are required: 1) auditory comprehension; 2) vocabulary retrieval; 3) phonological programming; 4) articulation; 5) breathing-voice production; and 6) prosodic regulation. Different patients perform differently at different stages, and training modules must be targeted; otherwise, it will be a case of "repetition at the good stages and zero stimulation at the bad stages."

[0054]

[0055] like Figure 1 As shown, the first embodiment of the present invention provides a method for generating a multi-dimensional speech training plan, which includes at least the following steps:

[0056] S1: Obtain the patient's age and gender information, and collect their voice samples for voice assessment to obtain assessment result parameters.

[0057] The evaluation parameters include: sound pressure level, shimmer, maximum duration of phonation (MPT), fundamental frequency perturbation (jitter), vowel space area (VSA), intonation impairment, and speech comprehension score.

[0058] S2: Determine the patient's sound pressure level.

[0059] If the sound pressure is less than the first sound pressure threshold, the loudness practice module is added to the speech training plan and the speech training plan generation ends; if the sound pressure is greater than or equal to the first sound pressure threshold and less than the second sound pressure threshold, or if the sound pressure is greater than the third sound pressure threshold, the loudness practice module is added to the speech training plan and the process proceeds to step S4; if the sound pressure is greater than or equal to the first sound pressure threshold and less than or equal to the third sound pressure threshold, the process proceeds to step S3.

[0060] S3: Determine the amplitude perturbation of the patient.

[0061] If the patient's amplitude perturbation is greater than or equal to the amplitude perturbation threshold, the loudness training module is added to the speech training plan and the process proceeds to step S4; if the patient's amplitude perturbation is less than the amplitude perturbation threshold, the process proceeds to step S4.

[0062] S4: Determine the patient's age.

[0063] If the patient's age is less than or equal to the age threshold, proceed to step S5; if the patient's age is greater than the age threshold, proceed to step S6.

[0064] S5: Determine the patient's maximum vocal duration.

[0065] If the patient's maximum vocalization duration is greater than the first maximum vocalization duration threshold for children, proceed to step S7; if the patient's maximum vocalization duration is less than the second maximum vocalization duration threshold for children, add the breath training module and pitch training module to the speech training plan and end the speech training plan generation; if the patient's maximum vocalization duration is greater than or equal to the second maximum vocalization duration threshold for children and less than or equal to the first maximum vocalization duration threshold for children, add the breath training module to the speech training plan and proceed to step S7.

[0066] S6: Determine the patient's gender.

[0067] If the gender is male, the maximum vocal duration for males is determined, training is performed based on the determination result, and then the process proceeds to step S7; if the gender is female, the maximum vocal duration for females is determined, training is performed based on the determination result, and then the process proceeds to step S7.

[0068] S7: Determine the patient's fundamental frequency perturbation.

[0069] If the patient's fundamental frequency perturbation is greater than the first fundamental frequency perturbation threshold, then the pitch practice module and singing practice module are added to the speech training plan, and the speech training plan generation ends; if the patient's fundamental frequency perturbation is less than the second fundamental frequency perturbation threshold, then proceed to step S8; if the patient's fundamental frequency perturbation is greater than or equal to the second fundamental frequency perturbation threshold, and less than or equal to the first fundamental frequency perturbation threshold, then the pitch practice module is added to the speech training plan, and proceed to step S8.

[0070] S8: Further branching decisions are made based on the patient's age.

[0071] If the patient's age is less than or equal to the age threshold, proceed to step S9; if the patient's age is greater than the age threshold, proceed to step S10.

[0072] S9: Determine the vowel space area of ​​the patient.

[0073] If the patient's vowel space area is greater than the threshold for the first vowel space area of ​​a child, proceed to step S11; if the patient's vowel space area is less than the threshold for the second vowel space area of ​​a child, add the vowel practice module to the speech training plan and end the speech training plan generation; if the patient's vowel space area is greater than or equal to the threshold for the second vowel space area of ​​a child and less than or equal to the threshold for the first vowel space area of ​​a child, add the vowel practice module to the speech training plan and proceed to step S11.

[0074] S10: Determine the patient's gender.

[0075] If the gender is male, the male vowel space area is determined, training is performed based on the determination result, and then the process proceeds to step S11; if the gender is female, the female vowel space area is determined, training is performed based on the determination result, and then the process proceeds to step S11.

[0076] S11: Assess the patient's intonation impairment.

[0077] If the patient has intonation impairment, the glissando practice module and tone practice module are added to the speech training plan, and the speech training plan generation ends; if the patient does not have intonation impairment, proceed to step S12.

[0078] S12: Assess the patient's verbal comprehension score.

[0079] If the patient's speech comprehension score is greater than or equal to the passing threshold, the speech training plan generation ends; if the patient's speech comprehension score is less than the passing threshold, the consonant practice module is added to the speech training plan, and the speech training plan generation ends.

[0080] In a preferred embodiment of the present invention, the first sound pressure threshold is 50 dBA; the second sound pressure threshold is 55 dBA; and the third sound pressure threshold is 90 dBA.

[0081] The core goal of loudness training is to help users achieve and maintain a stable vocal intensity at a set standard. It is suitable for speech rehabilitation patients with reduced volume due to stroke, Parkinson's disease, or other conditions; broadcasters and presenters who require precise vocal control; and ordinary users who wish to improve vocal stability and avoid excessively weak or fluctuating volume. The training forms a complete closed loop around training configuration, interactive processes, data recording, and dynamic adjustments, balancing professionalism with practicality.

[0082] Before training, two core parameter configurations must be completed. The training target is quantified in dBA, with a common setting range of 50 to 85 dBA, covering everything from daily conversation to slightly louder scenarios. In the early stages of rehabilitation, start with 50 dBA; for broadcast-related training, set it to around 70 dBA, while clearly defining the desired vocal intensity. Training intensity is defined by the number of fruits to collect. Beginners or those in the early stages of rehabilitation should choose 10 to 30 fruits, with each training session lasting 5 to 10 minutes to avoid fatigue. Advanced learners can increase the number of fruits to 50 to 100 to enhance volume maintenance.

[0083] refer to Figure 2 The training employs a scenario-based interactive design. Users must maintain a distance of 30 centimeters from the screen to ensure accurate detection and adjust their pronunciation as required, such as whispering specific numbers and exaggerating mouth movements. The height of the fruit in the training screen is fixed; when the pronunciation intensity meets the standard, a virtual bird will eat the fruit. Real-time visual feedback helps users intuitively judge whether their pronunciation is up to standard and adjust the volume accordingly.

[0084] After each training session, key data is automatically saved, including the maximum, minimum, and average fundamental frequencies related to fundamental frequency, the maximum, minimum, and average intensity related to intensity, and a comprehensive pronunciation score. Targeted feedback is also generated, providing information on the loudness stability range, comparisons with previous training sessions, and strengths and weaknesses, helping users identify areas for improvement.

[0085] The intelligent dynamic adjustment mechanism in this embodiment of the invention can optimize training difficulty. When the cumulative training time exceeds 300 seconds (5 minutes), it will automatically analyze recent data and pop up a prompt asking whether to update the plan. Adjustments are divided into two categories: reducing and increasing difficulty, distinguishing between 24-hour high-sensitivity and 48-hour low-sensitivity scenarios, prioritizing high-sensitivity conditions. When reducing difficulty, if the measured sound intensity is less than 65% or 85% of the set value and the duration condition is met, the new target is adjusted to 80% of the original set value. When increasing difficulty, if the measured sound intensity is greater than 160% or 130% of the set value and the duration condition is met, the new target is adjusted to 130% or 125% of the original set value, respectively, helping users improve their volume control ability while adapting to the rhythm.

[0086] In a preferred embodiment of the invention, the amplitude perturbation threshold is 3.08. The age threshold is 14 years old. The first maximum vocalization duration threshold for children is 10 seconds; the second maximum vocalization duration threshold for children is 3 seconds.

[0087] The core goal of breath control exercises is to improve a user's lung capacity, expiratory control, glottal closure efficiency, and vocal system coordination through sustained and stable pronunciation training. It is suitable for speech rehabilitation of patients with post-stroke aphasia, Parkinson's disease, and spastic speech disorders, and can also help ordinary users improve issues such as shortness of breath and unstable volume. The exercises are based on quantitative indicators, combined with scenario-based interaction and personalized dynamic adjustments, to help users extend pronunciation duration and improve stability.

[0088] Before training, three core parameters need to be configured: select a suitable training mode, set the training difficulty based on the minimum pronunciation duration (it is only effective if the pronunciation standard is met), and determine the number of effective training sessions to be completed. The configuration should be tailored to your own level. In the early stages of rehabilitation, you can set a shorter minimum pronunciation duration and 3-5 training sessions to avoid setting the goal too high and causing frustration. Gradually increase the number of sessions later.

[0089] refer to Figure 3 The training uses a scenario-based interactive design. Users take a deep breath, click to record, and then continuously and steadily pronounce the "a" sound. The screen will display a dynamic scene of a bird flying. The more stable and longer the pronunciation, the more planets the bird visits and the more flags it picks. The intuitive visual feedback helps users adjust their breathing rhythm and pronunciation intensity in real time. During training, users need to keep the volume stable and try to prolong the pronunciation duration.

[0090] After each training session, the system automatically saves key data such as pronunciation duration, average fundamental frequency, fundamental frequency standard deviation, average intensity, and intensity standard deviation. It also generates targeted feedback, informing users about volume stability, changes in pronunciation duration, and areas for improvement, helping them identify training priorities.

[0091] The dynamic adjustment mechanism is divided into two categories: increasing and decreasing difficulty. Increasing difficulty requires training duration exceeding 6 minutes, training for more than 5 days and at least 5 days since the last adjustment, and an average pronunciation duration exceeding the current difficulty setting. The minimum pronunciation duration is adjusted based on the difference: if the difference exceeds 5, it increases by 3; otherwise, it increases by 1. Decreasing difficulty is based on the user's previous maximum pronunciation duration and clinical reference standards: less than 5, it is set to 5; between 5 and 10, it is set to 10. This balances medical knowledge with the user's actual ability, helping users build confidence and continuously improve.

[0092] The core goal of pitch practice is to ensure that the user's fundamental frequency falls within a preset acceptable pitch range, improving pitch accuracy, leap ability, and sequence control. It is suitable for speech rehabilitation patients to improve pitch instability, broadcasting professionals to optimize pitch techniques, and ordinary users to improve pitch stability. The practice is based on quantitative standards, combined with scenario-based interaction and dynamic adjustments to achieve personalized progression.

[0093] refer to Figure 4 The training materials are divided into three categories: single-note materials require aligning with the frequency of a single piano key to train single-point alignment ability; two-note materials require continuously imitating the pitch of two piano keys to train the fluency of jumping; and three-note materials require continuously imitating the pitch of three piano keys to strengthen sequence control and coherence. The training adopts a "seed alignment with piano keys" interactive design. The screen displays piano keys of different pitches, and the user adjusts the pronunciation to make the virtual seed enter the blue qualified pitch range to pass. The intuitive feedback helps to adjust the fundamental frequency in real time.

[0094] The core evaluation metric is pitch accuracy, which is the percentage of valid fundamental frequency frames out of the total effective frames. The key parameter is the pronunciation accuracy per cent; the smaller the value, the higher the difficulty. Training configurations support single, double, and triple tones, with difficulty levels of 50, 100, and 120 cents. Beginners are advised to start with 120 cents. Training intensity is based on the number of seeds, with 200, 300, and 700 seeds corresponding to different training volumes.

[0095] After training, the system automatically saves data such as average fundamental frequency, fundamental frequency standard deviation, average intensity, intensity standard deviation, and accuracy, and provides targeted feedback to help users understand their weaknesses. Dynamic adjustments require training duration exceeding 6 minutes, training time exceeding 5 days and a period of more than 5 days since the last adjustment, and accuracy below 60% or above 90%. If accuracy exceeds 90% and precision is higher than the minimum, the range is narrowed by 10 cents; if accuracy is below 60% and precision is lower than the maximum, the range is widened by 10 cents. If training duration exceeds 10 minutes and intensity is higher than the minimum, the range is reduced by 60 seeds; if training duration is less than 5 minutes and intensity is lower than the maximum, the range is increased by 60 seeds, adapting to the user's steadily improving capabilities.

[0096] In one embodiment of the present invention, when determining the maximum vocal duration for a male in step S6, if the patient's maximum vocal duration is greater than the first maximum vocal duration threshold for a male, then proceed to step S7; if the patient's maximum vocal duration is less than the second maximum vocal duration threshold for a male, then add the breath training module and the pitch training module to the speech training plan, and end the speech training plan generation; if the patient's maximum vocal duration is greater than or equal to the second maximum vocal duration threshold for a male, and less than or equal to the first maximum vocal duration threshold for a male, then add the breath training module to the speech training plan, and proceed to step S7.

[0097] In a preferred embodiment of the present invention, the first maximum vocalization duration threshold for males is 20 seconds; the second maximum vocalization duration threshold for males is 5 seconds.

[0098] In step S6, when determining the maximum vocal duration for a female, if the patient's maximum vocal duration is greater than the first maximum vocal duration threshold for a female, proceed to step S7; if the patient's maximum vocal duration is less than the second maximum vocal duration threshold for a female, add the breath training module and pitch training module to the speech training plan and end the speech training plan generation; if the patient's maximum vocal duration is greater than or equal to the second maximum vocal duration threshold for a female, and less than or equal to the first maximum vocal duration threshold for a female, add the breath training module to the speech training plan and proceed to step S7.

[0099] In a preferred embodiment of the present invention, the first maximum vocalization duration threshold for women is 15 seconds; the second maximum vocalization duration threshold for women is 4 seconds. The first fundamental frequency perturbation threshold is 0.8; the second fundamental frequency perturbation threshold is 0.5.

[0100] The core objective of singing practice is to ensure that the fundamental frequency curve of the user's entire singing phrase matches the sample curve. This exercise trains pitch control, rhythm, and fundamental frequency following ability by having the user imitate the melody of a song segment and align their pronunciation with the sample curve provided by the system. The system offers song materials of varying vocal ranges (difficulty) and displays the user's fundamental frequency curve (red) and the sample curve (orange) in real time on the training interface to provide visual feedback.

[0101] The training configuration includes two core parameters that users can flexibly adjust. The training difficulty is related to the vocal range, represented by the width of the orange background behind the demonstration curve. A wider background indicates lower difficulty, while a narrower background indicates a stricter standard. The system offers different pitch level options; beginners can start with a wide pitch range, while those with some experience can directly select medium or higher difficulty. The training materials are complete song segments, including lyrics and melody. Users can choose content suitable for their own level, with materials covering various styles and difficulties to meet diverse needs.

[0102] refer to Figure 5The training employs a scenario-based interactive design, with an orange sample fundamental frequency curve as the background. As the user pronounces words, the system generates a red personal curve in real time. The core task is to align the red curve with the orange curve; the higher the overlap, the better the training effect. Real-time visual feedback helps users quickly perceive the gap, adjust pitch, rhythm, and intensity accordingly, and gradually improve their ability to follow the curve.

[0103] After training, the system automatically saves data such as average fundamental frequency, fundamental frequency standard deviation, average intensity, intensity standard deviation, and pronunciation score. The pronunciation score is calculated by comparing the individual's performance with the demonstration curve, directly reflecting the training effect. At the same time, the system generates targeted feedback, pointing out strengths and weaknesses and clarifying directions for improvement.

[0104] The dynamic adjustment mechanism is the core of personalized advancement. When two conditions are met—training for more than 5 days and more than 5 days since the last adjustment, and a pronunciation score of less than 60 or greater than 90—the system will pop up a window prompting whether to update the plan. If the score is less than 60, the orange background vocal range is widened to reduce the difficulty; if the score is greater than 90, the vocal range is narrowed to increase the difficulty, ensuring that the training difficulty matches the user's actual ability and helps continuous progress.

[0105] In a preferred embodiment of the present invention, the first vowel space area threshold for children is 20.67 bark squared; the second vowel space area threshold for children is 5.17 bark squared.

[0106] The core objective of vowel practice is to improve the accuracy, stability, and standardization of vowel pronunciation. Through quantitative training and dynamic adjustments, it helps speech rehabilitation patients, general users, and those seeking pronunciation optimization to improve their pronunciation quality. The practice uses vowel space area as the core indicator, combining scenario-based interaction and precise data tracking to form a closed-loop training process that balances professionalism and practicality.

[0107] Two core configurations must be completed before training. The training target is quantified using vowel space area: 0.272 kHz² for men, 0.394 kHz² for women, and 0.712 kHz² for children. These values ​​should be scientifically set based on individual circumstances. Training intensity is based on the number of plants planted (e.g., seeds). Each target vowel must reach the target for a single training session to be considered complete. Beginners are advised to start with 10 plants per vowel and gradually increase the number to avoid fatigue.

[0108] refer to Figure 6 The training employs a scenario-based interactive design, requiring users to pronounce the target vowels correctly as prompted on the screen, using voice input to help a virtual bird green the land. Intuitive visual feedback allows users to perceive their pronunciation status in real time, adjust their mouth shape and pronunciation intensity accordingly, and quickly establish correct pronunciation recognition.

[0109] After training, the system will automatically save data such as maximum fundamental frequency, minimum fundamental frequency, average fundamental frequency, maximum intensity, minimum intensity, average intensity, and vowel space area, and generate targeted feedback to inform users of pronunciation accuracy, changes in intensity and pitch, and comparisons with past training, helping users to identify areas for improvement.

[0110] The dynamic adjustment mechanism is a core highlight. When the cumulative training time exceeds 300 seconds (5 minutes), the system analyzes recent data and prompts a pop-up window asking if the plan needs updating. Adjustments are divided into two categories: reducing and increasing difficulty, distinguishing between 24-hour high-sensitivity and 48-hour low-sensitivity scenarios, prioritizing high-sensitivity conditions. In actual testing, when the vowel space area is below 65% or 85% of the set value, the new target is adjusted to 80% of the original set value; when it is above 160% or 130%, it is adjusted to 130% or 125% respectively, allowing training to adapt to the user's state and balancing effectiveness and motivation.

[0111] In one embodiment of the present invention, when determining the male vowel space area in step S10, if the patient's vowel space area is greater than the male first vowel space area threshold, then proceed to step S11; if the patient's vowel space area is less than the male second vowel space area threshold, then add the vowel practice module to the speech training plan and end the speech training plan generation; if the patient's vowel space area is greater than or equal to the male second vowel space area threshold and less than or equal to the male first vowel space area threshold, then add the vowel practice module to the speech training plan and proceed to step S11.

[0112] In a preferred embodiment of the present invention, the male first vowel space area threshold is 11.36 bark squared; the male second vowel space area threshold is 2.84 bark squared.

[0113] In step S10, when determining the area of ​​the female vowel space, if the patient's vowel space area is greater than the threshold for the area of ​​the first female vowel space, proceed to step S11; if the patient's vowel space area is less than the threshold for the area of ​​the second female vowel space, add the vowel practice module to the speech training plan and end the speech training plan generation; if the patient's vowel space area is greater than or equal to the threshold for the area of ​​the second female vowel space and less than or equal to the threshold for the area of ​​the first female vowel space, add the vowel practice module to the speech training plan and proceed to step S11.

[0114] In a preferred embodiment of the present invention, the spatial area threshold for the first female vowel is 13.44 bark squared; the spatial area threshold for the second female vowel is 3.36 bark squared.

[0115] In one embodiment of the present invention, the core objective of glissando exercises is to improve the range and flexibility of pitch gliding in pronunciation, achieve smooth transitions between high and low notes, and broaden the vocal range. It is suitable for speech rehabilitation patients to improve stiff pitch control, broadcasting professionals to optimize vocal flexibility, and ordinary users to improve pronunciation fluency and expressiveness, helping to overcome pitch limitations and achieve natural and smooth pitch changes.

[0116] The training configuration includes two core parameters that users can adjust flexibly. The training difficulty is based on the vocal range, which is the span between high and low frequencies of the sound, calculated as the number of semitones based on piano key frequencies. The system offers different levels of options; beginners can start with a smaller range, while those with some experience can choose medium or higher difficulty. The number of training sessions specifies the effective number of sounds for each mode. It is recommended to start with 3 sessions initially, gradually increasing to 8 sessions or higher to avoid training fatigue.

[0117] refer to Figure 7 The training employs a scenario-based interactive design. Users start with a normal pitch and continuously pronounce words while moving up and down the pitch along a lane on the screen, guiding a virtual car to the corresponding position. The screen displays pitch changes in real time, and users must strive to reach their own high and low pitch limits. The system provides instant feedback on changes in pitch range, maximum fundamental frequency, etc., guiding users to adjust their pronunciation.

[0118] After training, the system automatically saves data such as pronunciation duration, average fundamental frequency, fundamental frequency standard deviation, average intensity, and intensity standard deviation, reflecting the training effect from multiple dimensions and helping users identify key areas for improvement, such as pitch gliding range and stability.

[0119] The dynamic adjustment mechanism is divided into two categories: static and dynamic. Static adjustment requires training duration exceeding 6 minutes, training exceeding 5 days and the last adjustment exceeding 5 days, and an average vocal range greater than the current difficulty level. The number of semitones is adjusted based on the difference: 4 semitones are added if the difference exceeds 5 but does not exceed 32, otherwise 2 semitones are added. Dynamic adjustment addresses training difficulties by displaying a pop-up notification when total practice exceeds 15 seconds and there are more than 4 failures. The new difficulty level is calculated by combining the vocal range of failed pronunciations with the preset level, adapting to the user's ability and balancing training significance and motivation.

[0120] The core goal of tone practice is to improve users' ability to control the pronunciation of the four tones in Mandarin, ensuring that each tone value conforms to the standard and avoiding inaccurate or confused tones. It is suitable for speech rehabilitation patients to improve tone pronunciation disorders, Mandarin learners to correct tone deviations, and ordinary users to improve the clarity and expressiveness of their language. Through targeted practice materials, it helps to establish standard tone recognition and form stable pronunciation habits.

[0121] The training configuration focuses on the selection of training materials and offers flexible operation. After selecting tone exercises, users can access a dedicated page to choose content via the "Select Training Materials" button. The training materials cover 30 high-frequency everyday words, catering to different difficulties and scenarios. Users can determine the initial amount of materials based on their level, starting with a small amount for beginners or those in the early stages of rehabilitation, gradually increasing the amount of practice to ensure a progressive training experience.

[0122] refer to Figure 8 The training process is simple and intuitive, emphasizing practicality and immediate feedback. Users first need to read the text and pinyin of the materials, and then pronounce them according to the intonation and rhythm of the demonstration. The system is equipped with recording and playback functions. After completing the pronunciation, users can compare their own recordings with the demonstration speech to quickly perceive the differences in pitch fluctuations and tone lengths. The training interface clearly displays the practice progress, helping users plan their time and control their pace.

[0123] After training, the system automatically saves data such as pronunciation duration, average fundamental frequency, fundamental frequency standard deviation, average intensity, intensity standard deviation, and comprehensive score. The score is an average assessment of the pronunciation quality of the four tones, directly reflecting the degree of standardization. It also generates targeted feedback, highlighting strengths and weaknesses to help clarify areas for improvement and make training more targeted.

[0124] A dynamic adjustment mechanism facilitates personalized progress. The system analyzes recent data and will prompt users to update their plans when three conditions are met: training duration exceeds 6 minutes, training lasts for more than 5 days and more than 5 days have passed since the last adjustment, and the average score is below 60. A score of 60 corresponds to the passing grade; scores below this require increased training. Adjustments are determined by the current number of materials: less than 15 materials, add 10; less than 20, add 8; less than 25, add 5; and 30 or more materials, no further adjustments are made. This ensures training adapts to the user's abilities, steadily improving pronunciation accuracy.

[0125] In a preferred embodiment of the present invention, the passing threshold for verbal comprehension score is 85 points.

[0126] The core goal of consonant practice is to improve the standard and stability of consonant pronunciation, reduce mispronunciation of initials and confusion of finals, and make speech clearer. It is suitable for speech rehabilitation patients to correct articulation disorders, Mandarin learners to improve pronunciation deviations, and ordinary users to optimize the accuracy of expression. Through targeted training and dynamic adjustment, it helps to establish correct pronunciation habits.

[0127] refer to Figure 9 The training configuration is flexible and adaptable to different needs. The materials are divided into three categories: sentences, tongue twisters, and poems, all meticulously categorized by initial consonant. Sentences help users become familiar with pronunciation in daily contexts, tongue twisters enhance fluency and flexibility, and poems improve fluency and aesthetics. The materials are randomly generated from the selected initial consonant category; the higher the difficulty, the more training materials are available, allowing users to start from lower difficulty levels and gradually progress.

[0128] After training, the system calls a professional interface for a comprehensive evaluation, generating prosody scores, tone scores, completeness scores, as well as information on mispronounced initials and finals and accuracy data. It also records auxiliary data such as average volume and recording duration. Users can compare their own recordings with the demonstration speech, perceive the differences, adjust their pronunciation, and specifically strengthen weak initials.

[0129] After training, the system saves a complete record, including error classification statistics, various scores and pronunciation feedback. Errors are classified into categories such as incorrect initials and incorrect finals to help identify weak points and provide a basis for subsequent training optimization.

[0130] A dynamic adjustment mechanism ensures training adaptability. When three conditions are met—more than 5 training sessions, more than 3 incorrect initial consonants, and more than 5 days of training with an interval exceeding 5 days since the last adjustment—a pop-up window will prompt an update to the plan. The plan focuses on mastering the five initial consonants with the most errors in the selected materials, adjusting the difficulty based on rhythm and tone scores. 60 points is the passing grade, and 90 points is the excellent grade. Different difficulty levels are switched according to score ranges. The adjusted plan can be viewed or manually modified, forming a closed loop to facilitate steady improvement.

[0131] Second Embodiment

[0132] Based on the above method, the second embodiment of the present invention provides a system for generating multi-dimensional speech training plans. For example... Figure 10 As shown, the system includes one or more processors and a memory. The memory is coupled to the processor and is used to store computer programs that, when executed by the processor, can implement the methods described in the above embodiments.

[0133] The processor controls the overall operation of the system to complete all or part of the steps described above. This processor can be a central processing unit (CPU), graphics processing unit (GPU), field-programmable gate array (FPGA), application-specific integrated circuit (ASIC), digital signal processing (DSP) chip, etc. The memory stores various types of data to support the operation of the system. This data may include, for example, instructions for any application or method operating on the system, as well as application-related data. The memory can be implemented using any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, etc.

[0134] In one exemplary embodiment, the system may be implemented by a computer chip or physical entity, or by a product with certain functions, to perform the methods described above and achieve the same technical effects as those methods described above. A typical embodiment is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, an in-vehicle human-machine interface device, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0135] In another exemplary embodiment, the present invention also provides a computer-readable storage medium including program instructions that, when executed by a processor, implement the steps of the method in any of the above embodiments. For example, the computer-readable storage medium may be the memory including the program instructions described above, which can be executed by a processor to complete the above method and achieve the same technical effects as the above method.

[0136] It should be noted that the above embodiments are merely illustrative examples, and the technical solutions of each embodiment can be combined, and the order of each step can be changed, all of which are within the protection scope of this patent.

[0137] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0138] The method and system for generating multi-dimensional speech training plans provided by this invention have been described in detail above. Any obvious modifications made by those skilled in the art without departing from the essence of this invention will constitute an infringement of the patent rights of this invention and will incur corresponding legal liability.

Claims

1. A method for generating a multi-dimensional speech training plan, characterized in that... Includes the following steps: S1: Obtain the patient's age and gender information, and collect their voice samples for voice assessment to obtain assessment result parameters; The evaluation parameters include: sound pressure level, amplitude perturbation, maximum phonation duration, fundamental frequency perturbation, vowel space area, intonation impairment, and speech comprehension score. S2: Determine the patient's sound pressure level; If the sound pressure is less than the first sound pressure threshold, the loudness practice module is added to the speech training plan, and the speech training plan generation ends; if the sound pressure is greater than or equal to the first sound pressure threshold and less than the second sound pressure threshold, or if the sound pressure is greater than the third sound pressure threshold, the loudness practice module is added to the speech training plan, and the process proceeds to step S4; if the sound pressure is greater than or equal to the second sound pressure threshold and less than or equal to the third sound pressure threshold, the process proceeds to step S3. S3: Determine the amplitude perturbation of the patient; If the patient's amplitude perturbation is greater than or equal to the amplitude perturbation threshold, the loudness practice module is added to the speech training plan and the process proceeds to step S4; if the patient's amplitude perturbation is less than the amplitude perturbation threshold, the process proceeds to step S4. S4: Determine the patient's age; If the patient's age is less than or equal to the age threshold, proceed to step S5; if the patient's age is greater than the age threshold, proceed to step S6. S5: Determine the patient's maximum vocal duration; If the patient's maximum vocalization duration is greater than the first maximum vocalization duration threshold for children, proceed to step S7; if the patient's maximum vocalization duration is less than the second maximum vocalization duration threshold for children, add the breath training module and pitch training module to the speech training plan and end the speech training plan generation; if the patient's maximum vocalization duration is greater than or equal to the second maximum vocalization duration threshold for children and less than or equal to the first maximum vocalization duration threshold for children, add the breath training module to the speech training plan and proceed to step S7. S6: Determine the patient's gender; If the gender is male, the maximum vocal duration for males is determined, training is performed based on the determination result, and then the process proceeds to step S7; if the gender is female, the maximum vocal duration for females is determined, training is performed based on the determination result, and then the process proceeds to step S7. Specifically, when determining the maximum vocal duration for males, if the patient's maximum vocal duration is greater than the first maximum vocal duration threshold for males, proceed to step S7; if the patient's maximum vocal duration is less than the second maximum vocal duration threshold for males, add the breath training module and pitch training module to the speech training plan and end the speech training plan generation; if the patient's maximum vocal duration is greater than or equal to the second maximum vocal duration threshold for males and less than or equal to the first maximum vocal duration threshold for males, add the breath training module to the speech training plan and proceed to step S7. When determining the maximum vocal duration for a female, if the patient's maximum vocal duration is greater than the first maximum vocal duration threshold for a female, proceed to step S7; if the patient's maximum vocal duration is less than the second maximum vocal duration threshold for a female, add the breath training module and pitch training module to the speech training plan and end the speech training plan generation; if the patient's maximum vocal duration is greater than or equal to the second maximum vocal duration threshold for a female, and less than or equal to the first maximum vocal duration threshold for a female, add the breath training module to the speech training plan and proceed to step S7. S7: Determine the patient's fundamental frequency perturbation; If the patient's fundamental frequency perturbation is greater than the first fundamental frequency perturbation threshold, then the pitch practice module and singing practice module are added to the speech training plan, and the speech training plan generation ends; if the patient's fundamental frequency perturbation is less than the second fundamental frequency perturbation threshold, then proceed to step S8; if the patient's fundamental frequency perturbation is greater than or equal to the second fundamental frequency perturbation threshold, and less than or equal to the first fundamental frequency perturbation threshold, then the pitch practice module is added to the speech training plan, and proceed to step S8. S8: Further branching judgment based on the patient's age; If the patient's age is less than or equal to the age threshold, proceed to step S9; if the patient's age is greater than the age threshold, proceed to step S10. S9: Determine the area of ​​the patient's vowel space; If the patient's vowel space area is greater than the threshold for the first vowel space area of ​​a child, proceed to step S11; if the patient's vowel space area is less than the threshold for the second vowel space area of ​​a child, add the vowel practice module to the speech training plan and end the speech training plan generation; if the patient's vowel space area is greater than or equal to the threshold for the second vowel space area of ​​a child and less than or equal to the threshold for the first vowel space area of ​​a child, add the vowel practice module to the speech training plan and proceed to step S11. S10: Determine the patient's gender; If the gender is male, the male vowel space area is determined, training is performed based on the determination result, and then the process proceeds to step S11; if the gender is female, the female vowel space area is determined, training is performed based on the determination result, and then the process proceeds to step S11. Specifically, when determining the male vowel space area, if the patient's vowel space area is greater than the male first vowel space area threshold, then proceed to step S11; if the patient's vowel space area is less than the male second vowel space area threshold, then add the vowel practice module to the speech training plan and end the speech training plan generation; if the patient's vowel space area is greater than or equal to the male second vowel space area threshold and less than or equal to the male first vowel space area threshold, then add the vowel practice module to the speech training plan and proceed to step S11. When determining the vowel space area of ​​a female, if the patient's vowel space area is greater than the threshold for the first vowel space area of ​​a female, proceed to step S11; if the patient's vowel space area is less than the threshold for the second vowel space area of ​​a female, add the vowel practice module to the speech training plan and end the speech training plan generation; if the patient's vowel space area is greater than or equal to the threshold for the second vowel space area of ​​a female, and less than or equal to the threshold for the first vowel space area of ​​a female, add the vowel practice module to the speech training plan and proceed to step S11. S11: Assess the patient's intonation impairment; If the patient has intonation impairment, add the glissando practice module and tone practice module to the speech training plan and end the speech training plan generation; if the patient does not have intonation impairment, proceed to step S12. S12: Assess the patient's verbal comprehension score; If the patient's speech comprehension score is greater than or equal to the passing threshold, the speech training plan generation ends; if the patient's speech comprehension score is less than the passing threshold, the consonant practice module is added to the speech training plan, and the speech training plan generation ends.

2. A system for generating multi-dimensional speech training plans, characterized in that... It includes a processor and a memory; wherein the memory is coupled to the processor and is used to store a computer program that, when executed by the processor, causes the processor to implement the generation method as described in claim 1.

Images