Multi-logic interaction system for speech disorder rehabilitation training

Abstract

The invention relates to the technical field of rehabilitation training, in particular to a multi-logic interaction system for speech disorder rehabilitation training, which comprises a voice accuracy analysis module, a voice mouth shape comparison module, a training item distribution module, a training difficulty adjustment module and a review interval optimization module. By capturing and analyzing the pronunciation information of the user in real time, pronunciation errors can be accurately recognized and corrected, the pertinence and efficiency of voice training are enhanced, a patient can visualize the pronunciation errors by combining voice data with a mouth shape image, and meanwhile, by comparing standard pronunciation images, the pronunciation errors can be accurately recognized and corrected. According to the method, integration of voice and visual information is enhanced, each patient can obtain a rehabilitation scheme meeting target requirements and ability through intelligent distribution of training content and time and dynamic adjustment of training difficulty, training distribution and difficulty adjustment are optimized, a user can train in a proper rhythm, and the training efficiency is improved. And the individuation level and the overall effect of speech disorder rehabilitation training are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of rehabilitation training, and particularly to a multi-logic interaction system for speech disorder rehabilitation training. Background Art

[0002] The technical field of rehabilitation training involves the use of various technologies and methods to help individuals recover or improve physical or cognitive functions that have declined due to diseases, injuries, or chronic illnesses. The field of rehabilitation training is widely applied in clinical medicine, mental health, and physical therapy, focusing on improving the quality of life of patients through physical training, cognitive-behavioral training, and the use of assistive devices. The field of rehabilitation training utilizes computer technology, sensor technology, virtual reality, and artificial intelligence to design more personalized and interactive treatment programs. It helps rehabilitation experts more accurately assess the rehabilitation needs of patients, implement more effective treatment plans, and monitor the progress of patients.

[0003] Among them, the multi-logic interaction system for speech disorder rehabilitation training is a high system designed specifically for speech disorder patients. Its main purpose is to help patients improve their language expression ability and communication skills through interactive training and speech recognition technology. The system can adjust the training plan according to the specific situation of the patient, and use speech recognition technology and speech synthesis technology to provide feedback and guidance, so as to train for different types of speech disorders. Its main purpose is to help patients gradually recover normal speech functions, improve the quality of life, and assist professional rehabilitation therapists to conduct more effective training guidance through systematic training and real-time feedback; Traditional interaction systems are not precise enough in real-time feedback and error correction during speech training, resulting in patients being unable to immediately understand and correct their pronunciation errors, limiting the maximization of training effects. Traditional systems adopt a relatively fixed mode in training content distribution and difficulty adjustment, which is not flexible enough and difficult to optimize according to the specific needs of different patients, affecting the pertinence and effectiveness of training. For example, the training plan of traditional systems is too simple or too difficult for some patients, which will affect the learning motivation and training results of patients. Traditional systems fail to effectively integrate visual information, such as the comparison between mouth shape images and pronunciation, making the training lack intuitiveness and interactivity, limiting the improvement of rehabilitation effects, resulting in a slow and unstable rehabilitation process for patients. Summary of the Invention

[0004] The purpose of the present invention is to solve the deficiencies existing in the prior art, and to propose a multi-logic interaction system for speech disorder rehabilitation training.

[0005] To achieve the above purpose, the present invention adopts the following technical solution: A multi-logic interaction system for speech disorder rehabilitation training, the system includes: The voice accuracy analysis module, based on the language rehabilitation training platform, collects the voice sounds of the user during the rehabilitation training process, extracts the pronunciation information of the user, evaluates the accuracy of the user's voice by comparing with the standard voice, identifies the words with abnormal pronunciation, and obtains the abnormal pronunciation identification information; The voice mouth shape comparison module, based on the abnormal pronunciation identification information, during the user's voice rehabilitation training process, captures the user's mouth image in real time through an image capture device, compares it with the standard pronunciation mouth image, identifies the mouth shape image deviating from the normal pronunciation, gives wrong guidance to the user, and obtains the wrong voice guidance information; The training item allocation module, based on the wrong voice guidance information, by analyzing the voice characteristics of the user, combining with the user's training objectives, evaluates the importance of each rehabilitation training content, and combines with the total duration of the user's rehabilitation training to adjust the time allocation of each training content, and obtains the optimized rehabilitation training allocation information; The training difficulty adjustment module, based on the optimized rehabilitation training allocation information, counts the occurrence frequency of wrong voices in each training cycle, analyzes the change of the occurrence frequency of wrong voices, recommends a matching voice training difficulty for the user through the display interface, adjusts the user's rehabilitation training difficulty, and obtains the training difficulty optimized adjustment information; The review interval optimization module, based on the training difficulty optimized adjustment information, detects the pronunciation standard of the sentences that the user has trained, and adjusts the review interval of the trained content according to the change of the pronunciation standard, and obtains the optimized review training information.

[0006] The improvement of the present invention is that the acquisition step of the abnormal pronunciation identification information is as follows: Based on the language rehabilitation training platform, through displaying text, pronunciation images and playing voices, conduct voice rehabilitation training for the user, collect the voice sounds of the user during the rehabilitation training process, and obtain the user voice data record; Based on the user voice data record, conduct sound analysis on the user voice data, extract key pronunciation information, including the pronunciation and intonation of each word, and obtain the voice feature information; Based on the voice feature information, compare it with the standard voice corresponding to each word, through the formula: ; Calculate the standard score of each word, compare it with the preset standard threshold, mark the words with the standard score lower than the preset standard threshold as abnormal pronunciation words, and obtain the abnormal pronunciation identification information; Wherein, represents the standard score of the word, represents the weight of the th pronunciation feature, represents the performance value of the user in the th pronunciation feature, Indicates the standard value of the th pronunciation feature, Indicates the tolerance range of the th pronunciation feature, Indicates the total number of pronunciation features.

[0007] The improvement of the present invention is that the step of obtaining the incorrect voice guidance information is as follows: Based on the abnormal pronunciation recognition information, during the user's voice rehabilitation training, the image capture device is used to capture the user's mouth image in real time to obtain the user's mouth image record; Based on the user's mouth image record, combined with the user's voice capture information, through time comparison, the image corresponding to each pronunciation is marked, compared with the standard pronunciation mouth image, and through the formula: ; Calculate the deviation degree of the mouth shape; Wherein, Represents the deviation degree of the mouth shape, Represents the th mouth shape feature of the user, Represents the th feature of the standard mouth shape, Represents the th weight coefficient of the mouth shape feature, : is the exponential coefficient, Represents the total number of mouth shape features; Based on the deviation degree of the mouth shape, compared with the preset deviation threshold, the mouth shape exceeding the preset deviation threshold is marked as abnormal, combined with the abnormal pronunciation recognition information, the text content with incorrect pronunciation of the user is displayed through the display interface, and the user's mouth image and the standard pronunciation mouth image are compared and displayed, and the user is guided through language interaction by repeatedly playing the correct voice to obtain the incorrect voice guidance information.

[0008] The improvement of the present invention is that the step of evaluating the training importance of each rehabilitation training content is as follows: Based on the incorrect voice guidance information, analyze the user's voice characteristics, including counting the repeated occurrence times of words and phrases deviating from the normal sentence, recording the occurrence times of non-standard pronunciation words, and combining the training duration, analyze the occurrence frequency of each abnormal feature to obtain the abnormal feature frequency record; Based on the abnormal feature frequency record, extract the user's current training difficulty information, analyze the deviation between the current training difficulty and the target training difficulty at the end of the rehabilitation training to obtain the importance correlation data; Based on the importance correlation data, through the formula: ; Calculate the importance score of each rehabilitation training content to obtain the evaluation result of the importance of rehabilitation training content; Among them, represents the importance score of the rehabilitation training content, represents the occurrence frequency of the abnormal feature corresponding to the rehabilitation training content, represents the deviation between the target training difficulty and the current training difficulty, and are adjustment coefficients, is the base of the natural logarithm.

[0009] The improvement of the present invention is that the step of obtaining the optimized rehabilitation training allocation information is: Based on the evaluation result of the importance of rehabilitation training content, collect the importance scores of each rehabilitation training content, count the sum of the importance scores of each training content, and extract the total duration of rehabilitation training set by the user to obtain time allocation correlation data; Based on the time allocation correlation data, through the formula: ; Calculate the time allocated to each rehabilitation training content, adjust the time allocation of each training content, and obtain the optimized rehabilitation training allocation information; Among them, represents the time allocated to the rehabilitation training content, is the importance score of the rehabilitation training content, is the total duration of the user's rehabilitation training, is the reserved redundant time, is the sum of the importance scores of all rehabilitation training contents.

[0010] The improvement of the present invention is that the step of analyzing the change in the occurrence frequency of incorrect speech is: Based on the optimized rehabilitation training allocation information, collect the voice training information of the user within a period of time, count the occurrence frequency of incorrect speech in each training cycle, and obtain the incorrect speech frequency statistical information for each cycle; Based on the incorrect speech frequency statistical information for each cycle, through the formula: ; Calculate the change rate of the incorrect speech frequency to obtain the analysis result of the change in the occurrence of incorrect speech; Among them, is the change rate of the incorrect speech frequency, represents the difference in the incorrect speech frequency between adjacent cycles, represents the time interval of the corresponding cycle.

[0011] The improvement of the present invention is that the step of obtaining the optimized adjustment information of the training difficulty is: Based on the analysis result of the change in the incorrect speech, extract the incorrect speech frequency and the corresponding change rate in the current training cycle, and extract the training difficulty in the current training cycle to obtain difficulty-related data; Based on the difficulty-related data, through the formula: ; Calculate the recommended training difficulty; Wherein, is the recommended training difficulty, is the current training difficulty, is the incorrect speech frequency in the current cycle, and is the adjustment coefficient, is the difficulty change threshold, is the change rate of the incorrect speech frequency; Based on the recommended training difficulty, display the current training difficulty information and the recommended training difficulty information to the user through the display interface, and adjust the rehabilitation training difficulty of the user according to the user's interactive selection to obtain training difficulty optimization and adjustment information.

[0012] The improvement of the present invention is that the obtaining step of the optimized review training information is: Based on the training difficulty optimization and adjustment information, conduct review training for the user at a preset basic review interval, analyze the standardization of the pronunciation of each word in the review training to obtain review training detection information; Based on the review training detection information, evaluate the average value of the standardization of each word in each sentence according to the standard of the pronunciation of each word in the review training to obtain review-related data; Based on the review-related data, through the formula: ; Calculate the adjusted review interval, adjust the review interval of the trained content to obtain optimized review training information; Wherein, is the adjusted review interval, is the basic review interval, is the adjustment coefficient, is the average value of the standardization scores of each word in the review training sentence, is the target standardization score, is the base of the natural logarithm.

[0013] Compared with the prior art, the advantages and positive effects of the present invention are: In the present invention, by capturing and analyzing the user's pronunciation information in real time, pronunciation errors can be accurately identified and corrected, enhancing the pertinence and efficiency of speech training. By combining speech data with lip images, patients can visualize their pronunciation errors. At the same time, by comparing with standard pronunciation images, the integration of speech and visual information is strengthened, which is directly helpful for improving pronunciation accuracy. Through the intelligent allocation of training content time and the dynamic adjustment of training difficulty, each patient can obtain a rehabilitation plan that meets the target requirements and capabilities, optimizing the training allocation and difficulty adjustment, enabling users to train at a suitable rhythm, and improving the personalization level and overall effectiveness of speech disorder rehabilitation training. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 is the system flow chart of the present invention; Figure 2 is the flow chart for obtaining abnormal pronunciation recognition information of the present invention; Figure 3 is the flow chart for obtaining error speech guidance information of the present invention; Figure 4 is the flow chart for evaluating the importance of each rehabilitation training content of the present invention; Figure 5 is the flow chart for obtaining optimized rehabilitation training allocation information of the present invention; Figure 6 is the flow chart for analyzing the change in the occurrence frequency of error speech of the present invention; Figure 7 is the flow chart for obtaining optimized training difficulty adjustment information of the present invention; Figure 8 is the flow chart for obtaining optimized review training information of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0015] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0016] In the description of the present invention, it should be understood that the orientation or positional relationship indicated by the terms "length", "width", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. is based on the orientation or positional relationship shown in the accompanying drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation to the present invention. In addition, in the description of the present invention, the meaning of "a plurality of" is two or more unless otherwise specifically defined.

[0017] Please refer to Figure 1 , the present invention provides a technical solution: a multi-logic interaction system for speech disorder rehabilitation training, the system includes: Based on the language rehabilitation training platform, the speech accuracy analysis module conducts speech rehabilitation training on users by displaying texts, pronunciation images and playing voices, collects the speech sounds of users during the rehabilitation training process, extracts the pronunciation information of users, including the pronunciation and intonation of each word, and evaluates the accuracy of users' speech by comparing with the standard speech, identifies the words with abnormal pronunciation, and obtains the abnormal pronunciation recognition information; Based on the abnormal pronunciation recognition information, the speech lip shape comparison module captures the user's lip images in real time through an image capture device during the user's speech rehabilitation training process, combines the user's speech capture information, labels the images corresponding to each pronunciation, and compares them with the standard pronunciation lip images, identifies the lip images that deviate from the normal pronunciation, provides error guidance to the user, and obtains the error speech guidance information; Based on the error speech guidance information, the training item allocation module analyzes the user's speech characteristics, including inaccurate pronunciation and stuttering, evaluates the importance of each rehabilitation training content according to the occurrence frequency of each abnormal feature, combines the user's training objectives, and adjusts the time allocation of each training content in combination with the total duration of the user's rehabilitation training to obtain the optimized rehabilitation training allocation information; Based on the optimized rehabilitation training allocation information, the training difficulty adjustment module collects the user's speech training information within a period of time, counts the occurrence frequency of error speech in each training cycle, analyzes the change of the occurrence frequency of error speech, recommends a matching speech training difficulty for the user through the display interface, and adjusts the user's rehabilitation training difficulty according to the user's interactive selection to obtain the training difficulty optimized adjustment information; Based on the training difficulty optimized adjustment information, the review interval optimization module detects the pronunciation standardization of the sentences that the user has trained, adjusts the review interval of the trained content according to the change of pronunciation standardization, and obtains the optimized review training information.

[0018] The abnormal pronunciation recognition information includes the pronunciation accuracy score of each word, the pitch and rhythm deviation data, and the sentence marks with non-standard intonation. The error speech guidance information includes the lip images with inaccurate pronunciation, the comparison information between the correct and wrong lip shapes, and the improvement suggestions for each wrong lip shape. The optimized rehabilitation training allocation information includes the priority ranking of each training content, the recommended time allocation information for each content, and the corresponding training intensity. The training difficulty optimized adjustment information is specifically the dynamic adjustment standard of the training difficulty, the training objective setting for each stage, and the training parameters adjusted according to the error frequency. The optimized review training information adjusts the review interval time, the review frequency of each sentence type, and the difficulty level of the review content.

[0019] Please refer toFigure 2 , the steps for obtaining abnormal pronunciation recognition information are as follows: Based on the language rehabilitation training platform, through presenting text, pronunciation images, and playing voices, conduct voice rehabilitation training for users, collect the voice sounds during the users' rehabilitation training process, and obtain the user voice data record; Based on the language rehabilitation training platform, present text and pronunciation images. The user conducts pronunciation practice by referring to the pronunciation images through the platform, and the platform plays standard voices for the user to refer to. During the process, each pronunciation of the user is collected and recorded by the platform through a microphone. The data collection is mainly achieved through the voice acquisition unit. The cooperation of the microphone and the sound card ensures the accuracy of the data. The standard for data collection is to ensure that the frequency range of the voice signal is between 20 Hz and 20 kHz, and the sampling frequency is not lower than 16 kHz. After collection, the data will be preliminarily processed by the audio signal processing module to remove background noise, and the audio signal will be converted into clear audio data suitable for analysis through a filter to ensure the effectiveness of the audio data. The result of the audio signal processing will be stored in the form of audio features, providing basic data for subsequent pronunciation analysis.

[0020] Based on the user voice data record, conduct sound analysis on the user voice data, extract key pronunciation information, including the pronunciation and intonation of each word, to obtain the voice feature information; After obtaining the user voice data record, conduct sound analysis on the voice data of each word through an audio processing algorithm, extract the voice feature information of each word. The spectral analysis method of the audio signal is adopted. Calculate the frequency distribution of each word through the fast Fourier transform, extract parameters such as the frequency feature, pitch, and duration of each word, and compare with the standard voice to calculate the difference degree between the pronunciation feature of each word and the standard pronunciation feature. During the analysis process, if the deviation of the pitch of a certain word from the standard pronunciation pitch exceeds the set threshold, then the pronunciation feature of that word will be marked as an abnormal pronunciation feature. The marking threshold is that the pitch difference value is ±5 Hz. Suppose a user's pronunciation feature is 180 Hz and the standard pronunciation is 175 Hz, then the difference here is 5 Hz, which meets the threshold, and the word is marked as abnormal.

[0021] Based on the voice feature information, compare with the standard voice corresponding to each word, through the formula: ; Calculate the standardness score of each word, compare with the preset standardness threshold, and mark the words with a standardness score lower than the preset standardness threshold as abnormal pronunciation words to obtain the abnormal pronunciation recognition information; Among them, represents the standardness score of the word, represents the weight of the th pronunciation feature, Indicates the performance value of the user for the th pronunciation feature, represents the standard value of the th pronunciation feature, represents the tolerance range of the th pronunciation feature, represents the total number of pronunciation features; Formula: ; The advantage of the formula is that by calculating the weight parameters and pronunciation features, the standardness of each word can be accurately quantified, thereby helping the rehabilitation training platform to identify abnormal pronunciations and conduct more personalized speech correction.

[0022] Detailed explanation of the formula and the derivation process of formula calculation: In the formula: represents the standardness score of the word, representing the degree of difference between the pronunciation of the word and the standard pronunciation. represents the weight of the th pronunciation feature, and the weight reflects the influence degree of each pronunciation feature on the overall score. indicates the performance value of the user for the th pronunciation feature, that is, the actual pronunciation feature of the user. represents the standard value of the th pronunciation feature, usually referring to the pronunciation feature of the standard pronunciation. represents the tolerance range of the th pronunciation feature, used to define the acceptable range of the standard value. represents the total number of pronunciation features, that is, the total number of pronunciation features considered in the evaluation process.

[0023] Suppose there are 3 pronunciation features for scoring, and the given values are as follows: , , , , , , , , , , , .

[0024] Substitute into the formula to calculate the standardness score : ; The results show that the standard score of the user's pronunciation is 0.748. According to the preset standard threshold (set at 0.7), the pronunciation of this word is considered to be close to the standard, but there are still certain deviations and pronunciation correction is needed.

[0025] Please refer to Figure 3 , the steps for obtaining incorrect pronunciation guidance information are as follows: Based on the abnormal pronunciation recognition information, during the user's speech rehabilitation training, the image capture device is used to capture the user's mouth image in real time to obtain the user's mouth image record; Based on the abnormal pronunciation recognition information, the platform uses the image capture device to capture the user's mouth image in real time. The captured images are recorded during each pronunciation practice. A high-definition camera or other image capture device is used to monitor the user in real time to ensure that the movements of the user's mouth are captured, including the opening and closing of the lips, the position of the tongue, and other key parts of the oral cavity. The collected video stream is converted into an image sequence, and the mouth movement images of each key pronunciation are intercepted at regular intervals to extract the key information of the user's mouth, such as the mouth shape contour, the degree of lip opening and closing, etc. This information can help analyze whether the user's pronunciation is standard, and the images will be used as the basis for subsequent comparison with the standard pronunciation images to ensure that potential problems in the user's pronunciation can be accurately identified.

[0026] Based on the user's mouth image record, combined with the user's speech capture information, through time comparison, mark the image corresponding to each pronunciation and compare it with the standard pronunciation mouth image. Through the formula: ; Calculate the degree of deviation of the mouth shape; Among them, represents the degree of deviation of the mouth shape, represents the th mouth shape feature of the user, represents the th feature of the standard mouth shape, represents the th weight coefficient of the mouth shape feature, : is the exponential coefficient, represents the total number of mouth shape features; Formula: ; The advantage of the formula is that by quantifying the degree of deviation of the mouth shape, it can effectively detect and identify the inaccurate parts in the user's pronunciation, thus providing more accurate feedback in speech rehabilitation training.

[0027] Detailed explanation of the formula and the derivation process of the formula calculation: In the formula: represents the degree of deviation of the mouth shape, reflecting the difference between the user's mouth shape and the standard mouth shape. Represents the th mouth shape feature of the user, usually including features such as the opening and closing degree of the lips and the position of the tongue. Represents the th feature of the standard mouth shape, referring to the mouth shape movements of the standard speech. Represents the weight coefficient of the th mouth shape feature, and the weight coefficient can be set according to the importance of each mouth shape feature. Is the exponential coefficient, controlling the influence degree of the mouth shape deviation.

[0028] Suppose there are 3 mouth shape features for calculation, and the given values are as follows: , , , , , , , , , , .

[0029] Calculate the deviation degree of the mouth shape : ; The result shows that the deviation degree of the mouth shape is 0.0247. According to the preset deviation threshold (set to 0.03), this value indicates that the difference between the user's mouth shape and the standard pronunciation is small, within the normal range, and no correction is required. If the deviation degree exceeds the set threshold, it will be determined as an abnormal mouth shape, and adjustment suggestions will be provided to the user.

[0030] Based on the deviation degree of the mouth shape, compare it with the preset deviation threshold, mark the mouth shape that exceeds the preset deviation threshold as abnormal, combine the abnormal pronunciation recognition information, display the text content with incorrect pronunciation of the user through the display interface, and compare and display the user's mouth image and the standard pronunciation mouth image, and conduct language interaction guidance for the user by repeatedly playing the correct voice to obtain the incorrect voice guidance information; Based on the comparison between the deviation degree of the mouth shape and the preset deviation threshold, it is judged whether the user's mouth shape exceeds the preset deviation threshold. The core of the process is to compare based on the difference between the actually captured mouth shape feature data and the mouth shape data of standard pronunciation. Compare the deviation degree of the mouth shape during each pronunciation process. For example, if the mouth shape of the user's pronunciation deviates from the mouth shape of the standard pronunciation by more than the set threshold (for example, the deviation degree D value is greater than 0.03), the mouth shape during this pronunciation process will be marked as abnormal. The deviation threshold is usually obtained through statistical analysis of a large amount of user pronunciation data. For example, the deviation threshold can be set according to the characteristics of the target group of speech rehabilitation (such as rehabilitation patients or language learners). When abnormal pronunciation is detected, the specific text content during the user's pronunciation process is automatically identified, and the wrongly pronounced words of the user are marked on the display interface. At the same time, the corresponding standard pronunciation words will also be displayed, showing the comparison between the user's mouth image and the mouth image during standard pronunciation, helping the user intuitively see the gap between themselves and the standard pronunciation. Through image comparison, the user can better understand the deviation of their mouth shape during pronunciation, and then adjust their pronunciation and mouth shape movements to achieve a better rehabilitation effect. By playing the standard voice to help the user imitate the correct pronunciation, the standard voice is automatically repeated multiple times so that the user can correct the wrong pronunciation in continuous imitation and feedback, and the user is guided by voice prompts on how to change the mouth shape and pronunciation to improve the accuracy of speech pronunciation. Combining speech recognition technology, the similarity between the speech and the standard pronunciation is monitored in real time during the user's imitation pronunciation process, and instant feedback is provided to help the user gradually correct the pronunciation error until reaching the standard level. Through continuous training and feedback, the user can obtain detailed error speech guidance information, including adjustment suggestions for mouth shape and pronunciation.

[0031] Please refer to Figure 4 , the steps to evaluate the training importance of each rehabilitation training content are as follows: Based on the error speech guidance information, analyze the user's speech characteristics, including counting the repeated occurrence times of words deviating from normal sentences, recording the occurrence times of non-standard pronunciation words, and combining the training duration, analyze the occurrence frequency of each abnormal feature to obtain the abnormal feature frequency record; Based on the incorrect voice guidance information, the platform first analyzes the user's voice characteristics. It mainly identifies and records each pronounced word, especially those that deviate from the normal sentence, and counts the number of times the words repeat during the training process. Then, it obtains the occurrence times of each non-standard pronounced word. For example, during the training process, the system may identify that the word "hello" is frequently mispronounced. When recording the information, it will mark the word and calculate its occurrence frequency. Maybe a certain user mispronounces "hello" 5 times during a 10-minute training process. Combining the error occurrence frequency of each word and correlating it with the training duration for analysis, by analyzing the word frequency that deviates from the normal sentence, the occurrence frequency of abnormal characteristics can be obtained. The process is implemented through the data acquisition module and audio analysis algorithm, and the data is organized into frequency records for subsequent training difficulty analysis and adjustment.

[0032] Based on the abnormal feature frequency record, extract the user's current training difficulty information, analyze the deviation between the current training difficulty and the target training difficulty at the end of the rehabilitation training, and obtain the importance correlation data; Based on the abnormal feature frequency record, extract the user's current training difficulty information, and judge the deviation between the current training difficulty and the target training difficulty at the end of the rehabilitation training. For example, if the current training difficulty is represented by the number of times the user mispronounces, and the target training difficulty is based on the set standard and the user's target completion degree, the difference between the two will be calculated. The difference is quantified as a "deviation" value. Suppose the deviation value of the current training is 0.3 (indicating that the user has a certain degree of difficulty during the training process), then the system will compare this data with the target training difficulty to obtain the importance correlation data. Through analysis, the system can accurately judge the part that the user most needs to strengthen in the current training, so as to provide personalized guidance and feedback.

[0033] Based on the importance correlation data, through the formula: ; Calculate the importance score of each rehabilitation training content to obtain the importance evaluation result of the rehabilitation training content; Among them, represents the importance score of the rehabilitation training content, represents the occurrence frequency of the abnormal characteristics corresponding to the rehabilitation training content, represents the deviation between the target training difficulty and the current training difficulty, and are adjustment coefficients, is the base of the natural logarithm; Formula: ; The benefit of the formula is that by calculating the importance score of each training content, it can help to more precisely focus on the parts that most need improvement in the user's rehabilitation training process, thereby optimizing the training plan and improving the training effect.

[0034] Detailed explanation of the formula and the derivation process of formula calculation: In the formula: represents the importance score of the rehabilitation training content, measuring the contribution of each training content to the user's rehabilitation. represents the occurrence frequency of the abnormal features corresponding to the rehabilitation training content, that is, the frequency of abnormal pronunciations or error features in this training content. represents the deviation between the target training difficulty and the current training difficulty, quantifying the gap between the user's current difficulty and the target difficulty. and are adjustment coefficients, respectively adjusting the influence degree of the deviation on the importance of the training content. is the base of the natural logarithm.

[0035] Let the importance score of the training content be: (indicating that the occurrence frequency of the abnormal features of this training content is 60%), (indicating that the deviation between the current training difficulty and the target difficulty is 0.2), , .

[0036] Calculate the importance score of each training content : ; The result shows that the importance score of the training content is 0.0645, indicating that this training content is relatively low, and the deviation between the current training difficulty and the target difficulty is small. The adjustment focus of the training content may need to be concentrated on other training contents with greater difficulties. If the score is high, it means that the abnormal feature frequency of this training content is high and intensive training is required.

[0037] Please refer to Figure 5 , the steps to optimize the acquisition of rehabilitation training allocation information are as follows: Based on the results of the importance assessment of the rehabilitation training content, collect the importance scores of each rehabilitation training content, calculate the sum of the importance scores of each training content, and extract the total duration of the rehabilitation training set by the user to obtain time allocation correlation data; Based on the importance evaluation results of the rehabilitation training content, collect the importance scores of each training content, record the importance scores of each training content, and the scores reflect the priority of each training content in the rehabilitation process. By calculating the sum of these scores, obtain the total importance score sum of all training contents. Suppose the platform assigns different scores to different training contents according to the user's training history and personalized needs during the analysis process. For example, the training of pronunciation clarity may be assigned a higher score, while the score of rhythm training is lower. Extract the total duration of the rehabilitation training preset by the user, which is set by the user according to personal needs. Suppose the total duration of a certain user's rehabilitation training is 60 minutes. Combining the data can obtain the time allocation correlation data, that is, determine the time proportion occupied by each training content according to the relationship between the importance score of each training content and the total duration. Through the comprehensive calculation of the time and scores set by the user, ensure that each training content can be appropriately allocated time to achieve the best rehabilitation effect.

[0038] Based on the time allocation correlation data, through the formula: ; Calculate the time allocated to each rehabilitation training content, adjust the time allocation of each training content, and obtain the optimized rehabilitation training allocation information; Among them, represents the time allocated to the rehabilitation training content, is the importance score of the rehabilitation training content, is the total duration of the user's rehabilitation training, is the reserved redundant time, is the sum of the importance scores of all rehabilitation training contents; Formula: ; The benefit of the formula is that through this formula, the time allocation of each training content can be accurately calculated according to the importance score and total duration of each training content, so as to realize the reasonable optimization of the training content time and help users obtain the best rehabilitation effect within the limited time.

[0039] Detailed explanation of the formula and the derivation process of the formula calculation: In the formula: represents the time allocated to the rehabilitation training content, indicating the actual time amount allocated to each training content. represents the importance score of the rehabilitation training content, measuring the contribution degree of this training content to the rehabilitation process. is the total duration of the rehabilitation training set by the user, that is, the total time amount set by the user for the entire training process. is the reserved redundant time, used to deal with some unexpected situations or rest time, and ensure the flexibility of the training arrangement. It is the sum of the importance scores of all rehabilitation training contents, representing the total weight value of all training contents.

[0040] There are three training contents, and the total training duration set by the user minutes, and the redundant time minutes. The importance scores of the training contents are as follows: Training 1 (pronunciation clarity): ; Training 2 (rhythm training): ; Training 3 (language fluency): .

[0041] Calculate the sum of the importance scores of all training contents: ; Calculate the time allocated to each training content: ; ; ; The results show that according to the importance scores of each training content and the total training duration, the time allocation of the training contents is as follows: 30 minutes for pronunciation clarity training; 15 minutes for rhythm training; 5 minutes for language fluency training; The allocation results can help users focus on the training contents that need to be improved the most, while ensuring that other trainings also get appropriate practice time, ensuring the comprehensiveness and efficiency of the rehabilitation training.

[0042] Please refer to Figure 6 , and the steps to analyze the change in the occurrence frequency of incorrect voices are as follows: Based on the optimized rehabilitation training allocation information, collect the user's voice training information over a period of time, count the occurrence frequency of incorrect voices in each training cycle, and obtain the incorrect voice frequency statistical information for each cycle; Based on the optimized rehabilitation training allocation information, collect the user's voice training information over a period of time, mainly including the incorrect voice data in each training cycle, record all the voice data in each training cycle, and by identifying which pronunciations are incorrect, mark the errors, and calculate the occurrence frequency of incorrect voices in each cycle. By calculating the number of incorrect pronunciations in each training cycle and dividing it by the total number of pronunciations in that cycle, the incorrect voice frequency is obtained. For example, if a user pronounces 100 times in a 10 - minute training cycle and 10 of them are incorrect, then the incorrect voice frequency for that cycle is 10 / 100 = 0.1. By recording and counting the occurrence frequency of incorrect voices, it is possible to track the user's pronunciation performance in real - time, help adjust the subsequent training plan, and ensure that each user can focus on the pronunciation parts that need improvement within an appropriate time.

[0043] Based on the statistical information of the error speech frequency for each period, through the formula: ; Calculate the change rate of the error speech frequency to obtain the analysis result of the change in the occurrence of error speech; Among them, is the change rate of the error speech frequency, represents the difference in the error speech frequency between adjacent periods, represents the time interval of the corresponding period; Formula: ; The benefit of the formula is that by calculating the change rate of the error speech frequency, the change trend of the error speech frequency can be quantified, providing data support for subsequent training adjustment and helping to better identify the pronunciation features that need to be improved.

[0044] Detailed explanation of the formula and the derivation process of the formula calculation: In the formula: is the change rate of the error speech frequency, indicating the change rate of the error speech frequency between adjacent periods. represents the difference in the error speech frequency between adjacent periods, that is, the difference in the error speech frequency between two adjacent periods. represents the time interval of the corresponding period, that is, the time difference between two periods.

[0045] Suppose the error speech frequencies of two periods are as follows: Period 1: Error speech frequency ; Period 2: Error speech frequency ; The time interval between Period 1 and Period 2 is minutes.

[0046] Calculate the change rate of the error speech frequency : ; The result shows that the change rate of the error speech frequency between two periods is 0.01 per minute, indicating that the occurrence frequency of the error speech increases at a relatively slow speed during the training process. According to this change rate, the system can judge the current training effect. If the frequency changes too fast or too slow, the system can adjust the training content, such as increasing some pronunciation training with greater difficulty or adjusting the focus of the training, so as to ensure that the user's rehabilitation progress is consistent with the expectation.

[0047] Please refer to Figure 7 for the steps to obtain the training difficulty optimization adjustment information: Based on the analysis results of the changes in incorrect speech, extract the incorrect speech frequency and the corresponding change rate for the current training cycle, and extract the training difficulty of the current training cycle to obtain difficulty-related data; Based on the analysis results of the changes in incorrect speech, extract the incorrect speech frequency and the corresponding change rate for the current training cycle. The data comes from the incorrect speech frequency statistics information in the previous training cycle, and combined with the change rate of incorrect speech, it provides an important basis for adjusting the training difficulty. The extraction of training difficulty is quantified through the feedback provided by the user and the data analyzed by the system. The incorrect speech frequency in the training cycle represents the frequency of pronunciation deviations, while the change rate shows how the pronunciation deviations evolve over time. During the process, the pronunciation effect of each training cycle is monitored in real time, the number of incorrect speeches is recorded and analyzed, and combined with the duration of the cycle, the incorrect speech frequency and change rate within each cycle are obtained, thus ensuring the accuracy and timeliness of the data. Through the extraction and analysis of these data.

[0048] Based on the difficulty-related data, through the formula: ; Calculate the recommended training difficulty; Where, is the recommended training difficulty, is the current training difficulty, is the incorrect speech frequency of the current cycle, and are adjustment coefficients, is the difficulty change threshold, is the change rate of the incorrect speech frequency; Formula: ; The benefit of the formula is that through the quantitative adjustment of the training difficulty, the difficulty of the training content can be flexibly adjusted according to the user's actual pronunciation performance, so as to provide personalized training guidance and ensure that the user obtains the best results during the rehabilitation process.

[0049] Detailed explanation of the formula and the derivation process of the formula calculation: In the formula: is the recommended training difficulty, representing the adjusted difficulty level. is the current training difficulty, representing the difficulty level of the user's current training. is the incorrect speech frequency of the current cycle, representing the incorrect pronunciation frequency of the user within this cycle. and are adjustment coefficients, representing the relationship weights between the training difficulty and the incorrect speech frequency and change rate. is the difficulty change threshold, representing the standard of difficulty change set by the system. is the change rate of the error voice frequency, indicating the speed at which the error voice frequency changes over time.

[0050] Let: the current training difficulty ; the error voice frequency in the current cycle ; the change rate of the error voice frequency ; the difficulty change threshold ; the adjustment coefficient , ; Substitute into the formula to calculate the recommended training difficulty : ; The result shows that the recommended training difficulty is 3, indicating that under the influence of the error voice frequency and change rate in the current training cycle, it is recommended to maintain the current training difficulty. Through this adjustment, the system ensures that the training difficulty matches the user's progress, avoiding poor training effects caused by excessive difficulty increase or stagnant effects caused by being too simple.

[0051] Based on the recommended training difficulty, display the current training difficulty information and the recommended training difficulty information to the user through the display interface, and adjust the user's rehabilitation training difficulty according to the user's interaction selection to obtain the optimized adjustment information of the training difficulty; Based on the recommended training difficulty, the system will display the current training difficulty and the recommended training difficulty to the user through the display interface. Display both the current training difficulty and the recommended training difficulty adjusted according to factors such as the error voice frequency and change rate to ensure that the user can clearly understand their current training level and the recommended training goal. This display process includes two parts: The first part is the current training difficulty, usually displayed through an intuitive chart or progress bar, reflecting the user's current pronunciation situation and training status; the second part is the recommended training difficulty, indicating the difficulty recommendation suitable for the user based on the system's analysis and adjustment. The information will guide the user to understand the system's assessment of their rehabilitation progress. When displaying, the interface will prominently show the gap between the current and recommended difficulties to help the user quickly identify the direction of adjustment. The user will make a choice based on the current and recommended difficulty information provided by the system. The user can decide whether to accept the recommended difficulty adjustment according to their own feelings and needs. For example, the user may think that the current difficulty is suitable for them and choose to continue the current training; or, the user may choose to adjust the training difficulty to adapt to new challenges or reduce the training intensity. At this time, the system will record the user's choice and adjust the subsequent training plan accordingly.

[0052] Please refer to Figure 8 , the steps for obtaining the optimized review training information are: Optimize and adjust the information based on the training difficulty, conduct review training for the user according to the preset basic review interval, analyze the standardization of the pronunciation of each word in the review training, and obtain the review training detection information; Based on the training difficulty optimization adjustment information, conducting review training for the user according to the preset basic review interval will analyze the pronunciation standardization of each word to facilitate the detection of the user's pronunciation accuracy. During the review training process, the difference between each word and the standard pronunciation is analyzed through a speech recognition algorithm to evaluate its standardization. For example, during the review training, the deviation between the words pronounced by the user and the standard pronunciation will be checked, and the standardization of each word will be judged by calculating factors such as pitch, voice quality, and pronunciation clarity. The standardization of each word will be scored according to the pronunciation correctness to obtain the review training detection information, and these detection information will be used for subsequent analysis to adjust the content and strategy of the review training, thereby optimizing the user's training effect.

[0053] Based on the review training detection information, evaluate the average value of the standardization of each word in each sentence according to the standard of the pronunciation of each word in the review training, and obtain the review correlation data; Based on the review training detection information, calculate the average value of the standardization scores of each word in each sentence. First, analyze the standardization of each word word by word, and then comprehensively evaluate the standardization of the words in each sentence. Average the standardization scores of each word to obtain the comprehensive standardization score of each sentence, which reflects the accuracy of the overall pronunciation of the sentence. The data provides a basis for subsequent training difficulty adjustment. For example, if the pronunciation of multiple words in a paragraph is relatively accurate, the overall score of the sentence will be higher; otherwise, the system will feedback to the user the areas that need improvement. By continuously recording and analyzing these scores, more accurate feedback can be provided during the review training process to ensure the maximization of the effect of each training cycle.

[0054] Based on the review correlation data, through the formula: ; Calculate the adjusted review interval, adjust the review interval of the trained content, and obtain the optimized review training information; Among them, is the adjusted review interval, is the basic review interval, is the adjustment coefficient, is the average value of the standardization scores of each word in the review training sentence, is the target standardization score, is the base of the natural logarithm; Formula: ; The advantage of the formula is that by quantitatively calculating the adjustment of the review interval, it can dynamically optimize the review interval according to the gap between the user's pronunciation standard and the target standard, making the training process more personalized and improving the rehabilitation efficiency.

[0055] Detailed explanation of the formula and the derivation process of formula calculation: In the formula: is the adjusted review interval, representing the review time interval adjusted according to the pronunciation standard. is the basic review interval, representing the standard review interval time without adjustment. is the adjustment coefficient, controlling the relationship strength between the standardness score and the review interval. is the average value of the standardness scores of each word in the review training sentence, representing the pronunciation standardness of the user in this sentence. is the target standardness score, representing the standard pronunciation target set by the user. is the base of the natural logarithm.

[0056] Let: basic review interval days; the average value of the standardness scores of the user's current sentence ; target standardness score ; adjustment coefficient ; Substitute into the formula to calculate the adjusted review interval : ; The result shows that according to the gap between the pronunciation standardness score of the current sentence and the target standardness score, the adjusted review interval is 13.5 days, indicating that the review time needs to be increased to review when the pronunciation standardness reaches the target standard. The adjustment helps the system dynamically adjust the review plan according to the progress of the user's pronunciation, ensuring the personalization and efficiency of the training.

[0057] The above is only a preferred embodiment of the present invention, and it does not limit the present invention in other forms. Any person skilled in the art may use the disclosed technical content to make changes or modifications into equivalent embodiments with equivalent changes and apply them to other fields. However, as long as it does not depart from the technical solution content of the present invention, any simple modification, equivalent change, and modification made to the above embodiments based on the technical essence of the present invention still fall within the protection scope of the technical solution of the present invention.

Claims

1. A multi-logic interactive system for speech disorder rehabilitation training, characterized in that: The system comprises: The voice accuracy analysis module is based on the language rehabilitation training platform. It collects the user's voice sounds during the rehabilitation training process, extracts the user's pronunciation information, evaluates the accuracy of the user's voice by comparing it with the standard voice, identifies words with abnormal pronunciation, and obtains abnormal pronunciation recognition information; The speech mouth shape comparison module is based on the abnormal pronunciation recognition information. During the user's speech rehabilitation training, the user's mouth image is captured in real time by an image capture device, and compared with the standard pronunciation mouth image, to identify the mouth shape image that deviates from the normal pronunciation, and to provide error guidance to the user, thereby obtaining error speech guidance information; The training project allocation module analyzes the user's voice characteristics based on the erroneous voice guidance information, combines the user's training goals, evaluates the importance of each rehabilitation training content, and adjusts the time allocation of each training content based on the user's total rehabilitation training time to obtain optimized rehabilitation training allocation information; The training difficulty adjustment module counts the frequency of erroneous speech in each training cycle based on the optimized rehabilitation training allocation information, analyzes the change in the frequency of erroneous speech, recommends matching speech training difficulty to the user through the display interface, adjusts the rehabilitation training difficulty of the user, and obtains training difficulty optimization adjustment information; The review interval optimization module detects the user's pronunciation standardization of the trained sentences based on the training difficulty optimization adjustment information, adjusts the review interval of the trained content according to the change of pronunciation standardization, and obtains optimized review training information.

2. The multi-logic interactive system for speech disorder rehabilitation training according to claim 1, characterized in that: The steps for obtaining the abnormal pronunciation recognition information are as follows: Based on the language rehabilitation training platform, the user is given speech rehabilitation training by displaying text, pronunciation images and playing voice, and the voice sound of the user during the rehabilitation training is collected to obtain the user's voice data record; Based on the user voice data record, perform sound analysis on the user voice data to extract key pronunciation information, including the pronunciation and intonation of each word, to obtain voice feature information; Based on the voice feature information, the standard voice corresponding to each word is compared, and the formula is: ； Calculate the standardization score of each word, compare it with the preset standardization threshold, mark the words with standardization scores lower than the preset standardization threshold as abnormal pronunciation words, and obtain abnormal pronunciation recognition information; in, Indicates the standardness score of a word. Indicates The weight of the pronunciation feature, Indicates that the user The performance value of the pronunciation feature, Indicates The standard value of the pronunciation feature, Indicates The tolerance range of pronunciation features, Indicates the total number of pronunciation features.

3. The multi-logic interactive system for speech disorder rehabilitation training according to claim 1, characterized in that: The steps for obtaining the error voice guidance information are as follows: Based on the abnormal pronunciation recognition information, during the user's speech rehabilitation training, the user's mouth image is captured in real time by an image capture device to obtain a user's mouth image record; Based on the user's mouth image record, combined with the user's voice capture information, through time comparison, the image corresponding to each pronunciation is marked and compared with the standard pronunciation mouth image, through the formula: ； Calculate the degree of deviation of the mouth shape; in, Indicates the degree of deviation of the mouth shape. Indicates the user's Mouth shape features, Indicates the standard mouth shape Features, Indicates The weight coefficient of each mouth shape feature, : is the exponential coefficient, Indicates the total number of mouth features; Based on the degree of deviation of the mouth shape, it is compared with a preset deviation threshold, and the mouth shape that exceeds the preset deviation threshold is marked as abnormal. Combined with the abnormal pronunciation recognition information, the text content with incorrect pronunciation of the user is displayed through a display interface, and the user's mouth image and the standard pronunciation mouth image are compared and displayed. By repeatedly playing the correct voice, the user is guided in language interaction to obtain incorrect voice guidance information.

4. The multi-logic interaction system for speech disorder rehabilitation training according to claim 1, characterized in that: The steps for evaluating the importance of each rehabilitation training content are: Based on the erroneous voice guidance information, analyzing the user's voice features, including counting the number of repeated occurrences of words that deviate from normal sentences, recording the number of occurrences of words with non-standard pronunciations, and analyzing the frequency of occurrence of each abnormal feature in combination with the training time to obtain an abnormal feature frequency record; Based on the abnormal characteristic frequency record, extract the user's current training difficulty information, analyze the deviation between the current training difficulty and the target training difficulty at the end of the rehabilitation training, and obtain importance-related data; Based on the importance associated data, by the formula: ； Calculate the importance score of each rehabilitation training content to obtain the importance assessment results of rehabilitation training content; in, Indicates the importance score of rehabilitation training content, Indicates the frequency of occurrence of abnormal features corresponding to rehabilitation training content, Indicates the deviation between the target training difficulty and the current training difficulty, and is the adjustment factor, is the base of natural logarithms.

5. The multi-logic interactive system for speech disorder rehabilitation training according to claim 4, characterized in that: The steps for obtaining the optimized rehabilitation training allocation information are as follows: Based on the rehabilitation training content importance evaluation result, the importance score of each rehabilitation training content is collected, the sum of the importance score of each training content is calculated, and the total duration of rehabilitation training set by the user is extracted to obtain time allocation related data; Based on the time allocation associated data, by the formula: ； Calculate the time allocated to each rehabilitation training content, adjust the time allocation of each training content, and obtain optimized rehabilitation training allocation information; in, Indicates the time allocated to the rehabilitation training content, is the importance score of rehabilitation training content, is the total duration of the user's rehabilitation training, is the reserved redundant time, It is the sum of the importance scores of all rehabilitation training contents.

6. The multi-logic interaction system for speech disorder rehabilitation training according to claim 1, characterized in that: The step of analyzing the change of the frequency of occurrence of erroneous speech is: Based on the optimized rehabilitation training allocation information, voice training information of the user within a period of time is collected, and the frequency of occurrence of erroneous speech in each training cycle is counted to obtain erroneous speech frequency statistical information of each cycle; Based on the statistical information of the frequency of erroneous speech in each cycle, the formula is: ； Calculate the changing rate of the erroneous speech frequency to obtain the analysis result of the erroneous speech occurrence change; in, is the rate of change of the erroneous speech frequency, Represents the difference in the error speech frequency between adjacent cycles, Indicates the time interval of the corresponding cycle.

7. The multi-logic interaction system for speech disorder rehabilitation training according to claim 6, characterized in that: The steps for obtaining the training difficulty optimization adjustment information are as follows: Based on the analysis result of the occurrence and change of the erroneous speech, the erroneous speech frequency and the corresponding change rate of the current training cycle are extracted, and the training difficulty of the current training cycle is extracted to obtain difficulty-related data; Based on the difficulty correlation data, by the formula: ； Calculate the recommended training difficulty; in, For the recommended training difficulty, is the current training difficulty, is the error speech frequency of the current cycle, and is the adjustment factor, is the difficulty change threshold, is the rate of change of the erroneous speech frequency; Based on the recommended training difficulty, the current training difficulty information and the recommended training difficulty information are displayed to the user through a display interface, and according to the user's interactive selection, the user's rehabilitation training difficulty is adjusted to obtain training difficulty optimization adjustment information.

8. The multi-logic interaction system for speech disorder rehabilitation training according to claim 1, characterized in that: The steps for obtaining the optimized review training information are as follows: Based on the training difficulty optimization adjustment information, review training is performed on the user according to a preset basic review interval, and the standardization of pronunciation of each word in the review training is analyzed to obtain review training detection information; Based on the review training detection information, according to the pronunciation standard of each word in the review training, the average value of the standardization of each word in each sentence is evaluated to obtain review related data; Based on the review of the associated data, the formula: ； Calculate the adjusted review interval, adjust the review interval of the trained content, and obtain optimized review training information; in, is the adjusted review interval, It is the basic review interval. is the adjustment factor, is the average of the standardization scores of each word in the review training sentences, is the target standard score, is the base of natural logarithms.

Images