Language barrier analysis method and system based on voice information, medium and equipment

Abstract

The invention provides a voice barrier analysis method and system based on voice information, a medium and equipment. The method comprises the following steps: S1, acquiring voice to obtain user voice; s2, performing voice definition analysis on the user voice to obtain a word error rate; s3, if the error word rate is lower than a preset threshold value, determining that the voice definition is normal, and performing voice feature extraction preparation on the user voice to obtain a Mel-cepstrum coefficient; if the wrong word rate is higher than the preset threshold value, judging that the result is language barrier, and entering S5; s4, inputting the Mel-cepstrum coefficient into the trained language barrier classification model to obtain a judgment result of the voice fluency; if the judgment result of the voice fluency is smooth, the judgment result is normal; otherwise, the judgment result is language barrier; and S5, performing result feedback and interaction on a judgment result. According to the method, the speech recognition technology and the convolutional neural network trained by the pronunciation disorder data set are adopted, and the purpose of analyzing the speech clarity and fluency of the user is achieved.

Description

Technical Field

[0001] The present invention relates to the field of medical detection, and in particular to a method, system, medium and device for analyzing a language disorder based on voice information. Background Art

[0002] In the current medical field, the clinical determination of language disorders mainly relies on the evaluation operations conducted by speech pathologists or professional doctors, which are achieved through a series of standardized tests and comprehensive clinical observations. Clinical experts use various standardized assessment tools to test the patient's language comprehension and expression ability. These tests include many aspects, such as vocabulary tests, grammatical ability tests, sentence repetition tasks, etc.

[0003] However, the traditional assessment methods based on standardized tests and clinical observations have certain limitations. Although standardized tests provide relatively standardized and unified methods for the assessment of language disorders to a certain extent, they are difficult to comprehensively and accurately cover all types of language disorders. Especially when facing patients with specific cultural or language backgrounds, these standardized test tools often expose their shortcomings. Due to the significant differences in language structure, expression, semantic connotation, etc. in different cultural and language backgrounds, standardized test tools may not be able to fully adapt to these special situations, resulting in insensitive or inaccurate assessments of patients' language disorders. Similarly, for some patients who use minority languages ​​or have special language habits, standardized test tools may not be able to accurately capture the essential characteristics of their language disorders, thereby affecting the accuracy and pertinence of the diagnosis.

[0004] Through searching patent documents, it was found that the Chinese patent with application number CN113936799A disclosed a method for rapid stroke identification that combines TOPSIS with artificial intelligence. The patent method first determines whether there is a unilateral muscle strength decline by the degree to which the user raises both arms horizontally, and grades the muscle strength difference by the angle between the arms; then, the degree of slurred speech of the user is analyzed by artificial intelligence speech recognition and combined with the error rate calculation method; then, the position coordinates of the facial feature points are detected by the artificial intelligence facial feature point recognition model, and the degree of facial droop is calculated by the TOPSIS method. Finally, the risk of stroke is quickly identified by the TOPSIS method by combining the muscle strength difference classification, the degree of slurred speech and the degree of facial droop. Although this method of identifying speech disorders in stroke can calculate the error rate to quantify the accuracy of the patient's pronunciation, it is difficult to judge whether there is a pronunciation disorder problem based on language fluency.

[0005] In summary, in view of the above-mentioned problems of the prior art, studying a language barrier analysis method and system based on speech information has become a key task that needs to be solved urgently. Summary of the invention

[0006] In view of the defects in the prior art, the purpose of the present invention is to provide a method, system, medium and device for language barrier analysis based on voice information.

[0007] A method for analyzing language barriers based on speech information provided by the present invention comprises the following steps:

[0008] Step S1, voice collection, obtaining user voice;

[0009] Step S2, performing speech clarity analysis on the user's speech to obtain the word error rate;

[0010] Step S3, if the error rate is lower than the preset threshold, the speech clarity is normal, and the user's speech is subjected to speech feature extraction to obtain the Mel-frequency cepstral coefficient; if the error rate is higher than the preset threshold, the speech clarity is abnormal, and the result is a language disorder, and the process proceeds to step S5;

[0011] Step S4, inputting the Mel-frequency cepstral coefficient into the trained language disorder classification model to obtain a judgment result of speech fluency; if the judgment result of speech fluency is fluent, the judgment result is normal; otherwise, the judgment result is language disorder;

[0012] Step S5, feedback and interaction of the judgment result.

[0013] Preferably, step S1 includes: after the user sees a given specific sentence on the front end of the mini program, the user uses the recording function of the device to read the sentence and complete the voice recording to obtain the user voice.

[0014] Preferably, step S2 includes: uploading the user's voice to the cloud platform, the cloud platform uses voice recognition technology to extract the text content in the user's voice, and then compares it with a preset sentence, counts the number of incorrect words, and calculates the error rate.

[0015] Preferably, in step S3, the speech feature extraction preparation includes: performing fast Fourier transform processing on the user's speech and calculating Mel-frequency cepstral coefficients.

[0016] Preferably, in step S3, the threshold is 70%.

[0017] Preferably, step S4 includes the following sub-steps:

[0018] Step S4.1, using a convolutional neural network composed of multiple one-dimensional convolutional layers, batch normalization layers, and maximum pooling layers as a language disorder classification model, and using the pronunciation disorder dataset to train the language disorder classification model to obtain a trained model;

[0019] Step S4.2, input the Mel-frequency cepstral coefficients into the trained model, and output the judgment result of speech fluency.

[0020] Preferably, step S5 includes: sending the judgment result to the mini program page to achieve real-time interaction with the user.

[0021] The present invention also provides a language barrier analysis system based on voice information, comprising:

[0022] Module M1, voice collection, obtains user voice;

[0023] Module M2 performs speech clarity analysis on the user's speech to obtain the word error rate;

[0024] Module M3: If the error rate is lower than the preset threshold, the speech clarity is normal, and the user's speech is extracted to obtain the Mel-frequency cepstrum coefficient; if the error rate is higher than the preset threshold, the speech clarity is abnormal, and the result is a language disorder, and the module M5 is entered;

[0025] Module M4, inputting the Mel-frequency cepstral coefficient into the trained language disorder classification model to obtain the judgment result of speech fluency; if the judgment result of speech fluency is fluent, the judgment result is normal; otherwise, the judgment result is language disorder;

[0026] Module M5 provides feedback and interaction on the judgment results.

[0027] The present invention also provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned method for analyzing language barriers based on speech information are implemented.

[0028] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the steps of the above-mentioned method for analyzing language barriers based on speech information are implemented.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] 1. The present invention adopts speech recognition technology to first extract the specific text content in the speech, and then compare it with the preset sentence to obtain the error word rate of the speech, so as to achieve the purpose of analyzing the clarity of the user's speech.

[0031] 2. The present invention adopts a convolutional neural network trained with a pronunciation disorder dataset to extract the Mel-frequency cepstral coefficients of the user's voice, completes the voice feature extraction through multiple one-dimensional convolutional layers, and then classifies the user's voice, thereby achieving the purpose of analyzing the fluency of the user's voice. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0033] Figure 1 The present invention is a flow chart of a method for analyzing language barriers based on speech information in an embodiment of the present invention. DETAILED DESCRIPTION

[0034] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0035] The present invention adopts a method that combines artificial intelligence speech recognition technology and a deep learning language disorder diagnosis model to judge language disorders from multiple angles such as language clarity and fluency.

[0036] Embodiment 1:

[0037] Figure 1 The present invention is a flow chart of a method for analyzing language barriers based on speech information in an embodiment of the present invention.

[0038] like Figure 1 As shown, this embodiment provides a method for analyzing language barriers based on voice information, comprising the following steps:

[0039] Step S1: voice collection to obtain user voice.

[0040] Specifically, after the user sees a specific sentence given on the front end of the mini program, he uses the recording function of the device to read the sentence and complete the voice recording to obtain the user's voice.

[0041] Step S2, performing speech clarity analysis on the user's speech to obtain the word error rate.

[0042] Specifically, the user's voice is uploaded to the cloud platform, and the cloud platform uses voice recognition technology to extract the text content in the user's voice, and then compares it with the preset sentences, counts the number of incorrect words, and calculates the error rate.

[0043] Step S3, if the error rate is lower than the preset threshold, the speech clarity is normal, and the user's speech is prepared for speech feature extraction to obtain the Mel-frequency cepstral coefficient; if the error rate is higher than the preset threshold, the speech clarity is abnormal, and the judgment result is language disorder, and enter step S5.

[0044] Specifically, the speech feature extraction preparation includes: performing fast Fourier transform processing on the user's speech and calculating Mel-frequency cepstral coefficients as input to the language disorder classification model.

[0045] In this embodiment, the threshold is 70%.

[0046] Step S4, inputting the Mel-frequency cepstral coefficient into the trained language disorder classification model to obtain the judgment result of speech fluency; if the judgment result of speech fluency is fluent, the judgment result is normal; otherwise, the judgment result is language disorder.

[0047] Specifically, step S4 includes the following sub-steps:

[0048] Step S4.1, using a convolutional neural network consisting of multiple one-dimensional convolutional layers, batch normalization layers and maximum pooling layers as a language disorder classification model, and using the pronunciation disorder dataset to train the language disorder classification model to obtain a trained model.

[0049] Step S4.2, input the Mel-frequency cepstral coefficients into the trained model, and output the judgment result of speech fluency.

[0050] Step S5, feedback and interaction of the judgment result.

[0051] Specifically, the judgment result is sent to the mini program page to achieve real-time interaction with the user.

[0052] Embodiment 2:

[0053] This embodiment provides a language barrier analysis system based on voice information, including:

[0054] Module M1, voice collection, obtains user voice;

[0055] Module M2 performs speech clarity analysis on the user's speech to obtain the word error rate;

[0056] Module M3: If the error rate is lower than the preset threshold, the speech clarity is normal, and the user's speech is extracted to obtain the Mel-frequency cepstrum coefficient; if the error rate is higher than the preset threshold, the speech clarity is abnormal, and the result is a language disorder, and the module M5 is entered;

[0057] Module M4, inputting the Mel-frequency cepstral coefficient into the trained language disorder classification model to obtain the judgment result of speech fluency; if the judgment result of speech fluency is fluent, the judgment result is normal; otherwise, the judgment result is language disorder;

[0058] Module M5 provides feedback and interaction on the judgment results.

[0059] Embodiment 3:

[0060] This embodiment provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the steps of a method for analyzing a language barrier based on speech information in the above-mentioned embodiment 1 are implemented.

[0061] Embodiment 4:

[0062] This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the steps of a method for analyzing a language barrier based on speech information in the above-mentioned embodiment 1 are implemented.

[0063] Those skilled in the art know that, in addition to realizing the system and its various devices, modules, and units provided by the present invention in a purely computer-readable program code, it is entirely possible to realize the same functions in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered as a hardware component, and the devices, modules, and units included therein for realizing various functions can also be regarded as structures within the hardware component; the devices, modules, and units for realizing various functions can also be regarded as both software modules for realizing the method and structures within the hardware component.

[0064] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A method for analyzing language barriers based on speech information, characterized in that: The following steps are involved: Step S1, voice collection, obtaining user voice; Step S2, performing speech clarity analysis on the user's speech to obtain a word error rate; Step S3, if the error rate is lower than the preset threshold, the speech clarity is normal, and the user's speech is subjected to speech feature extraction to obtain Mel-frequency cepstral coefficients; if the error rate is higher than the preset threshold, the speech clarity is abnormal, and the result is a language disorder, and the process proceeds to step S5; Step S4, inputting the Mel-frequency cepstral coefficient into the trained language disorder classification model to obtain a judgment result of speech fluency; If the judgment result of the speech fluency is fluent, the judgment result is normal; Otherwise, the result is determined to be a language barrier; Step S5, providing feedback and interaction on the judgment result.

2. A method for analyzing language barriers based on speech information according to claim 1, characterized in that: The step S1 includes: after the user sees a specific sentence given on the front end of the mini program, the user uses the recording function of the device to read the sentence and complete the voice recording to obtain the user voice.

3. A method for analyzing language barriers based on speech information according to claim 1, characterized in that: The step S2 includes: uploading the user voice to the cloud platform, the cloud platform uses voice recognition technology to extract the text content in the user voice, and then compares it with the preset sentence, counts the number of wrong words, and calculates the wrong word rate.

4. The method for analyzing language barriers based on speech information according to claim 1, characterized in that: In the step S3, the speech feature extraction preparation includes: performing fast Fourier transform processing on the user speech and calculating Mel-frequency cepstral coefficients.

5. The method for analyzing language barriers based on speech information according to claim 1, characterized in that: In step S3, the threshold is 70%.

6. The method for analyzing language barriers based on speech information according to claim 1, characterized in that: The step S4 includes the following sub-steps: Step S4.1, using a convolutional neural network composed of multiple one-dimensional convolutional layers, batch normalization layers, and maximum pooling layers as a language disorder classification model, and training the language disorder classification model using a pronunciation disorder dataset to obtain a trained model; Step S4.2, inputting the Mel-frequency cepstral coefficients into the trained model, and outputting the judgment result of speech fluency.

7. The method for analyzing language barriers based on speech information according to claim 1, characterized in that: The step S5 includes: sending the judgment result to the mini program page to achieve real-time interaction with the user.

8. A language barrier analysis system based on voice information, characterized in that: include: Module M1, voice collection, obtains user voice; Module M2, performing speech clarity analysis on the user's speech to obtain a word error rate; Module M3: if the error rate is lower than the preset threshold, the speech clarity is normal, and the user's speech is subjected to speech feature extraction to obtain Mel-frequency cepstral coefficients; if the error rate is higher than the preset threshold, the speech clarity is abnormal, and the result is a language disorder, and the module M5 is entered; Module M4, inputting the Mel-frequency cepstral coefficient into the trained language disorder classification model to obtain a judgment result of speech fluency; If the judgment result of the speech fluency is fluent, the judgment result is normal; Otherwise, the result is determined to be a language barrier; Module M5 provides feedback and interaction on the judgment result.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of a method for analyzing a language disorder based on speech information according to any one of claims 1 to 7 are implemented.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the computer program is executed by a processor, the steps of a method for analyzing a language disorder based on speech information according to any one of claims 1 to 7 are implemented.

Images