A speech training hearing aid system equipped with an AI speech anomaly assessment algorithm

Abstract

The present invention discloses a speech training hearing aid system equipped with an AI speech anomaly evaluation algorithm, which relates to the AI speech anomaly evaluation algorithm. By collecting composite signals and formulating an age-adapted rule base, comprehensive hearing evaluation is carried out to obtain the hearing threshold dispersion and the signal-to-noise ratio attenuation amount, and then the hearing level evaluation coefficient is obtained; by collecting speech feature data and formulating a standard speech comparison library, daily speech evaluation is carried out, and a daily speech anomaly evaluation model is constructed using a convolutional neural network learning model, and then the daily speech evaluation coefficient is obtained; by collecting multi-dimensional data, professional speech evaluation is carried out, and then the professional speech evaluation coefficient is obtained; according to the hearing level evaluation coefficient, the daily speech evaluation coefficient and the professional speech evaluation coefficient, analysis is carried out to obtain the analysis result, and a comprehensive analysis plan is formulated, greatly improving the accuracy.

Description

Technical Field

[0001] The present invention relates to an AI speech abnormality assessment algorithm, and specifically to a speech training hearing aid system equipped with an AI speech abnormality assessment algorithm. Background Art

[0002] Congenital hearing loss is a disease that seriously affects the hearing and speech development of children. If not detected and intervened in a timely manner, it will have a long-term and even irreversible impact on the physical and mental health of the children. Without any intervention, hearing-impaired children may experience problems such as delayed language development and unclear speech. In severe cases, they may even develop into deaf-mutes, resulting in social barriers and psychological problems. However, through scientific intervention means, such as hearing aids or cochlear implants and other hearing compensation devices, the hearing condition of the children can be significantly improved, laying a foundation for the improvement of their language learning ability.

[0003] In the early stage of life (0 - 3 years old), which is a critical period for children's language development and also a key stage for the development of the auditory center. Research shows that if effective intervention can be carried out within 6 months after birth, including wearing hearing aids or cochlear implants and combining with systematic speech rehabilitation training, the language ability of the children can reach the development level of normal children. However, in practical applications, problems such as the real-time monitoring of the hearing level and speech level of hearing-impaired children, and the quantification of the speech training results by speech therapists and guardians make it difficult to fully utilize the effect of hearing compensation devices, resulting in a still lagging language development level.

[0004] In this context, it is particularly important to develop a speech training hearing aid system equipped with an AI speech abnormality assessment algorithm. Combining with hearing aids or cochlear implants, through intelligent speech recognition technology, personalized training program design, and real-time feedback mechanism, it helps guardians and speech training teachers carry out rehabilitation work more efficiently. For example, in the infantile period (0 - 3 years old), the device can evaluate the sensitivity of children to sounds and their auditory discrimination ability through simple audio stimulation and gamified interactive tests; in the preschool period (3 - 8 years old), more targeted language training tasks can be designed, such as pronunciation practice, vocabulary learning, etc.

[0005] In addition, the intervention strategy combined with the timeline can better help children gradually improve their language level. In the infantile period, the focus is on cultivating basic auditory perception ability; in the preschool period, it is necessary to strengthen the training of language understanding and expression ability; and after entering the school age (3 - 8 years old), the learning of social communication skills and grammar rules can be further strengthened. Through such phased and systematic intervention, the language development level of children will be significantly improved.

[0006] In summary, the early detection and scientific intervention of congenital hearing loss are of utmost importance. By using devices such as hearing aids or cochlear implants for hearing compensation and combining with professional speech training guidance, the language ability of children can be effectively improved. The development of intelligent and personalized evaluation and training devices provides strong tool support for guardians and speech training teachers, helping to maximize the development level of hearing-impaired children to reach that of normal children and laying a solid foundation for their future integration into society.

[0007] Therefore, a speech training hearing aid system equipped with an AI speech anomaly evaluation algorithm is provided. Summary of the Invention

[0008] To solve the above technical problems, the object of the present invention is to provide a speech training hearing aid system equipped with an AI speech anomaly evaluation algorithm.

[0009] According to one preferred embodiment of the present invention, a speech training hearing aid system equipped with an AI speech anomaly evaluation algorithm includes:

[0010] A hearing level evaluation module for collecting composite signals and formulating an age-adapted rule library, comprehensively evaluating the hearing of hearing-impaired children, obtaining the corresponding hearing threshold dispersion and signal-to-noise ratio attenuation amount, and further obtaining the hearing level evaluation coefficient of the corresponding hearing-impaired children;

[0011] A daily speech evaluation module for collecting speech feature data and formulating a standard speech comparison library, then evaluating the daily speech of hearing-impaired children, and using a convolutional neural network learning model to construct a daily speech anomaly evaluation model, and further obtaining the daily speech evaluation coefficient of the corresponding hearing-impaired children;

[0012] A professional speech evaluation module for collecting multi-dimensional data, professionally evaluating the speech of hearing-impaired children, and further obtaining the professional speech evaluation coefficient of the corresponding hearing-impaired children;

[0013] A speech level analysis module for obtaining the hearing level evaluation coefficient, daily speech evaluation coefficient, and professional speech evaluation coefficient respectively according to the above modules, analyzing them to obtain an analysis result, and formulating a comprehensive analysis plan for the corresponding hearing-impaired children according to the analysis result.

[0014] According to one preferred embodiment of the present invention, the process of collecting composite signals and formulating an age-adapted rule library for comprehensively evaluating the hearing of hearing-impaired children includes:

[0015] Setting an embedded sound field generating device to generate a composite signal, the composite signal including a pure tone signal and a speech noise signal; different frequency points are set for the pure tone signal; the speech noise signal generates a standard composite noise;

[0016] Set up a speech signal acquisition device to collect test sound signals of hearing-impaired children in real time;

[0017] Formulate an age adaptation rule library, which includes age ranges and age correction compensation factors;

[0018] Based on the pure tone signal, speech noise signal, and test sound signal, obtain the corresponding auditory threshold dispersion and signal-to-noise ratio attenuation amount.

[0019] According to one preferred embodiment of the present invention, the process of obtaining the hearing level evaluation coefficient of the corresponding hearing-impaired child based on the corresponding auditory threshold dispersion and signal-to-noise ratio attenuation amount includes:

[0020] Based on different frequency points of the pure tone signal and the test sound signals of hearing-impaired children collected in real time by the speech signal acquisition device, obtain the auditory threshold dispersion between adjacent frequency points;

[0021] Segment according to the standard composite noise and the dynamic range of -20dB to +10dB into a quiet environment, a noise environment, and a reverberation environment, and then obtain the corresponding signal-to-noise ratio attenuation amount;

[0022] Based on the threshold dispersion, signal-to-noise ratio attenuation amount, and age correction compensation factor, obtain the hearing level evaluation coefficient of the corresponding hearing-impaired child.

[0023] According to one preferred embodiment of the present invention, the process of collecting speech feature data, formulating a standard speech comparison library, and then conducting a daily speech evaluation of hearing-impaired children includes:

[0024] Formulate a standard speech comparison library to obtain standard speech data, which includes standard acoustic feature data and standard linguistic feature data;

[0025] Set up a high-precision microphone device to extract speech features of hearing-impaired children to obtain speech feature data; the speech feature extraction includes acoustic feature extraction and linguistic feature extraction, and the speech feature data includes acoustic feature data and linguistic feature data;

[0026] Based on the acoustic feature extraction, obtain acoustic feature data;

[0027] Based on the linguistic feature extraction, obtain linguistic feature data.

[0028] According to one preferred embodiment of the present invention, the process of constructing a daily speech anomaly evaluation model using a convolutional neural network learning model includes:

[0029] Obtain several groups of acoustic feature data and linguistic feature data of different age groups in historical collection periods;

[0030] Obtain the standard acoustic feature data and standard linguistic feature data of normal children of corresponding age groups in the standard speech comparison library;

[0031] According to the acoustic feature data, linguistic feature data of different age groups in several historical collection cycles, the standard acoustic feature data and standard linguistic feature data of the corresponding age groups, form a training sample set;

[0032] Based on the convolutional neural network learning model, construct a standard evaluation model;

[0033] And input the training sample set into the standard evaluation model, train the standard evaluation model, obtain the trained standard evaluation model, and then construct a daily speech abnormality evaluation model.

[0034] According to one preferred embodiment of the present invention, obtain a daily speech evaluation coefficient according to the daily speech abnormality evaluation model.

[0035] According to one preferred embodiment of the present invention, the process of collecting multi-dimensional data and conducting professional speech evaluation on hearing-impaired children includes:

[0036] According to the evaluation by the standardized language evaluation tool, conduct multi-dimensional quantitative evaluation on hearing-impaired children, including: language comprehension quantitative evaluation, language expression quantitative evaluation, vocabulary quantity quantitative evaluation, grammar application quantitative evaluation, and speech clarity quantitative evaluation;

[0037] Set up a multi-dimensional data synchronous acquisition device to obtain multi-dimensional data corresponding to the quantitative evaluation, including: instruction execution accuracy rate, reaction time standard deviation, average sentence length, dependency distance, proportion of age-appropriate words, edit distance of retold syntactic tree, and phoneme comparison correct rate.

[0038] According to one preferred embodiment of the present invention, the process of obtaining the professional speech evaluation coefficient of corresponding hearing-impaired children includes:

[0039] According to the instruction execution accuracy rate, reaction time standard deviation, average sentence length, dependency distance, proportion of age-appropriate words, edit distance of retold syntactic tree, and phoneme comparison correct rate, obtain the professional speech evaluation coefficient of the corresponding hearing-impaired children, denoted as , the professional speech evaluation coefficient is:

[0040] ;

[0041] Among them, is the weight coefficient of the language comprehension quantitative evaluation dimension, is the composite instruction execution accuracy rate, is the reaction time standard deviation, is the mean of the same-age norm, is the standard deviation of the same-age norm, is the weight coefficient of the language expression quantitative evaluation dimension, is the average sentence length, is the maximum compound instruction execution accuracy value corresponding to the age, is the anti-zero value, is the dependency distance, is the proportion of age-appropriate word count, is the weight coefficient of the vocabulary quantitative evaluation dimension, is the edit distance of the retold syntactic tree, is the weight coefficient of the grammar application quantitative evaluation dimension, is the phoneme comparison correct rate and is the weight coefficient of the speech clarity quantitative evaluation dimension.

[0042] According to one preferred embodiment of the present invention, the process of analyzing the hearing level evaluation coefficient, the daily speech evaluation coefficient, and the professional speech evaluation coefficient, obtaining the analysis result, and formulating a comprehensive analysis plan for the corresponding hearing-impaired children includes:

[0043] According to the hearing level evaluation coefficient, the daily speech evaluation coefficient, and the professional speech evaluation coefficient, the analysis result is denoted as , and the analysis result is:

[0044] ;

[0045] Among them, is the weight coefficient of the hearing level evaluation coefficient ; is the weight coefficient of the daily speech evaluation coefficient ; is the weight coefficient of the professional speech evaluation coefficient ;

[0046] Set the standard analysis result ;

[0047] If at this time, the hearing-impaired children can go to a normal school;

[0048] If at this time, formulate a comprehensive analysis plan for the corresponding hearing-impaired children, and the comprehensive analysis plan includes an adaptive speech training plan and a speech training result evaluation;

[0049] The adaptive speech training plan is applicable to the guardian as the main body to help the hearing-impaired children with speech training;

[0050] When the speech training achievement evaluation is applicable to a speech training teacher as the main body, the speech training hearing aid plays an evaluation role, provides an effect evaluation criterion for the speech training teacher, and further assists in completing the speech training course.

[0051] The present invention further provides a computer-readable storage medium storing a computer program executable by a processor to implement the speech training hearing aid system with an AI speech abnormality evaluation algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required in the embodiments. Obviously, the drawings described below are only some embodiments recorded in the present invention, and those of ordinary skill in the art can also obtain other drawings based on these drawings.

[0053] Figure 1 It shows a schematic diagram of the steps of a speech training hearing aid system with an AI speech abnormality evaluation algorithm according to the present invention.

[0054] Figure 2 It shows a schematic diagram of the judgment process of a speech training hearing aid system with an AI speech abnormality evaluation algorithm according to the present invention.

[0055] Figure 3 It shows a schematic diagram of the modules of a speech training hearing aid system with an AI speech abnormality evaluation algorithm according to the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0056] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art can think of other obvious variations. The basic principles defined in the following description can be applied to other implementation schemes, variant schemes, improvement schemes, equivalent schemes, and other technical schemes without departing from the spirit and scope of the present invention.

[0057] It can be understood that the term "one" should be understood as "at least one" or "one or more". That is, in one embodiment, the number of an element can be one, while in other embodiments, the number of the element can be multiple. The term "one" cannot be understood as a limitation on the number.

[0058] Please refer to Figure 1 、 Figure 2 and Figure 3 , a speech training hearing aid system with an AI speech abnormality evaluation algorithm, including a control center, the control center is communicatively connected to a hearing level evaluation module, a daily speech evaluation module, a professional speech evaluation module, and a speech level analysis module;

[0059] The hearing level evaluation module is used to comprehensively evaluate the hearing of children with hearing loss and obtain the hearing level evaluation coefficient corresponding to the children with hearing loss;

[0060] It should be further noted that in the specific implementation process, the specific process of the hearing level evaluation module for comprehensively evaluating the hearing of children with hearing loss includes:

[0061] Set up an embedded sound field generating device;

[0062] The embedded sound field generating device is used to generate a composite signal by using DDS (Direct Digital Synthesis) technology, and the composite signal includes a pure tone signal and a speech noise signal;

[0063] The pure tone signal is set with seven frequency points of 125Hz, 250Hz, 500Hz, 1kHz, 2kHz, 4kHz, and 8kHz. It should be further noted that the duration of each frequency point is 300ms, the rise and fall time is 50ms, the sound pressure level range is 20 - 90dBSPL, and the step size is 5dB;

[0064] The speech noise signal generates a composite noise background of 65dB based on the International Speech Equalization Corpus (IEEE C standard), denoted as the standard composite noise, and the dynamic range of the signal-to-noise ratio is adjustable from -20dB to +10dB;

[0065] Set up a speech signal acquisition device;

[0066] The speech signal acquisition device is composed of a bone conduction sensor array (including 6 piezoelectric ceramic units) arranged in the mastoid area, with a frequency response range of 80 - 10kHz, and is used to collect the test sound signals of children with hearing loss in real time;

[0067] Formulate an age adaptation rule library;

[0068] The age adaptation rule library includes an age interval and an age correction compensation factor. It should be further noted that when the age interval is 0 - 3 years old, the age correction compensation factor is +3dB; when the age interval is 4 - 7 years old, the age correction compensation factor is 0dB; when the age interval is greater than 7 years old, the age correction compensation factor is -2dB.

[0069] It should be further noted that in the specific implementation process, the specific process of the hearing level evaluation module for obtaining the hearing level evaluation coefficient corresponding to the children with hearing loss includes:

[0070] According to the different frequency points of the pure tone signal and the test sound signals of children with hearing loss collected in real time by the speech signal acquisition device, obtain the hearing threshold dispersion of adjacent frequency points. It should be further noted that the specific process of obtaining the hearing threshold dispersion of adjacent frequency points includes:

[0071] The seven frequency points are denoted as i = 1, 2, 3, 4, 5, 6, a total of 6 frequency bands; the hearing threshold corresponding to the test sound signal of the hearing-impaired children collected in real time in the i-th frequency band is denoted as ;

[0072] The threshold dispersion of the i-th frequency band is denoted as , and the threshold dispersion is:

[0073] ; where is the hearing threshold collected for the k-th time, is the average value of the hearing thresholds collected k times;

[0074] According to the standard composite noise and the dynamic range of -20dB to +10dB, it is segmented into a quiet environment, a noise environment, and a reverberation environment, denoted as j = 1, 2, 3 respectively;

[0075] The signal-to-noise ratio attenuation amount in different stages is denoted as , and the signal-to-noise ratio attenuation amount is:

[0076] ; where is the standard reference value, is the actual distance of the ear canal, is the standard ear canal distance;

[0077] The age correction compensation factor in the age adaptation rule library is denoted as , and it should be further explained that the age correction compensation factor is automatically loaded according to the actual age of the hearing-impaired children;

[0078] According to the threshold dispersion , the signal-to-noise ratio attenuation amount and the age correction compensation factor , the hearing level evaluation coefficient corresponding to the hearing-impaired children is obtained, denoted as ; the hearing level evaluation coefficient is:

[0079] ; where is the weight coefficient corresponding to the threshold dispersion , is the speech recognition correction factor in the environment corresponding to the signal-to-noise ratio attenuation amount , is the actual age compensation weight coefficient;

[0080] It should be further explained that according to the sum and average of the threshold dispersions corresponding to the seven frequency points, and the signal-to-noise ratio attenuation amounts and the actual age of the hearing-impaired children to obtain the hearing level evaluation coefficient 。

[0081] The daily speech evaluation module is used to conduct daily speech evaluation on hearing-impaired children, construct a daily speech abnormality evaluation model, and obtain the daily speech evaluation coefficient of the corresponding hearing-impaired children;

[0082] It should be further noted that in the specific implementation process, the specific process of the daily speech evaluation module for conducting daily speech evaluation on hearing-impaired children includes:

[0083] Formulate a standard speech comparison library;

[0084] By collecting the standard speech data of normal children of different ages, the standard speech data includes standard acoustic feature extraction data and standard linguistic feature extraction data;

[0085] Divide according to different ages, and perform noise reduction, frame segmentation, and endpoint detection on the standard speech data of normal children of different ages respectively, and extract the standard acoustic feature extraction data and standard linguistic feature extraction data corresponding to the ages of normal children to form a standard speech comparison library;

[0086] Set up a high-precision microphone device to receive the language performance of hearing-impaired children in natural communication scenarios;

[0087] According to the high-precision microphone device, extract speech features of hearing-impaired children to obtain speech feature data; it should be further noted that the speech feature extraction includes acoustic feature extraction and linguistic feature extraction, the speech feature data includes acoustic feature data and linguistic feature data; the acoustic feature extraction is used to extract the pronunciation clarity and accuracy of hearing-impaired children; the linguistic feature extraction is used to evaluate the vocabulary and grammar of hearing-impaired children;

[0088] It should be further noted that in the specific implementation process, the specific process of extracting acoustic features of hearing-impaired children to obtain acoustic feature data includes:

[0089] The acoustic feature data includes 20-dimensional MFCC coefficients, vowel pronunciation accuracy rate, syllable rate, and pause frequency;

[0090] Collect the speech signals of hearing-impaired children through the high-precision microphone device;

[0091] Perform frame windowing operation on the speech signal, segment the speech signal into 25-ms frames, shift the frames by 10 ms, weight them with a Hamming window, enhance the high-frequency energy of the weighted speech signal through a first-order high-pass filter, and perform FFT calculation to obtain the corresponding magnitude spectrum, and output according to the Mel filter bank; it should be further noted that the Mel filter bank consists of 40 triangular filters;

[0092] Perform DCT transformation on the output 40-dimensional logarithmic energy, and retain the first 20-dimensional discrete logarithmic energy, thereby obtaining 20-dimensional MFCC coefficients;

[0093] Detect the fundamental frequency of vocal cord vibration and the resonance characteristics of the vocal tract through the YIN algorithm and LPC analysis technology, and obtain the corresponding vowel pronunciation accuracy rate;

[0094] Segment the speech signal into syllables, find the energy minimum points in the speech signal, and regard the interval > 80 ms as an independent syllable; record the effective speech duration and pause frequency of the speech signal, and the effective speech duration is denoted as and the pause frequency is denoted as ;

[0095] Obtain the syllable rate according to the number of independent syllables and the effective speech duration as:

[0096] ; where is the number of independent syllables;

[0097] If and the pause frequency , it is recorded as too fast speech rate; and the pause frequency , it is recorded as normal speech rate; and the pause frequency , it is recorded as too slow speech rate;

[0098] It should be further noted that in the specific implementation process, the specific process of extracting linguistic features from hearing-impaired children to obtain linguistic feature data includes:

[0099] The linguistic feature data includes lexical richness, syntactic complexity, and pronunciation error rate;

[0100] According to the speech signal, count the number of diverse words, and through the children's language development norm database, perform word frequency distribution statistics on the number of diverse words to obtain the lexical richness of the corresponding hearing-impaired children;

[0101] According to the dependency parser (such as the CTB8 model), extract the average dependency distance and the number of clauses of the speech signal; according to the dependency distance and the number of clauses, obtain the corresponding syntactic complexity;

[0102] Using an automatic speech recognition (ASR) model, perform phoneme alignment on the speech signal, and perform pronunciation error detection on the speech signal after phoneme alignment to generate a pronunciation error rate; the pronunciation error detection includes substitution errors, omission errors, and distortion errors.

[0103] It should be further noted that in the specific implementation process, the specific process of constructing the daily speech abnormality assessment model includes:

[0104] Obtain acoustic feature data and linguistic feature data of different age groups in a number of historical acquisition periods;

[0105] It should be further noted that classify the acoustic feature data and linguistic feature data of a number of historical acquisition periods according to the age of children with hearing loss to facilitate data unity;

[0106] Obtain the standard acoustic feature data and standard linguistic feature data of normal children corresponding to the age groups in the standard speech comparison library;

[0107] According to the acoustic feature data and linguistic feature data of different age groups in a number of historical acquisition periods, and the standard acoustic feature data and standard linguistic feature data corresponding to the age groups;

[0108] The specific process of constructing the training sample set includes:

[0109] Group and label the acoustic feature data and linguistic feature data of different age groups in a number of historical acquisition periods, and the standard acoustic feature data and standard linguistic feature data corresponding to the age groups, denoted as is a natural number;

[0110] Take groups of acoustic feature data and linguistic feature data of different age groups in a number of historical acquisition periods, and the standard acoustic feature data and standard linguistic feature data corresponding to the age groups as sample data, and is a natural number less than , and use the sample data to obtain the sample data mean, denoted as the sample set;

[0111] Take the remaining groups of acoustic feature data and linguistic feature data of different age groups in a number of historical acquisition periods, and the standard acoustic feature data and standard linguistic feature data corresponding to the age groups as the test set;

[0112] According to the sample set and the test set, form the training sample set;

[0113] Based on the convolutional neural network, construct the standard evaluation model;

[0114] Input the training sample set into the standard evaluation model, train the standard evaluation model, obtain the trained standard evaluation model, and denote the trained standard evaluation model as the daily speech abnormality evaluation model.

[0115] It should be further noted that in the specific implementation process, the specific process of obtaining the daily speech evaluation coefficient of the corresponding hearing-impaired child includes:

[0116] Obtain the daily speech evaluation coefficient according to the daily speech abnormality evaluation model which is:

[0117] ;

[0118] where is the weight coefficient of the 20-dimensional MFCC coefficient, and the value range , is the 20-dimensional MFCC coefficient; is the weight coefficient of the vowel pronunciation accuracy rate, and the value range , is the vowel pronunciation accuracy rate; is the weight coefficient of the syllable rate, and the value range , is the syllable rate; is the weight coefficient of the pause frequency, and the value range , is the pause frequency; is the weight coefficient of the lexical richness, and the value range , is the lexical richness; is the weight coefficient of the syntactic complexity, and the value range , is the syntactic complexity; is the weight coefficient of the pronunciation error rate, and the value range , is the pronunciation error rate.

[0119] The professional speech evaluation module is used to conduct professional speech evaluation on hearing-impaired children and obtain the professional speech evaluation coefficient of the corresponding hearing-impaired children;

[0120] It should be further noted that in the specific implementation process, the specific process of the professional speech evaluation module conducting professional speech evaluation on hearing-impaired children includes:

[0121] Conduct multi-dimensional quantitative evaluation on hearing-impaired children according to the standardized language evaluation tool, and the multi-dimensional quantitative evaluation includes language comprehension quantitative evaluation, language expression quantitative evaluation, vocabulary quantity quantitative evaluation, grammar application quantitative evaluation, and speech clarity quantitative evaluation;

[0122] A multi-dimensional data synchronous acquisition device is set up to obtain multi-dimensional data, where the multi-dimensional data includes the accuracy rate of composite instruction execution, the standard deviation of reaction time, the average sentence length, the dependency distance, the proportion of age-appropriate words, the edit distance of the retold syntactic tree, and the correct rate of phoneme comparison.

[0123] For the quantitative evaluation of language understanding, the multi-dimensional data synchronous acquisition device acquires the accuracy rate of composite instruction execution and the standard deviation of reaction time according to the Z-score standardization technique; for the quantitative evaluation of language expression, the multi-dimensional data synchronous acquisition device acquires the average sentence length and the dependency distance according to the percentile ranking conversion technique; for the quantitative evaluation of vocabulary, the multi-dimensional data synchronous acquisition device acquires the proportion of age-appropriate words according to the logarithmic conversion technique and the linear scaling technique; for the quantitative evaluation of grammar application, the multi-dimensional data synchronous acquisition device acquires the edit distance of the retold syntactic tree according to the Levenshtein distance normalization technique; for the quantitative evaluation of speech clarity, the multi-dimensional data synchronous acquisition device acquires the correct rate of phoneme comparison according to the Gaussian mixture model probability density model.

[0124] It should be further noted that in the specific implementation process, the specific process of the professional speech evaluation module obtaining the professional speech evaluation coefficient corresponding to the hearing-impaired children includes:

[0125] Mark the accuracy rate of composite instruction execution, the standard deviation of reaction time, the average sentence length, the dependency distance, the proportion of age-appropriate words, the edit distance of the retold syntactic tree, and the correct rate of phoneme comparison, and record them as , , , , , and ;

[0126] According to the accuracy rate of composite instruction execution , the standard deviation of reaction time , the average sentence length , the dependency distance , the proportion of age-appropriate words , the edit distance of the retold syntactic tree and the correct rate of phoneme comparison , obtain the professional speech evaluation coefficient corresponding to the hearing-impaired children, and record it as ; The professional speech evaluation coefficient is:

[0127] ;

[0128] Among them, is the weight coefficient of the quantitative evaluation dimension of language understanding, is the mean value of the same-age norm, is the standard deviation of the norm for the same age, is the weight coefficient of the quantitative evaluation dimension of language expression, is the maximum accuracy value of the execution of the composite instruction corresponding to the age, is the anti-zero value, is the weight coefficient of the quantitative evaluation dimension of vocabulary, is the weight coefficient of the quantitative evaluation dimension of grammar application and is the weight coefficient of the quantitative evaluation dimension of speech clarity.

[0129] The speech level analysis module is used to analyze the hearing level evaluation coefficient, the daily speech evaluation coefficient, and the professional speech evaluation coefficient, obtain the analysis result, and formulate a comprehensive analysis plan for the corresponding hearing-impaired children according to the analysis result;

[0130] It should be further noted that in the specific implementation process, the specific process of formulating a comprehensive analysis plan for the corresponding hearing-impaired children includes:

[0131] According to the hearing level evaluation coefficient , the daily speech evaluation coefficient and the professional speech evaluation coefficient , the obtained analysis result is recorded as , and the analysis result is:

[0132] ; where is the weight coefficient of the hearing level evaluation coefficient ; is the weight coefficient of the daily speech evaluation coefficient ; is the weight coefficient of the professional speech evaluation coefficient ;

[0133] Set the standard analysis result ;

[0134] If , at this time, the hearing-impaired children can go to a normal school;

[0135] If , formulate a comprehensive analysis plan for the corresponding hearing-impaired children, and the comprehensive analysis plan includes an adaptive speech training plan and an evaluation of the results of speech training;

[0136] The adaptive speech training plan is applicable when the main body is the guardian. The speech training hearing aid cooperates with the mobile phone APP to guide the guardian to help the hearing-impaired children conduct speech training, which mainly includes the following parts:

[0137] 1. Auditory training: Gradually improve children's auditory ability from sound detection, discrimination, recognition to understanding. Play different types of scenarios (animals, means of transportation) through the mobile phone APP, and simultaneously play sounds in the ear to let children distinguish the sound sources and characteristics. Conduct auditory memory training, such as asking children to repeat a group of words after listening.

[0138] 2. Pronunciation training: Start with oral motor function training (such as pursing the lips, grinning, sticking out the tongue) to help children enhance their oral muscle control ability, and then conduct pronunciation exercises for single sounds, syllables, words, and sentences. For pronunciation errors, such as substitution errors, omission errors, and distortion errors, conduct targeted corrective training.

[0139] 3. Language comprehension training: Gradually improve children's language comprehension ability from simple vocabulary comprehension (recognizing common objects and people's names), to sentence comprehension (executing instructions, answering simple questions), and then to text comprehension (listening to stories and answering relevant questions).

[0140] 4. Language expression training: From imitating speech and actively expressing needs (such as "I want to drink water"), to describing things, telling events, and having dialogue exchanges, improve the accuracy, fluency, and richness of children's language expression.

[0141] 5. Cultivation of cognitive and social abilities: Combine with the laws of cognitive development, and expand children's cognitive scope through games, activities, etc., such as recognizing colors, shapes, numbers. At the same time, create social scenarios to let children interact with their peers or family members to improve their social skills and communication abilities.

[0142] When the subject of the language training achievement evaluation is a language training teacher, the language training hearing aid plays an evaluation role. Through the hearing level evaluation coefficient , the daily speech evaluation coefficient and the professional speech evaluation coefficient provide a criterion for the language training teacher to judge the effect, and then assist in completing the language training course.

[0143] Embodiments disclosed in the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. Embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program codes for performing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network through a communication part, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), the above-mentioned functions defined in the methods of the present application are performed. It should be noted that the computer-readable medium in the present application can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wire segments, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer-readable storage medium can be any tangible medium that contains or stores a program, and the program can be used by or combined with an instruction execution system, apparatus, or device. In the present application, the computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries the computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, and the computer-readable medium can send, propagate, or transmit a program for use by or combined with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted by any suitable medium, including but not limited to: wireless segments, wire segments, optical cables, RF, etc., or any suitable combination of the above.

[0144] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a portion of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks may occur in a different order than that noted in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0145] Those skilled in the art should understand that the embodiments of the present invention described above and shown in the accompanying drawings are only examples and do not limit the present invention. The objectives of the present invention have been fully and effectively achieved. The functions and structural principles of the present invention have been demonstrated and explained in the embodiments. Without departing from the said principles, the embodiments of the present invention may have any variations or modifications.

Claims

1. A speech training hearing aid system equipped with an AI speech anomaly assessment algorithm, characterized in that, Including: A hearing level assessment module, which is used to collect composite signals and formulate an age-adapted rule base, conduct a comprehensive hearing assessment on hearing-impaired children, obtain corresponding threshold dispersion and signal-to-noise ratio attenuation, and then obtain a hearing level assessment coefficient for the corresponding hearing-impaired children. The hearing level assessment coefficient is: ; wherein, is the hearing level evaluation coefficient; is the threshold dispersion; is the signal-to-noise ratio attenuation; is the age correction compensation factor; is the corresponding threshold dispersion weight coefficient, is the corresponding signal-to-noise ratio attenuation under the environment of speech recognition correction factor, is the actual age compensation weight coefficient; A daily speech assessment module, which is used to collect speech feature data and formulate a standard speech comparison library, then conduct a daily speech assessment on hearing-impaired children, and use a convolutional neural network learning model to construct a daily speech abnormality assessment model, and then obtain a daily speech assessment coefficient for the corresponding hearing-impaired children. The daily speech assessment coefficient is: ； Among them, is the daily speech evaluation coefficient; is the weight coefficient of the 20-dimensional MFCC coefficient, is the 20-dimensional MFCC coefficient; is the weight coefficient of the vowel pronunciation accuracy rate, is the vowel pronunciation accuracy rate; is the weight coefficient of the syllable rate, is the syllable rate; is the weight coefficient of the pause frequency, is the pause frequency; is the weight coefficient of the lexical richness, is the lexical richness; is the weight coefficient of the syntactic complexity, is the syntactic complexity; is the weight coefficient of the pronunciation error rate, is the pronunciation error rate; A professional speech assessment module, which is used to collect multi-dimensional data, conduct a professional speech assessment on hearing-impaired children, and then obtain a professional speech assessment coefficient for the corresponding hearing-impaired children, including: According to the standardized language assessment tool, conduct multi-dimensional quantitative assessments on hearing-impaired children, including: language comprehension quantitative assessment, language expression quantitative assessment, vocabulary quantitative assessment, grammar application quantitative assessment, and speech clarity quantitative assessment; Set up a multi-dimensional data synchronous acquisition device to obtain multi-dimensional data for the corresponding quantitative assessment, including: instruction execution accuracy rate, reaction time standard deviation, average sentence length, dependency distance, proportion of age-adapted words, retelling syntactic tree editing distance, and phoneme comparison correct rate; and obtain the professional speech evaluation coefficient of the corresponding hearing-impaired children, denoted as , the professional speech evaluation coefficient is as follows: ； Among them, is the weight coefficient of the language comprehension quantitative evaluation dimension, is the accuracy rate of composite instruction execution, is the standard deviation of reaction time, is the mean of the same-age norm, is the standard deviation of the same-age norm, is the weight coefficient of the language expression quantitative evaluation dimension, is the average sentence length, is the maximum accuracy rate value of composite instruction execution corresponding to age, is to prevent zero values, is the dependency distance, is the proportion of age-appropriate word count, is the weight coefficient of the vocabulary quantitative evaluation dimension, is the edit distance of the retold syntactic tree, is the weight coefficient of the grammar application quantitative evaluation dimension, is the correct rate of phoneme comparison and is the weight coefficient of the speech clarity quantitative evaluation dimension; A speech level analysis module, which is used to obtain the hearing level assessment coefficient, daily speech assessment coefficient, and professional speech assessment coefficient according to the above modules respectively, conduct analysis, obtain the analysis result, and formulate a comprehensive analysis plan for the corresponding hearing-impaired children according to the analysis result.

2. The language training hearing aid system equipped with the AI speech anomaly evaluation algorithm according to claim 1, characterized in that The process of collecting composite signals and formulating an age-adapted rule base to conduct a comprehensive hearing assessment on hearing-impaired children includes: Set up an embedded sound field generating device to generate composite signals, where the composite signals include pure tone signals and speech noise signals; different frequency points are set for the pure tone signals; the speech noise signals generate standard composite noise; Set up a speech signal acquisition device to collect the test sound signals of hearing-impaired children in real time; Formulate an age-adapted rule base, where the age-adapted rule base includes age intervals and age correction compensation factors; According to the pure tone signals, speech noise signals, and test sound signals, obtain the corresponding threshold dispersion and signal-to-noise ratio attenuation.

3. The language training hearing aid system equipped with the AI speech abnormality evaluation algorithm according to claim 2, characterized in that The process of obtaining the hearing level assessment coefficient for the corresponding hearing-impaired children according to the corresponding threshold dispersion and signal-to-noise ratio attenuation includes: According to the different frequency points of the pure tone signals and the test sound signals of hearing-impaired children collected in real time by the speech signal acquisition device, obtain the threshold dispersion of adjacent frequency points; Segment according to the standard composite noise and the dynamic range of -20dB to +10dB into a quiet environment, a noise environment, and a reverberation environment, and then obtain the corresponding signal-to-noise ratio attenuation; According to the threshold dispersion, signal-to-noise ratio attenuation, and age correction compensation factor, obtain the hearing level assessment coefficient for the corresponding hearing-impaired children.

4. The language training hearing aid system with an AI speech anomaly evaluation algorithm according to claim 3, characterized in that, The process of collecting speech feature data and formulating a standard speech comparison library to conduct a daily speech assessment on hearing-impaired children includes: Develop a standard speech comparison library and obtain standard speech data, where the standard speech data includes standard acoustic feature data and standard linguistic feature data; Set up a high-precision microphone device to extract speech features from children with hearing loss and obtain speech feature data; the speech feature extraction includes acoustic feature extraction and linguistic feature extraction, and the speech feature data includes acoustic feature data and linguistic feature data; Obtain acoustic feature data according to the acoustic feature extraction; Obtain linguistic feature data according to the linguistic feature extraction.

5. The language training hearing aid system equipped with the AI speech anomaly evaluation algorithm according to claim 4, characterized in that, The process of using a convolutional neural network learning model to construct a daily speech abnormality assessment model includes: Obtain several sets of acoustic feature data and linguistic feature data of different age groups in historical collection periods; Obtain the standard acoustic feature data and standard linguistic feature data of normal children of corresponding age groups in the standard speech comparison library; According to the several sets of acoustic feature data, linguistic feature data, standard acoustic feature data and standard linguistic feature data of different age groups in the historical collection period, form a training sample set; Based on the convolutional neural network learning model, construct a standard evaluation model; And input the training sample set into the standard evaluation model to train the standard evaluation model, and obtain the trained standard evaluation model, and then construct a daily speech abnormality assessment model.

6. The language training hearing aid system equipped with the AI speech abnormality evaluation algorithm according to claim 1, characterized in that The process of analyzing the hearing level evaluation coefficient, daily speech evaluation coefficient and professional speech evaluation coefficient, obtaining the analysis result, and formulating a comprehensive analysis plan for the corresponding children with hearing loss according to the analysis result includes: According to the hearing level evaluation coefficient, the daily speech evaluation coefficient, and the professional speech evaluation coefficient, the obtained analysis result is denoted as , and the said analysis result is as follows: ； Among them, is the hearing level evaluation coefficient weight coefficient; is the daily speech evaluation coefficient weight coefficient; is the professional speech evaluation coefficient weight coefficient; Set standard analysis results ; If At this time, children with hearing loss can go to a normal school to study; If When, formulate a comprehensive analysis plan for children with hearing loss, and the comprehensive analysis plan includes an adaptive speech training plan and an evaluation of speech training results; The adaptive speech training plan is applicable to the guardian as the subject to help children with hearing loss with speech training; The speech training result evaluation is applicable when the subject is a speech training teacher. The speech training hearing aid plays an evaluation role and provides an effect evaluation standard for the speech training teacher, so as to assist in completing the speech training course.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and the computer program can be executed by a processor to implement a speech training hearing aid system with an AI speech abnormality assessment algorithm according to any one of claims 1-6 above.

Images