A speech training recognition method and system

Abstract

The present invention relates to the field of speech training technology, and its purpose is to provide a speech training recognition method and system. The method includes: converting the training audio data into text to obtain training text data; performing word segmentation processing on the training text data to obtain multiple word segmentation texts, and then obtaining a word text sequence; obtaining the start time and end time of multiple word segmentation texts in the word text sequence in the training audio data, cutting the training audio data to obtain multiple word segmentation audios, and then obtaining a word audio sequence; obtaining an evaluation result based on the word text sequence and the word audio sequence. The present invention can improve the accuracy and consistency of the evaluation results in the speech training process, can greatly reduce the labor cost in the training process, the training and evaluation efficiency is higher, and the problem points of the training users can be accurately located.

Description

Technical Field

[0001] The present invention relates to the technical field of speech training, and in particular to a speech training recognition method and system. Background Art

[0002] Speech, the material shell of language, refers to the sound produced by humans through their vocal organs, which has a certain meaning and is used for social communication. Speech is the external form of language. Intonation mainly refers to the speaker's configuration and changes in speed, lightness and heaviness, and includes three sub-dimensions: speaking speed, pitch and intensity.

[0003] In the process of voice services such as mobile communication customer service, service personnel need to deal with service recipients with different emotions and make appropriate responses. In order for service personnel to provide voice services quickly, professionally and in a standardized manner, it is usually necessary to provide service skills training, such as voice training, to service personnel in advance. In the prior art, when conducting voice training evaluation, the tone of voice of service personnel is evaluated and recommended mainly through subjective judgment by trainers. However, in the process of using the prior art, the inventors found that there are at least the following problems in the prior art: manual voice evaluation of service personnel is subjective to a certain extent, and the evaluation results are inaccurate. Summary of the invention

[0004] The present invention aims to solve the above technical problems at least to a certain extent, and provides a speech training recognition method and system.

[0005] The technical solution adopted by the present invention is:

[0006] In a first aspect, the present invention provides a speech training recognition method, comprising:

[0007] Receiving the training audio data input by the training object in real time, and converting the training audio data into text to obtain the training text data;

[0008] Performing word segmentation processing on the training text data to obtain a plurality of word segmentation texts, and then sorting the plurality of word segmentation texts according to the order in which they appear in the training text data to obtain a word text sequence;

[0009] Obtaining the start time and end time of multiple word segmentation texts in the word text sequence in the training audio data;

[0010] According to the start time and end time of multiple word segmentation texts in the word text sequence in the training audio data, the training audio data is cut to obtain multiple word segmentation audios, and then the multiple word segmentation audios are sorted according to the order of their appearance in the training audio data to obtain a word audio sequence;

[0011] According to the word text sequence and the word audio sequence, the parameters to be evaluated of each word segmentation audio in the word audio sequence are obtained, and the parameters to be evaluated are compared with pre-stored standard parameters to obtain an evaluation result.

[0012] In the present invention, by converting the training audio data into training text data, obtaining multiple segmented texts in the training text data, and then obtaining the corresponding start time and end time of the multiple segmented texts in the training audio data in a text processing manner, the accuracy of obtaining the segmented text in the training audio data and the start time and end time of the segmented text can be improved.

[0013] The present invention can visualize the detailed defects of the language expression of the training users, which can help the trainers understand the defects in the voice training process and improve the voice service ability of the trainees by repeatedly practicing the defect details.

[0014] Compared with the prior art, the present invention can achieve the following effects: by adopting a standardized evaluation method, each evaluation follows the same standard, avoiding the problem of inconsistent evaluation standards at different times due to personal factors of the trainer, resulting in fluctuations in evaluation quality, thereby improving the accuracy and consistency of the judgment results in the voice training process, and facilitating the visualization of the service capability improvement effect; in addition, since the prepared standard case data can be permanently saved and reused, the labor cost in the training process can be greatly reduced. At the same time, the prepared standard case data can support simultaneous training of multiple people at the same time, breaking through the limitation that manual evaluation can only be conducted one-on-one, and the efficiency of training and evaluation is higher; furthermore, since the present invention performs fine dimensional quantification and comparison of the actual training audio data and standard cases input by the training subjects, it can accurately locate the problem points of the training users, facilitating targeted training of the training users.

[0015] In a possible design, after receiving the training audio data input by the training subject, the speech training recognition method further includes:

[0016] The training audio data is subjected to a de-mute process to obtain audio data after the de-mute process, and then the training audio data is converted into text.

[0017] In a possible design, the training audio data is subjected to a muting process to obtain the audio data after the muting process, including:

[0018] Acquire the amplitude sequence of the training audio data, and extract continuous audio segments in which all amplitude values ​​of the training audio data are less than a preset amplitude threshold;

[0019] Obtaining the duration of the continuous audio segment;

[0020] It is determined in turn whether the duration of the continuous audio segment is greater than a preset time threshold. If so, the continuous audio segment is determined to be a silent audio segment, and the silent audio segment in the training audio data is deleted until all continuous audio segments with a duration greater than the preset time threshold are deleted to obtain audio data after de-silencing.

[0021] In a possible design, obtaining the start time and end time of multiple word segmentation texts in the word text sequence in the training audio data includes:

[0022] Obtain the number of characters in the word text sequence and the duration of the training audio data, obtain the average duration of each character according to the number of characters and the duration of the training audio data, and use the average duration as an estimated value to mark the estimated start time and estimated end time of each character in the word text sequence in order;

[0023] Obtaining the position of the single audio data corresponding to each character in the training audio data according to the estimated start time and the estimated end time;

[0024] Obtain the specified text, and modify the estimated start time of the specified text to the start time of the specified text;

[0025] Scan the training audio data, and obtain the audio energy value of the training audio data before and after the estimated end time of the specified text according to the specified time window, find the inflection point where the audio energy value changes from decreasing to increasing, and then use the time of the inflection point as the end time of the specified text;

[0026] Modify the estimated start time of the next character after the specified character in the word text sequence to the end time of the specified character, and then update the specified character to the next character until the start time and end time of all characters in the word text sequence are obtained;

[0027] The start time and end time of the characters in all the segmented texts in the word text sequence are merged to obtain the start time and end time of all the segmented texts in the word text sequence in the training audio data.

[0028] In a possible design, the parameter to be evaluated includes a speech rate parameter, and the speech rate parameter of each word segmentation audio in the word audio sequence is obtained according to the word text sequence and the word audio sequence, including:

[0029] According to the word text sequence, the number of characters a corresponding to each word segmentation audio in the word audio sequence is obtained;

[0030] Obtaining the duration t1 of each word segmentation audio in the word audio sequence according to the word audio sequence;

[0031] According to the number of characters a corresponding to each word segmentation audio and the duration t1 of each word segmentation audio, the speaking speed parameter v of each word segmentation audio is obtained, wherein the speaking speed parameter v of each word segmentation audio = the number of characters a corresponding to each word segmentation audio ÷ the duration t1 of each word segmentation audio.

[0032] In a possible design, the parameter to be evaluated includes a pitch parameter, and according to the word text sequence and the word audio sequence, the pitch parameter of each word segmentation audio in the word audio sequence is obtained, including:

[0033] According to the word audio sequence, any word segmentation audio in the word audio sequence is subjected to frame processing to obtain multiple audio frames;

[0034] Calculating the fundamental frequency of a plurality of audio frames respectively;

[0035] According to the fundamental frequency of the multiple audio frames, obtaining the fundamental frequency with the strongest energy in the multiple audio frames;

[0036] The fundamental frequency with the strongest energy among the multiple audio frames is set as the pitch parameter of the current word segmentation audio;

[0037] The other word segmentation audios are subjected to frame processing until the pitch parameter of each word segmentation audio in the word audio sequence is obtained.

[0038] In a possible design, the parameter to be evaluated includes a sound intensity parameter, and the sound intensity parameter of each word segmentation audio in the word audio sequence is obtained according to the word text sequence and the word audio sequence, including:

[0039] According to the word text sequence and the word audio sequence, a sampling value, a sampling bit number and a sampling width of each word segmentation audio are obtained;

[0040] According to the sampling value, sampling bit number and sampling width of each word segmentation audio, the sound intensity parameter of each word segmentation audio is obtained, wherein the sound intensity parameter of any word segmentation audio is:

[0041] Where, bits = sample_width * 8;

[0042] Among them, sample is the sampling value of the word segmentation audio, bits is the sampling bit number of the word segmentation audio, and sample_width is the sampling width of the word segmentation audio.

[0043] In a possible design, after obtaining the evaluation result, the speech training recognition method further includes:

[0044] Visualize the evaluation results in real time.

[0045] In a second aspect, the present invention provides a speech training recognition system for implementing any of the above-mentioned speech training recognition methods; the speech training recognition system comprises:

[0046] An audio data receiving module is used to receive the training audio data input by the training object in real time;

[0047] An audio data conversion module, used for converting the training audio data into text to obtain training text data;

[0048] A text segmentation module, used for performing segmentation processing on the training text data to obtain a plurality of segmented texts;

[0049] A text sorting module, used for sorting a plurality of segmented texts according to the order of their appearance in the training text data to obtain a word text sequence;

[0050] A word segmentation time acquisition module is used to obtain the start time and end time of multiple word segmentation texts in the word text sequence in the training audio data;

[0051] An audio cutting module is used to cut the training audio data according to the start time and end time of multiple word segmentation texts in the word text sequence in the training audio data to obtain multiple word segmentation audios;

[0052] An audio sorting module, used to sort multiple word segmentation audios according to their order of appearance in the training audio data to obtain a word audio sequence;

[0053] The comparison module is used to obtain the parameters to be evaluated of each word segmentation audio in the word audio sequence according to the word text sequence and the word audio sequence, and compare the parameters to be evaluated with pre-stored standard parameters to obtain an evaluation result. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 is a flow chart of a speech training recognition method in the present invention;

[0055] Figure 2 It is a module block diagram of a speech training recognition system in the present invention. DETAILED DESCRIPTION

[0056] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

[0057] It should be understood that although the terms first, second, etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element.

[0058] It should be understood that the term "and / or" that may appear in this document is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, A and / or B may mean: A exists alone, B exists alone, and A and B exist at the same time.

[0059] It should be understood that it should also be noted that in some alternative embodiments, the functions / actions that appear may be different from the order in which the drawings appear. For example, depending on the functions / actions involved, two figures shown in succession may actually be performed substantially concurrently, or may sometimes be performed in reverse order.

[0060] Embodiment 1:

[0061] The first aspect of this embodiment provides a speech training recognition method, which can be executed by, but is not limited to, a computer device or a virtual machine with certain computing resources, such as a personal computer (referring to a multi-purpose computer with a size, price and performance suitable for personal use; desktops, laptops, small laptops, tablets and ultrabooks are all personal computers), smart phones, personal digital assistants or wearable devices, or executed by a virtual machine.

[0062] like Figure 1 As shown, a speech training recognition method may include but is not limited to the following steps:

[0063] S1. Receive the training audio data input by the trainee in real time; it should be understood that in this embodiment, the training video data and / or training text data input by the trainee can also be received at the same time, which is not limited here.

[0064] When a user speaks, there may be irregular pauses, which are affected by the individual speaking style and have a certain degree of randomness. If the audio data recorded during the pause is subsequently processed, the calculation result of the speech speed will be inaccurate. To solve this technical problem, the present application further makes the following improvements: after receiving the training audio data input by the training subject, the speech training recognition method also includes:

[0065] S2. The training audio data is muted to obtain the audio data after muting;

[0066] Specifically, the training audio data is subjected to a de-mute process to obtain the de-mute audio data, including:

[0067] S201. Acquire the amplitude sequence of the training audio data, and extract continuous audio segments in which all amplitude values ​​of the training audio data are smaller than a preset amplitude threshold;

[0068] S202. Obtain the duration of the continuous audio segment;

[0069] S203. Determine in sequence whether the duration of the continuous audio segment is greater than a preset time threshold. If so, determine that the continuous audio segment is a silent audio segment, and delete the silent audio segment in the training audio data, until all continuous audio segments with a duration greater than the preset time threshold are deleted, and obtain the audio data after the silence processing. In this embodiment, by obtaining all continuous audio segments in the training audio data whose amplitude values ​​are less than the preset amplitude threshold, and then determining whether they are pause audio data, that is, silent audio segments, based on the duration of these continuous audio segments, and finally deleting the silent audio segments, the accuracy of the subsequent speech rate calculation results is improved. It should be understood that after deleting the silent audio segment, the other audio data in the training audio data are reconnected, which will not be described in detail here.

[0070] S3. Convert the training audio data into text to obtain training text data. In this embodiment, the training audio data is converted into training text data by using the audio conversion technology in the prior art, which will not be described in detail here.

[0071] S4. Perform word segmentation processing on the training text data to obtain multiple word segmentation texts, and then sort the multiple word segmentation texts according to the order of their appearance in the training text data to obtain a word text sequence; in this embodiment, the word segmentation processing may include but is not limited to: first extracting punctuation marks in the training text data, such as spaces, commas and semicolons, and using the punctuation marks as word segmentation marks, thereby dividing the training text data into multiple texts.

[0072] S5. Obtain the start time and end time of multiple word segmentation texts in the word text sequence in the training audio data.

[0073] In this embodiment, obtaining the start time and end time of multiple segmented texts in the word text sequence in the training audio data includes:

[0074] S501. Obtain the number of characters in the word text sequence and the duration of the training audio data, obtain the average duration of each character according to the number of characters and the duration of the training audio data, and use the average duration as the estimated value to mark the estimated start time and estimated end time of each character in the word text sequence in order;

[0075] S502. Obtaining the position of the single audio data corresponding to each character in the training audio data according to the estimated start time and the estimated end time;

[0076] S503. Obtain the specified text, and modify the estimated start time of the specified text to the start time of the specified text; in this embodiment, the specified text is set to the text located at the first position in the word text sequence, so as to determine the end time of all the texts in the word text sequence in order;

[0077] S504. Scan the training audio data, and obtain the audio energy value of the training audio data before and after the estimated end time of the specified text according to the specified time window, find the inflection point of the audio energy value from falling to rising, and then use the time of the inflection point as the end time of the specified text; it should be noted that the time window can be set to a very small time period, such as 5 milliseconds, 10 milliseconds, etc. In this embodiment, the time window is set to 5 milliseconds to capture the inflection point of the audio energy value;

[0078] S505. Modify the estimated start time of the next character after the specified character in the word text sequence to the end time of the specified character, that is, determine the start time of the next character to be the end time of the specified character, and then update the specified character to the next character until the start time and end time of all characters in the word text sequence are obtained;

[0079] S506. The start time and end time of the characters in all the segmented texts in the word text sequence are merged to obtain the start time and end time of all the segmented texts in the word text sequence in the training audio data.

[0080] In this embodiment, since there is usually an audio energy change between multiple characters in each segmented text during the user's speaking process, if the start time and end time of each segmented text in the training audio data are directly obtained, a certain time error is likely to occur, resulting in audio segmentation errors in the training audio data, thereby causing problems such as the subsequent segmented audio and the segmented text not corresponding to each other, resulting in a low accuracy rate of the evaluation result. In this embodiment, by first obtaining the start time and end time of multiple characters in the word text sequence, and then obtaining the start time and end time of all segmented texts in the word text sequence in the training audio data based on the start time and end time of the multiple characters, the accuracy of obtaining the start time and end time of the segmented text can be improved.

[0081] S6. According to the start time and end time of multiple word segmentation texts in the word text sequence in the training audio data, the training audio data is cut to obtain multiple word segmentation audios, and then the multiple word segmentation audios are sorted according to the order of their appearance in the training audio data to obtain a word audio sequence.

[0082] S7. According to the word text sequence and the word audio sequence, obtain the parameters to be evaluated of each word segmentation audio in the word audio sequence, and compare the parameters to be evaluated with the pre-stored standard parameters to obtain an evaluation result. Specifically, the pre-stored standard parameters can be obtained through standard case data, and the acquisition process of the standard parameters can be consistent with the acquisition process of the parameters to be evaluated in this application, which is not limited here. It should be understood that comparing the voice data of the training user with the pre-stored standard parameters and obtaining the evaluation results can enable other training users to learn from the standard case data during training, so as to narrow the service capability gap of service personnel.

[0083] In this embodiment, the parameter to be evaluated includes a speech rate parameter. According to the word text sequence and the word audio sequence, the speech rate parameter of each word segmentation audio in the word audio sequence is obtained, including:

[0084] A1. Obtain the number of characters a corresponding to each segmented audio in the word audio sequence according to the word text sequence;

[0085] A2. The duration t1 of each word segment audio in the word audio sequence is obtained according to the word audio sequence;

[0086] A3. According to the number of characters a corresponding to each word segmentation audio and the duration t1 of each word segmentation audio, the speaking speed parameter v of each word segmentation audio is obtained, wherein the speaking speed parameter v of each word segmentation audio = the number of characters a corresponding to each word segmentation audio ÷ the duration t1 of each word segmentation audio.

[0087] In this embodiment, the parameters to be evaluated include pitch parameters. It should be noted that general sounds are composed of a series of vibrations with different frequencies and amplitudes emitted by the sound-producing body. Among these vibrations, the lowest frequency vibration is called the fundamental tone, and the rest are overtones. The fundamental tone determines the pitch of the speaker, which corresponds to the pitch parameter in this embodiment. In this embodiment, according to the word text sequence and the word audio sequence, the pitch parameters of each word segmentation audio in the word audio sequence are obtained, including:

[0088] B1. According to the word audio sequence, any word segmentation audio in the word audio sequence is framed to obtain multiple audio frames;

[0089] B2. Calculate the pitch frequency of multiple audio frames respectively;

[0090] B3. According to the pitch frequencies of multiple audio frames, obtain the pitch frequency with the strongest energy in multiple audio frames;

[0091] B4. The fundamental frequency with the strongest energy in multiple audio frames is set as the pitch parameter of the current word segmentation audio;

[0092] B5. Perform frame processing on other word segmentation audios until the pitch parameter of each word segmentation audio in the word audio sequence is obtained.

[0093] It should be noted that the frame processing is the segmentation processing of the word segmentation audio, which avoids the problems of large amount of calculation and lack of coherence caused by point-by-point processing of the word segmentation audio. In this embodiment, the length of the audio frame can be set according to the user's computing requirements, resolution requirements, etc., and adjacent frames are allowed to overlap partially, and the overlapping range is controlled by parameters.

[0094] In this embodiment, the number of signal frames is: Frames = (N-length) / gap+1, where length is the length of the audio frame, gap is the interval between adjacent audio frames, and N is the length of the segmented audio.

[0095] In this embodiment, the cepstrum method is used to calculate the fundamental frequency of multiple audio frames respectively. The cepstrum method is one of the fundamental pitch period detection algorithms. It uses the cepstrum characteristics of the speech signal to detect the fundamental pitch information of the sound, and has a good fundamental pitch estimation effect.

[0096] In this embodiment, the parameter to be evaluated includes a sound intensity parameter. According to the word text sequence and the word audio sequence, the sound intensity parameter of each word segmentation audio in the word audio sequence is obtained, including:

[0097] C1. According to the word text sequence and the word audio sequence, obtain the sampling value, sampling bit number and sampling width of each word segmentation audio;

[0098] C2. According to the sampling value, sampling bit number and sampling width of each word segmentation audio, the sound intensity parameter of each word segmentation audio is obtained, wherein the sound intensity parameter of any word segmentation audio is:

[0099] Where, bits = sample_width * 8;

[0100] Wherein, sample is the sampling value of the word segmentation audio, bits is the sampling bit number of the word segmentation audio, and sample_width is the sampling width of the word segmentation audio. It should be noted that the larger the sound intensity parameter dBFS is, the greater the sound intensity of the word segmentation audio is.

[0101] This embodiment uses speaking speed, pitch and sound intensity as sub-dimensions of speech assessment. By calculating the speaking speed, pitch and sound intensity of each word during the speech training of the training users, and comparing them with the pre-stored standard parameters, the differences in speaking speed, pitch and sound intensity among the training users are obtained. Finally, an evaluation result is formed based on the differences to facilitate targeted training for the training users.

[0102] In this embodiment, the evaluation result is:

[0103] Score = 1-norma(std(a,b));

[0104] Among them, norma() is the normalization calculation, std() is the standard deviation calculation, a is the parameter to be evaluated for any segmented audio, and b is the standard parameter corresponding to the segmented audio.

[0105] In this embodiment, the evaluation result Score is 0 or 1. When the score is 0, it means that the segmented audio in the word audio sequence is unqualified, and when the score is 1, it means that the segmented audio in the word audio sequence is qualified. It should be understood that in this embodiment, the evaluation result can also be visualized in the form of text or animation display, so that the user can understand the evaluation result in time.

[0106] In this embodiment, after obtaining the evaluation result, the speech training recognition method further includes:

[0107] S8. Visually display the evaluation results in real time so that the training users can understand their voice training status in real time and make corrections in time.

[0108] In this embodiment, by converting the training audio data into training text data, obtaining multiple segmented texts in the training text data, and then obtaining the corresponding start time and end time of the multiple segmented texts in the training audio data by text processing, the accuracy of obtaining the segmented text in the training audio data and the start time and end time of the segmented text can be improved.

[0109] This embodiment can visualize the detailed defects of the language expression of the training users, which can help the trainers understand the defects in the voice training process and improve the voice service capabilities of the trainees by repeatedly practicing the defect details.

[0110] Compared with the prior art, this embodiment can achieve the following effects: by adopting a standardized evaluation method, each evaluation follows the same standard, avoiding the problem of inconsistent evaluation standards at different times due to personal factors of the trainer, resulting in fluctuations in evaluation quality, thereby improving the accuracy and consistency of the judgment results in the voice training process, and facilitating the visualization of the service capability improvement effect; in addition, since the prepared standard case data can be permanently saved and reused, the labor cost in the training process can be greatly reduced. At the same time, the prepared standard case data can support multiple people to train at the same time, breaking through the limitation that manual evaluation can only be conducted one-on-one, and the training and evaluation efficiency is higher; furthermore, since this embodiment performs fine dimensional quantification and comparison of the actual training audio data and standard cases input by the training subjects, it can accurately locate the problem points of the training users, facilitating targeted training of the training users.

[0111] Embodiment 2:

[0112] This embodiment provides a speech training recognition system for implementing the speech training recognition method in Embodiment 1; Figure 2 As shown, the speech training recognition system includes:

[0113] An audio data receiving module is used to receive the training audio data input by the training object in real time;

[0114] An audio data conversion module, used for converting the training audio data into text to obtain training text data;

[0115] A text segmentation module, used for performing segmentation processing on the training text data to obtain a plurality of segmented texts;

[0116] A text sorting module, used for sorting a plurality of segmented texts according to the order of their appearance in the training text data to obtain a word text sequence;

[0117] A word segmentation time acquisition module is used to obtain the start time and end time of multiple word segmentation texts in the word text sequence in the training audio data;

[0118] An audio cutting module is used to cut the training audio data according to the start time and end time of multiple word segmentation texts in the word text sequence in the training audio data to obtain multiple word segmentation audios;

[0119] An audio sorting module, used to sort multiple word segmentation audios according to their order of appearance in the training audio data to obtain a word audio sequence;

[0120] The comparison module is used to obtain the parameters to be evaluated of each word segmentation audio in the word audio sequence according to the word text sequence and the word audio sequence, and compare the parameters to be evaluated with pre-stored standard parameters to obtain an evaluation result.

[0121] Embodiment 3:

[0122] Based on Embodiment 1 or 2, this embodiment discloses an electronic device, which may be a smart phone, a tablet computer, a laptop computer, or a desktop computer, etc. The electronic device may be referred to as a terminal, a portable terminal, a desktop terminal, etc., and the electronic device includes:

[0123] a memory for storing computer program instructions; and,

[0124] The processor is used to execute the computer program instructions to complete the operation of the speech training recognition method as described in any one of the embodiments 1.

[0125] Embodiment 4:

[0126] Based on any one of Embodiments 1 to 3, this embodiment discloses a computer-readable storage medium for storing computer program instructions readable by a computer, wherein the computer program instructions are configured to execute the operations of the speech training recognition method as described in Embodiment 1 when run.

[0127] It should be noted that if the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program code.

[0128] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0129] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention is described in detail with reference to the above embodiments, a person skilled in the art should understand that the technical solutions described in the above embodiments can still be modified, or some of the technical features can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

[0130] Finally, it should be noted that the present invention is not limited to the above optional implementations, and anyone can derive other various forms of products under the enlightenment of the present invention. The above specific implementations should not be understood as limiting the scope of protection of the present invention. The scope of protection of the present invention should be based on the definition in the claims, and the description can be used to interpret the claims.

Claims

1. A speech training recognition method, characterized in that: include: Receiving the training audio data input by the training object in real time, and converting the training audio data into text to obtain the training text data; Performing word segmentation processing on the training text data to obtain a plurality of word segmentation texts, and then sorting the plurality of word segmentation texts according to the order in which they appear in the training text data to obtain a word text sequence; Obtaining the start time and end time of multiple word segmentation texts in the word text sequence in the training audio data; According to the start time and end time of multiple word segmentation texts in the word text sequence in the training audio data, the training audio data is cut to obtain multiple word segmentation audios, and then the multiple word segmentation audios are sorted according to the order of their appearance in the training audio data to obtain a word audio sequence; According to the word text sequence and the word audio sequence, obtaining the parameters to be evaluated of each word segmentation audio in the word audio sequence, and comparing the parameters to be evaluated with pre-stored standard parameters to obtain an evaluation result; Obtaining the start time and end time of multiple word segmentation texts in the word text sequence in the training audio data includes: Obtain the number of characters in the word text sequence and the duration of the training audio data, obtain the average duration of each character according to the number of characters and the duration of the training audio data, and use the average duration as an estimated value to mark the estimated start time and estimated end time of each character in the word text sequence in order; Obtaining the position of the single audio data corresponding to each character in the training audio data according to the estimated start time and the estimated end time; Obtain the specified text, and modify the estimated start time of the specified text to the start time of the specified text; Scan the training audio data, and obtain the audio energy value of the training audio data before and after the estimated end time of the specified text according to the specified time window, find the inflection point where the audio energy value changes from decreasing to increasing, and then use the time of the inflection point as the end time of the specified text; Modify the estimated start time of the next character after the specified character in the word text sequence to the end time of the specified character, and then update the specified character to the next character until the start time and end time of all characters in the word text sequence are obtained; The start time and end time of the characters in all the segmented texts in the word text sequence are merged to obtain the start time and end time of all the segmented texts in the word text sequence in the training audio data.

2. A speech training recognition method according to claim 1, characterized in that: After receiving the training audio data input by the training subject, the speech training recognition method further includes: The training audio data is subjected to a de-mute process to obtain audio data after the de-mute process, and then the training audio data is converted into text.

3. A speech training recognition method according to claim 2, characterized in that: The training audio data is subjected to a mute removal process to obtain audio data after the mute removal process, including: Acquire the amplitude sequence of the training audio data, and extract continuous audio segments in which all amplitude values ​​of the training audio data are less than a preset amplitude threshold; Obtaining the duration of the continuous audio segment; It is determined in turn whether the duration of the continuous audio segment is greater than a preset time threshold. If so, the continuous audio segment is determined to be a silent audio segment, and the silent audio segment in the training audio data is deleted until all continuous audio segments with a duration greater than the preset time threshold are deleted to obtain audio data after de-silencing.

4. A speech training recognition method according to claim 1, characterized in that: The parameter to be evaluated includes a speech rate parameter. According to the word text sequence and the word audio sequence, the speech rate parameter of each word segmentation audio in the word audio sequence is obtained, including: According to the word text sequence, the number of characters a corresponding to each word segmentation audio in the word audio sequence is obtained; Obtaining the duration t1 of each word segmentation audio in the word audio sequence according to the word audio sequence; According to the number of characters a corresponding to each word segmentation audio and the duration t1 of each word segmentation audio, the speaking speed parameter v of each word segmentation audio is obtained, wherein the speaking speed parameter v of each word segmentation audio = the number of characters a corresponding to each word segmentation audio ÷ the duration t1 of each word segmentation audio.

5. A speech training recognition method according to claim 1, characterized in that: The parameter to be evaluated includes a pitch parameter. According to the word text sequence and the word audio sequence, the pitch parameter of each word segmentation audio in the word audio sequence is obtained, including: According to the word audio sequence, any word segmentation audio in the word audio sequence is subjected to frame processing to obtain multiple audio frames; Calculating the fundamental frequency of a plurality of audio frames respectively; According to the fundamental frequency of the multiple audio frames, obtaining the fundamental frequency with the strongest energy in the multiple audio frames; The fundamental frequency with the strongest energy among the multiple audio frames is set as the pitch parameter of the current word segmentation audio; The other word segmentation audios are subjected to frame processing until the pitch parameter of each word segmentation audio in the word audio sequence is obtained.

6. A speech training recognition method according to claim 1, characterized in that: The parameter to be evaluated includes a sound intensity parameter. According to the word text sequence and the word audio sequence, the sound intensity parameter of each word segmentation audio in the word audio sequence is obtained, including: According to the word text sequence and the word audio sequence, a sampling value, a sampling bit number and a sampling width of each word segmentation audio are obtained; According to the sampling value, sampling bit number and sampling width of each word segmentation audio, the sound intensity parameter of each word segmentation audio is obtained, wherein the sound intensity parameter of any word segmentation audio is: Where, bits = sample_width * 8; Among them, sample is the sampling value of the word segmentation audio, bits is the sampling bit number of the word segmentation audio, and sample_width is the sampling width of the word segmentation audio.

7. A speech training recognition method according to claim 1, characterized in that: After obtaining the evaluation result, the speech training recognition method further includes: Visualize the evaluation results in real time.

8. A speech training recognition system, characterized in that: Used to implement the speech training recognition method according to any one of claims 1 to 7; the speech training recognition system comprises: An audio data receiving module is used to receive the training audio data input by the training object in real time; An audio data conversion module, used for converting the training audio data into text to obtain training text data; A text segmentation module, used for performing segmentation processing on the training text data to obtain a plurality of segmented texts; A text sorting module, used for sorting a plurality of segmented texts according to the order of their appearance in the training text data to obtain a word text sequence; A word segmentation time acquisition module is used to obtain the start time and end time of multiple word segmentation texts in the word text sequence in the training audio data; An audio cutting module is used to cut the training audio data according to the start time and end time of multiple word segmentation texts in the word text sequence in the training audio data to obtain multiple word segmentation audios; An audio sorting module, used to sort multiple word segmentation audios according to their order of appearance in the training audio data to obtain a word audio sequence; The comparison module is used to obtain the parameters to be evaluated of each word segmentation audio in the word audio sequence according to the word text sequence and the word audio sequence, and compare the parameters to be evaluated with pre-stored standard parameters to obtain an evaluation result.

Images