Audio inspection method and electronic equipment

Abstract

The present application discloses an audio inspection method and electronic device, belonging to the field of electronic technology. After converting the target audio to be inspected into a target pinyin sequence, the electronic device can determine the wrong pinyin of the target pinyin sequence relative to the standard pinyin sequence of the standard text and the error type of the wrong pinyin, and mark the text corresponding to the wrong pinyin and the error type in the displayed standard text. Compared with manual inspection of the mastery of the article, the electronic device in the method provided in the embodiment of the present application can automatically inspect the target audio to realize the inspection of the mastery of the article, thereby improving the efficiency of the inspection of the mastery of the article.

Description

Technical Field

[0001] The present disclosure relates to the field of electronic technology, and in particular to an audio inspection method and electronic equipment. Background Art

[0002] In traditional Chinese teaching, teachers usually listen to students reciting articles to check their understanding of the articles. However, this manual checking method takes a lot of time and is inefficient. Summary of the invention

[0003] The embodiment of the present application provides an audio checking method and electronic device, which can solve the problem of low efficiency in the related art of manually checking students' understanding of the article. The technical solution is as follows:

[0004] In one aspect, an audio inspection method is provided, which is applied to an electronic device, and the method comprises:

[0005] Get the target audio to be checked;

[0006] Converting the target audio into a target pinyin sequence, wherein the target pinyin sequence includes the pinyin of multiple characters;

[0007] Determine the incorrect pinyin of the target pinyin sequence relative to the standard pinyin sequence of the standard text and the error type of the incorrect pinyin, wherein the error type includes at least one of: pinyin repetition, mispronunciation, pinyin omission and pinyin redundancy;

[0008] The standard text and an error mark are displayed, wherein the error mark is used to indicate the text corresponding to the incorrect pinyin and the error type.

[0009] In another aspect, an electronic device is provided, the electronic device comprising a processor and a display screen, the processor being configured to:

[0010] Get the target audio to be checked;

[0011] Converting the target audio into a target pinyin sequence, wherein the target pinyin sequence includes the pinyin of multiple characters;

[0012] Determine the incorrect pinyin of the target pinyin sequence relative to the standard pinyin sequence of the standard text and the error type of the incorrect pinyin, wherein the error type includes at least one of: pinyin repetition, mispronunciation, pinyin omission and pinyin redundancy;

[0013] The display screen is used to display the standard text and an error mark, wherein the error mark is used to indicate the text corresponding to the incorrect pinyin and the error type.

[0014] On the other hand, an electronic device is provided, comprising: a memory, a processor and a computer program stored in the memory, wherein the processor implements the audio inspection method as described in the above aspects when executing the computer program.

[0015] On the other hand, a computer-readable storage medium is provided, in which instructions are stored. The instructions are loaded and executed by a processor to implement the audio inspection method as described in the above aspects.

[0016] An embodiment of the present application provides a computer program product comprising instructions, and when the computer program product is run on a computer, the computer is enabled to execute the audio inspection method as described in the above aspects.

[0017] The beneficial effects brought by the technical solution provided by the embodiment of the present application include at least:

[0018] The embodiment of the present application provides an audio checking method and an electronic device. After the electronic device converts the target audio to be checked into a target pinyin sequence, it can determine the wrong pinyin of the target pinyin sequence relative to the standard pinyin sequence of the standard text and the error type of the wrong pinyin, and mark the text corresponding to the wrong pinyin and the error type in the displayed standard text. Compared with manual inspection of the mastery of the article, the method provided by the embodiment of the present application allows the electronic device to automatically check the target audio to realize the inspection of the mastery of the article, thereby improving the efficiency of the inspection of the mastery of the article.

[0019] Moreover, since the target audio is converted into a target pinyin sequence and the errors in the audio are checked in the form of pinyin, compared with converting the target audio into text and checking the errors in the audio based on the converted text, the method provided in the embodiment of the present application effectively improves the accuracy of audio checking. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 It is a schematic diagram of an implementation environment involved in an audio inspection method provided in an embodiment of the present application;

[0022] Figure 2 is a flow chart of an audio inspection method provided in an embodiment of the present application;

[0023] Figure 3is a flow chart of another audio inspection method provided in an embodiment of the present application;

[0024] Figure 4 is a schematic diagram of an audio upload interface provided in an embodiment of the present application;

[0025] Figure 5 is a schematic diagram of an audio collection interface provided in an embodiment of the present application;

[0026] Figure 6 is a schematic diagram of another audio collection interface provided in an embodiment of the present application;

[0027] Figure 7 is a schematic diagram of multiple audio clips provided in an embodiment of the present application;

[0028] Figure 8 is a schematic diagram of multiple spectrum graph segments provided in an embodiment of the present application;

[0029] Fig. 9 is a schematic diagram of a target pinyin sequence provided in an embodiment of the present application;

[0030] Fig.10 is a schematic diagram of a standard pinyin sequence provided in an embodiment of the present application;

[0031] Fig.11 is a schematic diagram of a longest common subsequence provided in an embodiment of the present application;

[0032] Fig.12 is a schematic diagram of an updated standard label digital sequence provided in an embodiment of the present application;

[0033] Fig.13 is a schematic diagram of an updated target tag digital sequence provided in an embodiment of the present application;

[0034] Fig.14 is a schematic diagram of a displayed error mark provided in an embodiment of the present application;

[0035] Fig.15 is a flow chart of another audio inspection method provided in an embodiment of the present application;

[0036] Fig.16 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application;

[0037] Fig.17 A schematic diagram of the structure of another electronic device provided in an embodiment of the present application;

[0038] Fig.18 It is a software structure block diagram of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0039] In order to make the objectives, technical solutions and advantages of the present application more clear, the embodiments of the present disclosure will be further described in detail below with reference to the accompanying drawings.

[0040] In the related art, after obtaining the audio of the user reciting an article, the electronic device can convert the audio into text. Then, the converted text is compared with the correct text one by one, and the text that does not match the correct text is determined as incorrect text, and then the mastery of the article can be evaluated based on the determined incorrect text. However, since different texts may be homophones, some of the converted texts may be incorrect in the process of converting the audio to text, resulting in low accuracy of the converted text, and then low accuracy of the evaluation of the mastery of the article.

[0041] Figure 1 Schematic diagram of an implementation environment involved in an audio inspection method provided in an embodiment of the present application. Figure 1 As shown, the implementation environment may include: an electronic device 10, which may be a smart phone, a computer, an electronic reader, etc. Figure 1 Take the electronic device 10 as an example, which is a smart phone.

[0042] Figure 2 This is a flowchart of an audio inspection method provided in an embodiment of the present application, which can be applied to Figure 1 The electronic device 10 shown. Figure 2 As shown, the method includes:

[0043] Step 201: Obtain target audio to be checked.

[0044] In an embodiment of the present application, the target audio may be audio collected when a user recites or reads a standard text.

[0045] The target audio to be checked may be pre-collected audio, and the electronic device may be installed with an audio check application. The audio check application may include an audio upload interface, and the audio upload interface is provided with an audio upload area and an audio upload button. After the electronic device detects audio in the audio upload area, if a selection operation for the audio upload button is received, the detected audio may be determined as the target audio to be checked, thereby obtaining the target audio to be checked.

[0046] The audio inspection application may include an audio collection interface, which is provided with an audio collection button and an audio collection completion button. The electronic device may collect audio in response to the audio collection operation, and may determine the audio collected within the target duration as the target audio to be inspected. Alternatively, the electronic device may collect audio in response to the audio collection operation, stop collecting audio in response to the audio collection completion operation, and determine the collected audio as the target audio to be inspected.

[0047] The audio collection operation may be a selection operation for the audio collection button, and the audio collection completion operation may be a selection operation for the audio collection completion button. The target duration may be a fixed duration pre-stored in the electronic device.

[0048] Step 202: Convert the target audio into a target pinyin sequence.

[0049] After acquiring the target audio to be checked, the electronic device can convert the target audio into a target pinyin sequence, wherein the target pinyin sequence can include the pinyin of multiple characters.

[0050] Step 203: Determine the incorrect pinyin of the target pinyin sequence relative to the standard pinyin sequence of the standard text and the error type of the incorrect pinyin.

[0051] After determining the target pinyin sequence, the electronic device can determine the wrong pinyin of the target pinyin sequence relative to the standard pinyin sequence of the standard text and the error type of the wrong pinyin. The standard pinyin sequence is a correct pinyin sequence, and the standard pinyin sequence includes the pinyin of each word in the standard text. The error type can include at least one of pinyin repetition, mispronunciation, pinyin omission and pinyin redundancy.

[0052] Step 204: Display standard text and error mark.

[0053] After determining the wrong pinyin of the target pinyin sequence relative to the standard pinyin sequence of the standard text and the error type of the wrong pinyin, the electronic device can also display the standard text and an error mark, wherein the error mark is used to indicate the text corresponding to the wrong pinyin and the error type. Optionally, the error mark may include at least one of a bold mark, an underline mark, a font mark, a color mark, and an annotation mark.

[0054] In summary, the embodiment of the present application provides an audio inspection method. After the electronic device converts the target audio to be inspected into a target pinyin sequence, it can determine the incorrect pinyin of the target pinyin sequence relative to the standard pinyin sequence of the standard text and the error type of the incorrect pinyin, and mark the text corresponding to the incorrect pinyin and the error type in the displayed standard text. Compared with manual inspection of the mastery of the article, the method provided by the embodiment of the present application allows the electronic device to automatically inspect the target audio to realize the inspection of the mastery of the article, thereby improving the efficiency of the inspection of the mastery of the article.

[0055] Furthermore, since the target audio is converted into a target pinyin sequence and the errors in the audio are checked in the form of pinyin, compared to converting the audio into text and checking the errors in the audio based on the converted text, the method provided in the embodiment of the present application effectively improves the accuracy of audio checking.

[0056] Figure 3 is a flowchart of another audio inspection method provided in an embodiment of the present application, which can be applied to Figure 1 The electronic device 10 shown. Figure 3 As shown, the method may include:

[0057] Step 301: Obtain target audio to be checked.

[0058] In an embodiment of the present application, the target audio may be audio collected when a user recites or reads a standard text.

[0059] In an optional implementation of the embodiment of the present application, the target audio to be checked may be pre-collected audio, and the electronic device may be installed with an audio checking application. Figure 4 The audio inspection application may include an audio upload interface 40, and the audio upload interface 40 may include an audio upload area 401 and an audio upload button 402. The name of the audio upload button 402 may be confirm upload. After the electronic device detects audio in the audio upload area 401, if a selection operation is received for the audio upload button 402, the detected audio may be determined as the target audio to be inspected, thereby obtaining the target audio to be inspected.

[0060] refer to Figure 4 The audio upload area 401 may also display a prompt message, which is used to prompt the user to upload the audio in the audio upload area. For example, the prompt message may be to click on this area to upload the audio.

[0061] refer to Figure 4The audio upload interface 40 may further include a first input box 403 for obtaining the name of the standard text and a start check button 404. The first input box 403 is used to obtain the name of the standard text. After obtaining the target audio to be checked, if the electronic device obtains the name of the standard text in the first input box 403 and detects a selection operation on the start check button 404, the target audio to be checked may be checked based on the standard text corresponding to the name of the standard text.

[0062] For example, if the target audio to be checked is the audio collected by reciting "Quiet Night Thoughts", the user can enter "Quiet Night Thoughts" in the first input box 403. Accordingly, the electronic device can obtain the name of the standard text corresponding to the target audio to be checked in the first input box 403: Quiet Night Thoughts.

[0063] In another optional implementation of the embodiment of the present application, the audio inspection application may also include an audio collection interface, referring to Figure 5 and Figure 6 The audio collection interface may display a second input box 501 of the name of the standard text, a recitation mode button 502, a text display area 503, and a recitation mode button 504. After the electronic device obtains the name of the standard text in the second input box 501, if a selection operation is received for the recitation mode button 502, the standard text corresponding to the name of the standard text may be displayed in the text display area 503. Figure 5 , the name of the standard text obtained by the electronic device in the second input box 501 is: Quiet Night Thoughts. After receiving the selection operation for the recitation mode button 502, the electronic device can display the standard text named "Quiet Night Thoughts" in the text display area 503. Figure 6 After the electronic device obtains the name of the standard text in the second input box 501, if a selection operation is received for the recitation mode button 504, there is no need to display the standard text corresponding to the obtained name.

[0064] The audio collection interface 50 is provided with an audio collection button 501 and an audio collection completion button 502. For example, refer to Figure 5 and Figure 6 , the name of the audio collection button 501 can be start, and the name of the audio collection completion button 502 can be end. The electronic device can collect audio in response to the audio collection operation, and determine the audio collected within the target duration as the target audio to be checked. Or the electronic device can collect audio in response to the audio collection operation, stop collecting audio in response to the audio collection completion operation, and determine the collected audio as the target audio to be checked.

[0065] The audio collection operation may be a selection operation for the audio collection button, and the audio collection completion operation may be a selection operation for the audio collection completion button. The target duration may be a fixed duration pre-stored in the electronic device.

[0066] Step 302: Segment the target audio at each slice duration to obtain multiple audio segments.

[0067] After obtaining the target audio to be checked, the electronic device can segment the target audio at each slice duration to obtain multiple audio segments. The slice duration can be a fixed duration pre-stored in the electronic device, for example, 50ms. Figure 7 The electronic device divides the target audio into 11 audio segments, from audio segment 1 to audio segment 11, at intervals of the slice duration.

[0068] In an embodiment of the present application, the electronic device may pre-store a pinyin recognition model. Optionally, the pinyin recognition model is obtained by training based on multiple sample data, each of which may include an audio sample and a pinyin sequence of the audio sample. Alternatively, the pinyin recognition model is obtained by training an automatic speech recognition (automatic speech recognition) model using multiple sample data, each of which may include an audio sample and a pinyin sequence of the audio sample.

[0069] Optionally, the pinyin recognition model may include a segmentation sub-model, and the electronic device may use the segmentation sub-model to segment the target audio file at each slice duration to obtain multiple audio segments.

[0070] In an embodiment of the present application, after acquiring the target audio, the electronic device may use a slicing sub-model to perform a Fourier transform on the target audio to obtain a spectrogram of the target audio. The electronic device may then use the slicing sub-model to slice the spectrogram at intervals of a slice duration to obtain a spectrogram sequence, which includes spectrograms of multiple audio segments. The Fourier transform may be a fast Fourier transform (FFT).

[0071] refer to Figure 8 If the electronic device segments the target audio every slice duration to obtain 11 audio segments from audio segment 1 to audio segment 11, the spectrogram sequence may include 11 spectrogram segments corresponding to the 11 audio segments.

[0072] Step 303: Determine a phoneme sequence based on the multiple audio clips.

[0073] After obtaining multiple audio clips, the electronic device may determine a phoneme sequence based on the multiple audio clips. The phoneme sequence may include multiple phonemes, which are the smallest units of pinyin. Each phoneme is determined based on an audio clip. That is, the multiple phonemes in the phoneme sequence correspond to the multiple audio clips one by one. Figure 7 The phoneme sequence determined by the electronic device based on the 11 audio clips may include 11 phonemes corresponding to the 11 audio clips one by one, and the 11 phonemes are: hhaaaisshii.

[0074] Optionally, the pinyin recognition model may further include a first neural network sub-model. The electronic device may input the multiple audio clips into the first neural network sub-model to obtain a phoneme sequence output by the first neural network sub-model.

[0075] refer to Figure 8 , the first neural network sub-model may include 5 neuron sub-models from h1 to h5, and the electronic device may use 5 neuron sub-models to process the multiple audio clips to obtain the phoneme sequence of the target audio. Each neuron sub-model may process the result output by the previous neuron sub-model and transmit the processed result to the next neuron sub-model.

[0076] In the embodiment of the present application, the duration T of the pinyin of each character satisfies: T=dt×n, where dt is the slice duration, n is the number of phoneme sequences included in the pinyin, and n is an integer greater than or equal to 1.

[0077] It is understandable that due to the different speaking speeds of the users, the number of audio segments obtained by the electronic device based on the fixed slicing duration to segment the target audio is also different, and accordingly, the number of phonemes included in the phoneme sequence determined by the electronic device is also different. Among them, the number of audio segments and the number of phonemes included in the phoneme sequence are both negatively correlated with the user's speaking speed. That is, the faster the user's speaking speed, the fewer the number of audio segments, and the fewer the number of phonemes included in the corresponding phoneme sequence. The slower the user's speaking speed, the more audio segments, and the more the number of phonemes included in the corresponding phoneme sequence.

[0078] For example, if the user's speech speed is slow, the phoneme sequence corresponding to the pinyin determined by the electronic device may include hh aa ai sshii, a total of 11 phonemes. If the user's speech speed is fast, the phoneme sequence corresponding to the pinyin determined by the electronic device may include haaishii, a total of 8 phonemes.

[0079] Step 304: Obtain a target pinyin sequence of the target audio based on the phoneme sequence.

[0080] After determining the phoneme sequence, the electronic device may obtain the target pinyin sequence of the target audio based on the phoneme sequence. Among them, the target pinyin sequence may include the pinyins of multiple characters. Refer to Figure 7 , if the phoneme sequence includes h h a aa i s s h ii, the target pinyin sequence of the target audio obtained by the electronic device based on the phoneme sequence may include haishi, and the target pinyin sequence includes the pinyin of the character "还是".

[0081] For example Fig. 9 As shown, the target pinyin sequence determined by the electronic device may also include the pinyin of the characters "床前明月明月光疑似地霜".

[0082] Optionally, the pinyin recognition model may further include a second neural network sub-model. The electronic device may input the determined phoneme sequence into the second neural network sub-model to obtain the target pinyin sequence of the target audio output by the second neural network sub-model.

[0083] It can be understood that after obtaining the target audio, the electronic device may input the target audio into the pinyin recognition model, so as to obtain the target pinyin sequence of the target audio output by the pinyin recognition model, as well as the phoneme sequence.

[0084] Step 305: Determine the incorrect pinyin and the error type of the incorrect pinyin of the target pinyin sequence relative to the standard pinyin sequence of the standard text.

[0085] After determining the target pinyin sequence, the electronic device may determine the incorrect pinyin and the error type of the incorrect pinyin of the target pinyin sequence relative to the standard pinyin sequence of the standard text. Among them, the standard text is a correct text, and the standard text includes multiple characters. The standard pinyin sequence is a correct pinyin sequence, and the standard pinyin sequence includes the pinyin of each character in the standard text. The error type may include at least one of pinyin repetition, pronunciation error, pinyin omission, and redundant pinyin. For example Fig.10 As shown, the standard text may include "床前明月光疑是地上霜", and the standard pinyin sequence of the standard text may include "chuang qian ming yue guang yi shi di shang shuang".

[0086] In the embodiments of the present application, the error type may include pinyin repetition. If the first pinyin in the target pinyin sequence is the same as the standard pinyin at the corresponding position in the standard pinyin sequence, it may be determined that the first pinyin is a correct pinyin.

[0087] It is understandable that if there is an alternative pinyin whose error type is missing pinyin in the alternative pinyin before the first pinyin, then the alternative pinyin before the first pinyin in the target pinyin sequence here includes the wrong pinyin of the missing pinyin. If there is an incorrect pinyin whose error type is repeated pinyin and redundant pinyin in the alternative pinyin before the first pinyin, then the incorrect pinyin of repeated pinyin and redundant pinyin is not included in the alternative pinyin before the first pinyin in the target pinyin sequence here.

[0088] If the first pinyin in the target pinyin sequence is different from the standard pinyin at the corresponding position in the standard pinyin sequence, and there is a pinyin identical to the first pinyin in the first candidate pinyin in the target pinyin sequence, the electronic device can determine that the first pinyin is an incorrect pinyin, and the error type of the first pinyin is a pinyin repetition. The first candidate pinyin is at least one pinyin located before the first pinyin and adjacent to the first pinyin.

[0089] If the first pinyin in the target pinyin sequence is different from the standard pinyin at the corresponding position in the standard pinyin sequence, and it is determined that the condition that there is a pinyin identical to the first pinyin in the first alternative pinyin in the target pinyin sequence is not met, the electronic device can re-determine the wrong pinyin based on the first pinyin, and whether the error type of the wrong pinyin is pronunciation error, pinyin omission or pinyin redundancy.

[0090] In an embodiment of the present application, the error type may also include a pronunciation error. If the second pinyin in the target pinyin sequence is the same as the second standard pinyin at the corresponding position in the standard pinyin sequence, the electronic device may determine that the second pinyin is the correct pinyin.

[0091] It is understandable that if there is an alternative pinyin whose error type is missing pinyin in the alternative pinyin before the second pinyin, then the target pinyin sequence here includes the wrong pinyin whose error type is missing pinyin in the alternative pinyin before the second pinyin. If there is an wrong pinyin whose error type is repeated pinyin and redundant pinyin in the alternative pinyin before the second pinyin, then the target pinyin sequence here does not include the wrong pinyin whose error type is repeated pinyin and redundant pinyin in the alternative pinyin before the second pinyin.

[0092] If the second pinyin in the target pinyin sequence is different from the first standard pinyin at the corresponding position in the standard pinyin sequence, and the second alternative pinyin in the target pinyin sequence is the same as the standard pinyin at the corresponding position in the standard pinyin sequence, and the third alternative pinyin is the same as the standard pinyin at the corresponding position in the standard pinyin sequence, then the electronic device can determine that the second pinyin is an incorrect pinyin, and the error type of the second pinyin is a pronunciation error.

[0093] Wherein, the second candidate pinyin is located before the second pinyin and adjacent to the second pinyin, and the third candidate pinyin is located after the second pinyin and adjacent to the second pinyin. It can be understood that the number of the second pinyin can be one or more. The number of the first standard pinyin is the same as the number of the second pinyin.

[0094] If the second pinyin in the target pinyin sequence is different from the first standard pinyin at the corresponding position in the standard pinyin sequence, and does not satisfy the conditions that the second alternative pinyin in the target pinyin sequence is the same as the standard pinyin at the corresponding position in the standard pinyin sequence, and the third alternative pinyin is the same as the standard pinyin at the corresponding position in the standard pinyin sequence, then the electronic device can re-determine the wrong pinyin based on the second pinyin, and whether the error type of the wrong pinyin is pinyin repetition, pinyin reduplication or pinyin omission.

[0095] In an embodiment of the present application, the error type may also include redundant pinyin. If the third pinyin in the target pinyin sequence is the same as the second standard pinyin at the corresponding position in the standard pinyin sequence, the electronic device may determine that the third pinyin is the correct pinyin.

[0096] It is understandable that if there is an alternative pinyin whose error type is missing pinyin in the alternative pinyin before the third pinyin, then the target pinyin sequence here includes the wrong pinyin whose error type is missing pinyin in the alternative pinyin before the third pinyin. If there is an wrong pinyin whose error type is repeated pinyin and redundant pinyin in the alternative pinyin before the third pinyin, then the target pinyin sequence here does not include the wrong pinyin whose error type is repeated pinyin and redundant pinyin in the alternative pinyin before the third pinyin.

[0097] If the third pinyin in the target pinyin sequence is different from the second standard pinyin at the corresponding position in the standard pinyin sequence, and the third alternative pinyin in the target pinyin sequence is the same as the second standard pinyin, the electronic device can determine that the third pinyin and the alternative pinyin between the third pinyin and the third alternative pinyin are erroneous pinyins, and the error type of the third pinyin and the alternative pinyin between the third pinyin and the third alternative pinyin is pinyin redundancy.

[0098] If the third pinyin in the target pinyin sequence is different from the third standard pinyin at the corresponding position in the standard pinyin sequence, and the condition that the third alternative pinyin in the target pinyin sequence is the same as the second standard pinyin is not satisfied, the electronic device can re-determine the wrong pinyin based on the third pinyin, and whether the error type of the wrong pinyin is pinyin repetition, pronunciation error or pinyin omission.

[0099] The third candidate pinyin is located after the third pinyin, and the number of pinyins separated from the third pinyin is less than a first threshold. The first threshold may be a fixed value pre-stored in the electronic device, for example, the first threshold may be 2, that is, the third candidate pinyin is located after the third pinyin and adjacent to the third pinyin.

[0100] In an embodiment of the present application, the error type may also include pinyin omission. If the fourth pinyin in the target pinyin sequence is the same as the third standard pinyin at the corresponding position in the standard pinyin sequence, the electronic device may determine that the fourth pinyin is the correct pinyin.

[0101] It is understandable that if there is an alternative pinyin whose error type is missing pinyin in the alternative pinyin before the fourth pinyin, then the target pinyin sequence here includes the wrong pinyin whose error type is missing pinyin in the alternative pinyin before the fourth pinyin. If there is an wrong pinyin whose error type is repeated pinyin and redundant pinyin in the alternative pinyin before the fourth pinyin, then the target pinyin sequence here does not include the wrong pinyin whose error type is repeated pinyin and redundant pinyin in the alternative pinyin before the fourth pinyin.

[0102] If the fourth pinyin in the target pinyin sequence is different from the third standard pinyin at the corresponding position in the standard pinyin sequence, and is the same as the fourth standard pinyin in the standard pinyin sequence, the electronic device can determine that the third standard pinyin and the standard pinyin between the third standard pinyin and the fourth standard pinyin are incorrect pinyins, and the error type of the third standard pinyin and the standard pinyin between the third standard pinyin and the fourth standard pinyin is pinyin omission.

[0103] If the fourth pinyin in the target pinyin sequence is different from the third standard pinyin at the corresponding position in the standard pinyin sequence, and does not meet the condition of being different from the fourth standard pinyin in the standard pinyin sequence, the electronic device can re-determine the wrong pinyin based on the fourth pinyin, and whether the error type of the wrong pinyin is pinyin repetition, pronunciation error or pinyin redundancy.

[0104] The fourth standard pinyin is located after the third standard pinyin, and the number of pinyins between the fourth standard pinyin and the third standard pinyin is less than a second threshold. The second threshold may be a fixed value pre-stored in the electronic device. For example, the second threshold may be 2, which means that the second standard pinyin is located after the first standard pinyin and is adjacent to the first standard pinyin.

[0105] In an embodiment of the present application, the sample data for training the pinyin recognition model may also include the label digits of each pinyin in the pinyin sequence of the audio sample. After the target audio is input into the pinyin recognition model, the electronic device may also obtain the target label digit sequence of multiple pinyins in the target pinyin sequence output by the pinyin recognition model. The electronic device may determine the public label digit sequence of the target label digit sequence and the standard label digit sequence, update the label digits that do not belong to the public label digit sequence in the target label digit sequence to the first target value, and obtain the updated target label digit sequence. And update the label digits that do not belong to the public label digit sequence in the standard label digit sequence to the second target value, and obtain the updated standard label digit sequence. It is understandable that the target pinyin sequence adopted here is the target pinyin sequence determined by the above-mentioned step 304.

[0106] Wherein, the target label digital sequence may include multiple label digitals, and the multiple label digitals correspond one-to-one with multiple pinyins in the target pinyin sequence. The standard label digital sequence may include multiple label digitals, and the multiple label digitals correspond one-to-one with multiple pinyins in the standard pinyin sequence. The public label digital sequence is the longest common subsequence of the target label digital sequence and the standard label digital sequence, and the longest subsequence of the target label digital sequence and the standard label digital sequence is the longest common subsequence of the target label digital sequence and the standard label digital sequence.

[0107] Optionally, in the process of determining the tag numbers in the target tag number sequence that do not belong to the public tag number sequence, the electronic device may traverse the target tag number sequence and the public tag number sequence simultaneously according to a target order, wherein the target order may be an order from front to back.

[0108] If the first target label number in the acquired target label number sequence is different from the first public label number in the public label number sequence, it can be determined that the first target label number does not belong to the public label number sequence, and the first target label number is updated to the first target value. Afterwards, based on the second target label number in the target label number sequence and the first public label number, it is determined whether the second target label number belongs to the public label number sequence. Among them, the position of the first target label number in the target label number sequence is the same as the position of the first public label number in the public label number sequence. The second target label number is located after the first target label number and is adjacent to the first target label number.

[0109] If the first target label number is the same as the first public label number, the electronic device can determine whether the second target label number belongs to the public label number sequence based on the second target label number and the second public label number. By analogy, the electronic device can use this method to determine all label numbers in the target label number sequence that do not belong to the public label number sequence. Among them, the second public label number is located after the first public label number and is adjacent to the first public label number.

[0110] Example, reference Fig. 9 and Fig.11 , if the first target tag number is the tag number 3 located at the 5th position of the target tag number sequence, the second target tag number is the tag number 4 located at the 6th position of the target tag number sequence, and the first public tag number is the tag number 5 located at the 5th position of the public tag number sequence. Since the first target tag number 3 is different from the first public tag number 5, and the second target tag number 4 is different from the first public tag number 5, the electronic device can update both the first target tag number 3 and the second target tag number 4 to the first target value.

[0111] It can be understood that the process of the electronic device determining the label numbers in the standard label number sequence that do not belong to the public label number sequence can refer to the process of determining the label numbers in the target label number sequence that do not belong to the public label number sequence, and the embodiments of the present application will not be repeated here.

[0112] The first target value and the second target value may be values ​​pre-stored in the electronic device, and the first target value and the second target value may be the same or different. For example, the first target value and the second target value may both be 0.

[0113] After the electronic device obtains the updated target label digital sequence and the updated standard label digital sequence, it can sequentially traverse the updated target label digital sequence. If the first target value is traversed, it can be determined that the first pinyin in the target pinyin sequence is different from the standard pinyin at the corresponding position in the standard pinyin sequence. The position of the first target value traversed this time in the updated target label digital sequence is the same as the position of the first pinyin in the target pinyin sequence.

[0114] If the electronic device detects that there is a pinyin identical to the first pinyin in the first candidate pinyin in the target pinyin sequence, it can be determined that the first pinyin is an incorrect pinyin, and the type of the incorrect pinyin is a pinyin repetition. If the electronic device does not detect that there is a pinyin identical to the first pinyin in the first candidate pinyin in the target pinyin sequence, the electronic device can determine that the condition that there is a pinyin identical to the first pinyin in the first candidate pinyin in the target pinyin sequence is not met.

[0115] Optionally, the label number corresponding to pinyin chuang is 1, the label number corresponding to qian is 2, the label number corresponding to ming is 3, the label number corresponding to yue is 4, the label number corresponding to guang is 5, and the label number corresponding to yi is 6. The label number corresponding to shi is 7, the label number corresponding to di is 8, the label number corresponding to shang is 9, the label number corresponding to shuang is 10, and the label number corresponding to si is 11.

[0116] Assuming that the first target value and the second target value are both 0, refer to Fig.10 , the standard label digital sequence L may include (1, 2, 3, 4, 5, 6, 7, 8, 9, 10), refer to Fig. 9 , the target tag digital sequence R can include (1, 2, 3, 4, 3, 4, 5, 6, 11, 8, 10). Fig.11 , the public tag digital sequence S may include (1, 2, 3, 4, 5, 6, 8, 10). Fig.12 , the updated standard label digital sequence SL may include (1, 2, 3, 4, 5, 6, 0, 8, 0, 10), refer to Fig.13 , the updated target tag digital sequence SR may include (1, 2, 3, 4, 0, 0, 5, 6, 0, 8, 10).

[0117] refer to Fig. 9 and Fig.13 , when the electronic device sequentially traverses the updated target label digital sequence, if it traverses to the fifth label digital 0 (i.e., the first target value), it can be determined that the fifth pinyin in the target pinyin sequence (i.e., the first pinyin ming) is different from the standard pinyin guang at the corresponding position in the standard pinyin sequence. The first target value 0 is at position 5 in the updated target label digital sequence, which is the same as the first pinyin ming at position 5 in the target pinyin sequence.

[0118] Assuming that the first candidate pinyin is the three pinyins located before and adjacent to the first pinyin, the electronic device can determine that the first candidate pinyin includes qian ming yue. Since the first candidate pinyin (qian ming yue) contains the same pinyin as the first pinyin ming, the electronic device can determine that the first pinyin ming is an incorrect pinyin, and the error type of the first pinyin ming is pinyin repetition.

[0119] Optionally, if the electronic device traverses to the first target value in the updated target tag digital sequence, it can be determined that the second pinyin in the target pinyin sequence is different from the first standard pinyin at the corresponding position in the standard pinyin sequence. The position of the first target value traversed this time in the updated target tag digital sequence is the same as the position of the second pinyin in the target pinyin sequence.

[0120] Afterwards, the electronic device can determine the first position of the label number corresponding to the second candidate pinyin in the updated standard label number sequence, and the second position of the label number corresponding to the third candidate pinyin in the updated standard label number sequence. If the difference between the second position and the first position is greater than the difference threshold, the electronic device can determine that the second candidate pinyin is the same as the standard pinyin at the corresponding position in the standard pinyin sequence, and the third candidate pinyin is the same as the standard pinyin at the corresponding position in the standard pinyin sequence. Then the electronic device can determine that the second pinyin is an incorrect pinyin, and the error type of the second pinyin is a pronunciation error.

[0121] If the difference between the second position and the first position is not greater than the difference threshold, the electronic device can determine that the second candidate pinyin is the same as the standard pinyin at the corresponding position in the standard pinyin sequence, and the third candidate pinyin is the same as the standard pinyin at the corresponding position in the standard pinyin sequence. The difference threshold can be a fixed value pre-stored in the electronic device, and for example, the difference threshold can be 1.

[0122] refer to Fig. 9 and Fig.13 , when the electronic device sequentially traverses the updated target label digital sequence, if it traverses to the 9th label digital 0 (i.e., the first target value), it can be determined that the 9th pinyin in the target pinyin sequence (i.e., the second pinyin si) is different from the standard pinyin shi at the corresponding position in the standard pinyin sequence. The first target value 0 is at position 9 in the updated target label digital sequence, which is the same as the second pinyin si at position 9 in the target pinyin sequence.

[0123] refer to Fig.12 and Fig.13 , the electronic device can determine that the digital label 6 corresponding to the second candidate pinyin yi is 6 in the first position of the updated standard label digital sequence, and the digital label 8 corresponding to the third candidate pinyin di is 8 in the second position of the updated standard label digital sequence. Since the difference 2 between the second position 8 and the first position 6 is greater than 1, the electronic device can determine that the second candidate pinyin yi is the same as the standard pinyin yi at the corresponding position in the standard pinyin sequence, and the third candidate pinyin di is the same as the standard pinyin di at the corresponding position in the standard pinyin sequence, and then the electronic device can determine that the second pinyin si is an incorrect pinyin, and the error type of the second pinyin si is a pronunciation error.

[0124] Optionally, after the electronic device traverses to the first target value in the updated target tag digital sequence, it can determine that the third pinyin in the target pinyin sequence is different from the second standard pinyin at the corresponding position in the standard pinyin sequence. The position of the first target value traversed this time in the updated target tag digital sequence is the same as the position of the third pinyin in the target pinyin sequence.

[0125] Afterwards, the electronic device can determine that the label number located before and adjacent to the first target value is at the third position in the updated standard label number sequence, and the label number corresponding to the third alternative pinyin is at the fourth position in the updated standard label number sequence.

[0126] If the difference between the fourth position and the third position is equal to the difference threshold, it can be determined that the third candidate pinyin is the same as the second standard pinyin, and then the electronic device can determine that the third pinyin and the candidate pinyin between the third pinyin and the third candidate pinyin are wrong pinyins, and the error type of the third pinyin and the candidate pinyin between the third pinyin and the third candidate pinyin is pinyin redundancy. If the difference between the fourth position and the third position is not equal to the difference threshold, it can be determined that the condition that the third candidate pinyin is the same as the second standard pinyin is not met.

[0127] Optionally, after traversing the updated target label digital sequence, the electronic device may also traverse the updated standard label digital sequence in sequence. If the second target value is traversed in the updated standard label digital sequence, it can be determined that the fourth pinyin in the target pinyin sequence is different from the third standard pinyin at the corresponding position in the standard pinyin sequence. The position of the second target value traversed this time in the updated standard label digital sequence is the same as the position of the third standard pinyin in the standard pinyin sequence.

[0128] Afterwards, the electronic device can determine that the label number located before and adjacent to the second target value is at the fifth position in the updated standard label number sequence, the label number corresponding to the fourth standard pinyin is at the sixth position in the updated standard label number sequence, the label number corresponding to the pinyin located before and adjacent to the fourth pinyin is at the seventh position in the updated target label number sequence, and the label number corresponding to the fourth pinyin is at the eighth position in the updated target label number sequence.

[0129] If the difference between the eighth position and the seventh position is less than the difference between the sixth position and the fifth position, it can be determined that the fourth pinyin is the same as the fourth standard pinyin. Then the electronic device can determine that the third standard pinyin and the standard pinyin between the third standard pinyin and the fourth standard pinyin are wrong pinyin, and the error type of the third standard pinyin and the standard pinyin between the third standard pinyin and the fourth standard pinyin is pinyin omission. If the difference between the eighth position and the seventh position is greater than or equal to the difference between the sixth position and the fifth position, the electronic device can determine that the condition that the fourth pinyin is the same as the fourth standard pinyin is not met.

[0130] refer to Fig.12 , when the electronic device sequentially traverses the updated standard label digital sequence, if it traverses to the 9th label digital 0 (i.e., the second target value), it can be determined that the fourth pinyin shuang in the target pinyin sequence is different from the fourth standard pinyin shang. The second target value 0 is at position 9 in the updated standard label digital sequence, which is the same as the position 9 of the fourth pinyin shang in the standard pinyin sequence.

[0131] refer to Fig.12 and Fig.13 , the label number 8 located before and adjacent to the second target value 0 is at the fifth position 8 in the updated standard label number sequence, and the label number 10 corresponding to the fourth standard pinyin shuang is at the sixth position 10 in the updated standard label number sequence. The label number 8 corresponding to the pinyin di located before and adjacent to the fourth pinyin shuang is at the seventh position 10 in the updated target label number sequence, and the label number 10 corresponding to the fourth pinyin shuang is at the eighth position 11 in the updated target label number sequence.

[0132] The difference between the eighth position 11 and the seventh position 10 is 1, and the difference between the sixth position 10 and the fifth position 8 is 2. Since 2 is greater than 1, the electronic device can determine that the third standard pinyin is an incorrect pinyin, and the error type of the third standard pinyin is pinyin omission.

[0133] In the embodiment of the present application, the electronic device may use the following procedure to determine the incorrect pinyin of the target pinyin sequence relative to the standard pinyin sequence, and the error type of the incorrect pinyin:

[0134] Use i to traverse between (1, R.length); / / That is, traverse the updated target label digital sequence in sequence, R.length represents the number of label numbers included in the target label digital sequence, i is an integer greater than or equal to 1 and less than or equal to R.length / /

[0135] If SR(i)!=0, then R(i) is correct; / / R(i) is correct, indicating that the pinyin at the position corresponding to R(i) in the target pinyin sequence is correct / / ;

[0136] Otherwise: if R(i)∈{R(i+j)|j>=-t and j<=t and j!=i}, then R(i) is a pinyin repetition; / / t is the number of the first candidate pinyins. If i is 5 and t is 2, then check whether R(5)∈{R(3), R(4), R(5), R(6), R(7)}. R(i) is a repetition error, indicating that the pinyin at the position corresponding to R(i) in the target pinyin sequence is an incorrect pinyin, and the type of the incorrect pinyin is a pinyin repetition. / /

[0137] Otherwise: record the number of non-zero label numbers in the updated target label number sequence SR(1:i-1) as n;

[0138] The position of S(n) in the updated standard label digital sequence SL is recorded as a, and the position of S(n+1) in the updated standard label digital sequence SL is recorded as b, that is, SL(a)==S(n), SL(b)==S(n+1);

[0139] If ba>1, then R(i) is mispronounced; / / R(i) is mispronounced, indicating that the pinyin at the position corresponding to R(i) in the target pinyin sequence is an incorrect pinyin, and the type of the incorrect pinyin is a repeated error / /

[0140] If ba==1, then R(i) is a redundant pinyin; / / R(i) is a mispronunciation, indicating that the pinyin at the position corresponding to R(i) in the target pinyin sequence is an incorrect pinyin, and the type of the incorrect pinyin is redundant pinyin / /

[0141] Use i to traverse between (1, L.length); / / That is, traverse the updated standard label digital sequence in sequence, L.length represents the number of pinyins included in the standard label digital sequence / /

[0142] If SL(i)==0 in the updated standard label digital sequence, the number of non-zero label numbers in the updated standard label digital sequence SL(1:j-1) is recorded as n;

[0143] The position of S(n) in the updated standard label digital sequence SL is recorded as a, and the position of S(n+1) in the updated standard label digital sequence SL is recorded as b;

[0144] The position of S(n) in the updated target label digital sequence SR is recorded as c, and the position of S(n+1) in the updated target label digital sequence SR is recorded as d;

[0145] If d - c < b - a, determine that the pinyin corresponding to L(a + 1:b - 1) is the pinyin missing between the pinyin corresponding to R(c) and the pinyin corresponding to R(d). / / L(a + 1:b - 1) includes L(a + 1) to L(b - 1) / /

[0146] Step 306, display the standard text and error marks.

[0147] After the electronic device determines the incorrect pinyin and the error type of the incorrect pinyin of the target pinyin sequence relative to the standard pinyin sequence of the standard text, it can also display the standard text and error marks, where the error marks are used to indicate the text corresponding to the incorrect pinyin and the error type. Among them, the error marks can include forms such as underlines, bold, and annotations.

[0148] Optionally, the electronic device can display the standard text, and the display effects of the texts corresponding to the incorrect pinyin of different error types in the standard text are different. Among them, the display effects can include at least one of the display color of the text, the font of the text, the size of the text, and the background color of the text.

[0149] Exemplarily, referring to Fig.14 , the font of the text "Mingyue" corresponding to the incorrect pinyin with the error type of duplicate pinyin is italic, the font of the text "shi" corresponding to the incorrect pinyin with the error type of pronunciation error is a large font, and the text "shang" corresponding to the incorrect pinyin with the error type of missing pinyin is bold.

[0150] Or, the background color of the text "Mingyue" corresponding to the incorrect pinyin with the error type of duplicate pinyin is yellow, the background color of the text "shi" corresponding to the incorrect pinyin with the error type of pronunciation error is green, and the background color of the text "shang" corresponding to the incorrect pinyin with the error type of missing pinyin is red.

[0151] In summary, the embodiments of the present application provide an audio inspection method. After the electronic device converts the target audio to be inspected into a target pinyin sequence, it can determine the incorrect pinyin and the error type of the incorrect pinyin of the target pinyin sequence relative to the standard pinyin sequence of the standard text, and mark the text corresponding to the incorrect pinyin and the error type in the displayed standard text. Compared with the manual inspection of the mastery of the article, the method provided by the embodiments of the present application enables the electronic device to automatically inspect the target audio to achieve the inspection of the mastery of the article, thereby improving the efficiency of the inspection of the mastery of the article.

[0152] Moreover, since the target audio is converted into a target pinyin sequence and the errors in the audio are inspected in the form of pinyin, compared with converting the target audio into text and inspecting the errors in the audio based on the converted text, the method provided by the embodiments of the present application effectively improves the accuracy of audio inspection.

[0153] Fig.15 This is another flow chart of an audio inspection provided by an embodiment of the present application. This method can be applied to Figure 1 In the electronic device shown. Fig.15 As shown, the method may include:

[0154] Step 307: Determine the longest common subsequence of the target pinyin sequence and the standard pinyin sequence.

[0155] After determining the target pinyin sequence, the electronic device can also determine the longest common subsequence of the target pinyin sequence and the standard pinyin sequence. The longest subsequence of the target pinyin sequence and the standard pinyin sequence is the longest common subsequence of the target pinyin sequence and the standard pinyin sequence.

[0156] refer to Fig.11 , Fig. 9 The target phonetic sequence shown is Fig.10 The longest common subsequence of the standard pinyin sequence shown may include the pinyin of the text "the moon shines brightly before the bed, and I suspect frost on the ground".

[0157] Step 308: Determine the target phoneme sequence corresponding to the longest common subsequence from the phoneme sequence.

[0158] After determining the longest common subsequence, the electronic device can determine the target phoneme sequence corresponding to the longest common subsequence from the phoneme sequence.

[0159] Optionally, the electronic device may determine the phoneme sequence corresponding to each pinyin of the longest common subsequence from the phoneme sequence, thereby obtaining the target phoneme sequence.

[0160] Step 309: Determine a first duration sequence corresponding to the target phoneme sequence.

[0161] After determining the target phoneme sequence corresponding to the longest common subsequence, the electronic device may determine a first duration sequence corresponding to the target phoneme sequence, wherein the first duration sequence may include a first number of slice durations, and the first number may be the number of phonemes included in the target phoneme sequence.

[0162] Optionally, the electronic device may replace each phoneme in the target phoneme sequence with a slice duration, and determine the updated target phoneme sequence as a first duration sequence.

[0163] Step 310: Determine the fluency of the target audio based on the similarity between the first time sequence and the second time sequence.

[0164] After determining the first time sequence, the electronic device may determine the similarity between the first time sequence and the second time sequence, and may determine the fluency of the target audio based on the similarity between the first time sequence and the second time sequence.

[0165] The second duration sequence may include a second number of slice durations, where the second number is the number of phonemes included in the standard phoneme sequence of the standard pinyin sequence. The second duration sequence may be pre-stored in the electronic device. The fluency is positively correlated with the similarity, that is, the higher the similarity, the smoother the target audio, and the lower the similarity, the less smooth the target audio.

[0166] Optionally, the electronic device may directly determine the similarity between the first time sequence A and the second time sequence B as the fluency of the target audio. The similarity R may satisfy: Here, |A| is the modulus of the first time-length sequence A, and |B| is the modulus of the second time-length sequence B.

[0167] Alternatively, the electronic device may determine the product of the similarity and the first coefficient as the fluency of the target audio, wherein the first coefficient may be a fixed value pre-stored in the electronic device.

[0168] Step 311: Determine the degree of completion of the target audio based on a first ratio of the total number of pinyins in the longest common subsequence to the total number of pinyins in the standard pinyin sequence.

[0169] In the embodiment of the present application, after determining the longest common subsequence, the electronic device can also determine the total number of pinyins in the longest common subsequence and the total number of pinyins in the standard pinyin sequence, and determine a first ratio of the total number of pinyins in the longest common subsequence to the total number of pinyins in the standard pinyin sequence. Then the electronic device can determine the degree of completion of the target audio based on the first ratio.

[0170] The degree of completion is positively correlated with the first ratio, that is, the larger the first ratio is, the higher the degree of completion of the target audio is, and the smaller the first ratio is, the lower the degree of completion of the target audio is.

[0171] Optionally, the electronic device may directly determine the first ratio as the completion of the target audio, or may determine the product of the first ratio and the second coefficient as the completion of the target audio, wherein the second coefficient may be a fixed value pre-stored in the electronic device.

[0172] refer to Fig.10 and Fig.11 , the total number of pinyins in the standard pinyin sequence is 10, the total number of pinyins in the longest common subsequence is 8, and the electronic device can directly use the first ratio The product of 80% and the second coefficient 100% is determined as the degree of completion of the target audio.

[0173] Step 312: determine the second ratio of the total number of erroneous pinyins to the total number of pinyins in the standard pinyin sequence as the error rate of the target audio.

[0174] After determining the erroneous pinyins of the target pinyin sequence relative to the standard pinyin sequence, the electronic device can determine the total number of erroneous pinyins and the total number of pinyins in the standard pinyin sequence, and determine the total number of erroneous pinyins and the total number of pinyins in the standard pinyin sequence. Then the electronic device can determine the error rate of the target audio based on the second ratio.

[0175] The error rate is positively correlated with the second ratio, that is, the larger the second ratio is, the higher the error rate of the target audio is, and the smaller the second ratio is, the lower the error rate of the target audio is.

[0176] Optionally, the electronic device may directly determine the second ratio as the error rate of the target audio, or may determine the error rate of the target audio as the product of the second ratio and a third coefficient, wherein the third coefficient may be a fixed value pre-stored in the electronic device.

[0177] The incorrect pinyin determined based on the above step 305 includes four incorrect pinyins: ming yue si shang. Fig.10 , the total number of pinyin in the standard pinyin sequence is 10, so the electronic device can convert the second ratio to the first ratio The product of 40% and the third coefficient 100% is determined as the error rate of the target audio.

[0178] Step 313: Display the fluency, completion and error rate of the target audio.

[0179] After displaying the standard text and error marks, the electronic device can also display the fluency, completion and error rate of the target audio. Fig.14 , the electronic device can display the fluency of the target audio: 60%, completion: 80%, and error rate: 40%.

[0180] In summary, the embodiment of the present application provides an audio inspection method. After the electronic device converts the target audio to be inspected into a target pinyin sequence, it can determine the incorrect pinyin of the target pinyin sequence relative to the standard pinyin sequence of the standard text and the error type of the incorrect pinyin, and mark the text corresponding to the incorrect pinyin and the error type in the displayed standard text. Compared with manual inspection of the mastery of the article, the method provided by the embodiment of the present application allows the electronic device to automatically inspect the target audio to realize the inspection of the mastery of the article, thereby improving the efficiency of the inspection of the mastery of the article.

[0181] Moreover, since the target audio is converted into a target pinyin sequence and the errors in the audio are checked in the form of pinyin, compared with converting the target audio into text and checking the errors in the audio based on the converted text, the method provided in the embodiment of the present application effectively improves the accuracy of audio checking.

[0182] It should be noted that the order of the steps of the audio inspection method provided in the embodiment of the present application can be appropriately adjusted, and the steps can also be deleted according to the situation, for example, steps 1501 to 1507 can be deleted according to the situation. Any technician familiar with the technical field can easily think of a change within the technical scope disclosed in this application, and it should be included in the protection scope of this application, so it will not be repeated.

[0183] The present application embodiment provides an electronic device, such as Fig.16 As shown, the electronic device may include a processor 101 and a display screen 102. The processor 101 is used to:

[0184] Get the target audio to be checked.

[0185] The target audio is converted into a target pinyin sequence, where the target pinyin sequence includes the pinyin of multiple characters.

[0186] Determine the incorrect pinyin of the target pinyin sequence relative to the standard pinyin sequence of the standard text and the error type of the incorrect pinyin, the error type including: at least one of pinyin repetition, pronunciation error, pinyin omission and pinyin redundancy.

[0187] The display screen 102 is used to display standard text and error marks. The error marks are used to indicate the text corresponding to the incorrect pinyin and the error type.

[0188] In summary, the embodiment of the present application provides an electronic device, which, after converting the target audio to be checked into a target pinyin sequence, can determine the wrong pinyin of the target pinyin sequence relative to the standard pinyin sequence of the standard text and the error type of the wrong pinyin, and mark the text corresponding to the wrong pinyin and the error type in the displayed standard text. Compared with manual inspection of the mastery of the article, the electronic device provided by the embodiment of the present application can automatically check the target audio to realize the inspection of the mastery of the article, thereby improving the efficiency of the inspection of the mastery of the article.

[0189] Furthermore, since the target audio is converted into a target pinyin sequence and errors in the audio are checked in the form of pinyin, the accuracy of audio checking is effectively improved compared to converting the target audio into text and checking errors in the audio based on the converted text.

[0190] Optionally, the display screen 102 is used for:

[0191] Display standard text, and the text corresponding to the incorrect pinyin of different error types in the standard text will have different display effects.

[0192] Optionally, the processor 101 is configured to:

[0193] If the first pinyin in the target pinyin sequence is different from the standard pinyin at the corresponding position in the standard pinyin sequence, and there is a pinyin that is the same as the first pinyin in the first alternative pinyin in the target pinyin sequence, then the first pinyin is determined to be an incorrect pinyin, and the error type of the first pinyin is pinyin repetition.

[0194] The first candidate pinyin is at least one pinyin that is located before the first pinyin and adjacent to the first pinyin.

[0195] Optionally, the processor 101 is configured to:

[0196] If the second pinyin in the target pinyin sequence is different from the first standard pinyin at the corresponding position in the standard pinyin sequence, and the second alternative pinyin in the target pinyin sequence is the same as the standard pinyin at the corresponding position in the standard pinyin sequence, and the third alternative pinyin is the same as the standard pinyin at the corresponding position in the standard pinyin sequence, then the second pinyin is determined to be an incorrect pinyin, and the error type of the second pinyin is a pronunciation error.

[0197] Among them, the second alternative pinyin is located before the second pinyin and is adjacent to the second pinyin, and the third alternative pinyin is located after the second pinyin and is adjacent to the second pinyin.

[0198] Optionally, the processor 101 is configured to:

[0199] If the third pinyin in the target pinyin sequence is different from the second standard pinyin at the corresponding position in the standard pinyin sequence, and the third alternative pinyin in the target pinyin sequence is the same as the second standard pinyin, then the third pinyin and the alternative pinyin between the third pinyin and the third alternative pinyin are determined to be incorrect pinyins, and the error type of the third pinyin and the alternative pinyin between the third pinyin and the third alternative pinyin is pinyin redundancy.

[0200] Among them, the third candidate pinyin is located after the third pinyin, and the number of pinyins between the third pinyin and the third pinyin is less than the first threshold.

[0201] Optionally, the processor 101 is configured to:

[0202] If the fourth pinyin in the target pinyin sequence is different from the third standard pinyin at the corresponding position in the standard pinyin sequence, and is the same as the fourth standard pinyin in the standard pinyin sequence, then the third standard pinyin and the standard pinyin between the third standard pinyin and the fourth standard pinyin are determined to be incorrect pinyins, and the error type of the third standard pinyin and the standard pinyin between the third standard pinyin and the fourth standard pinyin is pinyin omission.

[0203] Among them, the fourth standard pinyin is located after the third standard pinyin, and the number of pinyins between the fourth standard pinyin and the third standard pinyin is less than the second threshold.

[0204] Optionally, the processor 101 is configured to:

[0205] The target audio is segmented at every slice duration to obtain multiple audio segments.

[0206] A phoneme sequence is determined based on the plurality of audio segments, wherein the phoneme sequence includes a plurality of phonemes, and each phoneme is determined based on one audio segment.

[0207] A target pinyin sequence of the target audio is obtained based on the phoneme sequence.

[0208] Optionally, the processor 101 is further configured to:

[0209] After obtaining the target pinyin sequence of the target audio based on the phoneme sequence, the longest common subsequence of the target pinyin sequence and the standard pinyin sequence is determined.

[0210] Determine the target phoneme sequence corresponding to the longest common subsequence from the phoneme sequence.

[0211] A first duration sequence corresponding to the target phoneme sequence is determined, the first duration sequence including a first number of slice durations, where the first number is the number of phonemes included in the target phoneme sequence.

[0212] Based on the similarity between the first time sequence and the second time sequence, the fluency of the target audio is determined, the second time sequence includes a second number of slice durations, the second number is the number of phonemes included in the standard phoneme sequence of the standard pinyin sequence, and the fluency is positively correlated with the similarity.

[0213] Based on a first ratio of the total number of pinyins in the longest common subsequence to the total number of pinyins in the standard pinyin sequence, the completion of the target audio is determined, and the completion is positively correlated with the first ratio.

[0214] Displays the smoothness and completeness of the target audio.

[0215] Optionally, the processor 101 is further configured to:

[0216] After determining the erroneous pinyins in the target pinyin sequence relative to the standard pinyin sequence, a second ratio of the total number of erroneous pinyins to the total number of pinyins in the standard pinyin sequence is determined as the error rate of the target audio, and the error rate is positively correlated with the second ratio.

[0217] Displays the error rate of the target audio.

[0218] In summary, the embodiment of the present application provides an electronic device, which can determine the wrong pinyin of the target pinyin sequence relative to the standard pinyin sequence of the standard text and the error type of the wrong pinyin after converting the target audio to be checked into the target pinyin sequence, and mark the text corresponding to the wrong pinyin and the error type in the displayed standard text. Compared with manual inspection of the mastery of the article, the electronic device provided by the embodiment of the present application can automatically check the target audio to realize the inspection of the mastery of the article, thereby improving the efficiency of the inspection of the mastery of the article.

[0219] Furthermore, since the target audio is converted into a target pinyin sequence and errors in the audio are checked in the form of pinyin, the accuracy of audio checking is effectively improved compared to converting the target audio into text and checking errors in the audio based on the converted text.

[0220] Fig.17 A structural diagram of another electronic device provided in an embodiment of the present application is as follows: Fig.17 As shown, the electronic device also includes: a camera 130, a memory 140, a radio frequency (RF) circuit 150, an audio circuit 160, a wireless fidelity (Wi-Fi) module 170, a Bluetooth module 180, a power supply 190 and other components.

[0221] Among them, the camera 130 can be used to capture still pictures or videos. The object generates an optical picture through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the processor 101 to convert it into a digital picture signal.

[0222] The processor 101 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. It executes various functions of the electronic device and processes data by running or executing software programs stored in the memory 140 and calling data stored in the memory 140. In some embodiments, the processor 101 may include one or more processing units; the processor 101 may also integrate an application processor and a baseband processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the baseband processor mainly processes wireless communications. It is understandable that the above-mentioned baseband processor may not be integrated into the processor 101. In the present application, the processor 101 can run the operating system audio inspection method, application program, user interface display and touch response. In addition, the processor 101 is coupled to the input unit and the display screen 102.

[0223] The display unit can be used to receive input digital or character information, generate signal input related to user settings and function control of the electronic device, and optionally, the display unit can also be used to display information input by the user or information provided to the user and a graphical user interface (GUI) of various menus of the electronic device. The display unit may include a display screen 102 arranged on the front of the electronic device. Among them, the display screen 102 can be configured in the form of a liquid crystal display, a light emitting diode, etc. The display screen 102 can be used to display various graphical user interfaces described in this application.

[0224] The display unit includes: a display screen 102 and a touch screen 103 arranged on the front of the electronic device. The display screen 102 can be used to display preview pictures. The touch screen 103 can collect user touch operations on or near it, such as clicking a button, dragging a scroll box, etc. Among them, the touch screen 103 can be covered on the display screen 102, or the touch screen 103 can be integrated with the display screen 102 to realize the input and output functions of the electronic device. After integration, it can be simply called a touch display screen.

[0225] The memory 140 can be used to store software programs and data. The processor 101 executes various functions and data processing of the electronic device by running the software programs or data stored in the memory 140. The memory 140 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. The memory 140 stores an operating system that enables the electronic device to run. In the present application, the memory 140 can store an operating system and various application programs, and can also store the code for executing the audio inspection method described in the embodiment of the present application.

[0226] The RF circuit 150 can be used to receive and send signals during the process of sending and receiving information or making calls. It can receive downlink data from the base station and hand it over to the processor 101 for processing; it can send uplink data to the base station. Generally, the RF circuit includes but is not limited to antennas, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer and other devices.

[0227] The audio circuit 160, the speaker 161 and the microphone 162 can provide an audio interface between the user and the electronic device. The audio circuit 160 can transmit the electrical signal converted from the received audio data to the speaker 161, which is converted into a sound signal for output. The electronic device can also be equipped with a volume button for adjusting the volume of the sound signal. On the other hand, the microphone 162 converts the collected sound signal into an electrical signal, which is received by the audio circuit 160 and converted into audio data, and then outputs the audio data to the RF circuit 150 to send to, for example, another electronic device, or outputs the audio data to the memory 140 for further processing. In this application, the microphone 162 can obtain the user's voice.

[0228] Wi-Fi is a short-range wireless transmission technology. Electronic devices can help users send and receive emails, browse web pages, and access streaming media through the Wi-Fi module 170, which provides users with wireless broadband Internet access.

[0229] The Bluetooth module 180 is used to exchange information with other Bluetooth devices having Bluetooth modules through the Bluetooth protocol. For example, the electronic device can establish a Bluetooth connection with a wearable electronic device (such as a smart watch) that also has a Bluetooth module through the Bluetooth module 180 to exchange data.

[0230] The electronic device also includes a power supply 190 (such as a battery) for supplying power to various components. The power supply can be logically connected to the processor 101 through a power management system, so that the power management system can manage functions such as charging, discharging, and power consumption. The electronic device can also be configured with a power button for turning on and off the electronic device, as well as locking the screen and other functions.

[0231] The electronic device may further include at least one sensor 1110, such as a motion sensor 11101, a distance sensor 11102, and a fingerprint sensor 11103. The electronic device may also be configured with other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, and an infrared sensor.

[0232] Fig.18It is a software structure diagram of the electronic device provided in the embodiment of the present application. The layered architecture divides the software into several layers, each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, namely, the application layer, the application framework layer, the Android runtime (Android runtime) and the system library, and the kernel layer.

[0233] The application layer can include a series of application packages. Fig.18 As shown, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, etc. The application framework layer provides an application programming interface (API) and programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0234] like Fig.18 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0235] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0236] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, pictures, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0237] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0238] The phone manager is used to provide communication functions for electronic devices, such as the management of call status (including answering, hanging up, etc.).

[0239] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0240] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is prompted in the status bar, a prompt sound is emitted, a communication electronic device vibrates, an indicator light flashes, etc.

[0241] Android runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0242] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0243] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0244] The system library may include multiple functional modules, such as surface manager, media libraries, 3D graphics processing library (such as openGL ES), 2D graphics engine (such as SGL), etc.

[0245] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0246] The media library supports playback and acquisition of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0247] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing.

[0248] A 2D graphics engine is a drawing engine for 2D drawings.

[0249] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

[0250] The embodiment of the present application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory, and the processor implements the audio inspection method provided in the above embodiment when executing the computer program, for example, Figure 2 , Figure 3 or Fig.15 The method shown.

[0251] The present application embodiment provides a computer-readable storage medium, in which a computer program is stored. The computer program is loaded and executed by a processor to implement the audio inspection method provided in the above embodiment. For example, Figure 2 , Figure 3 or Fig.15 The method shown.

[0252] The embodiment of the present application provides a computer program product including instructions. When the computer program product is run on a computer, the computer executes the audio inspection method provided in the above embodiment, for example, Figure 2 , Figure 3 or Fig.15 The method shown.

[0253] It should be noted that the information (including but not limited to the target audio to be checked, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions. For example, the target audio to be checked involved in this application is obtained with full authorization.

[0254] It is to be understood that, in the embodiments of the present application, the terms "first", "second", "third", "fourth", "fifth", "sixth", "seventh" and "eighth" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance. The term "plurality" in the embodiments of the present application means two or more. The term "and / or" in the embodiments of the present application is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent three situations: A exists alone, A and B exist at the same time, and B exists alone.

[0255] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0256] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. An audio inspection method, characterized in that: Applied to electronic equipment, the method comprises: Get the target audio to be checked; The target audio is segmented by a segmentation sub-model in the pinyin recognition model at intervals of a segmentation duration to obtain a plurality of audio segments; Inputting the multiple audio clips into a first neural network sub-model in the pinyin recognition model to obtain a phoneme sequence output by the first neural network sub-model, wherein the phoneme sequence includes multiple phonemes, each phoneme being determined based on one of the audio clips; Input the phoneme sequence into the second neural network sub-model in the pinyin recognition model, obtain a target pinyin sequence of the target audio output by the second neural network sub-model, and obtain a target label digital sequence of multiple pinyins in the target pinyin sequence output by the pinyin recognition model, wherein the target pinyin sequence includes pinyins of multiple characters, and the label numbers in the target label digital sequence correspond one-to-one to the pinyins in the target pinyin sequence; Determine a common label digital sequence of the target label digital sequence and the standard label digital sequence, wherein the label digital sequence in the standard label digital sequence corresponds to the pinyin in the standard pinyin sequence one by one, and the common label digital sequence is the longest common subsequence of the target label digital sequence and the standard label digital sequence; Updating the label numbers in the target label digital sequence that do not belong to the public label digital sequence to a first target value to obtain an updated target label digital sequence, and updating the label numbers in the standard label digital sequence that do not belong to the public label digital sequence to a second target value to obtain an updated standard label digital sequence; Traversing the label numbers in the updated target label number sequence in sequence, if traversing to the first target value, determining that the target pinyin currently traversed in the target pinyin sequence is different from the standard pinyin at the corresponding position in the standard pinyin sequence, and the target pinyin currently traversed is an erroneous pinyin, based on the target pinyin currently traversed and the alternative pinyins in the target pinyin sequence, determining that the error type of the currently traversed pinyin is pinyin repetition, mispronunciation or redundant pinyin, the alternative pinyins include the pinyins in the target pinyin sequence that are adjacent to the currently traversed target pinyin, the position of the currently traversed target pinyin in the target pinyin sequence is the same as the position of the currently traversed label number in the updated target label number sequence; Traverse the label numbers in the updated standard label number sequence in sequence. If the second target value is traversed, determine that the standard pinyin currently traversed in the standard pinyin sequence is different from the target pinyin at the corresponding position in the target pinyin sequence. If the target pinyin at the corresponding position in the target pinyin sequence is the same as the reference standard pinyin in the standard pinyin sequence, determine that the currently traversed standard pinyin is an incorrect pinyin and the error type is pinyin omission. The reference standard pinyin is located after the currently traversed standard pinyin, and the number of pinyins separated from the currently traversed standard pinyin is less than a second threshold value. The position of the currently traversed standard pinyin in the standard pinyin sequence is the same as the position of the currently traversed label number in the updated standard label number sequence. The standard text and the error mark are displayed, wherein the error mark is used to indicate the text corresponding to the incorrect pinyin and the error type.

2. The method according to claim 1, characterized in that The displaying of the standard text and the error mark includes: The standard text is displayed, and the display effects of the texts corresponding to the incorrect pinyins of different error types in the standard text are different.

3. The method according to claim 1, characterized in that The target pinyin currently traversed is the first pinyin; The step of determining, based on the currently traversed target pinyin and the candidate pinyins in the target pinyin sequence, that the error type of the currently traversed pinyin is a repeated pinyin, a mispronunciation, or a redundant pinyin comprises: If there is a pinyin identical to the first pinyin in the first candidate pinyin in the target pinyin sequence, determining that the error type of the first pinyin is the repetition of the pinyin; The first candidate pinyin is at least one pinyin that is located before the first pinyin and adjacent to the first pinyin.

4. The method according to claim 1, characterized in that The target pinyin currently traversed is the second pinyin, and the standard pinyin at the corresponding position in the standard pinyin sequence is the first standard pinyin; The step of determining, based on the currently traversed target pinyin and the candidate pinyins in the target pinyin sequence, that the error type of the currently traversed pinyin is a repeated pinyin, a mispronunciation, or a redundant pinyin comprises: If the second candidate pinyin in the target pinyin sequence is the same as the first standard pinyin, and the third candidate pinyin is the same as the first standard pinyin, then determining that the error type of the second pinyin is the pronunciation error; Among them, the second candidate pinyin is located before the second pinyin and adjacent to the second pinyin, and the third candidate pinyin is located after the second pinyin and adjacent to the second pinyin.

5. The method according to claim 1, characterized in that The target pinyin currently traversed is the third pinyin, and the standard pinyin at the corresponding position in the standard pinyin sequence is the second standard pinyin; The step of determining, based on the currently traversed target pinyin and the candidate pinyins in the target pinyin sequence, that the error type of the currently traversed pinyin is a repeated pinyin, a mispronunciation, or a redundant pinyin comprises: If the third candidate pinyin in the target pinyin sequence is the same as the second standard pinyin, the third pinyin and the candidate pinyin between the third pinyin and the third candidate pinyin are determined to be wrong pinyins, and the error type of the third pinyin and the candidate pinyin between the third pinyin and the third candidate pinyin is that the pinyin is redundant; Among them, the third candidate pinyin is located after the third pinyin, and the number of pinyins between the third candidate pinyin and the third pinyin is less than the first threshold.

6. The method according to claim 1, characterized in that The currently traversed standard pinyin is the third standard pinyin, the reference standard pinyin is the fourth standard pinyin, and the target pinyin at the corresponding position in the target pinyin sequence is the fourth pinyin; the method further includes: If the fourth pinyin is different from the third standard pinyin and the same as the fourth standard pinyin, it is determined that the standard pinyin between the third standard pinyin and the fourth standard pinyin is an incorrect pinyin, and the error type of the standard pinyin between the third standard pinyin and the fourth standard pinyin is pinyin omission.

7. The method according to any one of claims 1 to 6, characterized in that: After obtaining the target pinyin sequence of the target audio output by the second neural network sub-model, the method further includes: Determine the longest common subsequence of the target pinyin sequence and the standard pinyin sequence; Determine a target phoneme sequence corresponding to the longest common subsequence from the phoneme sequence; Determine a first duration sequence corresponding to the target phoneme sequence, the first duration sequence including a first number of the slice durations, the first number being the number of phonemes included in the target phoneme sequence; Determine the fluency of the target audio based on the similarity between the first duration sequence and the second duration sequence, the second duration sequence includes a second number of the slice durations, the second number is the number of phonemes included in the standard phoneme sequence of the standard pinyin sequence, and the fluency is positively correlated with the similarity; Determining the degree of completion of the target audio based on a first ratio of the total number of pinyins in the longest common subsequence to the total number of pinyins in the standard pinyin sequence, wherein the degree of completion is positively correlated with the first ratio; Displays the smoothness and completeness of the target audio.

8. The method according to any one of claims 1 to 6, characterized in that: After determining the incorrect pinyin, the method further includes: Determine a second ratio of the total number of erroneous pinyins to the total number of pinyins in the standard pinyin sequence as the error rate of the target audio, wherein the error rate is positively correlated with the second ratio; Displays the error rate of the target audio.

9. An electronic device, characterized in that: The electronic device comprises a processor and a display screen, wherein the processor is used to: Get the target audio to be checked; The target audio is segmented by a segmentation sub-model in the pinyin recognition model at intervals of a segmentation duration to obtain a plurality of audio segments; Inputting the multiple audio clips into a first neural network sub-model in the pinyin recognition model to obtain a phoneme sequence output by the first neural network sub-model, wherein the phoneme sequence includes multiple phonemes, each phoneme being determined based on one of the audio clips; Input the phoneme sequence into the second neural network sub-model in the pinyin recognition model, obtain a target pinyin sequence of the target audio output by the second neural network sub-model, and obtain a target label digital sequence of multiple pinyins in the target pinyin sequence output by the pinyin recognition model, wherein the target pinyin sequence includes pinyins of multiple characters, and the label numbers in the target label digital sequence correspond one-to-one to the pinyins in the target pinyin sequence; Determine a common label digital sequence of the target label digital sequence and the standard label digital sequence, wherein the label digital sequence in the standard label digital sequence corresponds to the pinyin in the standard pinyin sequence one by one, and the common label digital sequence is the longest common subsequence of the target label digital sequence and the standard label digital sequence; Updating the label numbers in the target label digital sequence that do not belong to the public label digital sequence to a first target value to obtain an updated target label digital sequence, and updating the label numbers in the standard label digital sequence that do not belong to the public label digital sequence to a second target value to obtain an updated standard label digital sequence; Traversing the label numbers in the updated target label number sequence in sequence, if traversing to the first target value, determining that the target pinyin currently traversed in the target pinyin sequence is different from the standard pinyin at the corresponding position in the standard pinyin sequence, and the target pinyin currently traversed is an erroneous pinyin, based on the target pinyin currently traversed and the alternative pinyins in the target pinyin sequence, determining that the error type of the currently traversed pinyin is pinyin repetition, mispronunciation or redundant pinyin, the alternative pinyins include the pinyins in the target pinyin sequence that are adjacent to the currently traversed target pinyin, the position of the currently traversed target pinyin in the target pinyin sequence is the same as the position of the currently traversed label number in the updated target label number sequence; Traverse the label numbers in the updated standard label number sequence in sequence. If the second target value is traversed, determine that the standard pinyin currently traversed in the standard pinyin sequence is different from the target pinyin at the corresponding position in the target pinyin sequence. If the target pinyin at the corresponding position in the target pinyin sequence is the same as the reference standard pinyin in the standard pinyin sequence, determine that the currently traversed standard pinyin is an incorrect pinyin and the error type is pinyin omission. The reference standard pinyin is located after the currently traversed standard pinyin, and the number of pinyins separated from the currently traversed standard pinyin is less than a second threshold value. The position of the currently traversed standard pinyin in the standard pinyin sequence is the same as the position of the currently traversed label number in the updated standard label number sequence. The display screen is used to display standard text and error marks, and the error marks are used to indicate the text corresponding to the incorrect pinyin and the error type.

Images