A speech recognition content correction method and apparatus

Abstract

Embodiments of the present application provide a speech recognition content correction method and device, electronic equipment and computer readable storage medium, first continuously obtain audio data of a unit time length, and an encoding feature sequence corresponding to the audio data; according to the encoding feature sequence, obtain a plurality of candidate recognition sequences, sequence scores of the sequences, and a plurality of time sequences corresponding to the plurality of candidate recognition sequences; in the case of receiving complete audio data, re-score the candidate recognition sequences to generate a target candidate recognition sequence; finally, according to the paragraph weight ratio of the time node where the character is updated, correct the character in the target candidate recognition sequence, and output the corrected target candidate recognition sequence as the final recognition result; by introducing paragraph weight analysis of the content relationship between a single character in the sequence and the context, and correcting individual characters one by one, the final output recognition text content is more accurate.

Description

Technical Field

[0001] This invention relates to the field of speech recognition technology, and in particular to a speech recognition content correction method, apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] Speech recognition is a rapidly emerging information technology that captures and analyzes audio media data to accurately generate matching text content. It has great value in scenarios such as live subtitles, meeting minutes, and real-time voice chat software. How to make the text generated by speech recognition algorithms more accurate is an area that speech recognition technology urgently needs to improve.

[0003] In related technologies, the correction method for streaming speech content recognition mainly involves performing CTC feature decoding on the audio data stream after speech feature encoding to generate multiple candidate sequences of recognition results. Then, the accuracy of the generated text recognition results is evaluated, and the text with the highest accuracy is selected as the final recognition result.

[0004] However, in existing technologies, scoring and filtering the generated candidate recognition sequences can only be done by selecting from multiple generated complete candidate result sequences in the decoding and arrangement results. It is impossible to perform fine-grained correction on individual characters of the text content, and the correction method is too limited. Summary of the Invention

[0005] This invention provides a method, apparatus, electronic device, and computer-readable storage medium for correcting speech recognition content, in order to solve the problem that the correction of text content generated by speech recognition is not precise enough in the prior art.

[0006] In a first aspect, embodiments of the present invention provide a method for correcting speech recognition content, the method comprising:

[0007] Acquire audio data per unit time length and the corresponding encoded feature sequence of the audio data;

[0008] Based on the encoded feature sequence, a set of recognition sequences including multiple candidate recognition sequences, sequence scores of the candidate recognition sequences, and multiple time series corresponding to the multiple candidate recognition sequences are obtained. The time series are used to record the time nodes of character updates in the candidate sequences.

[0009] Upon receiving complete audio data, the target candidate recognition sequence is obtained based on the sequence score and the time points of character updates in the candidate recognition sequence.

[0010] The characters in the target candidate recognition sequence are corrected based on the paragraph weight ratio of the time node where the character is updated;

[0011] The corrected target candidate recognition sequence is output as the final recognition result.

[0012] Optionally, obtaining the target candidate recognition sequence based on the sequence score and the time nodes for character updates in the candidate recognition sequence includes:

[0013] According to the preset attention mechanism decoding algorithm, multiple candidate recognition sequences in the recognition sequence set are re-scored to obtain the scoring results;

[0014] Based on the scoring results, the candidate recognition sequence with the highest score in the collection of recognition sequences is selected as the target candidate recognition sequence.

[0015] Optionally, the step of re-scoring multiple candidate recognition sequences in the recognition sequence set according to a preset attention mechanism decoding algorithm to obtain a scoring result includes:

[0016] Based on the preset attention mechanism decoding algorithm, the encoded feature sequence, and the candidate recognition sequence, the attention score of the candidate recognition sequence is obtained;

[0017] By combining the attention score and the sequence score, a re-scoring result is obtained for multiple candidate recognition sequences in the recognition sequence set.

[0018] Optionally, obtaining the attention score of the candidate recognition sequence based on a preset attention mechanism decoding algorithm, encoded feature sequence, and candidate recognition sequence includes:

[0019] The character scores before the current time node and the character position at the current time node in the encoded feature sequence and the candidate recognition sequence are used as inputs, and the character score at the current time node position is obtained through a preset attention mechanism decoding algorithm.

[0020] The character scores at all time points in the candidate recognition sequence are summed to obtain the attention score of the candidate recognition sequence.

[0021] Optionally, when obtaining the target candidate recognition sequence based on the sequence score and the time node of character update in the candidate recognition sequence, the character score at the current time node position will be obtained through a preset attention mechanism decoding algorithm; the correction of characters in the target candidate recognition sequence by using the paragraph weight ratio of the time node where the character update occurs includes:

[0022] When obtaining the character score at the current time node position through the preset attention mechanism decoding algorithm, if the character score is less than the first score threshold, the position where the character is located is marked as pending confirmation.

[0023] For characters in a pending confirmation state, obtain the segment weight ratio of that character in the candidate recognition sequence;

[0024] If the paragraph weight ratio is less than the first weight threshold, the character at the current position is corrected.

[0025] Optionally, the paragraph weight ratio is used to characterize the relevance of a character to the content of the preceding and following paragraphs in the candidate recognition sequence;

[0026] The paragraph weight ratio is specifically defined as the ratio of the sum of the weights of the preceding context before the character at the current position in the candidate recognition sequence to the sum of the weights of the following context after the character at the current position.

[0027] The larger the value of the paragraph weight ratio, the more the character at the current position depends on the preceding text when the character is predicted by the preset attention mechanism decoding algorithm.

[0028] The smaller the value of the paragraph weight ratio, the more the character at the current position depends on the following content when the character is predicted by the preset attention mechanism decoding algorithm.

[0029] Optionally, the correction of the character at the current position includes:

[0030] The character with the highest score obtained from the character score obtained through the preset attention mechanism decoding algorithm is used to replace the character at the current position, thereby correcting the characters in the target candidate recognition sequence.

[0031] Optionally, for characters at positions in the pending confirmation state, calculating the segment weight ratio of the target character in the candidate recognition sequence includes:

[0032] The attention decoder is used to obtain the encoded feature vector of the target character's location;

[0033] Obtain the time node at which the target character is updated;

[0034] Based on the time node and the encoded feature vector, the paragraph weight ratio of the target character in the candidate recognition sequence is obtained.

[0035] Optionally, obtaining a set of recognition sequences including multiple candidate recognition sequences, sequence scores of the candidate recognition sequences, and multiple time series corresponding to the multiple candidate recognition sequences based on the encoded feature sequence includes:

[0036] The encoded feature sequence is decoded using a neural network time sequence decoder and a prefix beam search algorithm to obtain multiple candidate recognition sequences and sequence scores corresponding to the candidate recognition sequences, as well as multiple time series corresponding to the multiple candidate recognition sequences.

[0037] Secondly, embodiments of the present invention provide a speech recognition content correction device, the device comprising:

[0038] An audio data acquisition module is used to acquire audio data of a unit time length and the encoded feature sequence corresponding to the audio data;

[0039] The feature sequence decoding module is used to obtain, based on the encoded feature sequence, a set of recognition sequences including multiple candidate recognition sequences, sequence scores of the candidate recognition sequences, and multiple time series corresponding to the multiple candidate recognition sequences, wherein the time series are used to record the time nodes of character updates in the candidate sequences;

[0040] The target candidate recognition sequence generation module is used to obtain the target candidate recognition sequence based on the sequence score and the time node of character update in the candidate recognition sequence when receiving complete audio data.

[0041] The sequence character correction module is used to correct the characters in the target candidate recognition sequence according to the paragraph weight ratio of the time node where the character is updated;

[0042] The recognition result output module is used to output the corrected target candidate recognition sequence as the final recognition result.

[0043] Optionally, the feature sequence decoding module 302 further includes:

[0044] The feature sequence decoding submodule is used to decode the encoded feature sequence through a neural network time sequence decoder and a prefix beam search algorithm to obtain multiple candidate recognition sequences and sequence scores corresponding to the candidate recognition sequences, as well as multiple time series corresponding to the multiple candidate recognition sequences.

[0045] Optionally, the target candidate identification sequence generation module 303 further includes:

[0046] The candidate recognition sequence scoring module is used to re-score multiple candidate recognition sequences in the recognition sequence set according to a preset attention mechanism decoding algorithm to obtain a scoring result;

[0047] The target candidate recognition sequence generation submodule is used to select the candidate recognition sequence with the highest score in the recognition sequence set as the target candidate recognition sequence based on the scoring results.

[0048] Optionally, the candidate recognition sequence breaking module includes:

[0049] The attention score acquisition unit is used to obtain the attention score of the candidate recognition sequence based on a preset attention mechanism decoding algorithm, encoded feature sequence and candidate recognition sequence;

[0050] The scoring result acquisition unit is used to obtain a scoring result for re-scoring multiple candidate recognition sequences in the recognition sequence set by using the attention score and the sequence score.

[0051] Optionally, the attention score acquisition unit includes:

[0052] The character score acquisition subunit is used to take the character scores before the current time node and the character position at the current time node in the encoded feature sequence and the candidate recognition sequence as input, and obtain the character score at the current time node position through a preset attention mechanism decoding algorithm.

[0053] The character score accumulation subunit is used to accumulate the character scores at all time points in the candidate recognition sequence to obtain the attention score of the candidate recognition sequence.

[0054] Optionally, the sequence character correction module further includes:

[0055] The character status marking submodule is used to mark the position of the character as pending confirmation if the character score is less than a first score threshold when the character score at the current time node position is obtained through a preset attention mechanism decoding algorithm.

[0056] The paragraph weight ratio acquisition submodule is used to obtain the paragraph weight ratio of a character in the candidate recognition sequence for a character in a position that is in the pending confirmation state.

[0057] The character correction execution submodule is used to correct the character at the current position when the paragraph weight ratio is less than the first weight threshold.

[0058] Optionally, the paragraph weight ratio acquisition submodule further includes:

[0059] The feature encoding vector acquisition unit is used to acquire the encoded feature vector corresponding to the position of the target character through the attention decoder;

[0060] The update time node acquisition unit is used to acquire the time node at which the target character is updated;

[0061] The paragraph weight ratio acquisition unit is used to obtain the paragraph weight ratio of the target character in the candidate recognition sequence based on the time node and the encoded feature vector.

[0062] Optionally, the character correction execution submodule further includes:

[0063] The character correction execution unit is used to replace the character at the current position with the character that has the highest score obtained from the character score obtained through the preset attention mechanism decoding algorithm, so as to complete the correction of the characters in the target candidate recognition sequence.

[0064] Thirdly, embodiments of the present invention provide an electronic device, including: a processor;

[0065] Memory used to store the processor's executable instructions;

[0066] The processor is configured to execute the instructions to implement the method.

[0067] Fourthly, embodiments of the present invention provide a computer-readable storage medium that, when instructions in the computer-readable storage medium are executed by a processor of an electronic device, enables the electronic device to perform the method.

[0068] In this embodiment of the invention, audio data of a unit time length and the corresponding encoded feature sequence are first continuously acquired; based on the encoded feature sequence, a set of recognition sequences including multiple candidate recognition sequences, the sequence scores of the candidate recognition sequences, and multiple time sequences corresponding to the multiple candidate recognition sequences are obtained; when complete audio data is received, the candidate recognition sequences are re-scored to generate target candidate recognition sequences; finally, based on the paragraph weight ratio of the time node where the character is updated, the characters in the target candidate recognition sequence are corrected, and the corrected target candidate recognition sequence is output as the final recognition result; by introducing paragraph weight analysis to analyze the content relationship between individual characters in the sequence and the context, and correcting individual characters one by one, the final output recognized text content is more accurate.

[0069] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0070] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0071] Figure 1A simplified implementation flowchart of a speech recognition content correction method provided in an embodiment of the present invention;

[0072] Figure 2 A logic diagram of a simple streaming speech recognition method provided in an embodiment of the present invention;

[0073] Figure 3 A logical principle diagram of a rich streaming speech recognition method provided in an embodiment of the present invention;

[0074] Figure 4 A character position relationship diagram for a time series provided in an embodiment of the present invention;

[0075] Figure 5 This invention provides a scoring step relationship diagram based on attention mechanism decoding.

[0076] Figure 6 A complete flowchart of the steps of a speech recognition content correction method provided in this embodiment of the invention;

[0077] Figure 7 This is a schematic diagram of the functional modules of a speech recognition content correction device provided in an embodiment of the present invention;

[0078] Figure 8 This is a functional component relationship diagram of an electronic device provided in an embodiment of the present invention;

[0079] Figure 9 This is a functional component relationship diagram of another electronic device provided in an embodiment of the present invention. Detailed Implementation

[0080] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0081] Reference Figure 1 The diagram illustrates a simplified implementation flow of a speech recognition content correction method provided by an embodiment of the present invention; as shown below. Figure 1 As shown, the method includes:

[0082] Step 101: Obtain audio data per unit time length and the encoded feature sequence corresponding to the audio data.

[0083] This invention provides a speech recognition content correction method applicable to audio processing of six-sentence speech. Streaming speech recognition refers to the ability to return recognition results in real time during audio stream processing, without waiting for the entire sentence to be processed before returning the results.

[0084] For the recognition process of streaming speech, a complete audio stream can be divided into multiple time blocks. During the recognition process, the speech content within a unit time length can be continuously acquired and recognized.

[0085] Specifically, after obtaining the audio data, feature extraction is the first step. Audio features (e.g., frequency domain features) are extracted, resulting in a W×P two-dimensional feature matrix, where W is the length of the audio feature sequence and P is the dimension of the feature vector. The feature encoding model can be a series of encoding models incorporating an attention mechanism. After feature encoding, the two-dimensional feature matrix yields a W×P two-dimensional encoded feature matrix, where M is the length of the encoded feature sequence and Q is the dimension of the encoded feature vector, ultimately obtaining the encoded feature sequence of the audio.

[0086] Step 102: Based on the encoded feature sequence, obtain a set of recognition sequences including multiple candidate recognition sequences, sequence scores of the candidate recognition sequences, and multiple time series corresponding to the multiple candidate recognition sequences. The time series are used to record the time nodes of character updates in the candidate sequences.

[0087] Once the encoded feature sequence of the audio is obtained, the audio data can be quantitatively analyzed. (Refer to...) Figure 2 The diagram illustrates the logic principle of a simple streaming speech recognition method provided by an embodiment of the present invention.

[0088] like Figure 2 As shown in the diagram, following the direction of the arrows, feature extraction and encoding are first performed on the n acquired audio data units of duration: t1, t2, t3, ..., tn, to obtain the encoded speech feature sequence: e1, e2, e3, ..., en. The obtained encoded feature vectors are then decoded using a Connectionist Temporal Classification (CTC) decoder, and a prefix beam search algorithm is used for planning and calculation to generate K candidate recognition sequences and their scores. The dashed box represents a sequence set containing multiple candidate recognition sequences.

[0089] Among them, the prefix beam search algorithm is a dynamic programming algorithm that can greatly reduce the search space, increase search efficiency, and its error is within an acceptable range. It is often used in applications such as CTC decoding.

[0090] After generating candidate recognition sequences and their scores, the sequence with the highest score is selected as the recognition result. Once the system has acquired the speech data of a complete sentence, an attention mechanism decoder is used to re-evaluate the recognition results using the generated candidate recognition sequences and encoded feature sequences, and the sequence with the highest score is selected as the final recognition result.

[0091] Figure 2 The aforementioned speech recognition scheme is an existing technical method. The present invention provides a speech recognition content correction method that, based on the existing technology, adds correction and improvement for individual characters in the candidate recognition sequence.

[0092] Reference Figure 3 The diagram illustrates the logical principle of a rich streaming speech recognition method provided by an embodiment of the present invention; as shown below. Figure 3 As shown, the speech recognition content correction method provided by this embodiment of the invention adds time sequence recording for characters based on the existing solution.

[0093] Based on the encoded feature sequence, a set of recognition sequences including multiple candidate recognition sequences, sequence scores of the candidate recognition sequences, and multiple time series corresponding to the multiple candidate recognition sequences are obtained. The time series are used to record the time nodes of character updates in the candidate sequences.

[0094] For details, please refer to Figure 4 This illustrates a character position relationship diagram for a time series provided by an embodiment of the present invention; for example... Figure 4 As shown, in the current candidate recognition sequence, the first character C1 is updated in the first time block (or time step), while the lower 2, 3, and 4 characters C2, C3, and C4 are updated in the second time step.

[0095] By marking the update time of individual characters in the candidate recognition sequence, it is helpful to strengthen the predictive analysis of individual characters when generating the candidate recognition sequence, making the connection between the preceding and following text in the generated recognition sequence more natural and fluent.

[0096] Step 103: Upon receiving complete audio data, obtain the target candidate recognition sequence based on the sequence score and the time nodes of character updates in the candidate recognition sequence.

[0097] Due to the technical characteristics of streaming speech recognition, it can return recognition results without waiting to process a complete sentence of audio. Based on the above, steps 101 to 102 are based on the received audio data across multiple time steps for recognition and analysis. Therefore, after the system receives a complete sentence, further content correction is needed on the generated recognition results.

[0098] The attention mechanism decoder is used to re-score the obtained candidate recognition sequences, and the target candidate recognition sequence is obtained based on the re-score results.

[0099] Specifically, refer to Figure 5 This diagram illustrates a scoring step relationship diagram based on attention mechanism decoding provided by an embodiment of the present invention.

[0100] First, for the generated candidate recognition sequence, a sentence start marker is added to the front of the sequence to inform the system of the starting position of the characters in the candidate recognition sequence. The multiple candidate recognition sequences with the added start marker are used as the input sequence of the attention mechanism decoder. The final output result is the score of multiple predicted sequences that contain sentence end markers but do not contain start markers.

[0101] It is worth noting that, in this process, for a single input candidate recognition sequence, the attention mechanism decoder will base its input on the characters at the first m-1 positions (see reference). Figure 5 Specifically, characters C1 to C m-1 To predict the character C at the m-th position. m The score is calculated as follows. The output Pm is a k-dimensional vector, where k equals the length of the system dictionary. Each element in the k-dimensional vector represents the predicted score for each symbol in the dictionary. The system dictionary is a pre-built character dictionary. By predicting and scoring the matching results of characters in the system dictionary, the most suitable character for the current position can be selected based on the score.

[0102] For example, the decoder takes the start symbol as input and outputs the score P1 for each character at position 1, thus obtaining the score of character c1 at position 1 in the candidate sequence; the decoder takes the start symbol and character c1 as input and outputs the score vector P2 at position 2, thus obtaining the score of character c2 at position 2 in the candidate sequence; and so on, finally obtaining the score of each character at each position in the entire candidate sequence. The sum of the scores of all characters is the attention score of the candidate sequence.

[0103] The attention score in each candidate recognition sequence is weighted and summed with the sequence score of the candidate recognition sequence obtained through the CTC decoder to obtain a secondary score for the candidate recognition sequence. Based on the new score, the multiple candidate recognition sequences are sorted, and the sequence with the highest score is selected as the target candidate recognition sequence.

[0104] Step 104: Correct the characters in the target candidate recognition sequence according to the paragraph weight ratio of the time node where the character is updated.

[0105] Following step 103, after obtaining the target candidate recognition sequence based on the sequence score and the time node for updating the characters in the candidate recognition sequence, it is also necessary to perform targeted correction on the characters in the target candidate recognition sequence.

[0106] Specifically, when predicting a character at a certain position in a candidate sequence using the attention mechanism decoder, the encoded feature vector associated with that position is obtained. That is, the decoder attention layer calculates the encoded feature weight vector W = (w1, w2, ..., w...). n ), query the time step (assumed to be t6) recorded for the character at this position during the CTC decoding stage, and calculate the context relation weight ratio of the character at this position in the decoder attention layer as (w1+w2+...+w5):(w7+w8+...+w n If the attention mechanism decoder has a total of L attention layers, then the context relation weights of all layers are accumulated and then divided to obtain the final context relation weight ratio w for the character at that position in the sequence. left :w right The larger the weight ratio, the more the prediction of the character at that position depends on the preceding text, and vice versa.

[0107] By calculating the weight ratio of the current character in the candidate recognition sequence to its context, we can determine which part of the text the attention mechanism decoder tends to rely on as input reference when predicting the character at the current position. In practical applications, if the prediction of the character at this position relies more on the context, it indicates that the result of relying solely on the preceding context in the previous stage is less reasonable, and thus, correction of that character is necessary.

[0108] Step 105: Output the corrected target candidate recognition sequence as the final recognition result.

[0109] According to the method described in step 104, the characters at the positions that need to be corrected in the target candidate recognition sequence are judged and corrected, and finally the corrected target candidate recognition sequence is used as the final recognition result for output.

[0110] In summary, the speech recognition content correction method provided by this invention first continuously acquires audio data of a unit time length and the corresponding encoded feature sequence; based on the encoded feature sequence, it obtains a set of recognition sequences including multiple candidate recognition sequences, the sequence scores of the candidate recognition sequences, and multiple time sequences corresponding to the multiple candidate recognition sequences; upon receiving complete audio data, it re-scores the candidate recognition sequences to generate a target candidate recognition sequence; finally, it corrects the characters in the target candidate recognition sequence based on the paragraph weight ratio of the time node where the character is updated, and outputs the corrected target candidate recognition sequence as the final recognition result; by introducing paragraph weight analysis to analyze the content relationship between individual characters in the sequence and the context, and correcting individual characters one by one, the final output recognized text content is more accurate.

[0111] Reference Figure 6 This diagram illustrates a complete implementation flowchart of a speech recognition content correction method provided by an embodiment of the present invention; as follows: Figure 6 As shown, the method steps include:

[0112] Step 201: Obtain audio data per unit time length and the encoded feature sequence corresponding to the audio data.

[0113] This step can be referred to in detail in step 101 above, and will not be repeated here in the embodiments of the present invention.

[0114] Step 202: Decode the encoded feature sequence using a neural network temporal decoder and a prefix beam search algorithm to obtain multiple candidate recognition sequences and sequence scores corresponding to the candidate recognition sequences, as well as multiple time series corresponding to the multiple candidate recognition sequences. The time series are used to record the time nodes of character updates in the candidate sequences.

[0115] The specific steps can be referred to in step 102 above. This embodiment of the invention will not be repeated here.

[0116] Step 203: Upon receiving complete audio data, based on the sequence score and the time nodes of character updates in the candidate recognition sequences, a preset attention mechanism decoding algorithm is used to re-score multiple candidate recognition sequences in the recognition sequence set, and the candidate recognition sequence with the highest score in the recognition sequence set is taken as the target candidate recognition sequence.

[0117] This step can be referred to in detail in step 103 above, and will not be repeated here in the embodiments of the present invention.

[0118] Optionally, step 203 may specifically include:

[0119] Sub-step 2031: Obtain the attention score of the candidate recognition sequence based on the preset attention mechanism decoding algorithm, the encoded feature sequence and the candidate recognition sequence.

[0120] Reference Figure 5 Upon receiving complete audio data, the attention mechanism decoder is invoked to re-score the generated candidate recognition sequences.

[0121] In this process, a start marker is added to the front of each candidate recognition sequence. The multiple candidate recognition sequences with the added start marker are used as the input sequence of the attention mechanism decoder. The decoder output is the score of the predicted sequence of multiple candidate recognition sequences that contain the sentence termination symbol but do not contain the start marker.

[0122] Optionally, sub-step 2031 may specifically include:

[0123] Sub-step 20311: Using the encoded feature sequence and the candidate recognition sequence, the character scores before the current time node and the character position at the current time node are taken as input, and the character score at the current time node position is obtained through a preset attention mechanism decoding algorithm.

[0124] Specifically, such as Figure 5 As shown, in a candidate sequence, the decoder predicts the score of the character at the m-th position based on the input characters at the first m-1 positions. Note that the output Pm is a k-dimensional vector, where k equals the length of the system dictionary. Each element in the k-dimensional vector represents the predicted score of each symbol in the dictionary. For example, if the decoder takes the start symbol as input, it outputs the scores P1 for each character at the first position, thus obtaining the score of the character c1 at the first position in the candidate sequence. If the decoder takes the start symbol and the character C1 as input, it outputs the score vector P2 for the second position, thus obtaining the score of the character C2 at the second position in the candidate sequence.

[0125] Sub-step 20312: Accumulate the character scores of all time points in the candidate recognition sequence to obtain the attention score of the candidate recognition sequence.

[0126] Following sub-step 20311, the predicted sequence score of each character in the candidate recognition sequence is obtained. By analogy, the score of each character in the entire candidate sequence is finally obtained. The sum of the scores of all characters is the attention score of the candidate sequence.

[0127] Sub-step 2032: Obtain a re-scoring result for multiple candidate recognition sequences in the recognition sequence set by using the attention score and the sequence score.

[0128] The attention score of each candidate sequence and the sequence score generated by CTC decoding are weighted and summed to obtain the final re-scoring score. The multiple candidate recognition sequences are reordered according to the re-scoring score, and the candidate sequence with the highest re-scoring score is output as the final recognition result.

[0129] Step 204: When obtaining the character score at the current time node position through the preset attention mechanism decoding algorithm, if the character score is less than the first score threshold, the position where the character is located is marked as pending confirmation.

[0130] In step 203, during the process of scoring the characters in the candidate recognition sequence through the preset attention mechanism decoder, if the score obtained by the character at a certain position in the sequence in the attention mechanism decoder is less than the preset first score threshold threshold1, the system will then mark that position as pending confirmation.

[0131] If a character scores less than a preset first score threshold in the attention mechanism decoder, it means that the result obtained by the character at that position through the attention mechanism decoder prediction did not meet the expected effect (e.g., the generated character is not closely linked to the preceding paragraph, or the wording is inappropriate, etc.), and further observation and correction are required.

[0132] Step 205: For characters in the pending confirmation state, obtain the paragraph weight ratio of the character in the candidate recognition sequence; if the paragraph weight ratio is less than the first weight threshold, then correct the character at the current position.

[0133] For each position in the sequence marked as a state to be confirmed, calculate its contextual weight ratio w. left :w right When w left :w right If the value is less than the preset first weight threshold (threshold2), the character at that position needs to be corrected.

[0134] Optionally, the paragraph weight ratio is specifically the ratio of the sum of the weights of the preceding context before the character at the current position in the candidate recognition sequence to the sum of the weights of the following context after the character at the current position.

[0135] It is worth noting that if the attention mechanism decoder has a total of L attention layers, then the context relation weights of all layers are accumulated and then divided to obtain the final context relation weight ratio w for the character at that position in the sequence. left :w rightFor example: Suppose the two-dimensional encoded feature matrix obtained at one time step of the system is M×Q, where M is the sequence length of the encoded features and Q is the dimension of the encoded feature vector. Suppose that a total of n time steps are accumulated in the decoding stage of the attention mechanism, then the concatenated two-dimensional encoded feature matrix is ​​(n×M)×Q. Therefore, the decoder attention layer will calculate n×M encoded feature weight values, forming a weight vector of length (n×M). The system adds up the weights of the encoded features belonging to the same time step, that is, every M weights are accumulated into a weight representing that time step. In this way, the encoded feature weight vector W = (w1, w2, w3, ..., wn) in the text is obtained. Here, w1, w2...wn are all specific numerical values, not vectors. Therefore, the context relation weight ratio is to add up the weight values ​​belonging to the preceding context, add up the weight values ​​belonging to the following context, and then divide them.

[0136] The larger the value of the paragraph weight ratio, the more the character at the current position depends on the preceding text when the character is predicted by the preset attention mechanism decoding algorithm.

[0137] The smaller the value of the paragraph weight ratio, the more the character at the current position depends on the following content when the character is predicted by the preset attention mechanism decoding algorithm.

[0138] Optionally, step 205 may specifically include:

[0139] Sub-step 2051: Obtain the encoded feature vector corresponding to the position of the target character through the attention decoder.

[0140] By referring to the corresponding position in the encoded feature sequence based on the position of the target character in the sequence, the encoded feature vector corresponding to the position of the target character can be obtained.

[0141] Sub-step 2052: Obtain the time node where the target character is updated.

[0142] Based on the time series generated during the CTC decoding process to record the time nodes of character updates in the candidate sequence, the time node where the target character update occurs is obtained.

[0143] Specifically, as referred to Figure 4 In the current candidate recognition sequence, the first character C1 is updated in the first time block (or time step), and the lower 2, 3, 4 characters C2, C3, and C4 are updated in the second time step.

[0144] Sub-step 2053: Based on the time node and the encoded feature vector, obtain the paragraph weight ratio of the target character in the candidate recognition sequence.

[0145] Specifically, when the attention mechanism decoder predicts a character at a certain position in the candidate sequence, it obtains the encoded feature vector associated with that position. That is, the decoder attention layer calculates the encoded feature weight vector W = (w1, w2, ..., w...). n ), query the time node (time step) information (let's assume it's t6) recorded in the time series generated during the CTC decoding process for the character at that position, and calculate the context relation weight ratio of the character at that position in the decoder's attention layer as (w1+w2+...+w5):(w7+w8+...+w n ).

[0146] Step 206: Replace the character at the current position with the character that has the highest score obtained from the character score obtained through the preset attention mechanism decoding algorithm, so as to complete the correction of the characters in the target candidate recognition sequence.

[0147] During the character score acquisition process using the pre-defined attention mechanism decoding algorithm, the decoder outputs Pm as a k-dimensional vector, where k equals the length of the system dictionary. Each element in the k-dimensional vector represents the predicted score for each symbol in the dictionary. The system dictionary is a pre-defined character dictionary within the system. By predicting and scoring the matching results of characters in the system dictionary, the most suitable character for the current position can be selected based on the score.

[0148] By selecting the character with the highest score among all character results in the system dictionary during the decoding process, and replacing the character at the current position, the character in the target candidate recognition sequence is corrected and replaced.

[0149] After replacement, the character at the current position can be identified as the best result character in the candidate recognition sequence, thereby achieving precise word-by-word correction of the speech recognition result and greatly improving the accuracy of the speech recognition result.

[0150] Step 207: Output the corrected target candidate recognition sequence as the final recognition result.

[0151] This step can be referred to in detail in step 104 above, and will not be repeated here in the embodiments of the present invention.

[0152] In summary, the speech recognition content correction method provided by this invention first continuously acquires audio data of a unit time length and the corresponding encoded feature sequence; based on the encoded feature sequence, it obtains a set of recognition sequences including multiple candidate recognition sequences, the sequence scores of the candidate recognition sequences, and multiple time sequences corresponding to the multiple candidate recognition sequences; upon receiving complete audio data, it re-scores the candidate recognition sequences to generate a target candidate recognition sequence; finally, it corrects the characters in the target candidate recognition sequence based on the paragraph weight ratio of the time node where the character is updated, and outputs the corrected target candidate recognition sequence as the final recognition result; by introducing paragraph weight analysis to analyze the content relationship between individual characters in the sequence and the context, and correcting individual characters one by one, the final output recognized text content is more accurate.

[0153] Reference Figure 7 This diagram illustrates the functional module composition of a speech recognition content correction device 300 provided in an embodiment of the present invention; as shown below. Figure 7 As shown, the device includes:

[0154] The audio data acquisition module 301 is used to acquire audio data of a unit time length and the encoded feature sequence corresponding to the audio data;

[0155] The feature sequence decoding module 302 is used to obtain, based on the encoded feature sequence, a set of recognition sequences including multiple candidate recognition sequences, sequence scores of the candidate recognition sequences, and multiple time series corresponding to the multiple candidate recognition sequences, wherein the time series is used to record the time nodes of character updates in the candidate sequences;

[0156] The target candidate recognition sequence generation module 303 is used to obtain a target candidate recognition sequence based on the sequence score and the time node of character update in the candidate recognition sequence when receiving complete audio data.

[0157] The sequence character correction module 304 is used to correct the characters in the target candidate recognition sequence according to the paragraph weight ratio of the time node where the character is updated;

[0158] The recognition result output module 305 is used to output the corrected target candidate recognition sequence as the final recognition result.

[0159] Optionally, the feature sequence decoding module 302 further includes:

[0160] The feature sequence decoding submodule is used to decode the encoded feature sequence through a neural network time sequence decoder and a prefix beam search algorithm to obtain multiple candidate recognition sequences and sequence scores corresponding to the candidate recognition sequences, as well as multiple time series corresponding to the multiple candidate recognition sequences.

[0161] Optionally, the target candidate identification sequence generation module 303 further includes:

[0162] The candidate recognition sequence scoring module is used to re-score multiple candidate recognition sequences in the recognition sequence set according to a preset attention mechanism decoding algorithm to obtain a scoring result;

[0163] The target candidate recognition sequence generation submodule is used to select the candidate recognition sequence with the highest score in the recognition sequence set as the target candidate recognition sequence based on the scoring results.

[0164] Optionally, the candidate recognition sequence breaking module includes:

[0165] The attention score acquisition unit is used to obtain the attention score of the candidate recognition sequence based on a preset attention mechanism decoding algorithm, encoded feature sequence and candidate recognition sequence;

[0166] The scoring result acquisition unit is used to obtain a scoring result for re-scoring multiple candidate recognition sequences in the recognition sequence set by using the attention score and the sequence score.

[0167] Optionally, the attention score acquisition unit includes:

[0168] The character score acquisition subunit is used to take the character scores before the current time node and the character position at the current time node in the encoded feature sequence and the candidate recognition sequence as input, and obtain the character score at the current time node position through a preset attention mechanism decoding algorithm.

[0169] The character score accumulation subunit is used to accumulate the character scores at all time points in the candidate recognition sequence to obtain the attention score of the candidate recognition sequence.

[0170] Optionally, the sequence character correction module 304 further includes:

[0171] The character status marking submodule is used to mark the position of the character as pending confirmation if the character score is less than a first score threshold when the character score at the current time node position is obtained through a preset attention mechanism decoding algorithm.

[0172] The paragraph weight ratio acquisition submodule is used to obtain the paragraph weight ratio of a character in the candidate recognition sequence for a character in a position that is in the pending confirmation state.

[0173] Optionally, the paragraph weight ratio acquisition submodule further includes:

[0174] The feature encoding vector acquisition unit is used to acquire the encoded feature vector corresponding to the position of the target character through the attention decoder;

[0175] The update time node acquisition unit is used to acquire the time node at which the target character is updated;

[0176] The paragraph weight ratio acquisition unit is used to obtain the paragraph weight ratio of the target character in the candidate recognition sequence based on the time node and the encoded feature vector.

[0177] The character correction execution submodule is used to correct the character at the current position when the paragraph weight ratio is less than the first weight threshold.

[0178] Optionally, the character correction execution submodule further includes:

[0179] The character correction execution unit is used to replace the character at the current position with the character that has the highest score obtained from the character score obtained through the preset attention mechanism decoding algorithm, so as to complete the correction of the characters in the target candidate recognition sequence.

[0180] Figure 8 This is a block diagram illustrating an electronic device 600 according to an exemplary embodiment. For example, the electronic device 600 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0181] Reference Figure 8 The electronic device 600 may include one or more of the following components: a processing component 602, a memory 604, a power supply component 606, a multimedia component 608, an audio component 610, an input / output (I / O) interface 612, a sensor component 614, and a communication component 616.

[0182] Processing component 602 typically controls the overall operation of electronic device 600, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 602 may include one or more processors 620 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 602 may include one or more modules to facilitate interaction between processing component 602 and other components. For example, processing component 602 may include a multimedia module to facilitate interaction between multimedia component 608 and processing component 602.

[0183] Memory 604 is used to store various types of data to support the operation of electronic device 600. Examples of such data include instructions for any application or method operating on electronic device 600, contact data, phonebook data, messages, pictures, multimedia, etc. Memory 604 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0184] Power supply component 606 provides power to various components of electronic device 600. Power supply component 606 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 600.

[0185] Multimedia component 608 includes a screen that provides an output interface between the electronic device 600 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 608 includes a front-facing camera and / or a rear-facing camera. When the electronic device 600 is in an operating mode, such as a shooting mode or a multimedia mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0186] Audio component 610 is used to output and / or input audio signals. For example, audio component 610 includes a microphone (MIC) used to receive external audio signals when electronic device 600 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 604 or transmitted via communication component 616. In some embodiments, audio component 610 also includes a speaker for outputting audio signals.

[0187] I / O interface 612 provides an interface between processing component 602 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0188] Sensor assembly 614 includes one or more sensors for providing state assessments of various aspects of electronic device 600. For example, sensor assembly 614 can detect the on / off state of electronic device 600, the relative positioning of components such as the display and keypad of electronic device 600, changes in position of electronic device 600 or a component of electronic device 600, the presence or absence of user contact with electronic device 600, orientation or acceleration / deceleration of electronic device 600, and temperature changes of electronic device 600. Sensor assembly 614 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 614 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 614 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0189] Communication component 616 facilitates wired or wireless communication between electronic device 600 and other devices. Electronic device 600 can access wireless networks based on communication standards, such as WiFi, carrier networks (such as 2G, 3G, 4G, or 5G), or combinations thereof. In one exemplary embodiment, communication component 616 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 616 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0190] In an exemplary embodiment, the electronic device 600 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to implement a speech recognition content correction method provided in this embodiment of the invention.

[0191] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 604 including instructions, which can be executed by a processor 620 of an electronic device 600 to perform the above-described method. For example, the non-transitory storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0192] Figure 9 This is a block diagram illustrating an electronic device 700 according to an exemplary embodiment. For example, the electronic device 700 may be provided as a server. (Refer to...) Figure 9 The electronic device 700 includes a processing component 722, which further includes one or more processors, and memory resources represented by a memory 732 for storing instructions, such as application programs, that can be executed by the processing component 722. The application programs stored in the memory 732 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 722 is configured to execute instructions to perform a speech recognition content correction method provided in embodiments of the present invention.

[0193] Electronic device 700 may also include a power supply component 726 configured to perform power management of electronic device 700, a wired or wireless network interface 750 configured to connect electronic device 700 to a network, and an input / output (I / O) interface 758. Electronic device 700 may operate on an operating system stored in memory 732, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or similar.

[0194] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the following claims.

[0195] It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A method for correcting speech recognition content, characterized in that, The method includes: Acquire audio data per unit time length and the corresponding encoded feature sequence of the audio data; Based on the encoded feature sequence, a set of recognition sequences including multiple candidate recognition sequences, sequence scores of the candidate recognition sequences, and multiple time series corresponding to the multiple candidate recognition sequences are obtained. The time series are used to record the time nodes of character updates in the candidate sequences. Upon receiving complete audio data, the target candidate recognition sequence is obtained based on the sequence score and the time points of character updates in the candidate recognition sequence. The characters in the target candidate recognition sequence are corrected based on the paragraph weight ratio of the time node where the character is updated; The corrected target candidate recognition sequence is output as the final recognition result; When obtaining the target candidate recognition sequence based on the sequence score and the character update time point in the candidate recognition sequence, the character score at the current time point position will be obtained through a preset attention mechanism decoding algorithm; the correction of characters in the target candidate recognition sequence based on the paragraph weight ratio of the time point where the character update occurs includes: When obtaining the character score at the current time node position through the preset attention mechanism decoding algorithm, if the character score is less than the first score threshold, the position where the character is located is marked as pending confirmation. For characters in a pending confirmation state, obtain the segment weight ratio of that character in the candidate recognition sequence; If the paragraph weight ratio is less than the first weight threshold, the character at the current position is corrected.

2. The method according to claim 1, characterized in that, The step of obtaining the target candidate recognition sequence based on the sequence score and the time nodes for character updates in the candidate recognition sequence includes: According to the preset attention mechanism decoding algorithm, multiple candidate recognition sequences in the recognition sequence set are re-scored to obtain the scoring results; Based on the scoring results, the candidate recognition sequence with the highest score in the collection of recognition sequences is selected as the target candidate recognition sequence.

3. The method according to claim 2, characterized in that, The step of re-scoring multiple candidate recognition sequences in the recognition sequence set according to a preset attention mechanism decoding algorithm to obtain a scoring result includes: Based on the preset attention mechanism decoding algorithm, the encoded feature sequence, and the candidate recognition sequence, the attention score of the candidate recognition sequence is obtained; By combining the attention score and the sequence score, a re-scoring result is obtained for multiple candidate recognition sequences in the recognition sequence set.

4. The method according to claim 3, characterized in that, The step of obtaining the attention score of the candidate recognition sequence based on the preset attention mechanism decoding algorithm, the encoded feature sequence, and the candidate recognition sequence includes: The character scores before the current time node and the character position at the current time node in the encoded feature sequence and the candidate recognition sequence are used as inputs, and the character score at the current time node position is obtained through a preset attention mechanism decoding algorithm. The character scores at all time points in the candidate recognition sequence are summed to obtain the attention score of the candidate recognition sequence.

5. The method according to claim 1, characterized in that, The paragraph weight ratio is used to characterize the relevance of a character to the content of the preceding and following paragraphs in the candidate recognition sequence; The paragraph weight ratio is specifically defined as the ratio of the sum of the weights of the preceding context before the character at the current position in the candidate recognition sequence to the sum of the weights of the following context after the character at the current position. The larger the value of the paragraph weight ratio, the more the character at the current position depends on the preceding text when the character is predicted by the preset attention mechanism decoding algorithm. The smaller the value of the paragraph weight ratio, the more the character at the current position depends on the following content when the character is predicted by the preset attention mechanism decoding algorithm.

6. The method according to claim 1, characterized in that, The correction of the character at the current position includes: The character with the highest score obtained from the character score obtained through the preset attention mechanism decoding algorithm is used to replace the character at the current position, thereby correcting the characters in the target candidate recognition sequence.

7. The method according to claim 1, characterized in that, For characters at positions in the pending confirmation state, calculate the segment weight ratio of the target character in the candidate recognition sequence, including: The attention decoder is used to obtain the encoded feature vector of the target character's location; Obtain the time node at which the target character is updated; Based on the time node and the encoded feature vector, the paragraph weight ratio of the target character in the candidate recognition sequence is obtained.

8. The method according to claim 1, characterized in that, The step of obtaining a set of recognition sequences including multiple candidate recognition sequences, sequence scores of the candidate recognition sequences, and multiple time series corresponding to the multiple candidate recognition sequences based on the encoded feature sequence includes: The encoded feature sequence is decoded using a neural network time sequence decoder and a prefix beam search algorithm to obtain multiple candidate recognition sequences and sequence scores corresponding to the candidate recognition sequences, as well as multiple time series corresponding to the multiple candidate recognition sequences.

9. A speech recognition content correction device, characterized in that, The device includes: An audio data acquisition module is used to acquire audio data of a unit time length and the encoded feature sequence corresponding to the audio data; The feature sequence decoding module is used to obtain, based on the encoded feature sequence, a set of recognition sequences including multiple candidate recognition sequences, sequence scores of the candidate recognition sequences, and multiple time series corresponding to the multiple candidate recognition sequences, wherein the time series are used to record the time nodes of character updates in the candidate sequences; The target candidate recognition sequence generation module is used to obtain the target candidate recognition sequence based on the sequence score and the time node of character update in the candidate recognition sequence when receiving complete audio data. The sequence character correction module is used to correct the characters in the target candidate recognition sequence according to the paragraph weight ratio of the time node where the character is updated; The recognition result output module is used to output the corrected target candidate recognition sequence as the final recognition result; The sequence character correction module further includes: The character status marking submodule is used to mark the position of the character as pending confirmation if the character score is less than a first score threshold when the character score at the current time node position is obtained through a preset attention mechanism decoding algorithm. The paragraph weight ratio acquisition submodule is used to obtain the paragraph weight ratio of a character in the candidate recognition sequence for a character in a position that is in the pending confirmation state. Optionally, the paragraph weight ratio acquisition submodule further includes: The feature encoding vector acquisition unit is used to acquire the encoded feature vector corresponding to the position of the target character through the attention decoder; The update time node acquisition unit is used to acquire the time node at which the target character is updated; The paragraph weight ratio acquisition unit is used to obtain the paragraph weight ratio of the target character in the candidate recognition sequence based on the time node and the encoded feature vector; The character correction execution submodule is used to correct the character at the current position when the paragraph weight ratio is less than the first weight threshold.

10. The apparatus according to claim 9, characterized in that, The target candidate identification sequence generation module includes: The candidate recognition sequence scoring module is used to re-score multiple candidate recognition sequences in the recognition sequence set according to a preset attention mechanism decoding algorithm to obtain a scoring result; The target candidate recognition sequence generation submodule is used to select the candidate recognition sequence with the highest score in the recognition sequence set as the target candidate recognition sequence based on the scoring results.

11. The apparatus according to claim 10, characterized in that, The candidate recognition sequence breaking module includes: The attention score acquisition unit is used to obtain the attention score of the candidate recognition sequence based on a preset attention mechanism decoding algorithm, encoded feature sequence and candidate recognition sequence; The scoring result acquisition unit is used to obtain a scoring result for re-scoring multiple candidate recognition sequences in the recognition sequence set by using the attention score and the sequence score.

12. The apparatus according to claim 11, characterized in that, The attention score acquisition unit includes: The character score acquisition subunit is used to take the character scores before the current time node and the character position at the current time node in the encoded feature sequence and the candidate recognition sequence as input, and obtain the character score at the current time node position through a preset attention mechanism decoding algorithm. The character score accumulation subunit is used to accumulate the character scores at all time points in the candidate recognition sequence to obtain the attention score of the candidate recognition sequence.

13. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the method as described in any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is enabled to perform the method as described in any one of claims 1 to 11.

Images