Method, device and equipment for automatically labeling phoneme label on voice and medium

Abstract

The invention relates to a method, a device, equipment and a medium for automatically labeling phoneme labels for voice, and the method comprises the steps: obtaining a decoded image, recognizing a to-be-labeled audio based on a preset phoneme recognition model, and generating a recognition result, the preset phoneme recognition model is used for calculating the acoustic score of each audio frame in the to-be-labeled audio, an identification result is generated according to the acoustic score; the acoustic score is used for indicating a probability value that each audio frame is a preset phoneme frame, and the identification result is used for indicating a distribution condition of each audio frame in the to-be-labeled audio, and generating a phoneme label of the to-be-labeled audio according to the identification result and the decoded image, the phoneme label is used for indicating at least one of the start time, the end time and the duration of each phoneme in the audio to be labeled. Therefore, the recognition result of the preset phoneme recognition model is corrected based on the constraint condition in the decoded image, the phoneme tag of the audio is generated, and the accuracy of the phoneme tag corresponding to the audio data is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of speech recognition, and in particular to a method, device, equipment and medium for automatically labeling phoneme tags for speech. Background Art

[0002] Voice wake-up refers to the real-time detection of segments containing specific keywords in a continuous speech stream. In related technologies, an acoustic model is trained using language recognition data, speech is recognized based on the acoustic model, and the recognized speech is aligned and labeled using forced alignment. Summary of the invention

[0003] In order to overcome the problems existing in the related art, the present disclosure provides a method, device, equipment and medium for automatically labeling phoneme tags for speech.

[0004] According to a first aspect of an embodiment of the present disclosure, a method for automatically annotating a speech with a phoneme tag is provided, comprising:

[0005] Obtaining a decoding graph, wherein the decoding graph is generated based on a phoneme annotation correction rule corresponding to the audio to be annotated, wherein the phoneme annotation correction rule includes a constraint condition for the duration of each phoneme corresponding to the audio to be annotated or a constraint condition for the duration of each type of phoneme;

[0006] The audio to be labeled is recognized based on a preset phoneme recognition model to generate a recognition result, wherein the preset phoneme recognition model is used to calculate the acoustic score of each audio frame in the audio to be labeled, and the recognition result is generated according to the acoustic score; the acoustic score is used to indicate the probability value of each audio frame being a preset phoneme frame, and the recognition result is used to indicate the distribution of each audio frame in the audio to be labeled;

[0007] Generate a phoneme tag of the audio to be annotated according to the recognition result and the decoding graph, wherein the phoneme tag is used to indicate at least one of a start time, an end time, and a duration of each phoneme in the audio to be annotated.

[0008] Optionally, the decoding graph includes multiple paths, each path includes multiple nodes, and adjacent nodes include node connection paths and decoding rules. The node connection path indicates the path direction through a direction arrow, and the direction arrow points from the upstream node to the downstream node. The decoding rules include selection judgment conditions for each phoneme path and weight values ​​of the selection judgment conditions.

[0009] Optionally, the constraint conditions for the duration of each phoneme or the constraint conditions for the duration of each type of phoneme include: the duration of the phoneme is not longer than a first predetermined duration and / or the duration of the phoneme is not shorter than a second predetermined duration.

[0010] Optionally, the various types of phonemes include vowel phonemes and consonant phonemes;

[0011] The constraints on the duration of each type of phoneme include: the duration of the vowel phoneme is not higher than the third predetermined duration or not lower than the fourth predetermined duration, and / or the duration of the consonant phoneme is not higher than the fifth predetermined duration or not lower than the sixth predetermined duration.

[0012] Optionally, a preset sample set is obtained, the preset sample set includes a plurality of samples, each sample includes: a preset voice audio and a corresponding first label, the first label is used to indicate the duration of each phoneme in the preset voice audio;

[0013] Obtaining the statistical result of the duration of each phoneme corresponding to the preset voice audio in the preset sample set;

[0014] Determining the phoneme annotation correction rule according to the statistical results;

[0015] The decoding graph is generated according to the phoneme annotation correction rule.

[0016] Optionally, the identifying the audio to be annotated based on a preset phoneme recognition model to generate a recognition result includes:

[0017] Dividing the audio to be labeled into frames to obtain multiple audio frames;

[0018] Extracting acoustic features from the multiple audio frames to obtain multiple acoustic features corresponding to the audio frames;

[0019] The plurality of acoustic features are input into the preset phoneme recognition model to generate the recognition result.

[0020] Optionally, generating a phoneme label of the audio to be annotated according to the recognition result and the decoding graph includes:

[0021] Based on the decoding graph, the distribution in the recognition result is corrected by a phoneme annotation alignment module to generate a target distribution of each phoneme in the audio to be annotated;

[0022] The phoneme labels are generated according to the target distribution.

[0023] Optionally, generating the phoneme label according to the target distribution includes:

[0024] The target distribution is identified by an image acceleration function to generate the phoneme label.

[0025] Optionally, the statistical result includes audio length information, audio length ratio information, audio distribution position information and audio position ratio information of each phoneme, and the method further includes:

[0026] Weighting the audio length information according to the audio length ratio information to generate a plurality of target length information corresponding to each phoneme;

[0027] Weighting the audio distribution position information according to the audio position proportion information to generate a plurality of alignment positions corresponding to each phoneme frame one by one;

[0028] The phoneme tagging correction rule is determined according to the multiple target length information and the multiple alignment positions.

[0029] According to a second aspect of an embodiment of the present disclosure, there is provided a device for automatically labeling phoneme tags on speech, the device comprising:

[0030] an acquisition module configured to acquire a decoding graph of the audio to be annotated, wherein the decoding graph is generated based on a phoneme annotation correction rule corresponding to the audio to be annotated, wherein the phoneme annotation correction rule includes a constraint condition for the duration of each phoneme corresponding to the audio to be annotated or a constraint condition for the duration of each type of phoneme;

[0031] A first generating module is configured to recognize the audio to be labeled based on a preset phoneme recognition model and generate a recognition result, wherein the preset phoneme recognition model is used to calculate the acoustic score of each audio frame in the audio to be labeled, and generate the recognition result according to the acoustic score; the acoustic score is used to indicate the probability value of each audio frame being a preset phoneme frame, and the recognition result is used to indicate the distribution of each audio frame in the audio to be labeled;

[0032] The second generation module is configured to generate a phoneme tag of the audio to be annotated according to the recognition result and the decoding graph, wherein the phoneme tag is used to indicate at least one of the start time, end time and duration of each phoneme in the audio to be annotated.

[0033] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0034] a memory having a computer program stored thereon;

[0035] A processor is used to execute the computer program in the memory to implement the steps of any one of the methods described in the first aspect of the present disclosure.

[0036] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of any one of the methods described in the first aspect of the present disclosure are implemented.

[0037] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0038] Through the above method, a decoding graph is obtained, the decoding graph is generated based on the phoneme annotation correction rule corresponding to the audio to be annotated, the phoneme annotation correction rule includes the constraint condition of the duration of each phoneme corresponding to the audio to be annotated or the constraint condition of the duration of each type of phoneme, the audio to be annotated is recognized based on the preset phoneme recognition model, and a recognition result is generated, wherein the preset phoneme recognition model is used to calculate the acoustic score of each audio frame in the audio to be annotated, and the recognition result is generated according to the acoustic score, the acoustic score is used to indicate the probability value of each audio frame being a preset phoneme frame, and the recognition result is used to indicate the distribution of each audio frame in the audio to be annotated, and the phoneme label of the audio to be annotated is generated according to the recognition result and the decoding graph, wherein the phoneme label is used to indicate at least one of the start time, end time and duration of each phoneme in the audio to be annotated. Therefore, based on the constraint condition in the decoding graph, the recognition result of the preset phoneme recognition model is corrected to generate the phoneme label of the audio, thereby improving the accuracy of the phoneme label corresponding to the audio data.

[0039] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0041] Figure 1 It is a schematic diagram of a phoneme tag according to an exemplary embodiment.

[0042] Figure 2 is a schematic diagram of another phoneme tag according to an exemplary embodiment.

[0043] Figure 3 The present invention is a flowchart of a method for automatically labeling phoneme tags for speech according to an exemplary embodiment.

[0044] Figure 4 A statistical histogram of parameter distribution information is shown according to an exemplary embodiment.

[0045] Figure 5 The figure is an example diagram of a decoding diagram according to an exemplary embodiment.

[0046] Figure 6 It is a schematic diagram of another decoding diagram according to an exemplary embodiment.

[0047] Figure 7 is a schematic diagram of yet another phoneme tag according to an exemplary embodiment.

[0048] Figure 8 The present invention is a flowchart of a method for determining a phoneme tagging correction rule according to an exemplary embodiment.

[0049] Fig. 9 The invention is a block diagram of a device for automatically labeling phoneme tags for speech according to an exemplary embodiment.

[0050] Fig.10 It is a block diagram of an electronic device according to an exemplary embodiment.

[0051] Fig.11 is a block diagram of another electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0052] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0053] It should be noted that all actions of acquiring signals, information or data in the present disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the device is located and with the authorization given by the owner of the corresponding device.

[0054] In the related art, the acoustic model is trained with speech recognition data (including speech data and speech vocabulary), the emission probability of each speech vocabulary in the speech recognition data is calculated based on the acoustic model, and then the phonemes in the speech recognition data are aligned and annotated at the frame level by forced alignment. For example, Figure 1 is a schematic diagram of a phoneme tag according to an exemplary embodiment. Figure 1As shown, the speech recognition data is input into the annotation model. The speech vocabulary of the speech recognition data is: "Xiaoi Tongxue", and the corresponding phonemes are "xiao3 ai4 tong2 xue2", where the numbers are the tones of each speech vocabulary. The annotation model identifies the phoneme frames in the speech recognition data, determines the position and length range of each phoneme frame in the language recognition data, and generates alignment labels by aligning and annotating the phoneme frames in the speech recognition data based on the start position and duration of each phoneme in the speech recognition data, so as to determine the position of each phoneme frame in the speech recognition data through the alignment labels and identify the wake-up speech in the speech recognition data. The relevant language wake-up device wakes up the device based on the recognition result.

[0055] Under normal circumstances, the alignment label is generated based on the forward-backward algorithm, and this method cannot set separate state control parameters for the recognition of each phoneme frame during the generation process, which is likely to cause the forward or backward movement of the phoneme frame. For example, Figure 2 is a schematic diagram of another phoneme label shown according to an exemplary embodiment. As Figure 2 shown, the phonemes in the speech data: "Xiaoi Tongxue" are recognized based on the forward-backward algorithm, and the speech data is aligned and annotated based on the positions of the recognized phonemes. As Figure 2 shown, during the process of contrast annotation based on the forward-backward algorithm, because the state control parameters of each phoneme cannot be controlled during the generation process of the alignment annotation, the phoneme labels in the alignment label move backward, etc. For example, Figure 2 for the speech data in: "Xiaoi Tongxue", the first phoneme "x1", under normal circumstances, "x1" should appear at the position where the waveform of the speech data starts to be generated, that is, at the position of f1 in the figure. However, due to the interference of various environmental factors, when recognizing based on the forward-backward algorithm, the first phoneme "x1" is not recognized at the beginning stage of the speech data. After "x1" lasts for a period of time, the vowel phoneme "x1" in "xiao" is recognized, resulting in the backward movement of the alignment annotation of "x1", causing a delay in the recognition of the wake-up speech and a delay in the response of the wake-up device.

[0056] Based on the above problems, the present application provides a method for automatically annotating phoneme labels for speech. Figure 3 is a flowchart of a method for automatically annotating phoneme labels for speech shown according to an exemplary embodiment. As Figure 3 shown, this method can be applied to a terminal device and includes the following steps.

[0057] In step S11, a decoding graph is obtained. The decoding graph is generated based on the phoneme annotation correction rules corresponding to the audio to be annotated. The phoneme annotation correction rules include the constraint conditions for the duration of each phoneme corresponding to the audio to be annotated or the constraint conditions for the duration of each type of phoneme.

[0058] When phoneme annotation is performed on speech, due to the personalized characteristics of the speech uttered by different users, it is easy to cause deviations during recognition, resulting in incorrect phoneme label annotation and the situation of forward-backward movement. Therefore, it is necessary to constrain the recognition process of phoneme labels based on the decoding graph to make the obtained phoneme labels more accurate. Exemplarily, in this embodiment, the decoding graph is generated based on the phoneme annotation correction rules corresponding to the audio to be annotated, where the phoneme annotation correction rules include the constraint conditions for the duration of each phoneme in the audio to be annotated, or the constraint conditions for the duration of each type of phoneme. For example, when the audio to be annotated is "Xiaoi Tongxue", the corresponding phonemes are: "x", "iao3", "ai4", "t", "ong2", "x", "ue3". Through a large amount of data statistics and weighted averaging, the general phoneme position distribution of each phoneme corresponding to the audio to be annotated can be determined, and based on this distribution, phoneme annotation correction rules are generated.

[0059] Exemplarily, the audio to be annotated is a speech audio. In this embodiment, the audio to be annotated is used to test the environment of the terminal device. Therefore, the audio to be annotated includes wake-up words, such as "Xiaoi Tongxue", "Tmall Genie", "Xiaodu Xiaodu", etc.

[0060] The phoneme annotation correction rules include the duration constraint conditions of each phoneme or the duration constraint conditions of each type of phoneme. That is, through the phoneme annotation correction rules, the duration constraint conditions of "x", "iao3", "ai4", "t", "ong2", "x", "ue3" in the audio to be annotated can be determined, and the constraint conditions can be the minimum duration and / or maximum duration of each phoneme. Exemplarily, an audio word in the audio to be annotated can be composed of vowel phonemes and consonant phonemes. For example, "xiao" in the audio to be annotated can include the vowel phoneme "x" and the consonant phoneme "iao3". The phoneme annotation correction rules can indicate the duration constraint conditions of vowel phonemes and consonant phonemes in the audio to be annotated, and the constraint conditions can also be the minimum duration and / or maximum duration of each type of phoneme.

[0061] It should be noted that there are multiple types of decoding graphs configured in the terminal device, and different decoding graphs are used to annotate the audio to be annotated with different semantics. For example, decoding graph A is used to perform phoneme annotation on the audio to be annotated with the semantic information of "Xiaoi Tongxue"; decoding graph B is used to perform phoneme annotation on the audio to be annotated with the semantic information of "Start intelligent device". The corresponding decoding graph is obtained based on different application scenarios to perform phoneme annotation on the audio to be annotated.

[0062] Optionally, in one implementation manner, the method further includes:

[0063] Obtain a preset sample set, the preset sample set includes a plurality of samples, each sample includes: a preset voice audio and a corresponding first label, the first label is used to indicate the duration of each phoneme in the preset voice audio;

[0064] Obtain the statistical results of the duration of each phoneme corresponding to the preset voice audio in the preset sample set;

[0065] According to the statistical results, determine the phoneme annotation correction rules;

[0066] Generate a decoding graph based on the phoneme annotation correction rules.

[0067] It is worth mentioning that the present embodiment is applied to a terminal device, which can be a voice wake-up terminal or a voice recognition terminal. Based on the terminal device, the voice data to be annotated is aligned and annotated, and the position and duration of each phoneme frame in the voice data are determined, and then the voice data is recognized. Usually, the terminal device performs different operations based on a specific voice, and the specific voice includes a wake-up voice, a command voice, a verification voice, etc., and the duration, loudness, timbre, pitch and other parameters of the specific voice issued by different users are different, resulting in the terminal device being unable to correctly recognize the specific voice. Therefore, it is necessary to identify the normal characteristics of the preset voice audio based on a preset sample set, annotate the specific voice based on the normal characteristics, and generate a first label for indicating the duration of each phoneme in the preset voice audio. For example, in the present embodiment, the first label can be determined by the relevant recognition model recognition, and the duration of the phonemes in the preset voice audio can also be counted manually to generate a first label, wherein the first label can include information such as the start time, end time and duration range of each phoneme in the preset voice audio.

[0068] When performing normal feature recognition, multiple samples in the preset sample set are sample data generated by collecting different voice audios emitted by different users when emitting the same semantic information. The preset voice audio can be set based on actual needs. When the voice annotation requirement is a wake-up requirement, the preset voice audio can be: "Xiao Ai Tongxue", "Tmall Genie", "Xiaodu Xiaodu", etc., which is not limited in this embodiment.

[0069] By way of example, in this embodiment, based on the same specific semantics, preset voice audio generated when different users issue preset semantics is collected, and a first label is generated according to the duration of each phoneme in the preset voice audio. According to the first label, the duration of each phoneme corresponding to the preset voice audio in the preset sample set is counted, so as to determine the parameter distribution of each phoneme in the preset sample set, and generate the statistical result of the duration of each phoneme according to the parameter distribution. By way of example, a word in the preset voice audio may correspond to a vowel phoneme and a consonant phoneme, and the tone of the phoneme corresponding to each word is marked on the corresponding consonant phoneme; the parameter distribution of each phoneme frame may include the position of each phoneme frame in the entire voice sample data, the duration of each phoneme frame, and the interval duration between each phoneme frame. By way of example, Figure 4 is a statistical histogram of parameter distribution information according to an exemplary embodiment, such as Figure 4 As shown in the figure, based on the preset voice vocabulary "Xiao Ai Tongxue", the starting positions of the phoneme frames "x1", "iao", "aa", "ai", "t", "ong", "x2", and "ve" corresponding to each wake-up word in the voice sample data are counted to generate the following Figure 4 The histogram shown in the figure, wherein the horizontal axis of the histogram is the frame number axis corresponding to the speech sample data, and the vertical axis of the histogram is the proportion information of the corresponding frame number position in the speech sample data. For example, the number of samples where the phoneme "x1" appears in the 10th frame accounts for 1.2%.

[0070] For example, 3000 voice sample data of 3000 users uttering the preset voice vocabulary "Xiao Ai Tongxue" are collected, and the phoneme duration of the phoneme frame "xiao3 a ai t ong2 x ve2" corresponding to "Xiao Ai Tongxue" in each voice sample data is counted to determine the statistical results of the phoneme frames in each voice sample data. In some implementations, the parameters such as the start time, end time, and interval duration of adjacent phonemes of each phoneme frame "xiao3 a ai t ong2 x ve2" in the preset voice audio can also be counted to generate the statistical results of each phoneme.

[0071] It should be noted that in the process of counting the duration of each phoneme, the number of sample frames can be used as the minimum unit of measurement, and 1 frame is equal to 1 / 12 second. According to the duration of each phoneme in the sample data, the phoneme is counted at the frame level, the duration of each phoneme frame is determined, and the statistical method at the frame level is adopted. Under the premise of retaining the authenticity of the phoneme frame data, the statistical efficiency of the phoneme frame is improved.

[0072] Based on the statistical results generated in the above steps, the phoneme standard correction rule of each phoneme is determined. For example, in this embodiment, the statistical results can be the average duration of each phoneme in the preset voice audio, and then based on the phoneme type of each phoneme, the duration range is set on the basis of the average duration, and the phoneme annotation correction rule is determined according to the duration range. For example, according to the statistical results, in "xiao3 a ai t ong2 x ve2", the average duration of "x" is 12 frames, and then based on the feature that "x" is a vowel phoneme, the average duration is floated up and down by 2 frames on the basis of the average duration, and the duration range of "x" is obtained to be 10 frames-14 frames, then the phoneme annotation correction rule of the vowel phoneme "x" is that the duration range is 10 frames-14 frames. According to the phoneme annotation correction rules corresponding to each phoneme in the preset voice audio, the corresponding decoding graph is generated, and the audio to be annotated issued by the user can be corrected through the constraints in the decoding graph, thereby improving the accuracy of the phoneme label.

[0073] In this embodiment, after the parameter distribution of multiple phoneme frames of the same preset voice audio is counted based on the sample data, a statistical result is generated. It is necessary to generate a decoding graph of the voice audio to be annotated based on the statistical result. Therefore, based on the statistical result, multiple alignment positions and multiple annotation restrictions of each phoneme frame in the voice audio to be annotated can be determined. For example, based on the statistical result of the phoneme frame, the starting position of each phoneme frame in the sample data can be determined, and the sound source of the sample data is different users. Different users have different pronunciation habits based on the same preset voice audio, and there are differences in the starting positions of each phoneme frame. It is necessary to summarize and weight multiple different starting positions to determine the alignment position of the phoneme frame in the audio to be annotated.

[0074] For example, the speech length of the 3000 sample data collected is 40 frames. For the preset speech word "xiao" in the preset speech vocabulary "xiaoai tongxue", the vowel phoneme "x1" and the consonant phoneme "iao3" are included. Through the above steps, it is statistically determined that the start time of "x1" in the 3000 sample data is that "x1" appears at the position of 5 frames in 1000 speech sample data, "x1" appears at the position of 10 frames in 500 speech sample data, and "x1" appears at the position of 15 frames in 500 speech sample data. Based on the number of speech sample data corresponding to each result of the statistical result, the appearance position of "x1" is weighted, and the alignment position of "x1" is obtained as (1000×5+500×10+500×15) / 3000=5.8 frames, then it is determined that: the alignment position of "x1" is the position of 5.8 frames in the audio to be annotated.

[0075] It should be noted that, in the present embodiment, the alignment position of each phoneme frame is used to align the starting position of each phoneme frame in the audio to be annotated, and the annotation restriction condition limits the acquisition conditions of each phoneme frame after the starting position. For example, the duration of each phoneme frame can be determined by parameter distribution information, the starting annotation position of each phoneme frame in the audio to be annotated is determined based on the alignment position, and the minimum continuous detection duration of each phoneme frame in the audio to be annotated is limited based on the duration. The annotation restriction condition can also be the interval duration between adjacent phoneme frames. Usually, the interval duration is short, so the noise between phoneme frames can be eliminated based on the annotation restriction condition.

[0076] Optionally, in another embodiment, the decoding graph includes multiple paths, each path includes multiple nodes, adjacent nodes include node connection paths and decoding rules, the node connection paths indicate the path direction through direction arrows, the direction arrows point from upstream nodes to downstream nodes, and the decoding rules include selection judgment conditions for each phoneme path and weight values ​​of the selection judgment conditions.

[0077] It should be noted that in this embodiment, the Viterbi algorithm can be loaded in the annotation model of the terminal device, and a decoding graph from the first phoneme frame to the last phoneme frame in the preset voice audio is constructed based on the Viterbi algorithm. Figure 5 is an example diagram of a decoding diagram according to an exemplary embodiment. Figure 5 As shown, the nodes in the decoding graph are the various phoneme frames corresponding to the preset voice audio, and the arrows in the decoding graph point from the upstream nodes to the downstream nodes. Without adding any restrictions to the decoding graph, multiple recognition paths from the first phoneme frame "x1" to the last phoneme frame "ue" are determined in the phoneme frame recognition process, and then the shortest path is selected from the multiple recognition paths based on the Viterbi algorithm, and a decoding graph is generated according to the shortest path. In the process of generating the decoding graph, no restrictions are added to the recognition process of each phoneme frame, which may easily lead to deviations in the phoneme labels recognized based on the phoneme frames. Therefore, in this embodiment, a decoding graph is generated based on multiple alignment positions and multiple annotation restrictions of each phoneme frame. For example, Figure 6is a schematic diagram of another decoding graph according to an exemplary embodiment. In this embodiment, based on the above decoding graph, a decoding rule is set for the selection path from the upstream node to the downstream node based on the statistical results of each phoneme. The decoding rule may include the selection judgment condition of each phoneme path and the weight value of the selection judgment condition in the path selection. For example, taking the preset voice vocabulary "Xiao Ai Tongxue" as an example, the decoding rule of the preset voice vocabulary is determined according to the statistical results as follows: (1) the vowel phoneme lasts for at least two frames; (2) the consonant phoneme lasts for at least three frames; (3) noise data and / or negative sample data are not allowed to appear between wake-ups; (4) the weight values ​​of the above selection judgment conditions are 4:4:2 respectively. According to the decoding rule, a decoding graph is generated based on the Viterbi algorithm. Compared with the above decoding graph, the decoding graph in this embodiment is based on the decoding rule, reserves two frames of data after the vowel phoneme for phoneme detection, and reserves three frames of data after the consonant phoneme for phoneme detection, and then determines the shortest path from the first vowel phoneme frame to the tail consonant phoneme frame based on the Viterbi algorithm to generate the decoding graph.

[0078] Optionally, in another embodiment, the constraints on the duration of each phoneme or the duration of each type of phoneme include: the phoneme duration is not higher than a first predetermined duration and / or the phoneme duration is not lower than a second predetermined duration.

[0079] For example, the constraint condition in the phoneme standard correction rule in this embodiment is used to limit the duration of each phoneme in the audio to be annotated, and the average duration of each phoneme is determined based on the above statistical results, and the constraint condition is set based on the pronunciation characteristics of each phoneme on the basis of the average duration. Among them, the constraint condition can be that the phoneme duration is not higher than the first predetermined duration and / or the phoneme duration is not lower than the second predetermined duration.

[0080] Optionally, in another embodiment, the constraints on the duration of each type of phoneme include: the duration of a vowel phoneme is not higher than a third predetermined duration or not lower than a fourth predetermined duration, and / or the duration of a consonant phoneme is not higher than a fifth predetermined duration or not lower than a sixth predetermined duration.

[0081] For example, in this embodiment, the phonemes of the audio to be annotated are divided into vowel phonemes and consonant phonemes, and corresponding phoneme annotation correction rules are set based on the pronunciation rules of vowel phonemes and consonant phonemes. For example, the duration of the vowel phonemes can be set to be no higher than the third predetermined duration or no lower than the fourth predetermined duration, or the duration of the consonant phonemes can be limited to be set to be no higher than the fifth predetermined duration or no lower than the sixth predetermined duration.

[0082] In step S12, the audio to be annotated is recognized based on a preset phoneme recognition model to generate a recognition result, wherein the preset phoneme recognition model is used to calculate the acoustic score of each audio frame in the audio to be annotated, and generate a recognition result based on the acoustic score; the acoustic score is used to indicate the probability value of each audio frame being a preset phoneme frame, and the recognition result is used to indicate the distribution of each audio frame in the audio to be annotated.

[0083] For example, the terminal device of this embodiment is configured with a preset phoneme recognition model, which recognizes the meaning of the word represented by the phoneme in the audio to be annotated and generates a recognition result. Among them, the preset phoneme recognition model recognizes the audio frame in the audio to be annotated, determines the acoustic score of each audio frame, and the preset phoneme recognition model divides the audio to be standardized into multiple phonemes, annotates the recognized audio frame with the probability value of each phoneme according to the acoustic score, and then selects the phoneme with the highest probability value as the preset phoneme frame corresponding to the audio frame.

[0084] Each phoneme frame in the audio to be annotated is identified by a preset acoustic model, and the probability distribution of each phoneme frame corresponding to the preset phoneme frame is determined, wherein the probability distribution is used to represent the probability of each segment of the audio to be annotated corresponding to the preset phoneme frame. For example, taking the above-mentioned audio to be annotated as "xiao3 a ai t ong2 x ve2" as an example, the audio to be annotated is identified according to the preset phoneme recognition model, and the acoustic scores of the audio data of the 1-20 frame segments are determined to be "x"-95%, "iao3"-1%, "ai"-2%, "t"-0.5%, "ong2"-0.5%, "x"0.5%, "ve2"-0.5%, among which the probability score of "x" is the largest, then according to the acoustic score, it is determined that the audio data of the frame segment belongs to the audio frequency band of "x".

[0085] Optionally, in another implementation, the above step S12 includes:

[0086] Divide the audio to be annotated into frames to obtain multiple audio frames;

[0087] Extracting acoustic features from multiple audio frames to obtain multiple acoustic features corresponding to each audio frame;

[0088] Multiple acoustic features are input into a preset phoneme recognition model to generate a recognition result.

[0089] The audio to be annotated is processed by framing, the audio to be annotated is divided into multiple segments to generate multiple audio frames, and the audio to be annotated is divided in a frame-level recognition manner, thereby improving the recognition accuracy of the preset phoneme recognition model. The preset phoneme recognition model is used to extract acoustic features of the divided multiple audio frames, and multiple acoustic features corresponding to each audio frame are generated. Through the acoustic features, the recognition results corresponding to each audio frame are generated, wherein the recognition results are used to indicate the phoneme information of the audio frame. For example, the audio length of the audio to be annotated is determined to be 10s, and 10s is divided into frame-level data to obtain 120 frames of audio to be annotated. Then, based on the division rule, 20 frames are used as an identification frequency band, and the 120 frames of audio to be annotated are divided into 6 frequency bands of speech of 0-20 frames, 20-40 frames, 40-60 frames, 60-80 frames, 80-100 frames, and 100-120 frames. Then, the audio data of the 6 frequency bands are recognized based on the preset phoneme recognition model, and the acoustic scores of each frequency band are determined.

[0090] In step S13, a phoneme tag of the audio to be annotated is generated according to the recognition result and the decoding graph, wherein the phoneme tag is used to indicate at least one of the start time, end time and duration of each phoneme in the audio to be annotated.

[0091] For example, in this embodiment, according to the constraints of each phoneme in the decoding graph, the recognition result determined by the preset phoneme recognition model is corrected to generate a phoneme label for the audio to be annotated. The phoneme label is used to indicate at least one of the start time, end time and duration of each phoneme in the audio to be annotated. Figure 7 is a schematic diagram of another phoneme tag according to an exemplary embodiment. After the audio to be labeled is recognized through the above steps, the following is generated: Figure 7 The phoneme tags shown can be identified to determine that the marking labels of each phoneme in the phoneme tags are accurate, and there is no forward or backward shift of the phoneme tags.

[0092] Optionally, in another implementation, the above step S13 includes:

[0093] Based on the decoding graph, the distribution of the recognition results is corrected through the phoneme annotation alignment module to generate the target distribution of each phoneme in the audio to be annotated;

[0094] Generate phoneme labels based on the target distribution.

[0095] For example, in this embodiment, a phoneme annotation alignment module is configured in the terminal device, through which the distribution of each phoneme in the recognition result generated by the preset phoneme recognition mode is corrected based on the decoding graph, and the target distribution of each phoneme in the audio to be annotated is generated, wherein the target distribution is used to indicate the range of each phoneme in the audio to be annotated, and the phoneme label of the audio to be annotated is generated according to the target distribution.

[0096] Optionally, in one implementation, the above step of “generating phoneme labels according to target distribution” includes:

[0097] The target distribution is identified through an image acceleration function to generate phoneme labels.

[0098] For example, different from the prior art that generates alignment labels based on the CPU (Central Processing Unit) in the terminal device, this embodiment uses the image acceleration function in the GPU (graphics processing unit) to identify the target distribution and generate the phoneme labels corresponding to the audio to be annotated. This improves the efficiency of generating phoneme labels.

[0099] Through the above method, a decoding graph is obtained, the decoding graph is generated based on the phoneme annotation correction rule corresponding to the audio to be annotated, the phoneme annotation correction rule includes the constraint condition of the duration of each phoneme corresponding to the audio to be annotated or the constraint condition of the duration of each type of phoneme, the audio to be annotated is recognized based on the preset phoneme recognition model, and a recognition result is generated, wherein the preset phoneme recognition model is used to recognize the acoustic score of each audio frame in the audio to be annotated, and the recognition result is generated according to the acoustic score, the acoustic score is used to indicate the probability value of each audio frame being a preset phoneme frame, and the recognition result is used to indicate the distribution of each audio frame in the audio to be annotated, and the phoneme label of the audio to be annotated is generated according to the recognition result and the decoding graph, wherein the phoneme label is used to indicate at least one of the start time, end time and duration of each phoneme in the audio to be annotated. Therefore, based on the constraint condition in the decoding graph, the recognition result of the preset phoneme recognition model is corrected to generate the phoneme label of the audio, thereby improving the accuracy of the phoneme label corresponding to the audio data.

[0100] Figure 8 is a flowchart of a method for determining a phoneme annotation correction rule according to an exemplary embodiment. Figure 8 As shown, the parameter distribution information includes audio length information, audio length ratio information, audio distribution position information and audio position ratio information of multiple phoneme frames, and the method includes the following steps.

[0101] In step S121, the audio length information is weighted according to the audio length ratio information to generate a plurality of target length information corresponding to each phoneme frame in a plurality of phoneme frames.

[0102] For example, in this embodiment, the parameter distribution information of the phoneme frame is determined through the above steps, and the parameter distribution information includes the audio length information, audio length ratio information, audio distribution position information and audio position ratio information of multiple phoneme frames, wherein the ratio information represents the ratio of the number of phoneme frames of corresponding audio length in multiple voice sample data to the total number of language sample data. The audio length information is weighted by the audio length ratio information to generate the target length information of each phoneme frame.

[0103] It is worth mentioning that in this embodiment, based on the proportion of each audio length, the target length information of the corresponding phoneme frame is determined, the voice sample data is statistically analyzed, the general length pattern of each phoneme frame is determined, and the target length information of the phoneme frame is generated.

[0104] In step S122, the audio distribution position information is weighted according to the audio position ratio information to generate a plurality of alignment positions corresponding to each phoneme frame.

[0105] For example, in this embodiment, before generating the alignment label, the annotation restriction condition of the phoneme frame is determined according to the target length information corresponding to each phoneme frame, wherein the annotation restriction condition is used to limit the detection length of each phoneme frame when performing alignment annotation. For example, according to the target length information, the minimum annotation length of the phoneme frame in the annotation process is determined, and the annotation restriction condition is generated according to the minimum annotation length.

[0106] Optionally, the above step S122 includes:

[0107] Determining first frame number information of vowel frames and second frame number information of consonant frames in the plurality of phoneme frames according to the plurality of target length information;

[0108] Determining a first marking restriction condition for a vowel frame according to the first frame number information, wherein the first marking restriction condition is used to indicate a minimum length of a vowel frame in the speech data to be marked;

[0109] A second marking restriction condition for the consonant frame is determined according to the second frame number information, where the second marking restriction condition is used to indicate a minimum length of the consonant frame in the speech data to be marked.

[0110] For example, in this embodiment, the speech data to be annotated includes preset speech words, each speech word includes a vowel phoneme and a consonant phoneme, and the first frame number information corresponding to the vowel phoneme and the second frame number information corresponding to the consonant phoneme are screened out from the multiple target length information corresponding to each phoneme frame; according to the first frame number information, the first annotation restriction condition of the vowel frame is determined, and for example, the average value of multiple frame numbers in multiple first frame number information can be taken, and the first annotation restriction condition is determined according to the average value, wherein the first annotation restriction condition is used to indicate the minimum detection length of the vowel frame when the vowel frame in the speech data to be annotated is detected during the annotation of the speech data to be annotated. It is also possible to perform weighted averaging on multiple frame numbers in the second frame number information to generate a weighted average value, and determine the second annotation restriction condition according to the weighted average value, wherein the second annotation restriction condition is used to indicate the minimum detection length of the consonant frame when the consonant frame in the speech data to be annotated is detected during the annotation of the speech data to be annotated.

[0111] In step S123, a phoneme tagging correction rule is determined according to the multiple target length information and the multiple alignment positions.

[0112] By way of example, referring to the above-mentioned method for generating the target length information, in this embodiment, a phoneme annotation correction rule is generated according to a plurality of target length information and a plurality of alignment positions.

[0113] Through the above method, the alignment position and labeling restrictions are determined based on the position distribution and length distribution of each phoneme frame in the speech sample data, the starting position of each phoneme frame is restricted by the alignment position, and the phoneme frame is labeled based on the restrictions, thereby improving the accuracy of the phoneme label.

[0114] Fig. 9 is a block diagram of a device for automatically annotating phoneme labels for speech according to an exemplary embodiment. Fig. 9 As shown, the device 100 includes: an acquisition module 110 , a first generation module 120 and a second generation module 130 .

[0115] an acquisition module configured to acquire a decoding graph of the audio to be annotated, the decoding graph being generated based on a phoneme annotation correction rule corresponding to the audio to be annotated, the phoneme annotation correction rule including a constraint condition for the duration of each phoneme corresponding to the audio to be annotated or a constraint condition for the duration of each type of phoneme;

[0116] The first generating module is configured to recognize the audio to be annotated based on a preset phoneme recognition model and generate a recognition result, wherein the preset phoneme recognition model is used to calculate the acoustic score of each audio frame in the audio to be annotated, and generate the recognition result according to the acoustic score; the acoustic score is used to indicate the probability value of each audio frame being a preset phoneme frame, and the recognition result is used to indicate the distribution of each audio frame in the audio to be annotated;

[0117] The second generation module is configured to generate a phoneme tag of the audio to be annotated according to the recognition result and the decoding graph, wherein the phoneme tag is used to indicate at least one of the start time, end time and duration of each phoneme in the audio to be annotated.

[0118] Optionally, the decoding graph includes multiple paths, each path includes multiple nodes, and adjacent nodes include node connection paths and decoding rules. The node connection paths indicate the path direction through direction arrows, and the direction arrows point from upstream nodes to downstream nodes. The decoding rules include selection judgment conditions for each phoneme path and weight values ​​of the selection judgment conditions.

[0119] Optionally, the constraint conditions for the duration of each phoneme or the constraint conditions for the duration of each type of phoneme include: the duration of the phoneme is not longer than a first predetermined duration and / or the duration of the phoneme is not shorter than a second predetermined duration.

[0120] Optionally, each type of phoneme includes vowel phonemes and consonant phonemes; the constraints on the duration of each type of phoneme include: the duration of the vowel phoneme is not higher than the third predetermined duration or not lower than the fourth predetermined duration, and / or the duration of the consonant phoneme is not higher than the fifth predetermined duration or not lower than the sixth predetermined duration.

[0121] Optionally, the apparatus 100 further includes a third generating module, and the third generating module is configured to:

[0122] Obtain a preset sample set, the preset sample set includes a plurality of samples, each sample includes: a preset voice audio and a corresponding first label, the first label is used to indicate the duration of each phoneme in the preset voice audio;

[0123] Obtain the statistical results of the duration of each phoneme corresponding to the preset voice audio in the preset sample set;

[0124] According to the statistical results, determine the phoneme annotation correction rules;

[0125] Generate a decoding graph based on the phoneme annotation correction rules.

[0126] Optionally, the first generating module 120 is configured to:

[0127] Divide the audio to be annotated into frames to obtain multiple audio frames;

[0128] Extracting acoustic features from multiple audio frames to obtain multiple acoustic features corresponding to each audio frame;

[0129] Multiple acoustic features are input into a preset phoneme recognition model to generate a recognition result.

[0130] Optionally, the second generating module 130 includes:

[0131] The first generation submodule is configured to correct the distribution in the recognition result through the phoneme annotation alignment module based on the decoding graph to generate a target distribution of each phoneme in the audio to be annotated;

[0132] The second generation submodule is configured to generate phoneme labels according to the target distribution.

[0133] Optionally, the second generation submodule is configured as follows:

[0134] The target distribution is identified through an image acceleration function to generate phoneme labels.

[0135] Optionally, the statistical results include audio length information, audio length ratio information, audio distribution position information and audio position ratio information of each phoneme, and the device 100 further includes a fourth generation module, which is configured to:

[0136] The audio length information is weighted according to the audio length ratio information to generate a plurality of target length information corresponding to each phoneme;

[0137] The audio distribution position information is weighted according to the audio position ratio information to generate a plurality of alignment positions corresponding to each phoneme frame;

[0138] A phoneme tagging correction rule is determined according to the multiple target length information and the multiple alignment positions.

[0139] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0140] The present disclosure also provides a computer-readable storage medium on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of the method for automatically annotating phoneme tags for speech provided by the present disclosure are implemented.

[0141] Fig.10 1 is a block diagram of an electronic device 1000 according to an exemplary embodiment. For example, the electronic device 1000 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0142] Reference Fig.10 , the electronic device 1000 may include one or more of the following components: a processing component 1002 , a memory 1004 , a power component 1006 , a multimedia component 1008 , an audio component 1010 , an input / output interface 1012 , a sensor component 1014 , and a communication component 1016 .

[0143] The processing component 1002 generally controls the overall operation of the electronic device 1000, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 1002 may include one or more processors 1020 to execute instructions to complete all or part of the steps of the above-mentioned method for automatically annotating phoneme tags to speech. In addition, the processing component 1002 may include one or more modules to facilitate the interaction between the processing component 1002 and other components. For example, the processing component 1002 may include a multimedia module to facilitate the interaction between the multimedia component 1008 and the processing component 1002.

[0144] The memory 1004 is configured to store various types of data to support operations on the electronic device 1000. Examples of such data include instructions for any application or method operating on the electronic device 1000, contact data, phone book data, messages, pictures, videos, etc. The memory 1004 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0145] The power supply component 1006 provides power to various components of the electronic device 1000. The power supply component 1006 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 1000.

[0146] The multimedia component 1008 includes a screen that provides an output interface between the electronic device 1000 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 1008 includes a front camera and / or a rear camera. When the electronic device 1000 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and the rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0147] The audio component 1010 is configured to output and / or input audio signals. For example, the audio component 1010 includes a microphone (MIC), and when the electronic device 1000 is in an operation mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 1004 or sent via the communication component 1016. In some embodiments, the audio component 1010 also includes a speaker for outputting audio signals.

[0148] The input / output interface 1012 provides an interface between the processing component 1002 and the peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0149] The sensor assembly 1014 includes one or more sensors for providing various aspects of status assessment for the electronic device 1000. For example, the sensor assembly 1014 can detect the open / closed state of the electronic device 1000, the relative positioning of components, such as the display and keypad of the electronic device 1000, and the sensor assembly 1014 can also detect the position change of the electronic device 1000 or a component of the electronic device 1000, the presence or absence of user contact with the electronic device 1000, the orientation or acceleration / deceleration of the electronic device 1000, and the temperature change of the electronic device 1000. The sensor assembly 1014 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 1014 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 1014 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0150] The communication component 1016 is configured to facilitate wired or wireless communication between the electronic device 1000 and other devices. The electronic device 1000 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 1016 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 1016 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0151] In an exemplary embodiment, the electronic device 1000 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above-mentioned method of automatically annotating phoneme tags for speech.

[0152] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 1004 including instructions, and the instructions can be executed by the processor 1020 of the electronic device 1000 to complete the above-mentioned method of automatically annotating phoneme labels to speech. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0153] In addition to being an independent electronic device, the electronic device may also be a part of an independent electronic device. For example, in one embodiment, the electronic device may be an integrated circuit (IC) or a chip, wherein the integrated circuit may be an IC or a collection of multiple ICs; the chip may include but is not limited to the following types: GPU (Graphics Processing Unit), CPU (Central Processing Unit), FPGA (Field Programmable Gate Array), DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), SOC (System on Chip, SoC), etc. The above-mentioned integrated circuit or chip can be used to execute executable instructions (or codes) to implement the above-mentioned method of automatically labeling phoneme labels for speech. The executable instructions may be stored in the integrated circuit or chip, or may be obtained from other devices or equipment, such as the integrated circuit or chip including a processor, a memory, and an interface for communicating with other devices. The executable instruction can be stored in the memory, and when the executable instruction is executed by the processor, the above-mentioned method of automatically labeling phoneme tags for speech is implemented; alternatively, the integrated circuit or chip can receive the executable instruction through the interface and transmit it to the processor for execution, so as to implement the above-mentioned method of automatically labeling phoneme tags for speech.

[0154] In another exemplary embodiment, a computer program product is further provided. The computer program product includes a computer program executable by a programmable device, and the computer program has a code portion for executing the above-mentioned method for automatically annotating phoneme tags to speech when the computer program is executed by the programmable device.

[0155] Fig.11 1 is a block diagram of another electronic device 1100 according to an exemplary embodiment. For example, the electronic device 1100 may be provided as a server. Fig.11 The electronic device 1100 includes a processing component 1122, which further includes one or more processors, and a memory resource represented by a memory 1132 for storing instructions executable by the processing component 1122, such as an application. The application stored in the memory 1132 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1122 is configured to execute instructions to perform the above-mentioned method for automatically annotating phoneme tags for speech.

[0156] The electronic device 1100 may also include a power supply component 1126 configured to perform power management of the electronic device 1100, a wired or wireless network interface 1150 configured to connect the electronic device 1100 to a network, and an input / output interface 1158. The electronic device 1100 may operate based on an operating system stored in the memory 1132, such as Windows Server 2003. TM , Mac OS X TM , Unix TM , Linux TM , FreeBSD TM or similar.

[0157] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the present disclosure. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0158] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for automatically labeling phoneme tags for speech, characterized in that: include: Obtaining a decoding graph, wherein the decoding graph is generated based on a phoneme annotation correction rule corresponding to the audio to be annotated, wherein the phoneme annotation correction rule includes a constraint condition for the duration of each phoneme corresponding to the audio to be annotated or a constraint condition for the duration of each type of phoneme; The audio to be labeled is recognized based on a preset phoneme recognition model to generate a recognition result, wherein the preset phoneme recognition model is used to calculate the acoustic score of each audio frame in the audio to be labeled, and the recognition result is generated according to the acoustic score; the acoustic score is used to indicate the probability value of each audio frame being a preset phoneme frame, and the recognition result is used to indicate the distribution of each audio frame in the audio to be labeled; Generate a phoneme tag of the audio to be annotated according to the recognition result and the decoding graph, wherein the phoneme tag is used to indicate at least one of a start time, an end time, and a duration of each phoneme in the audio to be annotated.

2. The method according to claim 1, characterized in that The decoding graph includes multiple paths, each path includes multiple nodes, and adjacent nodes include node connection paths and decoding rules. The node connection path indicates the path direction through a direction arrow, and the direction arrow points from the upstream node to the downstream node. The decoding rules include selection judgment conditions for each phoneme path and the weight value of the selection judgment condition.

3. The method according to claim 1, characterized in that The constraint conditions for the duration of each phoneme or the constraint conditions for the duration of each type of phoneme include: the duration of the phoneme is not longer than a first predetermined duration and / or the duration of the phoneme is not shorter than a second predetermined duration.

4. The method according to claim 1, characterized in that: The various types of phonemes include vowel phonemes and consonant phonemes; The constraints on the duration of each type of phoneme include: the duration of the vowel phoneme is not higher than the third predetermined duration or not lower than the fourth predetermined duration, and / or the duration of the consonant phoneme is not higher than the fifth predetermined duration or not lower than the sixth predetermined duration.

5. The method according to claim 1, characterized in that The method further comprises: Acquire a preset sample set, the preset sample set comprising a plurality of samples, each sample comprising: a preset voice audio and a corresponding first label, the first label being used to indicate the duration of each phoneme in the preset voice audio; Obtaining the statistical result of the duration of each phoneme corresponding to the preset voice audio in the preset sample set; Determining the phoneme annotation correction rule according to the statistical results; The decoding graph is generated according to the phoneme annotation correction rule.

6. The method according to claim 1, characterized in that The step of recognizing the audio to be annotated based on a preset phoneme recognition model and generating a recognition result includes: Dividing the audio to be labeled into frames to obtain multiple audio frames; Extracting acoustic features from the multiple audio frames to obtain multiple acoustic features corresponding to the audio frames; The plurality of acoustic features are input into the preset phoneme recognition model to generate the recognition result.

7. The method according to claim 1, characterized in that Generating the phoneme label of the audio to be annotated according to the recognition result and the decoding graph includes: Based on the decoding graph, the distribution in the recognition result is corrected by a phoneme annotation alignment module to generate a target distribution of each phoneme in the audio to be annotated; The phoneme labels are generated according to the target distribution.

8. The generation method according to claim 7, characterized in that: Generating the phoneme label according to the target distribution includes: The target distribution is identified by an image acceleration function to generate the phoneme label.

9. The generation method according to claim 1, characterized in that: The statistical results include audio length information, audio length ratio information, audio distribution position information and audio position ratio information of each phoneme, and the method further includes: Weighting the audio length information according to the audio length ratio information to generate a plurality of target length information corresponding to each phoneme; Weighting the audio distribution position information according to the audio position proportion information to generate a plurality of alignment positions corresponding to each phoneme frame one by one; The phoneme tagging correction rule is determined according to the multiple target length information and the multiple alignment positions.

10. A device for automatically labeling phoneme tags on speech, characterized in that: The device comprises: an acquisition module configured to acquire a decoding graph of the audio to be annotated, wherein the decoding graph is generated based on a phoneme annotation correction rule corresponding to the audio to be annotated, wherein the phoneme annotation correction rule includes a constraint condition for the duration of each phoneme corresponding to the audio to be annotated or a constraint condition for the duration of each type of phoneme; A first generating module is configured to recognize the audio to be labeled based on a preset phoneme recognition model and generate a recognition result, wherein the preset phoneme recognition model is used to calculate the acoustic score of each audio frame in the audio to be labeled, and generate the recognition result according to the acoustic score; the acoustic score is used to indicate the probability value of each audio frame being a preset phoneme frame, and the recognition result is used to indicate the distribution of each audio frame in the audio to be labeled; The second generation module is configured to generate a phoneme tag of the audio to be annotated according to the recognition result and the decoding graph, wherein the phoneme tag is used to indicate at least one of the start time, end time and duration of each phoneme in the audio to be annotated.

11. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 9.

12. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the steps of the method described in any one of claims 1 to 9 are implemented.

Images