Audio processing method, device, electronic device, and storage medium

Abstract

The present invention discloses an audio processing method, device, electronic device, and storage medium. The audio processing method includes: determining the probability of the estimated corpus of each audio frame of the audio data and the corpus sequence of the estimated corpus; determining the position information of the target corpus in the corpus sequence; determining the audio frame of the estimated corpus that has a probability matching the target corpus in each estimated corpus as the initial audio frame of the target corpus; determining the target audio frame corresponding to the position information from the initial audio frame, and determining the target audio frame as the audio segment containing the target corpus. In this way, the position of the target corpus in the audio data can be accurately located, and the audio segment containing the target corpus in the audio data can be accurately determined. Compared with the method of manually determining the position of the target corpus, it not only saves labor costs, but also has the advantages of high efficiency and high accuracy.

Description

Technical Field

[0001] The present invention relates to the field of audio processing technology, and in particular to an audio processing method, device, electronic device, and storage medium. Background Art

[0002] The introduction of AI (artificial intelligence) technology into the field of intelligent speech has greatly improved recognition rates and reduced misrecognition rates. However, it requires a large amount of corpus support for training samples. For voice wake-up and multi-command word recognition, existing open source data is very limited. The acquisition method usually requires manual recording or manually determining the location of the target corpus in the open source audio data and manually intercepting audio clips as training samples. This method is time-consuming and labor-intensive. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to overcome the time-consuming and labor-intensive defects of the prior art in manually recording or manually determining the location of the target corpus in open source audio data and manually intercepting audio clips, and to provide an audio processing method, device, electronic device, and storage medium.

[0004] The present invention solves the above technical problems through the following technical solutions:

[0005] In a first aspect, an audio processing method is provided, comprising:

[0006] Determining a probability of an estimated corpus for each audio frame of the audio data and a corpus sequence of the estimated corpus;

[0007] Determining the position information of the target corpus in the corpus sequence;

[0008] Determining the audio frames of the estimated corpora that have a probability of matching the target corpora in each of the estimated corpora as the initial audio frames of the target corpora;

[0009] A target audio frame corresponding to the position information is determined from the initial audio frame, and the target audio frame is determined to be an audio segment containing the target corpus.

[0010] Optionally, determining the position information of the target corpus in the corpus sequence includes:

[0011] In the case where the target corpus includes multiple corpora, the position information of the phonemes in all the phonemes in the corpus sequence that match the sorting rule of the multiple corpora included in the target corpus is determined as the position information of the target corpus in the corpus sequence.

[0012] Optionally, determining a target audio frame corresponding to the position information from the initial audio frame includes:

[0013] Determining a non-silent frame from all audio frames of the audio data;

[0014] An audio frame in the initial audio frame whose frame coordinates fall within the frame coordinate range of the non-silent frame corresponding to the position information is determined as the target audio frame; the lower limit value of the frame coordinate range is the frame coordinate represented by the position information, and the upper limit value of the frame coordinate range is the sum of the frame coordinate represented by the position information and the frame number difference, and the frame number difference is the difference between the number of corpora in the corpus sequence and the number of non-silent frames determined from all the audio frames.

[0015] Optionally, determining the probability of each estimated corpus for each audio frame of the audio data and the corpus sequence of each estimated corpus includes:

[0016] The audio data is input into a pre-trained speech recognition model, and the probability of each estimated corpus and the corpus sequence are determined according to the output result of the speech recognition model.

[0017] In a second aspect, an audio processing device is provided, comprising:

[0018] a corpus determination module, configured to determine a probability of an estimated corpus for each audio frame of the audio data and a corpus sequence of the estimated corpus;

[0019] A position determination module, configured to determine the position information of the target corpus in the corpus sequence;

[0020] An audio frame determination module, configured to determine an audio frame of an estimated corpus that has a probability of matching the target corpus in each estimated corpus as an initial audio frame of the target corpus;

[0021] The segment determination module is configured to determine a target audio frame corresponding to the position information from the initial audio frame, and determine the target audio frame as an audio segment containing the target corpus.

[0022] Optionally, the location determination module is specifically configured to:

[0023] In the case where the target corpus includes multiple corpora, the position information of the phonemes in all the phonemes in the corpus sequence that match the sorting rule of the multiple corpora included in the target corpus is determined as the position information of the target corpus in the corpus sequence.

[0024] Optionally, the audio frame determination module is specifically configured to:

[0025] Determining a non-silent frame from all audio frames of the audio data;

[0026] An audio frame in the initial audio frame whose frame coordinates fall within the frame coordinate range of the non-silent frame corresponding to the position information is determined as the target audio frame; the lower limit value of the frame coordinate range is the frame coordinate represented by the position information, and the upper limit value of the frame coordinate range is the sum of the frame coordinate represented by the position information and the frame number difference, and the frame number difference is the difference between the number of corpora in the corpus sequence and the number of non-silent frames determined from all the audio frames.

[0027] Optionally, the corpus determination module is specifically configured to:

[0028] The audio data is input into a pre-trained speech recognition model, and the probability of each estimated corpus and the corpus sequence are determined according to the output result of the speech recognition model.

[0029] According to a third aspect, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above-described audio processing methods when executing the computer program.

[0030] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the audio processing method described in any one of the above items is implemented.

[0031] The positive progress effect of the present invention is that: in the embodiment of the present invention, the position of the target corpus in the audio data can be accurately located, and the audio segment containing the target corpus in the audio data can be accurately determined. Compared with the method of manually determining the position of the target corpus, it not only saves labor costs, but also has the advantages of high efficiency and high accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 A flowchart of an audio processing method provided by an exemplary embodiment of the present invention;

[0033] Figure 2 A module diagram of an audio processing device provided by an exemplary embodiment of the present invention;

[0034] Figure 3 The present invention provides a schematic structural diagram of an electronic device according to an exemplary embodiment of the present invention. DETAILED DESCRIPTION

[0035] The present invention is further described below by way of examples, but the present invention is not limited to the scope of the examples.

[0036] Figure 1The flowchart of an audio processing method provided by an exemplary embodiment of the present invention. This audio processing method can accurately determine the frame coordinate positions of the target corpus specified by the user in the audio data (i.e., the start and end frames of the target corpus in the audio data), so as to intercept the audio segment of the target corpus from the audio data. Refer to Figure 1 , the audio processing method includes the following steps:

[0037] Step 101: Determine the probability of the estimated corpus and the corpus sequence of the estimated corpus for each audio frame of the audio data.

[0038] The audio data contains multiple audio frames. The probability of the estimated corpus and the corpus sequence are obtained by performing speech recognition on the audio data. The estimated corpus is the corpus that may be represented by each audio frame predicted by performing speech recognition on each audio frame. Among them, the corpus can be, but is not limited to, pinyin characters or phonemes, etc.

[0039] In one embodiment, a probability matrix is used to represent the probability of each estimated corpus for all audio frames of a segment of audio data. The probability matrix P is expressed as follows:

[0040]

[0041] Where, T represents the number of audio frames contained in the audio data; N represents the number of estimated corpora; P tn represents the probability value that the t-th audio frame is the n-th estimated corpus. 1 ≤ t ≤ T, 1 ≤ n ≤ N.

[0042] For example, if the text corresponding to the audio data is "Today's weather conditions", due to environmental noise or user pronunciation, etc., when performing speech recognition on the above audio data, multiple pinyin characters (taking the corpus as pinyin characters) may be recognized for each audio frame. For example, for the audio frame corresponding to the character "今", the speech recognition results include "jin", "jing", "gi", etc. The first column in the probability matrix of "Today's weather conditions" respectively represents the probabilities of "jin", "jing", "gi"; for the audio frame corresponding to the character "天", the speech recognition results include "tián", "tiàn", "tian", etc. The second column in the probability matrix of "Today's weather conditions" respectively represents the probabilities of "tián", "tiàn", "tian"; and so on, each audio frame of the audio data is represented by probability.

[0043] It can be understood that due to different pronunciation durations, there is a situation where one character occupies multiple audio frames. For example, when a user pronounces "situation", the pronunciation of the character "kuàng" is particularly long and occupies 2 audio frames, resulting in the number of columns of the probability matrix being greater than the number of characters corresponding to the audio data. Similarly, pauses (no pronunciation) during the pronunciation process will cause some frames or several frames of the audio to be silent frames, resulting in the number of columns of the probability matrix being greater than the number of characters corresponding to the audio data.

[0044] In one embodiment, the corpus sequence is obtained according to the probability matrix. For the specific implementation process, refer to the related technical description and will not be elaborated here. The corpus sequence of the estimated corpus is represented by a vector. The vector representation of the corpus sequence R is as follows:

[0045] R = [R1, R2, …, R l ;

[0046] where l represents the number of corpora corresponding to the audio frames included in the audio data. R i represents each corpus included in the audio data. 1 ≤ i ≤ l.

[0047] In one embodiment, with the help of a pre-trained automatic speech recognition (ASR) model, the probabilities of each estimated corpus of the audio frames of the audio data and the corpus sequence of each estimated corpus are determined. Specifically, the audio data is input into the pre-trained speech recognition model, and the probabilities and corpus sequences of each estimated corpus are determined according to the output results of the speech recognition model. For the training process of the speech recognition model, refer to the related technical description and will not be elaborated here. The speech recognition model can be, but is not limited to, an end-to-end speech recognition model. By using an end-to-end speech recognition model, open-source data can be recognized, and the target corpus can be found and intercepted, thus solving the problem of lack of corpus for voice wake-up or multi-command word recognition.

[0048] In one embodiment, the input parameters of the pre-trained speech recognition model include audio data, and the output parameters include a probability matrix and a corpus sequence, that is, the probabilities of the estimated corpora of the audio data and the corpus sequence are directly output by the speech recognition model.

[0049] Step 102: Determine the position information of the target corpus in the corpus sequence.

[0050] The target corpus can be set according to actual needs. The target corpus can include one corpus or multiple corpora, and the embodiments of the present invention do not make special limitations on this. One corpus can correspond to one pinyin character or one phoneme, which corresponds to the corpora included in the corpus sequence. That is, if one corpus included in the corpus sequence corresponds to one phoneme, then one corpus included in the target corpus corresponds to one phoneme; if one corpus included in the corpus sequence corresponds to one pinyin character, then one corpus included in the target corpus corresponds to one pinyin character.

[0051] In one embodiment, by comparing the similarity between each corpus of the target corpus and each corpus included in the corpus sequence, it is determined whether the corpus sequence includes the target corpus, and further the position information of the target corpus in the corpus sequence is determined.

[0052] Determine the position of the target corpus in the corpus sequence, that is, determine the coordinates of the target corpus in the vector representing the corpus sequence. The coordinates of each corpus in the target corpus are represented by an array. For example, if the target corpus contains two corpora, the coordinate arrays of the two corpora are represented as follows:

[0053] A=[a1,a2,…,a k ];

[0054] B=[b1,b2,…,b k ];

[0055] Array A represents the coordinate set of a corpus in R, and array B represents the coordinate set of another corpus in R. It can be understood that the other corpus is always after the first corpus, so a j Always better than b j 1. Since a piece of audio data may contain multiple target corpora, the array size is not fixed. 1≤j≤k, where k represents the total number of times the target corpus appears in the corpus sequence.

[0056] Taking "today's weather conditions" as an example, assuming that the target corpus is "tian", the corpus sequence of "today's weather conditions" is expressed as R = [jin, tian, tian, qi, qing, kang], and the position information of the target corpus "tian" in the corpus sequence is expressed as A = [2, 3].

[0057] In one embodiment, when the target corpus contains multiple corpora, determining the position information of the target corpus also requires reference to the sorting rules of the multiple corpora. Specifically, the position information of the corpus that matches the sorting rules of the multiple corpora contained in the target corpus among all the corpora in the corpus sequence is determined as the position information of the target corpus in the corpus sequence.

[0058] For example, assuming the target corpus is "tian qi", which contains two corpora, "tian" and "qi", and the sorting rule is that "tian" comes before "qi" and "qi" comes after "tian", when determining the position information of the target corpus, it is necessary not only to see whether each corpus in the target corpus exists in the corpus sequence, but also to see whether the sorting rule matches. Based on the above two conditions, the position information of the target corpus "tian" in the corpus sequence is expressed as A = [3], and the position information of the target corpus "qi" in the corpus sequence is expressed as B = [4], rather than A = [2, 3] and B = [4]. Therefore, the sorting rule can exclude audio frames that do not meet the sorting rule of the target corpus, thereby improving the accuracy of the target corpus positioning.

[0059] Step 103: Determine the audio frames of the estimated corpus that have a probability of matching the target corpus in each estimated corpus as the initial audio frames of the target corpus.

[0060] In one embodiment, when determining the initial audio frame of the target corpus, a probability vector of the estimated corpus that matches each corpus in the target corpus is found in the probability matrix, representing the probability value of the corpus at different times in this audio data. If the probability vector exceeds a first probability threshold, it is considered that this audio frame may be the initial audio frame that matches the corpus of the target corpus. Based on the above method, the coordinates and probability values of the audio frames that may exist for each corpus in the time stream of the entire audio data are obtained.

[0061] Taking the target corpus containing two corpora, K1 and K2, as an example, the probability vector is expressed as P t K1 and P t K2 , the coordinates and probability values of the audio frames that may exist in the time stream of the entire audio data of the two corpora are expressed as:

[0062]

[0063]

[0064]

[0065]

[0066] Where X represents the coordinate array of the audio frame (initial audio frame) corresponding to corpus K1, and Y represents the coordinate array of the audio frame (initial audio frame) corresponding to corpus K2; the subscripts m of X and Y are m 、n n It is to distinguish it from the real frame coordinates described below. The probability matrix P is a two-dimensional matrix, one dimension is the time dimension t, one dimension is the pinyin character k, and the elements are probability values. t K1 represents the set of probability values at the position (t, K1) in P, Pt K2 Represents the set of probability values at the (t, K2) position in P.

[0067] Since there may be corpus similar to the target corpus in the audio data, misrecognition may occur, resulting in the coordinate array obtained based on the probability matrix in step 102 and the coordinate array of the initial audio frame obtained based on the corpus sequence in step 103 not being able to correspond one to one, indicating that there is misrecognition in the initial audio frame. Step 104 needs to be executed to determine the real audio frame of the target corpus from the initial audio frame and eliminate the misrecognized audio frame.

[0068] Step 104: Determine a target audio frame corresponding to the position information from the initial audio frame, and determine the target audio frame as an audio segment containing the target corpus.

[0069] This technical solution accurately locates the target corpus within the audio data, identifies the audio segments containing the target corpus, and extracts the desired audio segments. These extracted audio segments can then be used as corpus for voice wake-up and multi-command speech recognition, expanding the corpus.

[0070] In addition to speech frames, a piece of audio data generally also includes silent frames where the user pauses or does not pronounce a word. These silent frames need to be excluded.

[0071] In one embodiment, determining a target audio frame corresponding to the position information from an initial audio frame includes: determining a non-silent frame (i.e., a speech frame) from all audio frames of the audio data, and determining an audio frame in the initial audio frame whose frame coordinates fall within a frame coordinate range of the non-silent frame corresponding to the position information as the target audio frame. The lower limit of the frame coordinate range is the frame coordinate represented by the position information, and the upper limit of the frame coordinate range is the sum of the frame coordinate represented by the position information and a frame number difference, where the frame number difference is the difference between the number of corpora in the corpus sequence and the number of non-silent frames determined from all audio frames.

[0072] In one embodiment, using the silent mode tN Representing silent frames in the audio data allows for further precise location of the target corpus within the audio data. The silence dimension represents the probability that the audio data is silent. A column can be added after the probability matrix to represent the probability that the audio data is silent. When the probability of the silence dimension is lower than the second probability threshold, we consider the current audio frame to be a speech frame. Therefore, we can obtain the possible frame coordinate array S corresponding to the corpus of this audio data segment:

[0073] S=[S1,S2,…,S l+Δl ];

[0074] The frame coordinate array S represents the set of all frame coordinates that can identify the corpus in the audio data. Δl represents

[0075] The result of the probability matrix R is output by the speech recognition model. The speech recognition model outputs the result by referring to the relationship between the previous and next audio frames. When determining S, you only need to set a threshold filter based on the silence dimension of the probability matrix R to obtain S.

[0076] The second probability threshold can be set according to actual conditions. In order not to miss the coordinates of the speech segment, the second probability threshold is generally set to a larger value. The silence dimension can be obtained with the help of a speech recognition model.

[0077] Because silence frames can be mistakenly identified as speech frames, the length of S is generally greater than the length of R. To find the true frame coordinates in X and Y, the A, B, and S sequences must be used to narrow the range and accurately determine the true coordinates. If a silence frame is not mistakenly identified as a speech frame, the length of S is the same as that of R, and they correspond one-to-one.

[0078] The true frame coordinates of the i-th corpus of the target corpus are expressed as X i and Y i , X i and Y i respectively and In the range, Δl represents the length difference between S and R. Find the values that meet the conditions in arrays X and Y. If multiple and Within the available options, select the appropriate combination based on the following conditions:

[0079] 0<Y i -X i <L.

[0080] L is the maximum frame length of the target corpus. If there are still multiple groups that match, the corresponding, larger group is selected and considered to be the true frame coordinates of the i-th corpus of the target corpus. Otherwise, it is considered to be a misrecognition and needs to be excluded. Finally, the specific word is intercepted based on the output of the first and last frames of the specific word.

[0081] Let's take "today's weather conditions" as an example. Assuming that the target corpus we want is "weather", the real audio frame coordinates of "today's weather conditions" are r = [r1:r6] (this sequence cannot be obtained directly), and the real audio frame coordinates of the target corpus "weather" are [r3,r4]; if obtained according to the silence dimension, the speech frame coordinates of "today's weather conditions" are S = [s1:s8]; S should contain all r and two false positives; r1 can only be in [s1,s2,s3], so as to ensure that [r2:r6] has corresponding values in S, and similarly, other elements in r have corresponding values in S.

[0082] The length of the target corpus "weather" is 2. The true coordinates of the target corpus "weather" are within the range of [s3, s4, s5] and [s4, s5, s6]. However, there are two values for the possible coordinates of "tian". Combined with the sorting rule of "weather", the one within the range of [s3, s4, s5] among these two is determined as the frame coordinate of "weather".

[0083] Thus, by narrowing the range through the A, B, S sequences, it can be accurate to the true coordinates of the target. Searching within this range can improve the accuracy of target corpus positioning and avoid misrecognition of words with similar pronunciations.

[0084] After finding the true frame coordinate of the target corpus (i.e., the starting frame of the speech), a judgment will be made. If there are no other audio frames after the audio frame corresponding to the target corpus in the audio data, then the audio is intercepted from the starting frame of the first corpus of the target corpus to the end of the audio, and it is considered as the audio segment of the target corpus we need; if there are other pinyin characters in the target corpus besides "tian" and "qi", then use the same method to find the frame coordinate of the first pinyin character after the target corpus as the end frame coordinate for intercepting the audio.

[0085] At the same time, it makes full use of the silence dimension in the pinyin (phoneme) characters to accurately obtain the frame coordinates of the selected specific word in the audio.

[0086] When obtaining the frame coordinates of a specific word, due to the possible existence of similar words of the specific word in the recognized audio, errors may occur when searching for the frame coordinates of the specific word. Therefore, using the silence dimension can further accurately find the frame coordinates of the specific word, thereby achieving the interception of the specific word.

[0087] Corresponding to the foregoing embodiments of the audio processing method, the present invention also provides an embodiment of an audio processing device.

[0088] Figure 2 The following is a schematic diagram of the modules of an audio processing device provided by an exemplary embodiment of the present invention. The audio processing device includes:

[0089] A corpus determination module 21, configured to determine the probability of the estimated corpus of each audio frame of the audio data and the corpus sequence of the estimated corpus;

[0090] A position determination module 22, configured to determine the position information of the target corpus in the corpus sequence;

[0091] An audio frame determination module 23, configured to determine the audio frames of the estimated corpus whose probability matches the target corpus among each estimated corpus as the initial audio frames of the target corpus;

[0092] The segment determination module 24 is configured to determine a target audio frame corresponding to the position information from the initial audio frame, and determine the target audio frame as an audio segment containing the target corpus.

[0093] Optionally, the location determination module is specifically configured to:

[0094] In the case where the target corpus includes multiple corpora, the position information of the phonemes in all the phonemes in the corpus sequence that match the sorting rule of the multiple corpora included in the target corpus is determined as the position information of the target corpus in the corpus sequence.

[0095] Optionally, the audio frame determination module is specifically configured to:

[0096] Determining a non-silent frame from all audio frames of the audio data;

[0097] An audio frame in the initial audio frame whose frame coordinates fall within the frame coordinate range of the non-silent frame corresponding to the position information is determined as the target audio frame; the lower limit value of the frame coordinate range is the frame coordinate represented by the position information, and the upper limit value of the frame coordinate range is the sum of the frame coordinate represented by the position information and the frame number difference, and the frame number difference is the difference between the number of corpora in the corpus sequence and the number of non-silent frames determined from all the audio frames.

[0098] Optionally, the corpus determination module is specifically configured to:

[0099] The audio data is input into a pre-trained speech recognition model, and the probability of each estimated corpus and the corpus sequence are determined according to the output result of the speech recognition model.

[0100] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial description of the method embodiments. The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present invention. A person of ordinary skill in the art can understand and implement the present invention without inventive work.

[0101] Figure 3 This is a schematic structural diagram of an electronic device according to an exemplary embodiment of the present invention, showing a block diagram of an exemplary electronic device 30 suitable for implementing the embodiments of the present invention. Figure 3 The electronic device 30 shown is only an example and should not limit the functionality and scope of use of the embodiments of the present invention.

[0102] like Figure 3 As shown, the electronic device 30 may be a general-purpose computing device, such as a server device. Components of the electronic device 30 may include, but are not limited to, the at least one processor 31, the at least one memory 32, and a bus 33 connecting different system components (including the memory 32 and the processor 31).

[0103] The bus 33 includes a data bus, an address bus, and a control bus.

[0104] The memory 32 may include a volatile memory, such as a random access memory (RAM) 321 and / or a cache memory 322 , and may further include a read-only memory (ROM) 323 .

[0105] The memory 32 may also include a program tool 325 (or utility) having a set (at least one) of program modules 324, such program modules 324 including but not limited to: an operating system, one or more application programs, other program modules and program data, each of which or some combination may include an implementation of a network environment.

[0106] The processor 31 executes various functional applications and data processing by running the computer program stored in the memory 32, such as the method provided in any of the above embodiments.

[0107] The electronic device 30 can also communicate with one or more external devices 34 (e.g., a keyboard, pointing device, etc.). This communication can occur via an input / output (I / O) interface 35. Furthermore, the model-generated electronic device 30 can also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 36. As shown, the network adapter 36 communicates with other modules of the model-generated electronic device 30 via a bus 33. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with the model-generated electronic device 30, including but not limited to microcode, device drivers, redundant processors, external disk drive arrays, RAID (RAID) systems, tape drives, and data backup storage systems.

[0108] It should be noted that although several units / modules or sub-units / modules of the electronic device are mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of the present invention, the features and functions of two or more units / modules described above may be embodied in a single unit / module. Conversely, the features and functions of a single unit / module described above may be further divided and embodied by multiple units / modules.

[0109] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, and when the program is executed by a processor, the method provided in any of the above embodiments is implemented.

[0110] The readable storage medium may include, but is not limited to, a portable disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0111] In a possible implementation manner, the embodiment of the present invention may also be implemented in the form of a program product, which includes program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute a method for implementing any of the above embodiments.

[0112] The program code for executing the present invention may be written in any combination of one or more programming languages, and may be executed entirely on the user device, partially on the user device, as an independent software package, partially on the user device and partially on a remote device, or entirely on the remote device.

[0113] Although specific embodiments of the present invention have been described above, those skilled in the art will appreciate that these are merely illustrative and that the scope of the present invention is defined by the appended claims. Those skilled in the art may make various changes or modifications to these embodiments without departing from the principles and essence of the present invention, and such changes and modifications are intended to fall within the scope of the present invention.

Claims

1. An audio processing method, characterized in that: include: Determining a probability of an estimated corpus for each audio frame of the audio data and a corpus sequence of the estimated corpus; Determining the position information of the target corpus in the corpus sequence; Determining the audio frame of the estimated corpus that has a probability of matching the target corpus in each estimated corpus as the initial audio frame of the target corpus; Determining a target audio frame corresponding to the position information from the initial audio frame, and determining the target audio frame as an audio segment containing the target corpus; Determining the position information of the target corpus in the corpus sequence includes: In the case where the target corpus includes multiple corpora, determining the position information of the phonemes in all the phonemes in the corpus sequence that match the sorting rule of the multiple corpora included in the target corpus as the position information of the target corpus in the corpus sequence; and / or, Determining a target audio frame corresponding to the position information from the initial audio frame includes: Determining a non-silent frame from all audio frames of the audio data; An audio frame in the initial audio frame whose frame coordinates fall within the frame coordinate range of the non-silent frame corresponding to the position information is determined as the target audio frame; the lower limit value of the frame coordinate range is the frame coordinate represented by the position information, and the upper limit value of the frame coordinate range is the sum of the frame coordinate represented by the position information and the frame number difference, and the frame number difference is the difference between the number of corpora in the corpus sequence and the number of non-silent frames determined from all the audio frames.

2. The audio processing method according to claim 1, wherein: Determining a probability of an estimated corpus for each audio frame of the audio data and a corpus sequence of the estimated corpus includes: The audio data is input into a pre-trained speech recognition model, and the probability of each estimated corpus and the corpus sequence are determined according to the output result of the speech recognition model.

3. An audio processing device, characterized in that: include: a corpus determination module, configured to determine a probability of an estimated corpus for each audio frame of the audio data and a corpus sequence of the estimated corpus; A position determination module, configured to determine the position information of the target corpus in the corpus sequence; An audio frame determination module, configured to determine an audio frame of an estimated corpus that has a probability of matching the target corpus in each estimated corpus as an initial audio frame of the target corpus; a segment determination module, configured to determine a target audio frame corresponding to the position information from the initial audio frame, and determine the target audio frame as an audio segment containing the target corpus; The position determination module is specifically used for: In the case where the target corpus includes multiple corpora, determining the position information of the phonemes in all the phonemes in the corpus sequence that match the sorting rule of the multiple corpora included in the target corpus as the position information of the target corpus in the corpus sequence; and / or, The audio frame determination module is specifically used to: Determining a non-silent frame from all audio frames of the audio data; An audio frame in the initial audio frame whose frame coordinates fall within the frame coordinate range of the non-silent frame corresponding to the position information is determined as the target audio frame; the lower limit value of the frame coordinate range is the frame coordinate represented by the position information, and the upper limit value of the frame coordinate range is the sum of the frame coordinate represented by the position information and the frame number difference, and the frame number difference is the difference between the number of corpora in the corpus sequence and the number of non-silent frames determined from all the audio frames.

4. The audio processing device according to claim 3, characterized in that The corpus determination module is specifically used for: The audio data is input into a pre-trained speech recognition model, and the probability of each estimated corpus and the corpus sequence are determined according to the output result of the speech recognition model.

5. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the audio processing method according to claim 1 or 2 is implemented.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the audio processing method according to claim 1 or 2 is implemented.

Images