Audio processing method and apparatus, electronic device, and medium

Abstract

The application discloses an audio processing method and device, electronic equipment and medium, and belongs to the technical field of audio processing. The method comprises the following steps: determining N first sub-audio segments according to the obtained first audio, each first sub-audio segment corresponds to a text information of the first audio, N is a positive integer; obtaining N first audio features of the N first sub-audio segments respectively, the N first audio features and the N first sub-audio segments correspond to each other; obtaining N second sub-audio segments according to the N first audio features respectively, the N second sub-audio segments and the N first audio features correspond to each other, and each second sub-audio segment is determined according to the corresponding first audio feature; and generating a second audio according to the N second sub-audio segments; wherein the text information corresponding to each second sub-audio segment is matched with the text information corresponding to one first sub-audio segment respectively.

Description

Technical Field

[0001] This application belongs to the field of audio processing technology, specifically relating to an audio processing method, apparatus, electronic device, and medium. Background Technology

[0002] Typically, a user can first input the voice audio they want to send into their electronic device, and then trigger the electronic device to send the voice audio to other electronic devices, so that other users can listen to the voice audio on those other electronic devices and thus communicate with other users.

[0003] However, due to certain reasons (such as the user's accent, external noise, etc.), the user's voice audio input may be unclear. This may cause other users to be unable to hear the text information in the voice audio, thus requiring the user to perform multiple operations to input clear voice audio.

[0004] Therefore, it makes sending voice and audio messages using electronic devices less convenient. Summary of the Invention

[0005] The purpose of this application is to provide an audio processing method, apparatus, electronic device, and medium that can solve the problem of poor convenience for users to send voice audio using electronic devices.

[0006] In a first aspect, embodiments of this application provide an audio processing method, the method comprising: determining N first sub-audio segments based on an acquired first audio, each first sub-audio segment corresponding to a text information of the first audio, where N is a positive integer; acquiring N first audio features for each of the N first sub-audio segments, wherein the N first audio features correspond one-to-one with the N first sub-audio segments; determining N second sub-audio segments based on the N first audio features, wherein the N second sub-audio segments correspond one-to-one with the N first audio features, and each second sub-audio segment is determined based on a corresponding first audio feature; generating a second audio based on the N second sub-audio segments; wherein the text information corresponding to each second sub-audio segment is matched with the text information corresponding to a first sub-audio segment.

[0007] Secondly, embodiments of this application provide an audio processing apparatus, comprising: a determining module, an acquiring module, and a generating module. The determining module is configured to determine N first sub-audio segments based on the acquired first audio, each first sub-audio segment corresponding to a text message of the first audio, where N is a positive integer. The acquiring module is configured to acquire N first audio features of the N first sub-audio segments determined by the determining module, each of the N first audio features corresponding one-to-one with the N first sub-audio segments. The determining module is further configured to determine N second sub-audio segments based on the N first audio features acquired by the acquiring module, each of the N second sub-audio segments corresponding one-to-one with the N first audio features, and each second sub-audio segment is determined based on a corresponding first audio feature. The generating module is configured to generate second audio based on the N second sub-audio segments determined by the determining module. The text message corresponding to each second sub-audio segment is matched with the text message corresponding to a first sub-audio segment.

[0008] Thirdly, embodiments of this application provide an electronic device including a processor and a memory, wherein the memory stores programs or instructions executable on the processor, and the programs or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0009] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0010] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect.

[0011] In a sixth aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method described in the first aspect.

[0012] In this embodiment, the electronic device can first determine N first sub-audio segments (each first sub-audio segment corresponds to a text information of the first audio) based on the acquired first audio, and then obtain N first audio features corresponding one-to-one with the N first sub-audio segments. Based on these N first audio features, it can then obtain N second sub-audio segments (each second sub-audio segment is determined based on the corresponding first audio feature). Thus, the electronic device can generate a second audio based on these N second sub-audio segments (each second sub-audio segment's corresponding text information matches the text information corresponding to a first sub-audio segment). Since the electronic device can first determine the first sub-audio segment corresponding to each text information of the first audio, and then obtain the second sub-audio segment matching the text information corresponding to each first sub-audio segment based on the first audio feature of each first sub-audio segment—that is, a clear sub-audio segment matching the text information corresponding to each first sub-audio segment—to generate a clear second audio, it eliminates the need for the user to perform multiple operations on the electronic device to input clear voice audio. Therefore, the convenience of sending voice audio using the electronic device can be improved. Attached Figure Description

[0013] Figure 1 This is one of the flowcharts illustrating the audio processing method provided in the embodiments of this application;

[0014] Figure 2 This is a second schematic flowchart of the audio processing method provided in the embodiments of this application;

[0015] Figure 3 This is one of the schematic diagrams of the chat interface provided in the embodiments of this application;

[0016] Figure 4 This is the second schematic diagram of the chat interface provided in the embodiments of this application;

[0017] Figure 5 This is the third flowchart illustrating the audio processing method provided in the embodiments of this application;

[0018] Figure 6 This is a schematic diagram of the center frequency of the triangular filter provided in the embodiments of this application;

[0019] Figure 7 This is one of the schematic diagrams of the audio segment library provided in the embodiments of this application;

[0020] Figure 8 This is the second schematic diagram of the audio segment library provided in the embodiments of this application;

[0021] Figure 9 This is the third schematic diagram of the audio segment library provided in the embodiments of this application;

[0022] Figure 10This is a schematic diagram of the structure of the audio processing device provided in the embodiments of this application;

[0023] Figure 11 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application;

[0024] Figure 12 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0025] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0026] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0027] The audio processing method provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0028] Figure 1 A flowchart of an audio processing method provided in an embodiment of this application is shown. Figure 1 As shown in the embodiment of this application, the audio processing method may include the following steps 101 to 104.

[0029] Step 101: The electronic device determines N first sub-audio segments based on the acquired first audio.

[0030] It can be understood that each of the N first sub-audio segments is a part of the first audio segment.

[0031] In this embodiment of the application, each first sub-audio segment corresponds to a textual piece of information in the first audio, and N is a positive integer.

[0032] In this embodiment, the first audio includes N text information items, each of which can be a text string. The language families of these N text information items can be completely identical, partially identical, or completely different.

[0033] It should be noted that the "N pieces of text information" mentioned above can be understood as: the text information that can be obtained after converting the first audio into text.

[0034] Optionally, in the embodiments of this application, the content of the N textual information may be partially the same or completely different.

[0035] For example, suppose the first audio includes N text messages (e.g., four text messages), such as the text message "you", the text message "we", the text message "good", and the text message "ah". These four text messages belong to the same language family and have completely different contents.

[0036] Optionally, in this embodiment of the application, when the electronic device acquires the first audio, it can first preprocess the first audio, and then divide the preprocessed first audio into N first sub-audio segments according to the band characteristics of the preprocessed first audio.

[0037] The electronic device can delete the sub-audio segment corresponding to the silent part in the first audio to preprocess the first audio.

[0038] The electronic device can divide the pre-processed first audio into N first sub-audio segments based on the trough position of the pre-processed first audio.

[0039] In this process, after the preprocessed first audio is divided into N first sub-audio segments, the electronic device can label each first sub-audio segment to facilitate the processing of the N first sub-audio segments.

[0040] For example, the electronic device may label the first first sub-audio segment of N first sub-audio segments as D1, the second first sub-audio segment of N first sub-audio segments as D2, and the third first sub-audio segment of N first sub-audio segments as D3, and so on, until the last first sub-audio segment of N first sub-audio segments is labeled as Dn.

[0041] Optionally, in this embodiment of the application, when the electronic device displays the interface of the target application, the electronic device can obtain the first audio based on the user's input of the identifier of the first audio in the interface (e.g., the target audio identifier in the following embodiment) (e.g., the first input in the following embodiment), so that the electronic device can determine N first sub-audio segments based on the first audio segment.

[0042] The target application can be any of the following: chat application, music application, web application, payment application, lifestyle application, etc.

[0043] Optionally, in the embodiments of this application, combined with Figure 1 ,like Figure 2 As shown, prior to step 101 above, the audio processing method provided in this application embodiment further includes steps 201 to 204 below.

[0044] Step 201: When the electronic device displays the target audio identifier, it receives the user's first input for the target audio identifier.

[0045] Optionally, in this embodiment of the application, when the electronic device displays the main interface of the target application, the electronic device can display the chat interface corresponding to the target user identifier based on the user's click input of the target user identifier in at least one user identifier of the main interface. The chat interface includes the target audio identifier, so that the user can make the first input.

[0046] In one example, when the chat interface corresponding to the target user identifier is displayed, the user can press the "press to speak" control in the chat interface to input voice information into the electronic device, so that the electronic device can obtain the first audio and display the target audio identifier in the input box of the chat interface.

[0047] It is understood that in this example, the electronic device can be the sending device.

[0048] In another example, the electronic device can first receive the first audio from another electronic device corresponding to the target user identifier, so that when the user clicks on the target user identifier, the electronic device can display the chat interface corresponding to the target user identifier and display the target audio identifier in the chat interface.

[0049] It is understood that in this example, the electronic device can be the receiving device.

[0050] In this embodiment of the application, the target audio identifier is the identifier of the first audio.

[0051] Optionally, in this embodiment of the application, the target audio identifier can be any of the following: audio name, audio link, audio icon, etc.

[0052] Optionally, in the embodiments of this application, the first input can be any of the following: long press input, single click input, double click input, swipe input, etc.

[0053] It should be noted that the above "slide input" can be understood as: sliding input with the starting input position being the location of the target audio identifier.

[0054] For example, let's take a mobile phone as an example. Figure 3 As shown, when a user clicks on the target user identifier 10 (e.g., "Zhang San"), the electronic device can display the chat interface corresponding to that target user identifier, and display the target audio identifier 11 in the chat interface. The user can click on the target audio identifier 11 to listen to the voice message sent by the target user Zhang San.

[0055] Step 202: The electronic device responds to the first input and displays the target control.

[0056] In this embodiment, the target control is used to trigger the generation of a second audio.

[0057] Optionally, in this embodiment of the application, the electronic device can display the target control in a floating area adjacent to the area where the target audio identifier is located.

[0058] It should be noted that the "adjacent area of ​​the target audio identifier area" mentioned above can be understood as: the area within the predetermined range of the target audio identifier area.

[0059] Further optionally, in this embodiment of the application, the electronic device can display a first window in a floating manner, the first window including at least one control, each control being used to perform operation processing on the first audio, the at least one control including a target control.

[0060] For example, combined with Figure 3 ,like Figure 4 As shown, when a user presses and holds the target audio identifier 11, the electronic device can display a first window 12 floating in a predetermined area where the target audio identifier is located. The first window 12 includes at least one control, each of which is used to perform operation processing on the first audio. The at least one control includes a target control 13 (such as "correct playback").

[0061] Step 203: The electronic device receives the user's second input to the target control.

[0062] Optionally, in the embodiments of this application, the second input can be any of the following: long press input, single click input, double click input, swipe input, etc.

[0063] Step 204: The electronic device responds to the second input and acquires the first audio.

[0064] Optionally, in this embodiment of the application, the electronic device can read the first audio from the cache area of ​​the electronic device to obtain the first audio, so that the electronic device can determine N first sub-audio segments based on the first audio.

[0065] Thus, when an electronic device displays a target audio identifier, it can receive the user's first input to the target audio identifier, display a target control in an area adjacent to the area where the target audio identifier is located, and then receive the user's second input to the target control to obtain the first audio. This simplifies the user's operation of obtaining the first audio and improves the portability of sending or receiving audio content using the electronic device.

[0066] Step 102: The electronic device acquires the N first audio features of the N first sub-audio segments respectively.

[0067] In this embodiment of the application, N first audio features and N first sub-audio segments correspond one-to-one.

[0068] Optionally, in the embodiments of this application, the first audio feature mentioned above can be any of the following: zero-crossing rate, short-time energy, fundamental frequency, and Mel-scale frequency cepstral coefficients (MFCC).

[0069] Further optionally, in the embodiments of this application, when the first audio feature is MFCC, combined with Figure 1 ,like Figure 5 As shown, step 102 can be implemented through steps 102a to 102d.

[0070] Step 102a: The electronic device determines X consecutive audio frames based on the sixth sub-audio segment.

[0071] In this embodiment of the application, the aforementioned sixth sub-audio segment is any one of the N first sub-audio segments.

[0072] Optionally, in this embodiment of the application, the electronic device can perform algorithmic processing on the sixth sub-audio segment through a high-pass filter to pre-emphasize the sixth sub-audio segment.

[0073] Further, optionally, in this embodiment, the high-pass filter described above can be implemented using a first algorithm: H(Z) = 1 - μz -1 The algorithm processes a first sub-audio segment.

[0074] Where μ is a positive number greater than or equal to 0.9 and less than or equal to 1, for example, μ can be 0.97.

[0075] In this embodiment, the electronic device can pre-emphasize the sixth sub-audio segment to enhance the high-frequency portion of the signal, thereby flattening the signal spectrum. This allows the electronic device to calculate the spectrum using the same signal-to-noise ratio across the entire frequency band from low to high frequencies. Furthermore, by pre-emphasing the sixth sub-audio segment, the electronic device can eliminate the effects of the vocal cords and lips during phonation, compensating for the high-frequency suppression of the sixth sub-audio segment's signal by the user's vocal system, and thus highlighting the high-frequency resonant peaks of the sixth sub-audio segment's signal.

[0076] Optionally, in this embodiment of the application, after the electronic device pre-emphasizes the sixth sub-audio segment, it divides the signal of the sixth sub-audio segment into X audio frames.

[0077] The audio signal processing range is 20 to 40 milliseconds, and can generally be set to a frame size of 25 milliseconds.

[0078] In this embodiment of the application, adjacent audio frames among the X audio frames at least partially overlap in the time domain.

[0079] In order to avoid excessive changes in the audio signal between two adjacent frames, there is a required overlap area between two consecutive adjacent frames. Typically, there is a 50% (+ / -10%) overlap area between consecutive frames, usually set to 15 milliseconds.

[0080] Step 102b: The electronic device determines the target power spectrum based on X audio frames.

[0081] It should be noted that audio changes continuously over a long range and therefore lacks fixed characteristics, making it impossible to process. Therefore, after dividing the signal of the sixth sub-audio segment into X audio frames, a window function needs to be applied to each audio frame.

[0082] In this embodiment, applying a window function to each audio frame in the electronic device can counteract the assumption that the data in the FFT processing is infinite and reduce spectral leakage.

[0083] Optionally, in the embodiments of this application, the above-mentioned window function can be a square window, a Hamming window, or a Hanning window, etc. According to the frequency domain characteristics of the window function, the Hamming window is generally adopted.

[0084] Optionally, in this embodiment of the application, the aforementioned audio frame may specifically be a short-time frame.

[0085] Multiplying each short-time frame by a Hamming window function increases the continuity between the left and right ends of the short-time frame.

[0086] Here, we assume the framed signal is S(n), n = 0, 1, ..., N-1, where N is the frame size. After multiplying by the Hamming window function, S'(n) = S(n) × W(n), and W(n) has the following form:

[0087] Different values ​​of a will produce different Hamming windows. Generally, a can be 0.46.

[0088] It should be noted that since it is difficult to distinguish the characteristics of a signal by its transformation in the time domain, the transformation of a signal in the time domain is usually converted into an energy distribution in the frequency domain for observation. Different energy distributions can represent the characteristics of different audio frequencies.

[0089] In this embodiment, after the audio frame is multiplied by the Hamming window function, each audio frame needs to undergo a fast Fourier transform to obtain the energy distribution on the spectrum.

[0090] Optionally, in this embodiment of the application, the signal of the sixth sub-audio segment is divided into X audio frames and multiplied by a Hamming window function, and then a fast Fourier transform is performed on each frame signal to obtain the spectrum of each frame.

[0091] The target power spectrum of the audio signal is obtained by taking the modulus square of the spectrum of the audio signal by the electronic device. In this formula, the DFT of the audio signal is assumed to be:

[0092] In the formula, x(n) is the input audio signal, and N represents the number of points in the Fourier transform.

[0093] Step 102c: The electronic device obtains at least one logarithmic energy based on the target power spectrum.

[0094] In this embodiment of the application, the electronic device can pass the target power spectrum through a set of Mel-scale filters.

[0095] Optionally, in embodiments of this application, the electronic device can pass the target power spectrum through a set of filters of the Mel-Mel scale, including M filters (the number of filters is similar to the number of critical bands).

[0096] Further optionally, in this embodiment of the application, the filter can be a triangular filter, and the center frequency of the triangular filter can be expressed as f(m).

[0097] Where m is typically taken as 22-26. The intervals between each f(m) decrease as the value of m decreases and widen as the value of m increases, such as... Figure 6 As shown.

[0098] The frequency response of the triangular filter is defined as follows:

[0099] In the formula

[0100] In this embodiment, the triangular filter can smooth the spectrum and eliminate the effect of harmonics, highlighting the original formants of the speech. In this embodiment, the logarithmic energy of the output of each filter bank is calculated as follows:

[0101] Step 102d: The electronic device calculates the first audio feature of the sixth sub-audio segment based on at least one logarithmic energy.

[0102] In this embodiment, the logarithmic energy output by each filter bank is processed by DCT to obtain the MFCC coefficients as follows:

[0103]

[0104] The logarithmic energy is then substituted into the discrete cosine transform to obtain the L-order Mel parameters. L-order refers to the order of the MFCC coefficients, typically taken as 12-16. Here, M is the number of triangular filters.

[0105] In this embodiment of the application, after the above steps, the acoustic feature parameter MFCC of the sixth sub-audio segment is a two-dimensional matrix (T, N), where T is the duration and N is the set feature dimension, which is usually 13 or 39 by default.

[0106] Optionally, in this embodiment of the application, the electronic device extracts N first sub-audio segments of the first audio according to the above-described MFCC feature parameter extraction process to obtain N MFCC feature parameters corresponding to the N first sub-audio segments.

[0107] Thus, it can be seen that since the electronic device can divide a first sub-audio segment into X consecutive audio frames, determine the target power spectrum based on these X audio frames, and then obtain at least one logarithmic energy and calculate the first audio feature based on the target power spectrum, thereby acquiring N first audio features from N first sub-audio segments, the accuracy of the electronic device in acquiring N first audio features is improved.

[0108] Step 103: The electronic device determines N second sub-audio segments based on N first audio features.

[0109] In this embodiment of the application, N second sub-audio segments and N first audio features correspond one-to-one, and each second sub-audio segment is determined based on the corresponding first audio feature.

[0110] Optionally, in this embodiment of the application, each of the N first sub-audio features is compared with the target audio segment library (such as the cloud audio segment library in the following embodiment of the application) to match the second sub-audio segment corresponding to the second audio feature with the highest similarity to the first audio feature.

[0111] Alternatively, in this embodiment, the second audio feature can be obtained by applying a Dynamic Time Warping (DTW) algorithm to at least one MFCC feature corresponding to an audio segment and a first audio feature. The DTW algorithm is described in detail in the following embodiments and will not be repeated here.

[0112] Optionally, in this embodiment of the application, the electronic device can compare the N first audio features with the target audio segment library to obtain N second sub-audio segments that correspond one-to-one with the first audio features.

[0113] Step 104: The electronic device generates a second audio based on N second sub-audio segments.

[0114] In this embodiment of the application, the text information corresponding to each second sub-audio segment is matched with the text information corresponding to a first sub-audio segment.

[0115] For example, a first audio feature corresponds to a first sub-audio segment, and the text information corresponding to the first sub-audio segment is "you". In the target audio segment library, the second audio feature with the highest similarity to the first audio feature corresponds to a second sub-audio segment, and the text information corresponding to the second sub-audio segment is "you". The text information corresponding to each second sub-audio segment matches the text information corresponding to a first sub-audio segment.

[0116] Optionally, in this embodiment of the application, the text information corresponding to the N second sub-audio segments is matched one-to-one with the text information corresponding to the N first sub-audio segments.

[0117] It can be understood that the N second sub-audio segments are concatenated according to the original order of the text information corresponding to the N first sub-audio segments to generate the second audio.

[0118] Of course, the generated second audio is a standard tone. If the standard tone feels too mechanical, making it difficult for the other user to correctly understand the emotional state of the received audio, the electronic device can automatically identify the timbre, pitch, and other information of the user's recorded audio, and synthesize the generated standard tone with the corresponding timbre and pitch, so that the audio received by the other user is closer to the user's tone.

[0119] Optionally, in the embodiments of this application, after step 104 above, the audio processing method provided in the embodiments of this application further includes the following steps 301 to 302.

[0120] Step 301: The electronic device acquires the target audio parameters of the first audio.

[0121] Optionally, in the embodiments of this application, the target audio parameters mentioned above are at least one of the following: audio timbre and audio pitch.

[0122] It is understandable that when an electronic device acquires the first audio, it can automatically identify information such as the timbre and pitch of the recorded audio and extract and save it.

[0123] Step 302: The electronic device adjusts the audio parameters of the second audio according to the target audio parameters.

[0124] It should be noted that the above-mentioned "electronic device adjusts the audio parameters of the second audio according to the target audio parameters" can be understood as: after the electronic device acquires the second audio, it adjusts the audio parameters such as timbre and pitch corresponding to the second audio to the audio parameters such as timbre and pitch corresponding to the first audio that have been saved in advance.

[0125] Thus, it can be seen that since electronic devices can first obtain the target audio parameters of the first audio, and after obtaining the second audio, adjust the audio parameters corresponding to the second audio to the target audio parameters of the first audio, thereby making the timbre and pitch of the second audio closer to the first audio, users can better understand the emotional state of the received audio.

[0126] The audio processing method provided in this application embodiment allows an electronic device to first determine N first sub-audio segments (each first sub-audio segment corresponds to a text information of the first audio) based on the acquired first audio, and then obtain N first audio features corresponding one-to-one with each of the N first sub-audio segments. Based on these N first audio features, the electronic device then obtains N second sub-audio segments (each second sub-audio segment is determined based on its corresponding first audio feature). Thus, the electronic device can generate a second audio based on these N second sub-audio segments (each second sub-audio segment's corresponding text information matches the text information corresponding to a first sub-audio segment). Since the electronic device can first determine the first sub-audio segment corresponding to each text information of the first audio, and then obtain the second sub-audio segment matching the text information corresponding to each first sub-audio segment based on the first audio feature of each first sub-audio segment—that is, a clear sub-audio segment matching the text information corresponding to each first sub-audio segment—to generate a clear second audio, it eliminates the need for the user to perform multiple operations on the electronic device to input clear voice audio. Therefore, it improves the convenience of sending voice audio using the electronic device.

[0127] The following details how the electronic device obtains the second sub-audio segment from the target audio segment library.

[0128] Optionally, in the embodiments of this application, step 103 above can be implemented by steps 103a to 103c below.

[0129] Step 103a: The electronic device determines the N second audio features that have the highest similarity to the N first audio features.

[0130] In this embodiment of the application, N second audio features correspond one-to-one with N first audio features.

[0131] Optionally, in the embodiments of this application, the first audio feature and the second audio feature can specifically be MFCC feature parameters.

[0132] Optionally, in the embodiments of this application, each MFCC feature parameter belongs to time series data, and different MFCC feature parameters can be compared for similarity using the DTW algorithm.

[0133] Further optionally, in the embodiments of this application, the second audio feature can be obtained by using the Dynamic Time Warping (DTW) algorithm to combine the MFCC feature corresponding to at least one audio segment provided by the target audio segment library and a first audio feature.

[0134] It should be noted that the aforementioned "target audio segment library" can be an audio segment library provided by a network server, such as a massive audio segment library on a network cloud, or it can be an audio segment library that comes with the electronic device itself.

[0135] Optionally, in this embodiment of the application, the target audio segment library contains a massive collection of audio segments and records the acoustic MFCC features of each audio segment. Each audio segment also has its mapped content meaning and its language family information.

[0136] For example, such as Figure 7 As shown, the target audio segment library contains n collected audio segments. Each audio segment records its own acoustic MFCC features, and each audio segment has its mapped content meaning and its language family information. For example, audio segment 1: MFCC feature is (1000 / 10,13), content meaning is "you", and language family is "Chinese, Standard"; audio segment 2: MFCC feature is (2000 / 10,13), content meaning is "we", and language family is "Chinese, Henan accent"; audio segment 3: MFCC feature is (3000 / 10,13), content meaning is "good", and language family is "Chinese, Hebei accent", etc.

[0137] Further, optionally, in the embodiments of this application, the above-described DTW algorithm can measure the similarity between two time series of different lengths.

[0138] In this process, the two sequences are stretched or shortened to match the length of the reference template. During this process, the two sequences will be twisted or bent so that their feature quantities correspond to the standard pattern.

[0139] Here, we assume that the two MFCC sequences are Q and C, and that the lengths of the two MFCC sequences are n and m, respectively.

[0140] Q = q1, q2, ..., qn

[0141] C = q1, q2, ..., qm

[0142] Using an m×n matrix to compare two MFCC sequences, the warping path traverses this matrix, and the k-th element of the warping path represents w. k =(i,j) k The horizontal and vertical lines represent the points where the two sequences are aligned.

[0143] Optionally, in this embodiment of the application, the constraints on the warping path include boundary conditions, continuity conditions, and monotonicity conditions.

[0144] Further, optionally, in this embodiment of the application, the boundary conditions are: w1 = (1,1) and w k = (m, n).

[0145] In this case, the two sequences must match at the beginning and end, and the order of each part must also match.

[0146] Further optionally, in this embodiment of the application, the continuity constraint is: if w k = (a, b) and w k If -1 = (a', b'), then a - a' ≥ 1 and b - b' ≤ 1 must be satisfied.

[0147] The continuity condition means that in the many-to-one and one-to-many cases during the matching process, only the case of the surrounding time step can be matched. It is impossible to cross a certain point to match. It can only be aligned with the adjacent point. This can ensure that each coordinate in Q and C appears in the wrapping path.

[0148] Further optionally, in this embodiment of the application, the monotonicity constraint is: if w k -1 = (a', b'), and w k If =(a,b), then a-a'≥0 and b-b'≥0 must be satisfied.

[0149] The monotonicity constraint indicates that the warping path must monotonically increase over time.

[0150] It should be noted that there are many warping paths that satisfy the above constraints, so finding the second audio feature with the highest similarity to a first audio feature is essentially finding the optimal warping path.

[0151] This can be described in mathematical terms as follows:

[0152] γ(i,j)=d(q i ,c j )+min{γ(i-1,j-1),γ(i-1,j),γ(i,j-1)}.

[0153] The formula is applied to calculate the minimum warping path after matching the MFCC features of two audio segments. Similarly, by comparing each pair of matching pairs, the minimum warping paths of a first audio feature and the corresponding audio features of all audio segments in the audio segment library can be obtained, namely P1, P2, P3, P4, P5, P6...Pn. The smallest path among the matched minimum warping paths is found, min(P1,P2,P3,P4,...,Pn), thus identifying the audio feature with the highest matching probability, which is determined as a second audio feature. From this, the audio segment corresponding to the second audio feature, as well as the semantic information and language family information mapped within that audio segment, can be obtained.

[0154] For example, such as Figure 8 As shown, the MFCC feature corresponding to a first audio feature is (1200 / 10, 13). Using the DTW algorithm, the audio feature with the highest similarity to the first audio feature is obtained from the audio segment library as the second audio feature. The second audio feature (1000 / 10, 13) means "you" and belongs to the language family "Chinese, Henan".

[0155] Step 103b: The electronic device determines N target audio sets based on N second audio features.

[0156] In this embodiment of the application, N target audio sets correspond one-to-one with N second audio features, and a target audio set includes at least one third sub-audio segment.

[0157] Optionally, in the embodiments of this application, a second audio feature corresponds to the acoustic MFCC feature of an audio segment, and at least one third sub-audio segment can be obtained according to the content meaning mapped by the audio segment. The content meaning of the at least one third sub-audio segment is the same, but the language information to which they belong is different.

[0158] Optionally, in embodiments of this application, the at least one third sub-audio segment constitutes a target audio set.

[0159] For example, the target audio set includes: third sub-audio segment 1: "You, standard Chinese"; third sub-audio segment 2: "You, Henan accent Chinese"; third sub-audio segment 3: "You, Shaanxi accent Chinese"; third sub-audio segment 4: "You, standard Japanese"; third sub-audio segment 5: "You, standard English", etc.

[0160] Optionally, in the embodiments of this application, step 103b can be implemented by the following steps 103b1 and 103b2.

[0161] Step 103b1: The electronic device uses Q first correspondences to determine the target text information corresponding to the third audio feature.

[0162] In this embodiment of the application, each first correspondence is a correspondence between an audio feature and a text information, the third audio feature is any one of the N second audio features, and Q is a positive integer.

[0163] Optionally, in this embodiment of the application, the target text information can be text information represented by an audio segment corresponding to one of the N second audio features.

[0164] For example, combined with Figure 8 As shown, the first correspondence between audio features and text information in the target audio segment library is as follows: Audio segment 1: MFCC feature is (1200 / 10, 13), meaning "you", belonging to the language family "Chinese, Standard"; Audio segment 2: MFCC feature is (2000 / 10, 13), meaning "we", belonging to the language family "Chinese, Henan accent"; Audio segment 3: MFCC feature is (3000 / 10, 13), meaning "good", belonging to the language family "Chinese, Hebei accent", etc. The MFCC feature corresponding to a first audio feature is (1200 / 10, 13). After comparison with the target audio segment library using the DTW algorithm, the third audio feature with the highest similarity to this first audio feature is (1000 / 10, 13), meaning "you".

[0165] Step 103b2: The electronic device determines at least one third sub-audio segment that matches the target text information from at least one pre-stored sub-audio segment, thereby obtaining a first audio set.

[0166] In this embodiment of the application, the first audio set is the target audio set corresponding to the third audio feature in the N target audio sets.

[0167] Optionally, in this embodiment of the application, the aforementioned at least one pre-stored sub-audio segment can specifically be an audio segment library provided by the network.

[0168] For example, the electronic device compares and determines, from a target audio segment library provided by the network, at least one third sub-audio segment that matches the target text information "you" but belongs to a different language family, forming a first audio set: Third sub-audio segment 1: "you, standard Chinese"; Third sub-audio segment 2: "you, Henan accent Chinese"; Third sub-audio segment 3: "you, Shaanxi accent Chinese"; Third sub-audio segment 4: "you, standard Japanese"; Third sub-audio segment 5: "you, standard English", etc.

[0169] Thus, the electronic device uses the correspondence between Q audio features and one text information to determine the target text information corresponding to any one of the N second audio features, and determines at least one third sub-audio segment that matches the target text information from at least one pre-stored sub-audio segment, thereby accurately obtaining the first audio set. Therefore, the accuracy of the electronic device in obtaining the first audio set is improved.

[0170] Step 103c: The electronic device determines N second sub-audio segments from the third sub-audio segments of the N target audio sets.

[0171] In this embodiment of the application, the N second sub-audio segments correspond one-to-one with the N target audio sets.

[0172] Optionally, in this embodiment of the application, the electronic device determines N second audio features that have the highest similarity to N first audio features, and determines N target audio sets based on the N second audio features, thereby determining N second sub-audio segments from the third sub-audio segments of the N target audio sets.

[0173] Thus, since the electronic device can determine the N second audio features with the highest similarity to the N first audio features, and determine the N target audio sets based on the N second audio features, and thus determine the N second sub-audio segments from the third sub-audio segments of the N target audio sets, the electronic device can accurately match the second sub-audio segments.

[0174] Optionally, in this embodiment of the application, before step 103c, the audio processing method further includes step 401, and step 103c can be implemented by steps 103c1 and 103c2.

[0175] Step 401: The electronic device uses T second correspondences to determine the target language information corresponding to the fourth audio feature.

[0176] In this embodiment of the application, each second correspondence is a correspondence between an audio feature and a language information, and the fourth audio feature is any one of the N second audio features.

[0177] Optionally, in this embodiment of the application, the target language information mentioned above can specifically be "Chinese".

[0178] For example, such as Figure 7 As shown, the second correspondence between audio features and language family information in the target audio segment library is as follows: Audio segment 1: MFCC feature is (1200 / 10,13), meaning "you", and belongs to the language family "Chinese, Henan"; Audio segment 2: MFCC feature is (2000 / 10,13), meaning "we", and belongs to the language family "Chinese, Henan accent"; Audio segment 3: MFCC feature is (3000 / 10,13), meaning "good", and belongs to the language family "Chinese, Hebei accent"; Audio segment 4: MFCC feature is (1500 / 10,13), meaning "ah", and belongs to the language family "Japanese, Standard", etc. The MFCC feature corresponding to a first audio feature is (1200 / 10,13). After comparison with the target audio segment library using the DTW algorithm, the fourth audio feature with the highest similarity to the first audio feature is (1000 / 10,13), which means "you" and its corresponding target language information is "Chinese".

[0179] Step 103c1: The electronic device determines M fourth sub-audio segments from the third sub-audio segments of the second audio set that match the target language information.

[0180] In this embodiment of the application, the second audio set is the target audio set corresponding to the fourth audio feature in the N target audio sets, where M is a positive integer.

[0181] For example, such as Figure 9As shown, in the target audio segment library provided by the network, the second audio set corresponding to the fourth audio feature is the audio set containing the target text information "you" and the target language information "Chinese". The second audio set consists of at least one fourth sub-audio segment, for example: audio segment 1: MFCC feature (1000 / 10,13), meaning "you", belonging to the language family "Chinese, Standard"; audio segment 2: MFCC feature (2500 / 10,13), meaning "you", belonging to the language family "Chinese, Henan accent"; audio segment 3: MFCC feature (2800 / 10,13), meaning "you", belonging to the language family "Chinese, Shaanxi accent"; audio segment 4: MFCC feature (1500 / 10,13), meaning "you", belonging to the language family "Chinese, Hebei accent"; audio segment m: MFCC feature (1800 / 10,13), meaning "you", belonging to the language family "Chinese, Shanxi accent", etc.

[0182] Step 103c2: The electronic device determines the fifth sub-audio segment that matches the predetermined language from the M fourth sub-audio segments.

[0183] In this embodiment of the application, the fifth sub-audio segment is the second sub-audio segment that corresponds to the second audio set among the N second sub-audio segments.

[0184] Optionally, in this embodiment of the application, the aforementioned predetermined language can specifically be "Chinese, Standard".

[0185] For example, combined with Figure 9 ,from Figure 9 The second audio set shown identifies the fifth sub-audio segment with the language "Chinese, Standard" (e.g., audio segment 1: MFCC feature is (1000 / 10,13), the content means "you", and the language family is "Chinese, Standard").

[0186] Thus, it can be seen that because the electronic device identifies M fourth sub-audio segments from the third sub-audio segments of the second audio set that match the target language information, and then identifies a fifth sub-audio segment from the M fourth sub-audio segments that matches the predetermined language, the accuracy of the electronic device in obtaining the first audio set is improved.

[0187] The audio processing method provided in this application can be executed by an audio processing device. This application uses an audio processing device to perform the audio processing method as an example to illustrate the audio processing device provided in this application.

[0188] Figure 10 A possible structural schematic diagram of an audio processing device involved in the above embodiments is shown. Figure 10As shown in the embodiment of this application, the audio processing apparatus 60 includes: a determining module 61, an acquiring module 62, and a generating module 63. The determining module 61 is configured to determine N first sub-audio segments based on the acquired first audio, each first sub-audio segment corresponding to a text message of the first audio, where N is a positive integer. The acquiring module 62 is configured to acquire N first audio features of the N first sub-audio segments determined by the determining module, where each of the N first audio features corresponds one-to-one with the N first sub-audio segments. The determining module is further configured to determine N second sub-audio segments based on the N first audio features acquired by the acquiring module, where each of the N second sub-audio segments corresponds one-to-one with the N first audio features, and each second sub-audio segment is determined based on a corresponding first audio feature. The generating module 63 is configured to generate second audio based on the N second sub-audio segments determined by the determining module. The text message corresponding to each second sub-audio segment is matched with the text message corresponding to a first sub-audio segment.

[0189] In one possible implementation, the aforementioned determining module is specifically used to determine N second audio features that have the highest similarity to N first audio features, wherein the N second audio features correspond one-to-one with the N first audio features, and to determine N target audio sets based on the N second audio features, wherein the N target audio sets correspond one-to-one with the N second audio features, and each target audio set includes at least one third sub-audio segment, and to determine N second sub-audio segments from the third sub-audio segments of the N target audio sets, wherein the N second sub-audio segments correspond one-to-one with the N target audio sets.

[0190] In one possible implementation, the aforementioned determining module is further specifically used to use Q first correspondences to determine the target text information corresponding to the third audio feature, and to determine at least one third sub-audio segment that matches the target text information from at least one pre-stored sub-audio segment to obtain a first audio set.

[0191] In this context, each first correspondence is a correspondence between an audio feature and a text information, the third audio feature is any one of the N second audio features, the first audio set is the target audio set in the N target audio sets that corresponds to the third audio feature, and Q is a positive integer.

[0192] In one possible implementation, the determining module is further configured to use T second correspondences to determine the target language information corresponding to the fourth audio feature. Each second correspondence is a correspondence between an audio feature and a language information, and the fourth audio feature is any one of the N second audio features. Specifically, the determining module is further configured to determine M fourth sub-audio segments matching the target language information from the third sub-audio segments of the second audio set, where M is a positive integer, and to determine a fifth sub-audio segment matching a predetermined language from the M fourth sub-audio segments. Here, the second audio set is the target audio set corresponding to the fourth audio feature in the N target audio sets, and the fifth sub-audio segment is the second sub-audio segment from the N second sub-audio segments that corresponds to the second audio set.

[0193] In one possible implementation, the acquisition module includes a determination submodule and a calculation submodule.

[0194] The module comprises several sub-modules: a determination sub-module, which determines X consecutive audio frames based on the sixth sub-audio segment, wherein adjacent audio frames at least partially overlap in the time domain; a target power spectrum based on the X audio frames; and at least one logarithmic energy based on the target power spectrum determined by the determination sub-module. A calculation sub-module is used to calculate a first audio feature of the sixth sub-audio segment based on the at least one logarithmic energy obtained by the determination sub-module. The sixth sub-audio segment is any one of N first sub-audio segments, and X is a positive integer.

[0195] In one possible implementation, the audio processing device further includes an adjustment module. The acquisition module is further configured to acquire target audio parameters of the first audio. The adjustment module is configured to adjust the audio parameters of the second audio based on the target audio parameters acquired by the acquisition module.

[0196] The audio processing device provided in this application embodiment can first determine the first sub-audio segment corresponding to each text information of the first audio, and then obtain the second sub-audio segment that matches the text information corresponding to each first sub-audio segment, i.e., the clear sub-audio segment that matches the text information corresponding to each first sub-audio segment, based on the first audio feature of each first sub-audio segment, so as to generate a clear second audio, without requiring the user to perform multiple operations on the audio processing device to input clear voice audio. Therefore, it can improve the convenience of sending voice audio using the audio processing device.

[0197] The audio processing device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television set (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the scope of the device.

[0198] The audio processing device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit the specific operating system used.

[0199] The audio processing device provided in this application embodiment can achieve... Figures 1 to 9 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0200] Optionally, such as Figure 11 As shown, this application embodiment also provides an electronic device 80, including a processor 81 and a memory 82. The memory 82 stores a program or instructions that can run on the processor 81. When the program or instructions are executed by the processor 81, they implement the various process steps of the above-described audio processing method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0201] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0202] Figure 12 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0203] The electronic device 100 includes, but is not limited to, components such as: radio frequency unit 101, network module 102, audio output unit 103, input unit 104, sensor 105, display unit 106, user input unit 107, interface unit 108, memory 109, and processor 110.

[0204] Those skilled in the art will understand that the electronic device 100 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 110 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 12 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0205] The processor 110 is configured to: determine N first sub-audio segments based on the acquired first audio, where each first sub-audio segment corresponds to a text information of the first audio, and N is a positive integer; acquire N first audio features for each of the N first sub-audio segments, where each of the N first audio features corresponds one-to-one with the N first sub-audio segments; acquire N second sub-audio segments based on the N first audio features, where each of the N second sub-audio segments corresponds one-to-one with the N first audio features, and each second sub-audio segment is determined based on a corresponding first audio feature; and generate a second audio based on the N second sub-audio segments; wherein the text information corresponding to each second sub-audio segment is matched with the text information corresponding to a first sub-audio segment.

[0206] The electronic device provided in this application embodiment can first determine the first sub-audio segment corresponding to each text information of the first audio, and then obtain the second sub-audio segment that matches the text information corresponding to each first sub-audio segment, i.e., the clear sub-audio segment that matches the text information corresponding to each first sub-audio segment, based on the first audio feature of each first sub-audio segment, so as to generate a clear second audio, without requiring the user to perform multiple operations on the electronic device to input clear voice audio. Therefore, it can improve the convenience of sending voice audio using the electronic device.

[0207] Optionally, in this embodiment of the application, the processor 110 is specifically configured to: determine N second audio features that have the highest similarity to N first audio features, wherein the N second audio features correspond one-to-one with the N first audio features; determine N target audio sets based on the N second audio features, wherein the N target audio sets correspond one-to-one with the N second audio features, and each target audio set includes at least one third sub-audio segment; and determine N second sub-audio segments from the third sub-audio segments of the N target audio sets, wherein the N second sub-audio segments correspond one-to-one with the N target audio sets.

[0208] Thus, since the electronic device can determine the N second audio features with the highest similarity to the N first audio features, and determine the N target audio sets based on the N second audio features, and thus determine the N second sub-audio segments from the third sub-audio segments of the N target audio sets, the electronic device can accurately match the second sub-audio segments.

[0209] Optionally, in this embodiment of the application, the processor 110 is specifically configured to use Q first correspondences to determine the target text information corresponding to the third audio feature; determine at least one third sub-audio segment that matches the target text information from at least one pre-stored sub-audio segment to obtain a first audio set; wherein each first correspondence is a correspondence between an audio feature and a text information; the third audio feature is any one of N second audio features; the first audio set is the target audio set corresponding to the third audio feature in the N target audio sets; and Q is a positive integer.

[0210] Thus, it can be seen that since the electronic device uses the correspondence between Q audio features and one text information to determine the target text information corresponding to any one of the N second audio features, and determines at least one third sub-audio segment that matches the target text information from at least one pre-stored sub-audio segment, the first audio set is accurately obtained. Therefore, the accuracy of the electronic device in obtaining the first audio set is improved.

[0211] Optionally, in this embodiment, the processor 110 is specifically configured to use T second correspondences to determine the target language information corresponding to the fourth audio feature, where each second correspondence is a correspondence between an audio feature and a language information, and the fourth audio feature is any one of the N second audio features; determine M fourth sub-audio segments that match the target language information from the third sub-audio segments of the second audio set, where M is a positive integer; and determine a fifth sub-audio segment that matches a predetermined language from the M fourth sub-audio segments; wherein the second audio set is the target audio set corresponding to the fourth audio feature in the N target audio set; and the fifth sub-audio segment is the second sub-audio segment corresponding to the second audio set from the N second sub-audio segments.

[0212] Thus, it can be seen that because the electronic device identifies M fourth sub-audio segments from the third sub-audio segments of the second audio set that match the target language information, and then identifies a fifth sub-audio segment from the M fourth sub-audio segments that matches the predetermined language, the accuracy of the electronic device in obtaining the first audio set is improved.

[0213] Optionally, in this embodiment of the application, the processor 110 is specifically configured to: determine X consecutive audio frames based on the sixth sub-audio segment, wherein adjacent audio frames in the X audio frames at least partially overlap in the time domain; determine a target power spectrum based on the X audio frames; obtain at least one logarithmic energy based on the target power spectrum; and calculate a first audio feature of the sixth sub-audio segment based on the at least one logarithmic energy; wherein the sixth sub-audio segment is any one of N first sub-audio segments; and X is a positive integer.

[0214] Thus, it can be seen that since the electronic device can divide a first sub-audio segment into X consecutive audio frames, determine the target power spectrum based on these X audio frames, and then obtain at least one logarithmic energy and calculate the first audio feature based on the target power spectrum, thereby acquiring N first audio features from N first sub-audio segments, the accuracy of the electronic device in acquiring N first audio features is improved.

[0215] Optionally, in this embodiment of the application, the processor 110 is specifically used to obtain the target audio parameters of the first audio; and adjust the audio parameters of the second audio according to the target audio parameters.

[0216] Thus, it can be seen that since electronic devices can first obtain the target audio parameters of the first audio, and after obtaining the second audio, adjust the audio parameters corresponding to the second audio to the target audio parameters of the first audio, thereby making the timbre and pitch of the second audio closer to the first audio, users can better understand the emotional state of the received audio.

[0217] It should be understood that, in this embodiment, the input unit 104 may include a graphics processing unit (GPU) 1041 and a microphone 1042. The GPU 1041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 106 may include a display panel 1061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 107 includes at least one of a touch panel 1071 and other input devices 1072. The touch panel 1071 is also called a touch screen. The touch panel 1071 may include a touch detection device and a touch controller. Other input devices 1072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0218] The memory 109 can be used to store software programs and various data. The memory 109 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 109 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus RAM (DRRAM). The memory 109 in the embodiments of this application includes, but is not limited to, these and any other suitable types of memory.

[0219] Processor 110 may include one or more processing units; optionally, processor 110 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 110.

[0220] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described audio processing method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0221] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0222] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described audio processing method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0223] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0224] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the audio processing method embodiments described above, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0225] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0226] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0227] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An audio processing method, characterized by, The method comprises: According to the obtained first audio, determine N first sub-audio segments, each first sub-audio segment corresponds to a text information of the first audio respectively, N is a positive integer; Respectively acquire N first audio features of the N first sub-audio segments, the N first audio features and the N first sub-audio segments correspond one by one; Respectively determine N second audio features with the maximum similarity with the N first audio features, the N second audio features correspond one by one with the N first audio features; According to the N second audio features, respectively determine N target audio sets, the N target audio sets correspond one by one with the N second audio features, a target audio set includes at least one third sub-audio segment, the content of the at least one third sub-audio segment is the same, and the belonging language system information is different; From the third sub-audio segments of the N target audio sets, respectively determine N second sub-audio segments, the N second sub-audio segments correspond one by one with the N target audio sets; According to the N second sub-audio segments, generate a second audio; Wherein, the text information corresponding to each second sub-audio segment respectively matches the text information corresponding to a first sub-audio segment; each second sub-audio segment is a target audio set corresponding to a second audio feature, and the third sub-audio segment in the target audio set matches the predetermined language and the language system information corresponding to the second audio feature.

2. The method of claim 1, wherein, The method comprises: Determine the target text information corresponding to the third audio feature by using Q first corresponding relationships; From at least one pre-stored sub-audio segment, determine at least one third sub-audio segment matching the target text information to obtain a first audio set; Wherein, each first corresponding relationship is a corresponding relationship between an audio feature and a text information; the third audio feature is any one of the N second audio features; the first audio set is a target audio set in the N target audio sets corresponding to the third audio feature; Q is a positive integer.

3. The method of claim 1, wherein, Before the method comprises: Determine the target language system information corresponding to the fourth audio feature by using T second corresponding relationships, each second corresponding relationship is a corresponding relationship between an audio feature and a language system information, and the fourth audio feature is any one of the N second audio features; The method comprises: From the third sub-audio segments of the second audio set, determine M fourth sub-audio segments matching the target language system information, M is a positive integer; From the M fourth sub-audio segments, determine a fifth sub-audio segment matching the predetermined language; Wherein, the second audio set is a target audio set in the N target audio sets corresponding to the fourth audio feature; the fifth sub-audio segment is a second sub-audio segment in the N second sub-audio segments corresponding to the second audio set.

4. The method of claim 1, wherein, The N first audio features of the N first sub-audio segments are respectively obtained, including: determining, according to the sixth sub-audio segment, continuous X audio frames, adjacent audio frames in the X audio frames at least partially overlap in the time domain; determining a target power spectrum according to the X audio frames; obtaining at least one logarithmic energy based on the target power spectrum; calculating a first audio feature of the sixth sub-audio segment according to the at least one logarithmic energy; wherein the sixth sub-audio segment is any one of the N first sub-audio segments; and X is a positive integer.

5. The method of claim 1, wherein, After the second audio is generated according to the N second sub-audio segments, the method further includes: obtaining a target audio parameter of the first audio; adjusting an audio parameter of the second audio according to the target audio parameter.

6. An audio processing apparatus, characterized by comprising: The audio processing apparatus includes a determination module, an obtaining module, and a generation module. The determination module is configured to determine N first sub-audio segments from the obtained first audio, each first sub-audio segment corresponding to a text information of the first audio, and N being a positive integer. The obtaining module is configured to obtain N first audio features of the N first sub-audio segments determined by the determination module respectively, the N first audio features and the N first sub-audio segments corresponding one by one. The determination module is further configured to: determine N second audio features with the greatest similarity to the N first audio features respectively, the N second audio features corresponding to the N first audio features one by one; determine N target audio sets from the N second audio features respectively, the N target audio sets corresponding to the N second audio features one by one, one target audio set including at least one third sub-audio segment, the content of the at least one third sub-audio segment being the same, and the belonging language family information being different; and determine N second sub-audio segments from the third sub-audio segments of the N target audio sets respectively, the N second sub-audio segments corresponding to the N target audio sets one by one. The generation module is configured to generate a second audio according to the N second sub-audio segments determined by the determination module. Each text information corresponding to each second sub-audio segment matches the text information corresponding to one first sub-audio segment respectively; and each second sub-audio segment is a third sub-audio segment in a target audio set corresponding to one second audio feature, which matches a predetermined language and the language family information corresponding to the one second audio feature.

7. The audio processing apparatus of claim 6, wherein, The determination module is specifically configured to determine target text information corresponding to a third audio feature by using Q first corresponding relationships; and determine at least one third sub-audio segment matching the target text information from at least one pre-stored sub-audio segment to obtain a first audio set. Each first corresponding relationship is a corresponding relationship between one audio feature and one text information; the third audio feature is any one of the N second audio features; the first audio set is a target audio set in the N target audio sets corresponding to the third audio feature; and Q is a positive integer.

8. The audio processing apparatus of claim 6, wherein The determining module is further configured to determine target language information corresponding to a fourth audio feature by using T second corresponding relationships, each of which is a corresponding relationship between an audio feature and language information, the fourth audio feature being any one of the N second audio features. The determining module is further configured to determine M fourth sub-audio segments matching the target language information from a third sub-audio segment of the second audio set, M being a positive integer, and determine a fifth sub-audio segment matching a predetermined language from the M fourth sub-audio segments. The second audio set is a target audio set corresponding to the fourth audio feature in the N target audio sets, and the fifth sub-audio segment is a second sub-audio segment corresponding to the second audio set in the N second sub-audio segments.

9. The audio processing apparatus of claim 6, wherein, The obtaining module comprises a determining sub-module and a calculating sub-module. The determining sub-module is configured to determine X continuous audio frames from the sixth sub-audio segment, adjacent audio frames in the X audio frames at least partially overlapping in time domain, determine a target power spectrum from the X audio frames, and obtain at least one logarithmic energy based on the target power spectrum determined by the determining sub-module. The calculating sub-module is configured to calculate a first audio feature of the sixth sub-audio segment based on the at least one logarithmic energy obtained by the determining sub-module. The sixth sub-audio segment is any one of the N first sub-audio segments, and X is a positive integer.

10. The audio processing apparatus of claim 6, wherein, The audio processing apparatus further comprises an adjusting module. The obtaining module is further configured to obtain a target audio parameter of the first audio. The adjusting module is configured to adjust an audio parameter of the second audio based on the target audio parameter obtained by the obtaining module.

11. An electronic device, comprising: The apparatus comprises a processor, a memory, and a program or instructions stored in the memory and executable on the processor, and the program or instructions are executed by the processor to implement the steps of the audio processing method according to any one of claims 1 to 5.

12. A readable storage medium, characterized by, The readable storage medium stores a program or instructions, and the program or instructions are executed by the processor to implement the steps of the audio processing method according to any one of claims 1 to 5.

Images