Audio data processing method and device, electronic equipment, storage medium and program product

Abstract

The embodiment of the invention provides an audio data processing method and device, electronic equipment, a storage medium and a program product. The method comprises the following steps: acquiring to-be-processed first audio data, and determining a first audio duration of the first audio data; in response to the fact that the first audio duration is smaller than or equal to a target processing duration threshold value of a target audio processing model, the first audio data are prolonged, prolonged second audio data are obtained, and the second audio duration of the second audio data is larger than the target processing duration threshold value; inputting the second audio data into the target audio processing model, and obtaining third audio data output by the target audio processing model; and generating a processing result of the first audio data based on the third audio data. According to the technical scheme, the processing effect of the audio processing model on the audio data and the accuracy of the processing result of the audio data can be improved.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of computer technology, and in particular, to a method, device, electronic device, storage medium, and program product for processing audio data. Background Art

[0002] At present, audio data can be processed by an audio processing model, such as performing music source separation (MSS) on audio data by an audio processing model based on machine learning. However, in the related art, the audio processing model has a poor processing effect on audio data. Summary of the invention

[0003] The embodiments of the present disclosure provide a method, device, electronic device, storage medium and program product for processing audio data to improve the processing effect of an audio processing model on audio data.

[0004] In a first aspect, an embodiment of the present disclosure provides a method for processing audio data, including:

[0005] Acquire first audio data to be processed, and determine a first audio duration of the first audio data;

[0006] In response to the first audio duration being less than or equal to a target processing duration threshold of a target audio processing model, extending the first audio data to obtain extended second audio data, wherein a second audio duration of the second audio data is greater than the target processing duration threshold;

[0007] Inputting the second audio data into the target audio processing model, and obtaining third audio data output by the target audio processing model;

[0008] A processing result of the first audio data is generated based on the third audio data.

[0009] In a second aspect, the present disclosure also provides an audio data processing device, including:

[0010] An audio acquisition module, used to acquire first audio data to be processed and determine a first audio duration of the first audio data;

[0011] an audio extension module, configured to, in response to the first audio duration being less than or equal to a target processing duration threshold of a target audio processing model, extend the first audio data to obtain extended second audio data, wherein the second audio duration of the second audio data is greater than the target processing duration threshold;

[0012] an audio processing module, configured to input the second audio data into the target audio processing model, and obtain third audio data output by the target audio processing model;

[0013] A result generating module is used to generate a processing result of the first audio data based on the third audio data.

[0014] In a third aspect, an embodiment of the present disclosure further provides an electronic device, including:

[0015] one or more processors;

[0016] a memory for storing one or more programs,

[0017] When the one or more programs are executed by the one or more processors, the one or more processors implement the audio data processing method as described in the embodiment of the present disclosure.

[0018] In a fourth aspect, the embodiments of the present disclosure further provide a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the method for processing audio data as described in the embodiments of the present disclosure is implemented.

[0019] In a fifth aspect, the embodiments of the present disclosure further provide a computer program product. When the computer program product is executed by a computer, the computer implements the method for processing audio data as described in the embodiments of the present disclosure.

[0020] The audio data processing method, apparatus, electronic device, storage medium and program product provided by the embodiments of the present disclosure extend the audio data to be processed to a duration greater than the processing duration threshold of the audio processing model when the duration of the audio data to be processed is less than or equal to the processing duration threshold of the audio processing model, and generate a processing result of the audio data to be processed based on the audio data obtained by processing the extended audio data by the audio processing model. This can avoid the situation where the audio processing model outputs silent audio data due to insufficient audio information when performing audio processing, thereby improving the processing effect of the audio processing model on the audio data and the accuracy of the processing result of the audio data. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0022] Figure 1 A flowchart of a method for processing audio data provided by an embodiment of the present disclosure;

[0023] Figure 2 A flowchart of another method for processing audio data provided by an embodiment of the present disclosure;

[0024] Figure 3 A schematic diagram of a process for determining a processing time threshold provided in an embodiment of the present disclosure;

[0025] Figure 4 A schematic diagram of an audio data processing process provided by an embodiment of the present disclosure;

[0026] Figure 5 A structural block diagram of an audio data processing device provided by an embodiment of the present disclosure;

[0027] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0029] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0030] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0031] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0032] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0033] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0034] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0035] Figure 1 A flowchart of a method for processing audio data provided in an embodiment of the present disclosure. The method can be executed by an audio data processing device, wherein the device can be implemented by software and / or hardware and can be configured in an electronic device, typically, a computer, a mobile phone or a tablet computer. The method for processing audio data provided in an embodiment of the present disclosure is suitable for scenarios where audio data is processed using an audio processing model, such as scenarios where audio data is separated from the source using a sound source separation model. Figure 1 As shown, the audio data processing method provided in this embodiment may include:

[0036] S101. Acquire first audio data to be processed, and determine a first audio duration of the first audio data.

[0037] Among them, the first audio data can be understood as the audio data currently to be processed. Exemplarily, the first audio data can be the audio data currently to be processed based on the audio processing model. The audio processing method corresponding to the audio data currently to be processed is not limited. For example, the audio data currently to be processed can be the audio data currently to be processed for sound source separation, special effects to be added, sound conversion to be performed, speed change and / or pitch change, etc. The first audio duration can be understood as the duration of the first audio data.

[0038] Specifically, the first audio data to be processed can be obtained, and the first audio duration of the first audio data can be determined based on the data information of the first audio data; or, the first audio data can be detected to determine the first audio duration of the first audio data.

[0039] S102. In response to the first audio duration being less than or equal to a target processing duration threshold of a target audio processing model, extending the first audio data to obtain extended second audio data, wherein a second audio duration of the second audio data is greater than the target processing duration threshold.

[0040] Among them, the target audio processing model can be understood as the audio processing model currently used to process the first audio data. It can be understood that when the first audio data is currently to be processed differently, the target audio processing models used may be the same or different. Exemplarily, in a sound source separation scenario, the target audio processing model may be a sound source separation model; in a special effect adding scenario, the target audio processing model may be a special effect adding model; in a sound conversion scenario, the target audio processing model may be a sound conversion model; in an audio speed change scenario, the target audio processing model may be an audio speed change model; in an audio pitch change scenario, the target audio processing model may be an audio pitch change model, and so on.

[0041] Taking the sound source separation scenario as an example, optionally, the target audio processing model includes a sound source separation model. The sound source separation model can be a model for performing sound source separation on the input audio data. Sound source separation (Music Source Separation, MSS) refers to the process of separating signals from different sound sources from a mixed audio signal. Sound source separation aims to separate the various sound source components (such as human voice, accompaniment and / or musical instruments, etc.) in the mixed audio from each other, so that each sound source can be extracted and processed separately. Exemplarily, the characteristic patterns of different sound sources can be identified and distinguished by using various algorithms and techniques to analyze the characteristics of the mixed audio, such as spectrum, time-frequency characteristics, etc., to achieve sound source separation. The type of sound source separation model is not limited. For example, the sound source separation model can include but is not limited to a vocal accompaniment separation model, a de-harmony model and / or a de-reverberation model, etc. This vocal accompaniment separation model can be used to separate the accompaniment and basic vocals in the audio data. This de-harmony model can be used to separate the harmony and disharmony in the audio data. This de-reverberation model can be used to remove the reverberation in the audio data.

[0042] In some examples, the audio processing model can be a model that performs audio processing based on machine learning, which can be pre-trained, and the training method of the audio processing model is not limited. Each audio processing model can be used alone or in series for different audio processing task types. When used in series, the input of the current audio processing model can be determined based on the output of the previous audio processing model. For example, it can be determined whether the audio data output by the previous audio processing model needs to be extended based on whether the duration of the audio data input by the previous audio processing model is less than or equal to the processing duration threshold corresponding to the current audio processing model, and the processing result of the first audio data to be processed can be generated based on the audio data output by the last audio processing model used in series.

[0043] The target processing duration threshold can be understood as the processing duration threshold of the target audio processing model. The processing duration threshold of an audio processing model (including a target audio processing model) can be understood as the dividing point between the audio durations that the model can stably process and the audio durations that it cannot stably process. In some audio processing scenarios, the processing effects of certain audio processing models on audio data with shorter durations are unstable. For example, when processing audio data with shorter durations, the processed audio data may be silent due to insufficient audio information. The processing duration threshold of an audio processing model can be used to indicate the maximum duration that the processed audio data may be silent due to insufficient audio information, or the minimum duration that the processed audio data will not be silent due to insufficient audio information.

[0044] The second audio data can be understood as audio data obtained by extending the first audio data. The second audio duration can be understood as the duration of the second audio data. The second audio duration is greater than the target processing duration threshold of the target audio processing model.

[0045] Exemplarily, after determining the first audio duration of the first audio data, it can be determined whether the first audio duration is less than or equal to the target processing duration threshold of the target audio processing model to be currently adopted. If the first audio duration is less than or equal to the target processing duration threshold, it means that the duration of the first audio data is relatively short, and the target audio processing model may process the first audio data in a manner that causes the processed audio data to be silent due to insufficient audio information. At this point, the first audio data can be extended to be the second audio data whose audio duration is greater than the target processing duration threshold, so as to obtain the processing result of the first audio data based on the processing of the extended second audio data.

[0046] There is no limit to the way the first audio data can be extended. For example, one or more extended audio segments can be pre-set for extending the audio data to be processed, and different extended audio segments can have the same or different durations. Thus, when the first audio data is extended, based on the difference between the target processing duration threshold and the first audio duration, a segment of extended audio with a duration greater than the difference can be obtained, and it can be spliced ​​to the starting position or the end position of the first audio data to obtain the extended second audio data; or based on the difference between the target processing duration threshold and the first audio duration, one or more extended audio segments can be obtained, and they can be spliced ​​before or after the first audio data in turn to obtain the second audio data with an audio duration greater than the target processing duration threshold, and so on.

[0047] In addition, if the first audio duration of the first audio data is greater than the target processing duration threshold of the target audio processing model, it means that the first audio data can provide sufficient audio information for the target audio processing model. In other words, when the target audio processing model processes the first audio data, the processed audio data will not be silent due to insufficient audio information. At this time, the first audio data can be input into the target audio processing model, and the processing result of the target audio processing model can be obtained as the processing result of the first audio data, without extending the processing of the first audio data.

[0048] S103: Input the second audio data into the target audio processing model, and obtain third audio data output by the target audio processing model.

[0049] Among them, the third audio data can be understood as audio data obtained by processing the second audio data by the target audio processing model, such as audio data obtained after performing sound source separation, feature addition, sound conversion, audio speed change and / or audio pitch change on the second audio processing model.

[0050] Specifically, after extending the first audio data to obtain the second audio data, the second audio data can be input into the target audio processing model, the second audio data can be processed by the target audio processing model, and the third audio data obtained by processing the second audio data output by the target audio processing model can be obtained.

[0051] S104: Generate a processing result of the first audio data based on the third audio data.

[0052] Specifically, the processing result of the first audio data can be generated based on the processed third audio data, such as dividing the third audio data based on the first audio duration of the first audio data to obtain an audio data segment in the third audio data corresponding to the first audio data as the processing result of the first audio data.

[0053] In some examples, considering that the audio duration of the second audio data is greater than the target processing duration threshold of the target audio processing model, after processing the second audio data to obtain the third audio data, energy detection may not be performed on the third audio data, and the processing result of the first audio data is generated based on the third audio data to simplify the operations required to process the audio data and improve the processing speed of the audio data.

[0054] In some examples, after processing the second audio data to obtain the third audio data, energy detection may also be performed on the third audio data to ensure that there is no silence in the processing result of the first audio data generated based on the third audio data, thereby further improving the reliability of the processing result of the first audio data.

[0055] At this time, optionally, before generating the processing result of the first audio data based on the third audio data, it also includes: performing energy detection on the third audio data to generate a first energy detection result, and the first energy detection result is used to indicate whether the third audio data is silent audio data; generating the processing result of the first audio data based on the third audio data includes: in response to the first energy detection result indicating that the third audio data is not silent audio data, generating the processing result of the first audio data based on the third audio data.

[0056] Among them, energy detection can be understood as detecting the energy of audio data to determine whether there is sound in the audio data, that is, determining whether the audio data is silent audio data. Silent audio data, that is, silent audio data, can be understood as audio data without sound. In other words, audio data where the energy at each location is zero or less than a preset threshold. The first energy detection result can be understood as the energy detection result generated by detecting the third audio data, and this energy detection result can be used to indicate whether there is sound in the third audio data, that is, to indicate whether the third audio data is silent audio data.

[0057] Exemplarily, after obtaining the third audio data output by the target audio processing model, energy detection can be performed on the third audio data to determine whether there is sound in the third audio data, and a first energy detection result for the third audio data can be generated. Furthermore, when the first energy detection result indicates that the third audio data is not silent audio data, that is, when the detection determines that there is sound in the third audio data, a processing result of the first audio data can be generated based on the third audio data.

[0058] In addition, when the first energy detection result indicates that the third audio data is silent audio data, that is, when the detection determines that there is no sound in the third audio data, it can be prompted that the processing of the first audio data has failed, so that the first audio data can be analyzed and processed separately; and / or, the processing log of the first audio data is returned, and the relevant processing data of the first audio data is recorded through this processing log, so as to facilitate subsequent analysis of whether the processing of the first audio data failed due to insufficient audio information, and then adjust and optimize the target processing time threshold of the target audio processing model.

[0059] The energy detection method used for performing energy detection on the third audio data is not limited. The energy detection methods used for audio data output by different audio processing models and / or different audio data output by audio processing models can be the same or different.

[0060] In some examples, taking the third audio data as audio data containing human voice as an example, the voice activity detection (VAD) technology can be used to detect whether the third audio data contains a voice signal, and then determine whether the third audio data is silent audio data. Among them, voice activity detection is a technology for detecting whether there is voice in audio data. When performing voice activity detection, exemplarily, various features can be first extracted from the third audio signal, and these features can include energy, zero crossing rate and / or spectrum features, etc. For example, the voice signal usually has higher energy and a specific spectrum distribution, while the energy of the background noise is relatively low and the spectrum is relatively flat; based on the extracted features, a specific algorithm and threshold are used to determine whether there is voice in the third audio data. Specifically, if the feature value exceeds the preset threshold, it can be determined that there is voice activity, that is, there is voice in the audio data; otherwise, it can be determined to be silence or background noise, and the third audio data is determined to be silent audio data.

[0061] In some examples, considering that a sound signal is essentially an energy wave, an energy detection technique can be used to calculate the energy of the third audio data to determine whether there is sound, such as by calculating the energy of the third audio data within a time window corresponding to the first audio data to determine whether there is sound in the third audio data.

[0062] In this embodiment, the processing time thresholds of different audio processing models may be the same or different. The method for determining the processing time threshold of each audio processing model (including the target audio processing model) is not limited. Exemplarily, the processing time threshold of the audio processing model can be determined in advance based on the processing results of the audio processing model on audio data segments of different lengths; and / or, based on the processing results of the audio processing model for audio data of different lengths, the processing time threshold of the audio processing model is adjusted, and so on.

[0063] In some embodiments, before obtaining the first audio data to be processed, it also includes: dividing the audio data samples to obtain multiple original audio data segments with different audio durations; inputting each of the original audio data segments into the target audio processing model, and obtaining the target audio data segments obtained by the target audio processing model from processing each of the original audio data segments; performing energy detection on each of the target audio data segments to generate a second energy detection result, and determining the target processing duration threshold of the target audio processing model based on the second energy detection result.

[0064] Wherein, the audio data sample can be pre-acquired audio data used as a sample to determine the target processing duration threshold of the target audio processing model. Exemplarily, the audio data sample can be long audio data with a duration greater than a preset duration threshold. This preset duration threshold can be set as needed, for example, this preset duration threshold can be set to values ​​such as 180s, 200s or 240s. The number of audio data samples can be one or more, and optionally, the number of audio data samples can be multiple to improve the accuracy of the determined target processing duration threshold. The original audio data segment can be understood as the audio data segment obtained by segmenting the audio data sample. The target audio data segment can be the audio segment obtained by processing the original audio data through the target audio processing model. The second energy detection result can be understood as the energy detection result of the target audio data segment.

[0065] Exemplarily, the audio data samples may be segmented, for example, for each audio data sample, a slicing process is performed on it to obtain original audio data segments of different durations, such as original audio data segments of durations t, 2t, ..., nt may be obtained respectively. n is a positive integer, and the specific value of t is not limited. For example, t may be set to a duration of 5s, 10s, or 15s.

[0066] After obtaining original audio data segments of different lengths, each original audio data segment can be input into the target audio processing model respectively, and each original audio data segment can be processed by the target audio data processing model to obtain a target audio data segment corresponding to each original audio data segment.

[0067] For each target audio data segment, energy detection is performed on the target audio data segment based on voice activity detection technology or energy detection technology to determine whether the target audio data segment is a silent audio segment, and a second energy detection result of the target audio data segment is obtained.

[0068] According to the second energy detection results of each target audio data segment, the silent audio segment in each target audio data segment is determined, and according to the duration distribution of each silent audio segment, the target processing duration threshold of the target audio processing model is determined. For example, the maximum duration of the silent audio segment obtained by processing is determined as the target processing duration threshold of the target audio processing model, or the minimum duration of all the target audio data segments obtained by processing that are not silent audio data segments is determined as the target processing duration threshold of the target audio processing model, and so on.

[0069] The audio data processing method provided in this embodiment obtains the first audio data to be processed and determines the first audio duration of the first audio data; in response to the first audio duration being less than or equal to the target processing duration threshold of the target audio processing model, the first audio data is extended to obtain the extended second audio data, wherein the second audio duration of the second audio data is greater than the target processing duration threshold; the second audio data is input into the target audio processing model, and the third audio data output by the target audio processing model is obtained; and the processing result of the first audio data is generated based on the third audio data. This embodiment utilizes the above technical solution, when the duration of the audio data to be processed is less than or equal to the processing duration threshold of the audio processing model, the audio data to be processed is extended to a duration greater than the processing duration threshold of the audio processing model, and the processing result of the audio data to be processed is generated based on the audio data obtained by processing the extended audio data by the audio processing model, which can avoid the situation where the audio processing model outputs silent audio data due to insufficient audio information when performing audio processing, and improve the processing effect of the audio processing model on the audio data and the accuracy of the processing result of the audio data.

[0070] Figure 2 A flowchart of another method for processing audio data provided in an embodiment of the present disclosure. The solution in this embodiment can be combined with one or more optional solutions in the above embodiments. Optionally, the extending processing of the first audio data includes: determining the number of loops of the first audio data according to the first audio duration and the target processing duration threshold; and performing loop extension processing on the first audio data according to the number of loops.

[0071] Correspondingly, such as Figure 2 As shown, the audio data processing method provided in this embodiment may include:

[0072] S201. Acquire first audio data to be processed, and determine a first audio duration of the first audio data.

[0073] S202: In response to the first audio duration being less than or equal to a target processing duration threshold of a target audio processing model, determining a number of loops of the first audio data according to the first audio duration and the target processing duration threshold.

[0074] In this embodiment, a cyclic extension method can be used to extend the first audio data whose audio duration is less than or equal to the target processing duration threshold to the second audio data whose audio duration is greater than the target processing duration threshold.

[0075] Specifically, when the first audio duration of the first audio data is less than or equal to the target processing duration threshold, the number of loops of the first audio data can be determined based on the first audio duration and the target processing duration threshold, such as by calculating the ratio between the target processing duration threshold and the first audio duration. If the ratio is an integer, the ratio is used as the number of loops of the first audio data; if the ratio is not an integer, the ratio is rounded up as the number of loops of the first audio data.

[0076] S203: Perform a loop extension process on the first audio data according to the number of loops to obtain extended second audio data, wherein a second audio duration of the second audio data is greater than the target processing duration threshold.

[0077] In this embodiment, after the number of cycles of the first audio data is obtained, a cycle extension process may be performed on the first audio data according to the number of cycles to obtain extended second audio data.

[0078] When the first audio data is subjected to loop extension processing, the first audio data may be copied and the copied first audio data may be spliced. For example, the first audio data may be copied one copy at a time and spliced, and after the splicing is completed, the first audio data may be copied again and the copied first audio data may be spliced ​​again until the number of loops of the first audio data reaches the number of loops determined above; multiple copies of the first audio data may also be copied according to the number of loops, and the multiple copies of the first audio data may be spliced ​​in sequence, and so on.

[0079] In some embodiments, when splicing two segments of first audio data, for example, one of the two segments of first audio data can be directly spliced ​​to the starting position or the ending position of the other segment of first audio data, without adding a preset silent audio segment at the splicing position of the two segments of first audio data, so as to simplify the operations required to be performed when performing loop extension processing on the first audio data, thereby improving the processing speed of the first audio data.

[0080] In some embodiments, when two segments of first audio data are spliced ​​together, a preset silent audio segment can be added at the splicing position of the two segments of first audio data, so as to distinguish the different segments of the spliced ​​first audio data through the preset silent audio segment, and increase the difficulty of generating the processing result of the first audio data based on the third audio data, so as to avoid the situation where the processing result of the first audio data is inaccurate due to segmentation error. At this time, optionally, the loop extension processing of the first audio data according to the number of loops includes: splicing multiple segments of the first audio data according to the number of loops, and adding a preset silent audio segment at the splicing position of the first audio data.

[0081] Among them, the preset silent audio segment can be an audio frequency band in which there is no sound. The duration of the preset silent audio segment can be a set duration or a random duration. Optionally, the duration of the preset silent audio segment is a preset audio duration. The specific value of this preset audio duration is not limited. For example, this preset audio duration can be 100ms or 200ms, etc. The preset silent audio segment can be a preset silent audio segment; it can also be a silent audio segment generated in real time when the first audio data is subjected to a loop extension process. For example, when the first audio data is subjected to a loop extension process, a preset sound source can be called to generate a preset silent audio segment. At this time, optionally, the preset silent audio segment is generated by a preset sound source. The preset sound source can be a sound source that can be used to generate silent audio. The type of the preset sound source is not limited. For example, this preset sound source can be a virtual sound source, such as the anullsrc sound source of the ffmpeg.input function. This anullsrc sound source can be an audio filter of ffmpeg, which can be used to generate silent audio without any sound.

[0082] Specifically, the first audio data can be spliced ​​according to the determined number of loops, and a preset silent audio segment can be added at each splicing position. Taking another segment of first audio data obtained by splicing and copying at the end of a segment of first audio data as an example, a preset silent audio segment can be spliced ​​at the end of the previous segment of first audio data, and after the splicing of this preset silent audio segment is completed, the above-mentioned other segment of first audio data can be spliced ​​at the end of this preset silent audio segment. Among them, the preset silent audio segment can be generated by calling the ffmpeg.input function anullsrc virtual sound source in advance according to the number of loops after determining the number of loops of the first audio data; it can also be generated by calling the ffmpeg.input function anullsrc virtual sound source in real time when splicing two segments of first audio data, and this embodiment does not limit this.

[0083] S204: Input the second audio data into the target audio processing model, and obtain third audio data output by the target audio processing model.

[0084] In this embodiment, after the first audio data is processed by loop extension to obtain the second audio data, the second audio data can be input into the target audio processing model, and the second audio data can be processed by the target audio processing model to obtain the third audio data.

[0085] S205: Generate a processing result of the first audio data based on the third audio data.

[0086] In this embodiment, after the target audio processing model processes the second audio data to obtain the third audio data, the processing result of the first audio data can be generated based on the third audio data.

[0087] In some embodiments, generating the processing result of the first audio data based on the third audio data includes: segmenting the third audio data according to the first audio duration and the preset audio duration of the preset silent audio segment, and obtaining audio data segments corresponding to the first audio data as the processing result of the first audio data.

[0088] Exemplarily, the segmentation point of the third audio data can be determined based on the first audio duration and the preset audio duration of the preset silent audio segment. For example, an audio duration within the range from the first audio duration to the sum of the first audio duration and the preset audio duration can be used as the segmentation duration, and the time point corresponding to the segmentation duration in the third audio data can be used as the segmentation point. After determining the segmentation point of the third audio data, segmentation can be performed at this segmentation point, and the audio data segment obtained by segmentation, from the starting position of the third audio data to the position of this segmentation point, can be used as the processing result of the first audio data. Among them, the segmentation duration is within the range from the first audio duration to the sum of the first audio duration and the preset audio duration, and the specific determination method is not limited. For example, the midpoint of this range, that is, the sum of the first audio duration and 1 / 2 of the preset audio duration, can be used as the segmentation duration to further improve the fault tolerance rate during segmentation, and avoid the situation where the audio data segment corresponding to the first audio data obtained by segmentation is incomplete or introduced into the audio data obtained by processing the next segment of the first audio data due to segmentation errors.

[0089] The audio data processing method provided in this embodiment can simplify the operations required to extend the first audio data, improve the processing effect of the audio processing model on the audio data and the accuracy of the processing results of the audio data.

[0090] In an optional embodiment, taking the sound source separation scenario as an example, in some examples, the audio data can be separated from the sound source, and the separated clean dry sound track is used for sound conversion, arrangement adjustment, cover and other scenarios to assist in music creation. Among them, the dry sound (Dry Voacl) is the original sound that has not been processed with any effects. This sound has not been processed by effects such as reverberation, delay, and chorus, maintaining its purity and authenticity during recording. Because it can provide the clearest audio signal, dry sound is often used in the recording and mixing process. However, some sound source separation models cannot effectively process shorter audio.

[0091] In view of this, this embodiment provides a method for processing audio data, which can be specifically described as follows:

[0092] The abnormal separation results are detected in advance by VAD, such as abnormal volume separation results mainly composed of silence, and the silence duration threshold (i.e., processing duration threshold) of each sound source separation model is determined based on this result.

[0093] Figure 3 A schematic diagram of a process for determining a processing time threshold provided in this embodiment is shown in FIG. Figure 3 As shown, long audio (i.e., audio data samples) with a duration greater than or equal to nt can be sliced ​​to obtain multiple original audio data segments 1 with a duration of t, multiple original audio data segments 2 with a duration of 2t, ..., multiple original audio data segments n with a duration of nt; each original audio data segment is pushed to the sound source separation service respectively; for each sound source separation model of the sound source separation service, each original audio data segment is subjected to sound source separation by means of this sound source separation model to obtain the separation result of each original audio data segment (i.e., the target audio segment), and VAD is used to detect whether the separation result is silent; according to the duration distribution of the silent audio segment (i.e., the silent audio segment), the segment duration boundary (i.e., the processing duration threshold) corresponding to the silent result of this sound source separation model is determined.

[0094] Among them, VAD is a technology used to detect whether there is speech in an audio signal, which generally includes feature extraction and decision judgment.

[0095] Feature extraction: The VAD system first extracts various features from the input audio signal, which may include energy, zero-crossing rate, spectral characteristics, etc. For example, speech signals usually have higher energy and a specific spectral distribution, while background noise has relatively low energy and a flat spectrum.

[0096] Decision-making: Based on the extracted features, a specific algorithm and threshold are used to determine whether there is speech in the current audio segment. If the feature value exceeds the preset threshold, it is determined to be speech active, that is, there is speech in the segment; otherwise, it is considered to be silence or background noise.

[0097] In addition, for the model that only performs instrument separation, VAD can be replaced by energy detection to detect the loudness of non-human voice audio. Among them, energy detection technology can determine whether there is sound based on the energy of the sound signal. The sound signal is essentially an energy wave, and energy detection technology can determine whether there is sound by calculating the energy of the audio signal within a certain time window.

[0098] Figure 4 A schematic diagram of an audio data processing process provided in this embodiment, such as Figure 4 As shown, the processing of audio data of each model unit can be described as:

[0099] A1. Perform duration detection on the input audio (such as the first audio without extension or the audio output by the previous model) to determine whether its duration is greater than the boundary of the current model (i.e., the processing duration threshold). If so, input it into the current model; if not, perform cyclic expansion (i.e., cyclic extension) on the audio through the cyclic expansion module to obtain the expanded audio, and return to execute the operation of determining whether its duration is greater than the boundary of the current model.

[0100] When performing loop expansion through the loop expansion model, for example, the ratio between the boundary of the current model and the audio duration audio_duration (i.e., the first audio duration) of the audio input to the current model unit can be calculated and rounded up as the number of loops of the input audio (assuming it is k), and the input audio loop is extended to a duration of k×audio_duration, and the audio after loop extension (i.e., the second audio data) is output.

[0101] In addition, in order to prevent the original audio content from being lost due to the time accuracy error during the final cutting and restoration, a signal-free empty state (i.e., a preset silent audio segment) can be added between two segments (i.e., two adjacent segments of the first audio data) when the loop expansion product is spliced. For example, the ffmpeg.input function can be used to obtain an audio segment with a duration of 100ms (i.e., the preset audio duration) from anullsrc. Among them, anullsrc is a virtual audio source of FFmpeg, which can generate a silent signal.

[0102] A2. Obtain the audio output by the current model after sound source separation, and perform VAD detection on it to determine whether the VAD detection is qualified (i.e., determine whether there is sound). If so, continue with subsequent processing; if not, return a log recording the abnormality of the audio source separation to facilitate subsequent optimization of the boundary of the current model.

[0103] A3. After completing the sound source separation through at least one model, if the separated audio data output by the last model has no abnormality (i.e., the VAD test is qualified), it can be determined whether the audio data output by this model is the expanded audio data. If so, the audio data output by this model can be segmented, restored to the original audio length and output as the result; if not, the audio data can be output as the processing result of the original audio data.

[0104] For example, when cutting the final result of the model output, you can choose to add 1 / 2 of the preset audio length (such as adding 50ms) to the original audio length for cutting to prevent content loss due to time error.

[0105] The method for processing audio data provided in this embodiment can effectively solve the problem of reasoning failure of the machine learning audio processing model due to insufficient target audio information by supplementing audio content. For example, by extending the audio content through a loop replication method, it can effectively solve the problem of reasoning failure of the sound source separation model due to insufficient audio information in the audio data to be separated, thereby improving the reliability of the audio processing model when processing audio data.

[0106] Figure 5 This is a structural block diagram of an audio data processing device provided by an embodiment of the present disclosure. The device can be implemented by software and / or hardware, and can be configured in an electronic device, typically, a computer, a mobile phone or a tablet computer, and can process audio data by using an audio processing model by executing an audio data processing method, such as using a sound source separation model to separate the audio source of audio data. Figure 5 As shown, the audio data processing device provided in this embodiment may include: an audio acquisition module 501, an audio extension module 502, an audio processing module 503 and a result generation module 504, wherein:

[0107] The audio acquisition module 501 is used to acquire first audio data to be processed and determine a first audio duration of the first audio data;

[0108] The audio extension module 502 is used for extending the first audio data to obtain extended second audio data in response to the first audio duration being less than or equal to the target processing duration threshold of the target audio processing model, wherein the second audio duration of the second audio data is greater than the target processing duration threshold;

[0109] An audio processing module 503, configured to input the second audio data into the target audio processing model, and obtain third audio data output by the target audio processing model;

[0110] The result generating module 504 is configured to generate a processing result of the first audio data based on the third audio data.

[0111] The audio data processing device provided in this embodiment obtains the first audio data to be processed through the audio acquisition module, and determines the first audio duration of the first audio data; in response to the first audio duration being less than or equal to the target processing duration threshold of the target audio processing model, the audio extension module extends the first audio data to obtain the extended second audio data, wherein the second audio duration of the second audio data is greater than the target processing duration threshold; the second audio data is input into the target audio processing model through the audio processing module, and the third audio data output by the target audio processing model is obtained; and the processing result of the first audio data is generated based on the third audio data by the result generation module. This embodiment uses the above technical solution, when the duration of the audio data to be processed is less than or equal to the processing duration threshold of the audio processing model, the audio data to be processed is extended to a duration greater than the processing duration threshold of the audio processing model, and the processing result of the audio data to be processed is generated based on the audio data obtained by processing the extended audio data by the audio processing model, which can avoid the situation that the audio processing model outputs silent audio data due to insufficient audio information when performing audio processing, and improve the processing effect of the audio processing model on the audio data and the accuracy of the processing result of the audio data.

[0112] Optionally, the audio extension module 502 may include: a loop number determination unit, used to determine the number of loops of the first audio data according to the first audio duration and the target processing duration threshold; and an extension processing unit, used to perform loop extension processing on the first audio data according to the loop number.

[0113] Optionally, the extension processing unit is specifically configured to: splice multiple segments of the first audio data according to the number of loops, and add a preset silent audio segment at a splicing position of the first audio data.

[0114] Optionally, the duration of the preset silent audio segment is a preset audio duration, and / or the preset silent audio segment is generated by a preset sound source.

[0115] Optionally, the result generation module 504 can be specifically used to: segment the third audio data according to the first audio duration and the preset audio duration of the preset silent audio segment, and obtain audio data segments corresponding to the first audio data as the processing results of the first audio data.

[0116] Furthermore, the audio data processing device may also include: an energy detection module, used to perform energy detection on the third audio data to generate a first energy detection result before generating the processing result of the first audio data based on the third audio data, and the first energy detection result is used to indicate whether the third audio data is silent audio data; the result generation module 504 may be specifically used to: in response to the first energy detection result indicating that the third audio data is not silent audio data, generate the processing result of the first audio data based on the third audio data.

[0117] Furthermore, the audio data processing device may also include: a sample segmentation module, used to segment the audio data sample before obtaining the first audio data to be processed, so as to obtain multiple original audio data segments with different audio durations; an audio segment acquisition module, used to input each of the original audio data segments into the target audio processing model respectively, and obtain the target audio data segments obtained by the target audio processing model from processing each of the original audio data segments; a threshold determination module, used to perform energy detection on each of the target audio data segments respectively, generate a second energy detection result, and determine the target processing duration threshold of the target audio processing model based on the second energy detection result.

[0118] Optionally, the target audio processing model includes a sound source separation model.

[0119] The audio data processing device provided in the embodiment of the present disclosure can execute the audio data processing method provided in any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the audio data processing method. For technical details not described in detail in this embodiment, please refer to the audio data processing method provided in any embodiment of the present disclosure.

[0120] Reference below Figure 6 , which shows a schematic diagram of the structure of an electronic device (e.g., terminal device) 600 suitable for implementing the embodiment of the present disclosure. The terminal device in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (e.g., vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 6 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0121] like Figure 6As shown, the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0122] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 6 The electronic device 600 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0123] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 609, or installed from a storage device 608, or installed from a ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0124] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0125] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0126] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0127] The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: obtains the first audio data to be processed and determines the first audio duration of the first audio data; in response to the first audio duration being less than or equal to the target processing duration threshold of the target audio processing model, extends the first audio data to obtain extended second audio data, wherein the second audio duration of the second audio data is greater than the target processing duration threshold; inputs the second audio data into the target audio processing model and obtains the third audio data output by the target audio processing model; and generates the processing result of the first audio data based on the third audio data.

[0128] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0129] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0130] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a module does not, in some cases, constitute a limitation on the unit itself.

[0131] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0132] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0133] According to one or more embodiments of the present disclosure, Example 1 provides a method for processing audio data, including:

[0134] Acquire first audio data to be processed, and determine a first audio duration of the first audio data;

[0135] In response to the first audio duration being less than or equal to a target processing duration threshold of a target audio processing model, extending the first audio data to obtain extended second audio data, wherein a second audio duration of the second audio data is greater than the target processing duration threshold;

[0136] Inputting the second audio data into the target audio processing model, and obtaining third audio data output by the target audio processing model;

[0137] A processing result of the first audio data is generated based on the third audio data.

[0138] According to one or more embodiments of the present disclosure, Example 2 is a method according to Example 1, wherein the extending processing of the first audio data includes:

[0139] Determining the number of loops of the first audio data according to the first audio duration and the target processing duration threshold;

[0140] The first audio data is subjected to a loop extension process according to the loop number.

[0141] According to one or more embodiments of the present disclosure, Example 3 is the method according to Example 2, wherein the loop extension processing of the first audio data according to the number of loops includes:

[0142] The multiple segments of the first audio data are spliced ​​according to the number of cycles, and a preset silent audio segment is added at the splicing position of the first audio data.

[0143] According to one or more embodiments of the present disclosure, Example 4 is based on the method described in Example 3, wherein the duration of the preset silent audio segment is a preset audio duration, and / or the preset silent audio segment is generated by a preset sound source.

[0144] According to one or more embodiments of the present disclosure, Example 5 is the method according to Example 3, wherein the step of generating a processing result of the first audio data based on the third audio data includes:

[0145] The third audio data is segmented according to the first audio duration and the preset audio duration of the preset silent audio segment to obtain audio data segments corresponding to the first audio data as processing results of the first audio data.

[0146] According to one or more embodiments of the present disclosure, Example 6 is the method according to Example 1, and before generating the processing result of the first audio data based on the third audio data, further includes:

[0147] Performing energy detection on the third audio data to generate a first energy detection result, where the first energy detection result is used to indicate whether the third audio data is silent audio data;

[0148] The step of generating a processing result of the first audio data based on the third audio data includes:

[0149] In response to the first energy detection result indicating that the third audio data is not silent audio data, a processing result of the first audio data is generated based on the third audio data.

[0150] According to one or more embodiments of the present disclosure, Example 7 is a method according to any one of Examples 1-6, and before obtaining the first audio data to be processed, further includes:

[0151] The audio data sample is segmented to obtain multiple original audio data segments with different audio durations;

[0152] Inputting each of the original audio data segments into a target audio processing model respectively, and obtaining a target audio data segment obtained by processing each of the original audio data segments by the target audio processing model;

[0153] Energy detection is performed on each of the target audio data segments to generate a second energy detection result, and a target processing duration threshold of the target audio processing model is determined based on the second energy detection result.

[0154] According to one or more embodiments of the present disclosure, Example 8 is a method according to any one of Examples 1-4, wherein the target audio processing model includes a sound source separation model.

[0155] According to one or more embodiments of the present disclosure, Example 9 provides a device for processing audio data, including:

[0156] An audio acquisition module, used to acquire first audio data to be processed and determine a first audio duration of the first audio data;

[0157] an audio extension module, configured to, in response to the first audio duration being less than or equal to a target processing duration threshold of a target audio processing model, extend the first audio data to obtain extended second audio data, wherein the second audio duration of the second audio data is greater than the target processing duration threshold;

[0158] an audio processing module, configured to input the second audio data into the target audio processing model, and obtain third audio data output by the target audio processing model;

[0159] A result generating module is used to generate a processing result of the first audio data based on the third audio data.

[0160] According to one or more embodiments of the present disclosure, Example 10 provides an electronic device, including:

[0161] one or more processors;

[0162] a memory for storing one or more programs,

[0163] When the one or more programs are executed by the one or more processors, the one or more processors implement the audio data processing method as described in any one of Examples 1-8.

[0164] According to one or more embodiments of the present disclosure, Example 11 provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for processing audio data as described in any one of Examples 1-8.

[0165] According to one or more embodiments of the present disclosure, Example 12 provides a computer program product. When the computer program product is executed by a computer, the computer implements the audio data processing method as described in any one of Examples 1-8.

[0166] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0167] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0168] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A method for processing audio data, characterized in that: include: Acquire first audio data to be processed, and determine a first audio duration of the first audio data; In response to the first audio duration being less than or equal to a target processing duration threshold of a target audio processing model, extending the first audio data to obtain extended second audio data, wherein a second audio duration of the second audio data is greater than the target processing duration threshold; Inputting the second audio data into the target audio processing model, and obtaining third audio data output by the target audio processing model; A processing result of the first audio data is generated based on the third audio data.

2. The method according to claim 1, characterized in that The extending the first audio data comprises: Determining the number of loops of the first audio data according to the first audio duration and the target processing duration threshold; The first audio data is subjected to a loop extension process according to the loop number.

3. The method according to claim 2, characterized in that The performing a loop extension process on the first audio data according to the number of loops includes: The multiple segments of the first audio data are spliced ​​according to the number of cycles, and a preset silent audio segment is added at the splicing position of the first audio data.

4. The method according to claim 3, characterized in that The duration of the preset silent audio segment is a preset audio duration, and / or the preset silent audio segment is generated by a preset sound source.

5. The method according to claim 3, characterized in that: The step of generating a processing result of the first audio data based on the third audio data includes: The third audio data is segmented according to the first audio duration and the preset audio duration of the preset silent audio segment to obtain audio data segments corresponding to the first audio data as processing results of the first audio data.

6. The method according to claim 1, characterized in that Before generating the processing result of the first audio data based on the third audio data, the method further includes: Performing energy detection on the third audio data to generate a first energy detection result, where the first energy detection result is used to indicate whether the third audio data is silent audio data; The step of generating a processing result of the first audio data based on the third audio data includes: In response to the first energy detection result indicating that the third audio data is not silent audio data, a processing result of the first audio data is generated based on the third audio data.

7. The method according to any one of claims 1 to 6, characterized in that: Before obtaining the first audio data to be processed, the method further includes: The audio data sample is segmented to obtain multiple original audio data segments with different audio durations; Inputting each of the original audio data segments into a target audio processing model respectively, and obtaining a target audio data segment obtained by processing each of the original audio data segments by the target audio processing model; Energy detection is performed on each of the target audio data segments to generate a second energy detection result, and a target processing duration threshold of the target audio processing model is determined based on the second energy detection result.

8. The method according to any one of claims 1 to 6, characterized in that: The target audio processing model includes a sound source separation model.

9. An audio data processing device, characterized in that: include: An audio acquisition module, used to acquire first audio data to be processed and determine a first audio duration of the first audio data; an audio extension module, configured to, in response to the first audio duration being less than or equal to a target processing duration threshold of a target audio processing model, extend the first audio data to obtain extended second audio data, wherein the second audio duration of the second audio data is greater than the target processing duration threshold; an audio processing module, configured to input the second audio data into the target audio processing model, and obtain third audio data output by the target audio processing model; A result generating module is used to generate a processing result of the first audio data based on the third audio data.

10. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the audio data processing method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the audio data processing method according to any one of claims 1 to 8 when executed.

12. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the computer program implements the method for processing audio data according to any one of claims 1 to 8.

Images