Audio editing method

Abstract

The invention discloses an audio editing method, and belongs to the technical field of multi-user audio editing. The method comprises the following steps: segmenting a to-be-edited target audio into a plurality of pieces of audio particle data, and obtaining voiceprints of the plurality of pieces of audio particle data; determining voiceprint similarity between voiceprints of two adjacent pieces of audio particle data in the multiple pieces of audio particle data; according to the voiceprint similarity, merging the multiple pieces of audio particle data to obtain a target audio clip; and distributing the target audio clip to the editing object for editing to obtain an edited audio corresponding to the target audio. Through the mode, the utterance of the same speaker can be prevented from being cut into unreasonable audio clips, and the fluency of the voice is ensured, so that the editing object does not need to consume too much time to check the fluency of the audio clips, and the audio editing efficiency is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of multi-person audio editing, and in particular to an audio editing method. Background Art

[0002] In scenarios such as meetings, users will record target audio for a long time. However, the quality of the recorded target audio is poor and needs to be further edited so that it can be organized into high-quality files for use by other users.

[0003] To improve the efficiency of editing, a long audio can be cut into multiple short audio segments, each short audio segment is distributed to each user for editing, and finally summarized. However, in the related art, cutting a long audio into multiple short audio segments mainly relies on the user's experience, which is easy to cut off the continuous words of the same person, and it may happen that a complete sentence is cut into two short audio segments. When editing audio, the short audio needs to be frequently re-proofread, resulting in low audio editing efficiency. Summary of the invention

[0004] The embodiment of the present application provides an audio editing method, which can at least solve the problem of lack of speech fluency and low editing efficiency after long audio is cut.

[0005] In order to solve the above technical problems, this application is implemented as follows: In a first aspect, an embodiment of the present application provides an audio editing method, the method comprising: dividing a target audio to be edited into multiple audio particle data, obtaining voiceprints of the multiple audio particle data; determining the voiceprint similarity between the voiceprints of two adjacent audio particle data among the multiple audio particle data; merging the multiple audio particle data according to the voiceprint similarity to obtain a target audio segment; and obtaining an edited audio corresponding to the target audio by assigning the target audio segment to an editing object for editing.

[0006] In a second aspect, an embodiment of the present application provides an audio editing device, which includes: a first acquisition module, used to divide the target audio to be edited into multiple audio particle data, and obtain the voiceprints of the multiple audio particle data; a determination module, used to determine the voiceprint similarity between the voiceprints of two adjacent audio particle data in the multiple audio particle data; a merging module, used to merge the multiple audio particle data according to the voiceprint similarity to obtain a target audio segment; a second acquisition module, used to assign the target audio segment to an editing object for editing, and obtain an edited audio corresponding to the target audio.

[0007] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the method described in the first aspect above are implemented.

[0008] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the method described in the first aspect above are implemented.

[0009] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer executes the steps of the method described in the first aspect above.

[0010] The technical solution provided by this application may have the following beneficial effects: In an embodiment of the present application, the target audio to be edited is divided into multiple audio particle data, and the voiceprints of the multiple audio particle data are obtained; then the voiceprint similarity between the voiceprints of two adjacent audio particle data in the multiple audio particle data is determined; and then the multiple audio particle data can be merged according to the voiceprint similarity to obtain the target audio segment; finally, the target audio segment is assigned to the editing object for editing to obtain the edited audio corresponding to the target audio. Through the above method, it is possible to avoid cutting the words of the same speaker into unreasonable audio segments, ensure the fluency of the speech, so that the editing object does not need to spend too much time proofreading the fluency of the audio segment, and improve the efficiency of audio editing.

[0011] In the embodiments of the present application, it should be understood that the above general description and the following detailed description are merely exemplary and explanatory and cannot limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0013] Figure 1 A schematic diagram of a flow chart of an audio editing method provided in an embodiment of the present application is shown; Figure 2 A schematic diagram of the structure of a voiceprint network provided in an embodiment of the present application is shown; Figure 3 A schematic diagram of the structure of a speech synthesis model provided in an embodiment of the present application is shown; Figure 4A schematic diagram of the structure of an audio editing device provided in an embodiment of the present application is shown; Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown; Figure 6 A schematic structural diagram of another electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0014] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0015] In scenarios such as meetings, users will record long audios, but the quality of the recorded long audios is poor, and subsequent editing such as cutting, deleting, noise reduction, volume adjustment, and text proofreading are required in order to organize them into high-quality files for use by other users.

[0016] In order to improve editing efficiency, most of the current practices are to cut long audio into multiple short audio segments of equal length, distribute each short audio segment to individual users for editing, and then summarize them together.

[0017] In the related art, cutting long audio into multiple short audio segments mainly relies on the user's experience, which is a blind operation. It is easy to cut off the continuous words spoken by the same person, that is, a complete sentence in the long audio will be cut into two short audio segments, so that a complete sentence is cut into two audio segments, resulting in the need to frequently re-proofread the short audio, increasing the workload. In addition, the length of the short audio is at the segment level, which is still relatively long compared to a sentence spoken by a person (i.e., sentence level). The editing of short audio depends on the user repeatedly listening to the audio and frequently dragging the progress bar in a relatively long short audio. The operation is cumbersome and time-consuming, resulting in low efficiency in editing short audio. In addition, cutting long audio into multiple short audio segments and editing each short audio segment are offline operations, which makes it difficult to manage long audio.

[0018] Figure 1 The flowchart of an audio editing method provided by an exemplary embodiment of the present application is shown, and the method can be executed by an electronic device. The electronic device can be a terminal such as a mobile phone, a computer, or a server. Figure 1 As shown, the method mainly includes the following steps: S101: Divide the target audio to be edited into a plurality of audio particle data, and obtain the voiceprints of the plurality of audio particle data.

[0019] In an embodiment of the present application, the target audio to be edited can be divided into multiple audio particle data, and then the voiceprint of multiple audio particle data is obtained. In practical applications, there are usually multiple speakers, and different speakers take turns to speak. The recorded target audio includes recording the content of different speakers taking turns to speak in sequence. After completing the recording, the user can upload the target audio to the cloud disk for storage. At this time, the server of the cloud disk can divide the target audio into multiple audio particle data with smaller granularity. Since in practical applications, the speech content of each speaker is long or short, and the length of its speech is highly random, it is not known which speaker has said how long before the segmentation. Therefore, in an embodiment of the present application, in order to cover the speaker's speech content as much as possible, an initial smaller scale can be set, which can be less than the length of a single word. This scale can be expressed as n frames of speech signals in the target audio, and n is a variable. In practical applications, n frames of speech signals can be sequentially segmented from the target audio along the time axis as audio particle data, and the speech signal with less than n frames at the end is regarded as an audio particle data. In an embodiment of the present application, the target audio can be divided into multiple audio particle data with smaller granularity according to the scale, so as to cover the speech content as much as possible. Each speaker has the sound wave spectrum characteristics of the sound, and a unique voiceprint can be generated for the speaker, so that the identity of the speaker can be identified by the voiceprint. In an embodiment of the present application, the voiceprint of the speaker corresponding to the audio particle data can be obtained, so as to merge the audio particle data through the voiceprint.

[0020] In an optional implementation, the step of obtaining the voiceprints of the plurality of audio particle data includes: The voiceprints of the multiple audio particle data are obtained by inputting the multiple audio particle data into a voiceprint network for voiceprint recognition.

[0021] In an embodiment of the present application, audio particle data can be input into a voiceprint network for voiceprint recognition to obtain voiceprints of multiple audio particle data. The voiceprint network in the embodiment of the present application involves a speaker confirmation (SV) task, that is, the task of verifying whether an input speech belongs to a specific speaker. There are two concepts here: enrollment utterance and verification utterance. The former can be understood as a reserved "voiceprint", while the latter is a speech used for verification. The embodiment of the present application mainly uses enrollment utterance and does not need to use verification utterance. The voiceprint network in the embodiment of the present application can use a simple two-layer neural network model for voiceprint recognition, sacrificing a certain degree of accuracy and improving processing speed. Inputting audio particle data into a voiceprint network for voiceprint recognition can increase the speed of obtaining voiceprints of audio particle data and shorten the time to obtain voiceprints. In practical applications, other methods may also be used for voiceprint recognition (for example, voiceprint recognition is performed using a voiceprint recognition model composed of a long short-term memory network (LSTM) model + a generalized end-to-end loss function (GE2E)), which is not specifically limited in the embodiments of the present application.

[0022] In the embodiment of the present application, the voiceprint network includes a double-layer structure, which can increase the receptive field by alternately using convolution and a long short-term memory network (LSTM) with a self-attention mechanism, and can also enhance the weight of the local audio signal to achieve a balance between the double-layer structure, thereby improving the robustness of the system and significantly inhibiting the generation of bad cases, thereby ensuring the quality of the voiceprint while improving the computing speed.

[0023] In an optional implementation, the step of inputting the plurality of audio particles into a voiceprint network for voiceprint recognition to obtain the voiceprints of the plurality of audio particle data may include the following steps: Step 1, by performing Fourier transform on the multiple audio particle data, the mel spectra corresponding to the multiple audio particle data are obtained; in the embodiment of the present application, the multiple audio particle data can be Fourier transformed to obtain the mel spectrograms (mel spectrograms are characteristic data of sound, and are spectrograms under melscale) corresponding to the multiple audio particle data. In practical applications, the fast Fourier transform (FFT) algorithm can be used to extract the mel spectrum from each audio signal in the audio particle data, or other methods can be used to obtain the mel spectrum corresponding to the audio particle data, which is not specifically limited in the embodiment of the present application.

[0024] Step 2, obtaining a first spectrum feature by inputting the Mel spectrum into the first layer structure of the voiceprint network, wherein the first layer structure includes PreNet, convolution Conv, long short-term memory network LSTM with self-attention mechanism Self Attention, and LSTM; in this embodiment of the present application, the Mel spectrum can be input into the first layer structure of the voiceprint network, and processed by PreNet, Conv, LSTM with Self Attention, and LSTM in sequence to obtain the first spectrum feature.

[0025] Step 3, obtaining a second spectral feature by inputting the first spectral feature into the second layer structure of the voiceprint network, wherein the second layer structure has the same structure as the first layer structure; in this embodiment of the present application, the first spectral feature is input into the second layer structure of the voiceprint network, and processed sequentially by PreNet, Conv, LSTM with Self Attention, and LSTM to obtain the second spectral feature.

[0026] Step 4: Input the second spectral feature into the fully connected layer of the voiceprint network to obtain the voiceprint of the plurality of audio particle data. In the embodiment of the present application, the second spectral feature is input into the fully connected layer of the voiceprint network to be mapped into the voiceprint of the speaker, thereby obtaining the voiceprint of the plurality of audio particle data.

[0027] In the embodiments of the present application, Figure 2As shown in the figure, the PreNet in the voiceprint network includes two layers of networks, each of which includes a fully connected layer (FC), a linear rectified function (ReLU), and a dropout structure. The first PreNet can perform nonlinear transformation on the Mel Spectrogram, and the second PreNet can perform nonlinear transformation on the first spectrum feature output by the (LSTM). Conv effectively models the current two-dimensional vector sequence and context information for the input two-dimensional vector sequence (i.e., the above-mentioned Mel Spectrogram) through multiple 1D convolution operations, and outputs the spectrum features of the low-dimensional space. The first LSTM extracts the spectrum features in the time series from the spectrum features under the self-attention mechanism. The so-called self-attention mechanism refers to executing the self-attention module on the output input of the LSTM, maintaining the order between the features and the attention monotony, and not skipping the input sequence, which can enhance the robustness of the extracted features. The self-attention mechanism and the output of the LSTM are concat to form new features. The second LSTM normally extracts the context features of the sequence in the time series to obtain the spectrum features.

[0028] In an optional implementation, the voiceprint network may be determined according to the following method: Constructing a speech synthesis model, wherein the speech synthesis model includes an acoustic model and a vocoder, and the acoustic model and the vocoder are cascaded with each other; By training the speech synthesis model, obtaining the trained speech synthesis model; The voiceprint network is determined according to the trained acoustic model in the speech synthesis model.

[0029] In practical applications, the voiceprint is essentially the acoustic feature of the speaker, and the voiceprint network is a network for extracting features. Because it is difficult to label the features, it is difficult to perform supervised training on the voiceprint network. Therefore, an unsupervised method can be used to train the voiceprint network. The so-called unsupervised means that the input itself does not carry a tag, and the input is used as a sample of fitting loss. Under unsupervised training, the voiceprint is an intermediate output. In an embodiment of the present application, in order to obtain a voiceprint network, a speech synthesis model can be constructed, wherein the speech synthesis model includes an acoustic model and a vocoder with the same structure as the voiceprint network. The acoustic model and the vocoder are cascaded with each other, and the overall structure is as follows: Figure 3As shown. Then, by training the speech synthesis model, a trained speech synthesis model is obtained; finally, according to the acoustic model in the trained speech synthesis model, the voiceprint network is determined. In practical applications, the acoustic model and the vocoder can form a classic architecture from text to speech (Text To Speech, TTS). Among them, the vocoder may include HiFi-GAN (a generative adversarial network (GAN) model for efficient and high-quality speech synthesis), WaveNet (a text-to-speech system), etc. In the embodiment of the present application, the embodiment of the present application does not specifically limit the specific structure of the vocoder. In the embodiment of the present application, the construction of the speech synthesis model is not for the purpose of real speech synthesis, but in this way, an acoustic model with the same structure as the voiceprint network is trained. Finally, when the training is completed, the voiceprint network can inherit the parameters of the acoustic model, thereby realizing unsupervised training of the acoustic network.

[0030] In an optional implementation, the step of training the speech synthesis model to obtain the trained speech synthesis model may include the following steps: Step 1, divide multiple sample audios into sample audio granular data; in this embodiment of the present application, some audio data can be selected as sample audio, and the sample audio can be divided into multiple sample audio granular data according to different scales.

[0031] Step 2, perform Fourier transform on the sample audio particle data to obtain a first Mel spectrum; in an embodiment of the present application, the sample audio particle data can be Fourier transformed to obtain a first Mel spectrum. In practical applications, an FFT algorithm can be used to extract the first Mel spectrum from each audio signal of the sample audio particle data.

[0032] Step 3, by inputting the first Mel spectrum into the acoustic model, the voiceprint of the sample audio particle is obtained; in the embodiment of the present application, the acoustic model is responsible for understanding and compressing all relevant context information of the input Mel Spectrogram (first Mel spectrum), encoding the input first Mel spectrum into a new, dense vector representation, and the encoded information can usually be highly compressed and capture the key features and semantic information of the first Mel spectrum, so that the vocoder can generate a new target sequence based on these decodings. By inputting the first Mel spectrum into the acoustic model, a representation of the voiceprint of the sample audio particle data can be obtained.

[0033] Step 4, obtaining a second Mel spectrum by inputting the voiceprint of the sample audio particle into the vocoder; in this embodiment of the present application, the vocoder can receive a hidden representation or feature vector from the acoustic model, and then remap the abstract representation obtained by the acoustic model back to the original data space, convert it into the form of original data, and obtain a new Mel Spectrogram (second Mel spectrum).

[0034] Step 5: Update the training parameters of the acoustic model and the vocoder by calculating the loss value between the first mel spectrum and the second mel spectrum. In an embodiment of the present application, the loss value between the first mel spectrum and the second mel spectrum can be calculated, and the training parameters of the acoustic model and the vocoder can be updated based on the loss value.

[0035] Step 6, when the number of training times meets the preset number of training times or the loss value meets the preset conditions, obtain the trained speech synthesis model. In the embodiment of the present application, when the number of training times reaches the preset number of training times or the loss value is stable for multiple consecutive rounds, it can be determined that the training is completed, otherwise, it is necessary to return to execute the above step 3 until the training is completed. When the training is completed, the voiceprint network can inherit the parameters of the acoustic model, thereby realizing unsupervised training of the acoustic network, so that the performance of the voiceprint network is more inclined to parse out the characteristics that distinguish speakers.

[0036] In an embodiment of the present application, for the above step 5, a loss function such as cross entropy may be used to calculate the loss value between the first mel spectrum and the second mel spectrum.

[0037] Exemplarily, the loss value is calculated as follows:

[0038] in, is the number of frames. The number of frames of the first Mel spectrum is equal to the number of frames of the second Mel spectrum. is the first Mel spectrum, is the second Mel spectrum.

[0039] In practical applications, optimization methods such as stochastic gradient descent can be used to perform directional propagation based on the loss value to update the parameters of the acoustic model and the parameters of the vocoder.

[0040] S102: Determine the voiceprint similarity between two adjacent audio particle data among the plurality of audio particle data.

[0041] In an embodiment of the present application, the voiceprint similarity between the voiceprints of two adjacent audio particle data in a plurality of audio particle data can be determined based on the voiceprint of the audio particle data. In practical applications, the voiceprint similarity between the voiceprints of two adjacent audio particle data can be calculated in the time direction using algorithms such as cosine angle. Determining the voiceprint similarity helps to merge the audio particle data based on the voiceprint similarity, which can improve the accuracy of subsequent audio particle data merging.

[0042] S103: According to the voiceprint similarity, a plurality of audio particle data are merged to obtain a target audio segment.

[0043] In the embodiment of the present application, multiple audio particle data can be merged according to the voiceprint similarity to obtain the target audio segment. The target audio segment obtained in this way is similar in voiceprint and is spoken by the same person. This can not only avoid dividing the speech of the same person into two segments, but also avoid merging the speech of different people together, which can improve the accuracy of segmenting the target audio.

[0044] In an optional implementation, the merging of the plurality of audio particle data according to the voiceprint similarity to obtain the target audio segment may include the following steps: Step 1: According to the voiceprint similarity, multiple audio particle data are merged to obtain a first initial audio segment; in the embodiment of the present application, a threshold can be set in advance. In order to improve the resistance to background noise, the threshold can be set lower, for example, the threshold can be set to 0.6. If the voiceprint similarity of two adjacent audio particle data is greater than or equal to the threshold, the two audio particle data can be spliced ​​together. After traversing all the audio particle data, multiple first initial audio segments can be obtained.

[0045] Step 2: According to the length of the first initial audio segment and the voiceprint similarity, the first initial audio segment is merged to obtain a second initial audio segment; in practical applications, since the voiceprint network and the threshold in the above step 1 may have a certain false detection rate, the first initial audio segment can be post-processed to smooth the fluctuations between the first initial audio segments and improve the accuracy of the final target audio segment. In an embodiment of the present application, the first initial audio segment can be merged to obtain a second initial audio segment according to the length of the first initial audio segment and the voiceprint similarity, which can make up for possible errors and help obtain an accurate target audio segment.

[0046] Step 3, sorting the second initial audio segment to obtain a target audio segment, wherein the sorting includes merging the second initial audio segment and splitting the second initial audio segment. In the embodiment of the present application, the second initial audio segment can be sorted according to a preset sorting rule to obtain a target audio segment, which can make the user's editing more efficient.

[0047] In the embodiment of the present application, for the above step 2, in an optional implementation, the merging the first initial audio segments according to the length of the first initial audio segments and the voiceprint similarity to obtain the second initial audio segment may include: Obtaining the length of the first initial audio segment; When the length of the first initial audio segment is less than the first length, the initial audio segments are merged according to the voiceprint similarity between the last audio particle data and the first audio particle data of the adjacent initial audio segments to obtain a second initial audio segment.

[0048] In the embodiment of the present application, the length of each first initial audio segment can be obtained. If the length of the first initial audio segment A is less than the first length (which can be the length of a word, which is an adjustable empirical value), the voiceprint similarity between the last audio particle data in the first initial audio segment B and the first audio particle data in the first initial audio segment C can be calculated using algorithms such as cosine angle. If the voiceprint similarity is greater than or equal to the threshold, the first initial audio segment A, the first initial audio segment B, and the first initial audio segment C can be merged into a second initial audio segment. Among them, the first initial audio segment B is sorted before the first initial audio segment A, and the first initial audio segment C is sorted after the first initial audio segment A. In practical applications, this situation is mostly an error in the voiceprint network, and the first initial audio segment A, the first initial audio segment B, and the first initial audio segment C belong to the speech of the same speaker.

[0049] In an optional implementation, after obtaining the second initial audio segment, the method further includes: Acquire text information corresponding to the second initial audio segment by performing speech recognition on the second initial audio segment; identifying, according to text information corresponding to the second initial audio segment, whether the second initial audio segment is a complete sentence; In the case that the second initial audio segment is an incomplete sentence, the target audio is segmented again to obtain a new second initial audio segment.

[0050] In an embodiment of the present application, speech recognition can be performed independently on the second initial audio segment to obtain the text information corresponding to the second initial audio segment. In addition to being used to verify the integrity of the second initial audio segment, the text information can also be reused for subsequent editing, that is, the editing object can add the speaker's information (such as name, identity, etc.) on the basis of the text information and correct the text information itself, the speaking time and other information, and the final speech content to form a meeting record. In practical applications, a speaker speaks one sentence at a time, so the complete syntactic analysis technology (such as N-gram (a language model), neural network (Neural Net), etc.) in natural language processing (Natural Language Processing, NLP) can be used to analyze whether the text information is one or more complete sentences one by one. If the second initial audio segment is an incomplete sentence (that is, the text information of the second initial audio segment is not a complete sentence), the scale of segmentation can be reduced, the target audio can be re-segmented, new audio particle data can be obtained, and then a new second initial audio segment can be obtained. Optionally, the way to reduce the scale of segmentation can be: n=nx*m frames, where x is the number of reductions and m is the step size. This ensures that the final target audio segment is a complete sentence. In the embodiment of the present application, if the text information of a second initial audio segment is empty, it means that the second initial audio segment is a silent point, and the second initial audio segment can be deleted to save storage space.

[0051] In an optional implementation, the arranging the second initial audio segment to obtain the target audio segment may include the following steps: Step 1-1, determine the target number according to the number of editing objects; in the embodiment of the present application, the sorting rule may include assuming that the number of editing objects (collaborative editing users) is P, and the number of target audio clips needs to be greater than 2P+Q. 2P ensures that the target audio clips can be staggered and distributed to users later, and Q can ensure that there is a certain surplus for each user to edit.

[0052] Step 1-2, when the number of the second initial audio segments is less than the target number, starting from the second initial audio segment with the longest length, the second initial audio segments are segmented according to the text information corresponding to the second initial audio segments to obtain target audio segments; in this embodiment of the present application, if the number of the second initial audio segments is less than the target number, starting from the longest second initial audio segment, according to the text information obtained by speech recognition, some longer second initial audio segments are segmented with a complete sentence as a node until the number of second initial audios reaches 2P+Q, and the second initial audio can be determined as the target audio segment.

[0053] Step 2-1, obtain the length of the second initial audio segment; in the embodiment of the present application, the arrangement rule may also include that the length of the target audio segment is between [T1, T2], to ensure that the workload of a single edit is close, and to provide a basis for fairness for collaborative editing by multiple people. When arranging, the length of the second initial audio segment may be obtained first.

[0054] Step 2-2, when the length of the second initial audio segment is less than the second length, the second initial audio segment is merged with the adjacent second initial audio segment of smaller length to obtain the target audio segment; in the embodiment of the present application, if the length of the second initial audio segment is less than the second length (T1), the lengths of the previous second initial audio segment and the subsequent second initial audio segments are compared, and the current second initial audio segment is spliced ​​to the second initial audio segment of smaller length to obtain the target audio segment. This situation may be caused by the same user speaking and pausing slightly in the middle, or it may be that the users alternately say a simple word, such as the host of the meeting saying "next".

[0055] Step 2-3, when the length of the second initial audio segment is greater than the third length, the second initial audio segment is segmented according to the text information corresponding to the second initial audio segment to obtain the target audio segment, wherein the second length is less than the third length. In an embodiment of the present application, if the length of the second initial audio segment is greater than the third length (T2), the second audio segment can be segmented according to the text information of speech recognition with a complete sentence as a node until the length of each segmented second initial audio segment is between [T1, T2], and then the second initial audio segment is determined to be the target audio segment.

[0056] S104: Assigning the target audio segment to an editing object for editing to obtain an edited audio segment corresponding to the target audio segment.

[0057] In an embodiment of the present application, the target audio segment can be assigned to an editing object for editing to obtain an edited audio corresponding to the target audio. In a specific implementation, the editing object, that is, the user, can edit the target audio segment directly on the cloud disk. In actual applications, users can directly see the complete target audio on their respective cloud disks, wherein the segmentation points of each target audio segment can be displayed on the target audio, and the silent audio segments are directly displayed as frozen and uneditable, but the user can choose to delete the silent audio segments. In the above manner, the cumbersomeness of the operation can be effectively reduced, time can be saved, and the efficiency of editing audio can be improved.

[0058] In an optional implementation, the step of assigning the target audio segment to an editing object for editing to obtain the edited audio corresponding to the target audio segment may include the following steps: Step 1, randomly assign the target audio segment to the editing object for collaborative editing; in the embodiment of the present application, according to the principle of breaking up, some target audio segments can be synchronously assigned to each user for editing using a random method. The so-called breaking up means that there is at least one target audio segment between the target audio segments assigned to the user. This is because after editing the current target audio segment, the user will be more accustomed to editing other adjacent target audio segments, which can ensure the consistency of the content of the initially edited target audio segments. Initially, breaking up the target audio segments and assigning them to the user is in line with the user's behavioral habits and can improve the efficiency of the initial editing. After the user gradually becomes familiar with the content of the target audio data, there will not be too many obstacles to jumping to edit the target audio segment.

[0059] Step 2, lock the target audio clip; in the embodiment of the present application, an idle thread can be allocated to each user in the thread pool, and this thread can be responsible for locking, unlocking and editing the target audio clip allocated to the user. For the target audio clip allocated to the user, the thread will lock it, allowing the current user to edit it, but preventing other users from editing it, so as to avoid interference with the target audio clip being edited. Step 3, after the editing object completes editing the target audio segment, cancel the lock on the target audio segment and determine the review object from the editing object; in this embodiment of the present application, when the current target audio segment is completed editing, the thread will unlock it, and can select the one that meets the requirements from the editing object as the review object, so that the review object can review the edited target audio segment.

[0060] Step 4: When the audit object completes the audit, the edited audio corresponding to the target audio is obtained. In the embodiment of the present application, after the audit object completes the audit, the final edited audio can be obtained.

[0061] In an optional implementation, after randomly allocating the plurality of target audio segments to the plurality of editing objects for collaborative editing, the method further includes: In response to the request of the editing object, if the editing object does not have the target audio segment being edited and the target audio segment corresponding to the request is not locked, the target audio segment is allocated to the editing object.

[0062] In an embodiment of the present application, initially, a target editing segment can be assigned to an editing object. In a subsequent process, in response to a request from the editing object, when the editing object does not have a target audio segment being edited and the target audio segment corresponding to the request is not locked, the target audio segment can be assigned to the editing object. If the editing object itself has a target audio segment being edited and / or the target audio segment applied for is locked, the application of the editing object can be rejected and the reason for the rejection can be displayed. The above-mentioned thread pool can also be responsible for locking, unlocking and editing the target audio segment selected by the user. For the target audio segment selected by the user, the thread will lock it, allowing the current user to edit it, preventing other users from editing it, and when the current target audio segment completes editing, the thread will unlock it.

[0063] In an optional implementation, the method further includes: During the process of the editing object editing the target audio segment, adding an operation time limit to the target audio segment; If the editing object fails to complete editing of the target audio segment within the operation time limit, generating an alarm signal, wherein the alarm signal is used to instruct to shorten the operation time limit; When the operation time limit is lower than a preset time limit threshold, the editing object's editing of the target audio segment is canceled and the lock on the target audio segment is canceled.

[0064] In an embodiment of the present application, in order to ensure the efficiency of editing, an operation time limit can be added to each target audio segment. When the operation time limit is exceeded, an alarm signal is generated, and the alarm signal is used to indicate the reduction of the operation time limit. In practical applications, if the editing object does not complete the editing operation when the operation time limit is reached, most of the time it is offline or away, the operation time limit can be decayed with a dynamically increasing step size to accelerate the decay, and this is repeated until the operation time limit is 0, the editing object is canceled from editing the target audio segment, the target audio segment is unlocked, and other editing objects are allowed to edit. This can prevent the editing object from being offline, leaving, etc., from occupying the target audio segment, thereby improving the overall efficiency of editing audio data.

[0065] Exemplarily, after the alarm signal is generated, the operation time limit may be calculated using the following formula:

[0066] in, is the time limit for the t+1th operation, is the tth operation time limit, is a constant, and , The length of the target audio segment.

[0067] In an optional implementation, the adding of the operation time limit for the target audio segment may include the following two situations: Scenario 1: When the frequency of editing the target audio segment triggered by the editing object is less than the preset threshold, the operation time limit is determined according to the length of the target audio segment and the preset amplification factor; if the frequency of editing the target audio segment triggered by the editing object is less than the preset threshold, the corresponding operation time limit can be calculated according to the length of the target audio segment and the preset amplification factor, which can be understood as amplification based on the length of the target audio segment to ensure that the editing object has time to audition the target audio segment (that is, the editing time is at least equal to the length of the target audio segment), and the remaining time is left for the editing object to edit. Exemplarily, the formula can be: , ; in, The frequency with which the target audio clip is edited is triggered for the editing object. is the preset threshold, For the operation time limit, is the magnification factor, is the length of the target audio segment. In practical applications, the value of the amplification factor can be .

[0068] Scenario 2: When the frequency is greater than or equal to the preset threshold, the operation time limit is determined according to the length of the target audio segment, the amplification factor, the frequency, the preset threshold and the time step. In an embodiment of the present application, if the frequency is greater than or equal to the preset threshold, the operation time limit can be calculated according to the length of the target audio segment, the amplification factor, the frequency, the preset threshold and the time step. This is because some target audio segments are more complex, which may be sound effect problems, may be fuzzy sound quality, or may be important speakers, etc., which makes editing more cumbersome. When the editing object triggers editing operations on the target audio segment at a high frequency, the editing time can be increased proportionally to reserve more editing time for the editing object to ensure that the editing is completed. Exemplarily, the formula can be: , ; in, The frequency with which the target audio clip is edited is triggered for the editing object. is the preset threshold, For the operation time limit, is the magnification factor, is the length of the target audio segment, is the time step. In practical applications, the value of the amplification factor can be .

[0069] In an optional implementation, when all target audio segments have been edited, at least two editing objects may be selected from all editing objects as review objects, and review authority may be granted to them to review the target audio segments edited by other editing objects.

[0070] Optionally, the audit object can be determined by: Step 1: Calculate the average score of the scores of the recent multiple times of collaborative editing of audio data by multiple people for each editing object (user) to obtain the average score, sort the users from large to small according to the average score, and select the first K (K>I) users. The average score represents the attitude and ability of the user to a certain extent, and select users with better attitude and ability as the review objects.

[0071] Step 2: Sort the K users by the number of times they have revoked the editing rights of the audio clips, and select the first M (M>I) users. Regardless of whether it is a network problem or it is inconvenient for the user to edit, they are not suitable as review objects and are eliminated.

[0072] Step 3: Count the number of audio clips edited by the M users respectively, sort the M users from large to small according to the number of audio clips edited by the M users, select the first I users as the audit objects, and grant them the audit authority.

[0073] In practical applications, all target audio clips can be divided into J (J is a multiple of I) equal parts, each of which contains multiple continuous target audio clips, which can ensure the continuity of the review. Based on the principle of avoiding suspicion, for a certain review object, if a certain audio clip does not contain the target audio clip edited by the review object himself, the audio clip can be assigned to the review object. In extreme cases, when a certain audio clip contains audio clips edited by all review objects, it will be split into multiple audio clips until it can be assigned to the review object. The review object can browse the editing operations left by other users on the audio clips, and delete the editing operations when there are errors or omissions in the editing operations, so as to submit the correct editing operations.

[0074] When all review objects have reviewed all audio clips, they can obtain the final edited audio and perform subsequent business operations, such as submitting for approval, publishing to a specified range of users, etc.

[0075] In an optional implementation, after acquiring the edited audio, the method further includes: The editing object is scored according to the operation time limit of the editing object to obtain a scoring result corresponding to the editing object.

[0076] In an embodiment of the present application, in addition to being used to eliminate invalid occupation of the target audio segment, the operation time limit can also be used to score the editing object (user) for editing the target audio segment to obtain a corresponding scoring result, which can be used as a reference for the editing object's workload.

[0077] Exemplarily, the formula is as follows:

[0078] in, is the scoring result corresponding to the edited object, , , and are weights, The total duration of all target audio clips auditioned for the edited object, The total number of edit operations triggered for this edit object. The total number of edit operations triggered for the edit object that were retained (i.e. not deleted by the audit object), The number of times the operation time limit is exceeded.

[0079] In the embodiment of the present application, the target audio to be edited can be divided into multiple audio particle data to obtain the voiceprints of the multiple audio particle data; then the voiceprint similarity between the voiceprints of two adjacent audio particle data in the multiple audio particle data is determined; then the multiple audio particle data can be merged according to the voiceprint similarity to obtain the target audio segment; finally, the target audio segment is assigned to the editing object for editing to obtain the edited audio corresponding to the target audio. In the above manner, it is possible to avoid cutting the speech of the same speaker into unreasonable audio segments, ensure the fluency of the speech, so that the editing object does not need to spend too much time proofreading the fluency of the audio segment, and improve the efficiency of audio editing. .

[0080] The audio editing method provided in the embodiment of the present application can be executed by an audio editing device. In the embodiment of the present application, an audio editing device executing the audio editing method is taken as an example to illustrate the audio editing device provided in the embodiment of the present application.

[0081] Figure 4 A schematic diagram of the structure of an audio editing device provided by an exemplary embodiment of the present application is shown. The audio editing device can implement the following Figure 1 All or part of the contents of the illustrated embodiment, the audio editing device includes: a first acquisition module 401, a determination module 402, a merging module 403 and a second acquisition module 404.

[0082] In an embodiment of the present application, a first acquisition module 401 is used to divide the target audio to be edited into multiple audio particle data and obtain the voiceprints of the multiple audio particle data; a determination module 402 is used to determine the voiceprint similarity between the voiceprints of two adjacent audio particle data among the multiple audio particle data; a merging module 403 is used to merge the multiple audio particle data according to the voiceprint similarity to obtain a target audio segment; a second acquisition module 404 is used to obtain the edited audio corresponding to the target audio by assigning the target audio segment to an editing object for editing.

[0083] In an optional implementation, when the first acquisition module 401 is used to acquire the voiceprints of the plurality of audio particle data, it is specifically used to: The voiceprints of the multiple audio particle data are obtained by inputting the multiple audio particle data into a voiceprint network for voiceprint recognition.

[0084] In an optional implementation, when the first acquisition module 401 is used to obtain the voiceprints of the plurality of audio particle data by inputting the plurality of audio particles into a voiceprint network for voiceprint recognition, it is specifically used to: By performing Fourier transform on the plurality of audio particle data, Mel spectra corresponding to the plurality of audio particle data are obtained; The first spectrum feature is obtained by inputting the Mel spectrum into a first layer structure of a voiceprint network, wherein the first layer structure includes PreNet, convolution Conv, a long short-term memory network LSTM with a self-attention mechanism Self Attention, and LSTM; Obtaining a second spectral feature by inputting the first spectral feature into a second layer structure of the voiceprint network, wherein the second layer structure has the same structure as the first layer structure; The voiceprints of the plurality of audio particle data are obtained by inputting the second spectrum feature into the fully connected layer of the voiceprint network.

[0085] In an optional implementation, the voiceprint network is determined according to the following method: Constructing a speech synthesis model, wherein the speech synthesis model includes an acoustic model and a vocoder, and the acoustic model and the vocoder are cascaded with each other; By training the speech synthesis model, obtaining the trained speech synthesis model; The voiceprint network is determined according to the trained acoustic model in the speech synthesis model.

[0086] In an optional implementation, the acquiring the trained speech synthesis model by training the speech synthesis model includes: Split multiple sample audios into sample audio granular data; Performing Fourier transform on the sample audio particle data to obtain a first Mel spectrum; Obtaining a voiceprint of the sample audio particle by inputting the first Mel-spectrogram into the acoustic model; Obtaining a second Mel spectrum by inputting the voiceprint of the sample audio particle into the vocoder; Updating the training parameters of the acoustic model and the vocoder by calculating the loss value between the first Mel spectrum and the second Mel spectrum; When the number of training times meets the preset number of training times or the loss value meets the preset conditions, the trained speech synthesis model is obtained. In an optional implementation, when the merging module 403 is used to merge the plurality of audio particle data according to the voiceprint similarity to obtain the target audio segment, it is specifically used to: According to the voiceprint similarity, a plurality of the audio particle data are merged to obtain a first initial audio segment; According to the length of the first initial audio segment and the voiceprint similarity, the first initial audio segment is merged to obtain a second initial audio segment; The second initial audio segments are sorted to obtain target audio segments, wherein the sorting includes merging the second initial audio segments and segmenting the second initial audio segments.

[0087] In an optional implementation, when the merging module 403 is used to merge the first initial audio segments according to the length of the first initial audio segments and the voiceprint similarity to obtain the second initial audio segment, it is specifically used to: Obtaining the length of the first initial audio segment; When the length of the first initial audio segment is less than the first length, the initial audio segments are merged according to the voiceprint similarity between the last audio particle data of the adjacent initial audio segment and the first audio particle data to obtain the second initial audio particle.

[0088] In an optional implementation, the device also includes: a recognition module, used to obtain text information corresponding to the second initial audio segment by performing speech recognition on the second initial audio segment; identify whether the second initial audio segment is a complete sentence based on the text information corresponding to the second initial audio segment; if the second initial audio segment is an incomplete sentence, re-segment the target audio to obtain a new second initial audio segment.

[0089] In an optional implementation, when the merging module 403 is used to sort the second initial audio segment to obtain the target audio segment, it is specifically used to: Determining a target quantity according to the quantity of the editing objects; When the number of the second initial audio segments is less than the target number, starting from the second initial audio segment with the longest length, segmenting the second initial audio segments according to the text information corresponding to the second initial audio segments to obtain target audio segments; Obtaining the length of the second initial audio segment; When the length of the second initial audio segment is less than the second length, merging the second initial audio segment with an adjacent second initial audio segment with a smaller length to obtain the target audio segment; When the length of the second initial audio segment is greater than the third length, the second initial audio segment is segmented according to text information corresponding to the second initial audio segment to obtain the target audio segment, wherein the second length is less than the third length.

[0090] In an optional implementation, the second acquisition module 404, when used to assign the target audio segment to an editing object for editing to obtain the edited audio corresponding to the target audio, is specifically used to: Randomly assigning the plurality of target audio segments to the plurality of editing objects for collaborative editing; Locking the target audio segment; After the editing object completes editing the target audio segment, unlocking the target audio segment and determining a review object from the editing object; When the audit object completes the audit, the edited audio corresponding to the target audio is obtained.

[0091] In an optional implementation, the second acquisition module 404 is further configured to: In response to the request of the editing object, if the editing object does not have the target audio segment being edited and the target audio segment corresponding to the request is not locked, the target audio segment is allocated to the editing object.

[0092] In an optional implementation, the second acquisition module 404 is further configured to: add an operation time limit to the target audio segment during the process in which the editing object edits the target audio segment; If the editing object fails to complete editing of the target audio segment within the operation time limit, generating an alarm signal, wherein the alarm signal is used to instruct to shorten the operation time limit; When the operation time limit is lower than a preset time limit threshold, the editing object's editing of the target audio segment is canceled and the lock on the target audio segment is canceled.

[0093] In an optional implementation, the second acquisition module 404, when used for adding an operation time limit to the target audio segment, is specifically used to: When the frequency of editing the target audio segment triggered by the editing object is less than a preset threshold, determining the operation time limit according to the length of the target audio segment and a preset amplification factor; When the frequency is greater than or equal to the preset threshold, the operation time limit is determined according to the length of the target audio segment, the amplification factor, the frequency, the preset threshold and the time step.

[0094] In an optional implementation, the second acquisition module 404 is further configured to score the editing object according to the operation time limit of the editing object to obtain a scoring result corresponding to the editing object.

[0095] The audio editing device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a car-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0096] The audio editing device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0097] The audio editing device provided in the embodiment of the present application can achieve Figure 1 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0098] Alternatively, if Figure 5 As shown, the embodiment of the present application also provides an electronic device 500, including a processor 501 and a memory 502, the memory 502 stores a program or instruction that can be run on the processor 501, and the program or instruction is executed by the processor 501 to implement the above Figure 1 The various steps of the audio editing method shown in the figure can achieve the same technical effect. To avoid repetition, they will not be repeated here.

[0099] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0100] Figure 6FIG. 6 is a block diagram of another electronic device 600 according to an exemplary embodiment of the present application. The electronic device 600 may be implemented as a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart watch, a TV, etc. The electronic device 600 may also be called a user device, a portable terminal, a laptop terminal, a desktop terminal, etc.

[0101] Typically, the electronic device 600 includes a processor 601 and a memory 602 .

[0102] The processor 601 may include one or more processing cores, such as a 4-core processor, a 10-core processor, etc. The processor 601 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 601 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 601 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 601 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0103] The memory 602 may include one or more computer-readable storage media, which may be non-transitory. The memory 602 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 602 is used to store at least one instruction, which is used to be executed by the processor 601 to implement all or part of the steps in the audio editing method shown in the method embodiment of the present application.

[0104] In some embodiments, the electronic device 600 may further optionally include: a peripheral device interface 603 and at least one peripheral device. The processor 601, the memory 602 and the peripheral device interface 603 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 603 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 604, a display screen 605, a camera assembly 606, an audio circuit 607 and a power supply 608.

[0105] In some embodiments, the electronic device 600 further includes one or more sensors 609 , including but not limited to: an acceleration sensor 610 , a gyroscope sensor 611 , a pressure sensor 612 , an optical sensor 613 , and a proximity sensor 614 .

[0106] Those skilled in the art will understand that Figure 6 The structure shown in the figure does not constitute a limitation on the electronic device 600, and may include more or less components than those shown in the figure, or combine some components, or adopt a different component arrangement.

[0107] An embodiment of the present application also provides a computer-readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned audio editing method are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0108] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0109] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned audio editing method, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0110] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0111] The present application also provides a computer program product, wherein the computer program product includes a computer program stored on a non-transitory computer-readable storage medium, wherein the computer program includes program instructions, and when the program instructions are executed by a computer, the above Figure 1 The steps of the audio editing method shown in the figure can achieve the same technical effect. To avoid repetition, they will not be repeated here.

[0112] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the claims.

[0113] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. An audio editing method, characterized in that: include: Dividing the target audio to be edited into a plurality of audio particle data, and obtaining the voiceprints of the plurality of audio particle data; Determine the voiceprint similarity between the voiceprints of two adjacent audio particle data among the plurality of audio particle data; According to the voiceprint similarity, a plurality of the audio particle data are merged to obtain a target audio segment; The target audio segment is assigned to an editing object for editing, thereby obtaining an edited audio segment corresponding to the target audio segment.

2. The method according to claim 1, characterized in that The step of obtaining the voiceprint of the plurality of audio particle data comprises: The voiceprints of the multiple audio particle data are obtained by inputting the multiple audio particle data into a voiceprint network for voiceprint recognition.

3. The method according to claim 2, characterized in that The step of inputting the plurality of audio particles into a voiceprint network for voiceprint recognition to obtain the voiceprints of the plurality of audio particle data includes: By performing Fourier transform on the plurality of audio particle data, Mel spectra corresponding to the plurality of audio particle data are obtained; The first spectrum feature is obtained by inputting the Mel spectrum into a first layer structure of a voiceprint network, wherein the first layer structure includes PreNet, convolution Conv, a long short-term memory network LSTM with a self-attention mechanism Self Attention, and LSTM; Obtaining a second spectral feature by inputting the first spectral feature into a second layer structure of the voiceprint network, wherein the second layer structure has the same structure as the first layer structure; The voiceprints of the plurality of audio particle data are obtained by inputting the second spectrum feature into the fully connected layer of the voiceprint network.

4. The method according to claim 2, characterized in that: The voiceprint network is determined in the following manner: Constructing a speech synthesis model, wherein the speech synthesis model includes an acoustic model and a vocoder, and the acoustic model and the vocoder are cascaded with each other; By training the speech synthesis model, obtaining the trained speech synthesis model; The voiceprint network is determined according to the trained acoustic model in the speech synthesis model.

5. The method according to claim 4, characterized in that The step of training the speech synthesis model to obtain the trained speech synthesis model includes: Split multiple sample audios into sample audio granular data; Performing Fourier transform on the sample audio particle data to obtain a first Mel spectrum; Obtaining a voiceprint of the sample audio particle by inputting the first Mel-spectrogram into the acoustic model; Obtaining a second Mel spectrum by inputting the voiceprint of the sample audio particle into the vocoder; Updating the training parameters of the acoustic model and the vocoder by calculating the loss value between the first Mel spectrum and the second Mel spectrum; When the number of training times meets the preset number of training times or the loss value meets the preset conditions, the trained speech synthesis model is obtained.

6. The method according to claim 1, characterized in that The merging of the plurality of audio particle data according to the voiceprint similarity to obtain a target audio segment includes: According to the voiceprint similarity, a plurality of the audio particle data are merged to obtain a first initial audio segment; According to the length of the first initial audio segment and the voiceprint similarity, the first initial audio segment is merged to obtain a second initial audio segment; The second initial audio segments are sorted to obtain target audio segments, wherein the sorting includes merging the second initial audio segments and segmenting the second initial audio segments.

7. The method according to claim 6, characterized in that The merging the first initial audio segments according to the length of the first initial audio segments and the voiceprint similarity to obtain the second initial audio segment includes: Obtaining the length of the first initial audio segment; When the length of the first initial audio segment is less than the first length, the initial audio segments are merged according to the voiceprint similarity between the last audio particle data of the adjacent initial audio segment and the first audio particle data to obtain the second initial audio particle.

8. The method according to claim 6 or 7, characterized in that: After obtaining the second initial audio segment, the method further includes: Acquire text information corresponding to the second initial audio segment by performing speech recognition on the second initial audio segment; identifying, according to text information corresponding to the second initial audio segment, whether the second initial audio segment is a complete sentence; In the case that the second initial audio segment is an incomplete sentence, the target audio is segmented again to obtain a new second initial audio segment.

9. The method according to claim 8, characterized in that The step of arranging the second initial audio segment to obtain a target audio segment includes: Determining a target quantity according to the quantity of the editing objects; When the number of the second initial audio segments is less than the target number, starting from the second initial audio segment with the longest length, segmenting the second initial audio segments according to the text information corresponding to the second initial audio segments to obtain target audio segments; Obtaining the length of the second initial audio segment; When the length of the second initial audio segment is less than the second length, merging the second initial audio segment with an adjacent second initial audio segment with a smaller length to obtain the target audio segment; When the length of the second initial audio segment is greater than the third length, the second initial audio segment is segmented according to text information corresponding to the second initial audio segment to obtain the target audio segment, wherein the second length is less than the third length.

10. The method according to any one of claims 1 to 9, characterized in that: The step of obtaining edited audio corresponding to the target audio by assigning the target audio segment to an editing object for editing includes: Randomly assigning the plurality of target audio segments to the plurality of editing objects for collaborative editing; Locking the target audio segment; After the editing object completes editing the target audio segment, unlocking the target audio segment and determining a review object from the editing object; When the audit object completes the audit, the edited audio corresponding to the target audio is obtained.

11. The method according to claim 10, characterized in that After randomly allocating the plurality of target audio segments to the plurality of editing objects for collaborative editing, the method further includes: In response to the request of the editing object, if the editing object does not have the target audio segment being edited and the target audio segment corresponding to the request is not locked, the target audio segment is allocated to the editing object.

Images