Audio track data marking method, device, equipment and storage medium

Abstract

This application discloses a method, apparatus, device, and storage medium for annotating audio track data, pertaining to the field of audio processing. The method comprises: invoking an audio track separation model to separate at least one predicted track signal corresponding to an audio track from the audio to be processed; determining the audio to be processed as valid audio if the energy of the at least one predicted track signal meets a threshold; obtaining a track annotation result for the audio to be processed if the audio to be processed is determined to be valid based on the energy of the predicted track signal corresponding to the at least one audio track; and training the audio track separation model based on the audio to be processed and the track annotation result. This method can improve the efficiency of audio track data annotation.

Description

Technical Field

[0001] The present application relates to the field of audio processing, and in particular to a method, apparatus, device, and storage medium for labeling audio track data. Background Art

[0002] Audio usually includes track signals of multiple tracks. For example, song audio includes vocal audio and accompaniment audio, or accompaniment audio includes track audio of different instruments such as drum audio, bass audio, piano audio, etc.

[0003] In related technologies, it is necessary to manually label some track signals in the audio so that the labeled track signals can be used to perform corresponding functions. For example, in a song recording scenario, the accompaniment track audio in the song audio needs to be played separately in order to generate the recorded audio based on the recorded user voice audio and the accompaniment track audio.

[0004] However, relying on manual audition and labeling of track signals has high labor costs and low labeling efficiency. Summary of the Invention

[0005] This application provides a method, apparatus, device, and storage medium for annotating audio track data, which can improve the efficiency of audio track data annotation. The technical solution is as follows:

[0006] According to one aspect of the present application, a method for labeling audio track data is provided, the method comprising:

[0007] Calling the audio track separation model to separate the audio to be processed to obtain a predicted track signal corresponding to at least one audio track;

[0008] When the energy of the at least one predicted track signal satisfies a threshold condition, determining the audio to be processed as valid audio;

[0009] When it is determined that the audio to be processed is valid based on the energy of the predicted track signal corresponding to the at least one audio track, obtaining a track labeling result of the audio to be processed;

[0010] The audio track segmentation model is trained based on the audio to be processed and the track annotation result.

[0011] According to another aspect of the present application, a device for labeling audio track data is provided, the device comprising:

[0012] A separation module is used to call the audio track separation model to separate the audio to be processed to obtain a predicted track signal corresponding to at least one audio track;

[0013] a determination module, configured to determine the audio to be processed as valid audio if the energy of the at least one predicted track signal meets a threshold condition;

[0014] a labeling module, configured to obtain a track labeling result of the audio to be processed when it is determined that the audio to be processed is valid based on the energy of the predicted track signal corresponding to the at least one audio track;

[0015] A training module is used to train the audio track segmentation model based on the audio to be processed and the track annotation result.

[0016] According to another aspect of the present application, a computer device is provided, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the audio track data labeling method described above.

[0017] According to another aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores at least one instruction, at least one program, code set, or instruction set. The at least one instruction, at least one program, code set, or instruction set is loaded and executed by a processor to implement the audio track data labeling method described above.

[0018] According to another aspect of the present application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the audio track data labeling method provided in various optional implementations of the above aspects.

[0019] The beneficial effects of the technical solution provided by this application include at least:

[0020] The audio track separation model is used to perform audio track separation on the audio to be processed, and at least one predicted track signal corresponding to at least one audio track is obtained. Subsequently, the energy value of each predicted track signal is calculated, and based on the threshold conditions corresponding to the audio track, it is determined whether the audio to be processed contains the track signal of the audio track that needs to be labeled. If it does, the audio to be processed is determined to be valid data, and the original track signal of the valid data is manually listened to to label the audio track. If it does not contain, it means that the audio to be processed does not contain the track signal of the audio track that needs to be labeled, and there is no need to manually listen to the original track signal of the audio to be processed, thereby improving the efficiency of track labeling. In addition, after obtaining the track labeling results, the track labeling results can be used to train the audio track separation model, and the track labeling results can be used to improve the separation accuracy of the audio track separation model, more accurately screen valid data, and form a virtuous cycle of track data labeling. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0022] Figure 1 is a schematic diagram of the structure of a computer system provided by an exemplary embodiment of the present application;

[0023] Figure 2 is a flowchart of a method for labeling audio track data provided by an exemplary embodiment of the present application;

[0024] Figure 3 is a flowchart of a method for labeling audio track data provided by an exemplary embodiment of the present application;

[0025] Figure 4 is a schematic diagram of a method for labeling audio track data provided by an exemplary embodiment of the present application;

[0026] Figure 5 is a schematic diagram of a method for labeling audio track data provided by an exemplary embodiment of the present application;

[0027] Figure 6 is a schematic diagram of a method for labeling audio track data provided by an exemplary embodiment of the present application;

[0028] Figure 7 is a schematic diagram of a method for labeling audio track data provided by an exemplary embodiment of the present application;

[0029] Figure 8is a schematic diagram of a method for labeling audio track data provided by an exemplary embodiment of the present application;

[0030] Figure 9 is a flowchart of a method for labeling audio track data provided by an exemplary embodiment of the present application;

[0031] Figure 10 is a schematic diagram of a method for labeling audio track data provided by an exemplary embodiment of the present application;

[0032] Figure 11 This is a schematic structural diagram of an audio track data tagging device provided by an exemplary embodiment of the present application;

[0033] Figure 12 It is a structural diagram of a computer device provided by an exemplary embodiment of the present application.

[0034] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application. DETAILED DESCRIPTION

[0035] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0036] Figure 1 A schematic diagram of a computer system provided by an exemplary embodiment of the present application is shown. The computer system may include a terminal device 101 and a server 103.

[0037] For example, the audio track data annotation method shown in the embodiment of the present application can be applied to a terminal device 101 running an application 102 with audio track data annotation requirements. The terminal device may include a mobile phone, tablet computer, notebook computer, laptop computer, desktop computer, all-in-one computer, Internet of Things device, intelligent robot workstation, television, set-top box, smart glasses, smart watch, digital camera, MP4 player, MP5 player, learning machine, point reading machine, electronic paper book, electronic dictionary, vehicle-mounted device, virtual reality (VR) player or augmented reality (AR) player, etc.

[0038] Exemplarily, the audio track data annotation method provided in this application can be executed by a client in a terminal device. The client is a client of an application that requires audio track data annotation. For example, the application can include at least one of the following: an audio player, an audio application, a video application, a social application, a shopping application, a live broadcast application, an in-car audio playback application, an information application, a browser, a game application, or a recording application.

[0039] For example, an audio player provides a sing-along function, which requires separating the vocal audio and the accompaniment audio from the original audio and annotating the accompaniment audio. The audio player can use the audio track data annotation method provided in this application to separate the accompaniment audio from the original audio and annotate it for use by the sing-along function.

[0040] The terminal device 101 includes a first memory and a first processor. The first memory stores a program for labeling audio track data; the program for labeling audio track data is called and executed by the first processor to implement the audio track data labeling method provided in the present application. The first memory may include, but is not limited to, the following: random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), and electrically erasable programmable read-only memory (EEPROM).

[0041] The first processor may be composed of one or more integrated circuit chips. Optionally, the first processor may be a general-purpose processor, such as a central processing unit (CPU) or a network processor (NP). Optionally, the first processor may implement the audio track data annotation method provided in this application by running a program or code.

[0042] In an optional embodiment, the terminal device 101 and the server 103 may be connected to each other via a wired or wireless network.

[0043] Exemplarily, the audio track data annotation method provided in this application can also be performed by a server. Server 103 is used to provide background services for the target use case system of terminal device 101. Optionally, server 103 performs primary computing tasks, and terminal device 101 performs secondary computing tasks; alternatively, server 103 performs secondary computing tasks, and terminal device 101 performs primary computing tasks; alternatively, server 103 and terminal device 101 utilize a distributed computing architecture for collaborative computing.

[0044] The server 103 may be a single server, a server cluster consisting of multiple servers, or a cloud computing service center.

[0045] Optionally, server 103 includes a second memory and a second processor. The second memory stores a program for annotating audio track data; this program is invoked by the second processor to implement the audio track data annotation method provided herein. Optionally, the second memory may include, but is not limited to, RAM, ROM, PROM, EPROM, and EEPROM. Optionally, the second processor may be a general-purpose processor, such as a CPU or NP.

[0046] Figure 2 FIG. 1 is a flow chart of a method for labeling audio track data provided by an exemplary embodiment of the present application. The method can be used for Figure 1 The terminal device or server shown in FIG.

[0047] Step 210: Calling the audio track separation model to separate the audio to be processed to obtain a predicted track signal corresponding to at least one audio track.

[0048] Exemplarily, the audio to be processed is composed of at least one original track signal, and the audio track to which the at least one original track signal belongs is unknown. The method provided in the embodiment of the present application can label the original track signal of the target audio track that needs to be labeled.

[0049] For example, a requirement is to label the accompaniment track audio in all processed audio, but the processed audio may include at least one of the following: vocal track audio, accompaniment track audio, and noise track audio. Using methods from related art, manual auditioning of each original track of each processed audio is required to identify the accompaniment track audio, resulting in inefficient labeling. However, using the method provided in an embodiment of the present application, an audio track segmentation model can be used to first separate the predicted accompaniment track audio from each processed audio. The predicted accompaniment track audio is then roughly screened using the accompaniment track audio's energy threshold to determine whether the energy of the predicted accompaniment track audio meets the accompaniment track audio energy threshold. If so, the processed audio likely contains the accompaniment track audio, and the processed audio is labeled as valid data. If not, the processed audio likely does not contain the accompaniment track audio, and the processed audio is labeled as invalid data. Manual auditioning only requires auditioning the original track audio of the valid data to label the accompaniment track audio; auditioning of the original track audio of the invalid data is no longer necessary, significantly improving the efficiency of audio track data labeling.

[0050] The audio track separation model is a neural network model that is used to separate the predicted track signals corresponding to at least one audio track from the input audio to be processed. At least one audio track is an audio track that needs to be labeled (i.e., the target audio track). For example, the audio track separation model is trained to separate the predicted track signals of the first audio track, the second audio track, and the third audio track. After the audio to be processed is input into the audio track separation model, a first predicted track signal corresponding to the first audio track, a second predicted track signal corresponding to the second audio track, and a third predicted track signal corresponding to the third audio track can be obtained.

[0051] Exemplarily, the audio track separation model can be trained using a small number of training samples. For example, the training samples can use the open source dataset musdbhq. Each training sample includes: a sample audio signal (mixture), and the original track signals of four audio tracks of vocals, drums, bass, and other contained in the sample audio signal. During the training process, the sample audio signal is input into the audio track separation model to obtain the predicted track signals of the four audio tracks, and then the audio track separation model is trained based on the loss between the predicted track signal and the original track signal in the training sample.

[0052] Step 220: When the energy of at least one predicted track signal meets a threshold condition, the audio to be processed is determined to be valid audio.

[0053] For example, each audio track's track signal has a corresponding energy threshold. If the predicted track signal reaches the energy threshold, the predicted track signal is likely the track signal for that audio track. Therefore, the predicted track signal can be roughly screened based on its energy to determine if it is the track signal for that audio track.

[0054] If at least one predicted track signal contains a track signal of an audio track that needs to be marked, the audio to be processed is determined to be valid audio, so that the original track audio of the valid audio can be manually listened to and the track signal of the audio track that needs to be marked can be marked therefrom.

[0055] Exemplarily, the at least one predicted track signal includes a first predicted track signal corresponding to a first audio track separated by the audio track separation model. Valid data may be determined using one of the following methods:

[0056] 1) When the energy of the first prediction track signal meets the first threshold condition corresponding to the first audio track, the audio to be processed is determined to be valid data.

[0057] Specifically, when the energy of one of the at least one predicted track signals meets a threshold condition, the audio to be processed can be marked as valid data. For example, the threshold value in the threshold condition corresponding to an audio track can be set based on the average energy value of the audio track's historical audio signals, or based on the lowest energy value of the audio track's historical audio signals. Alternatively, the threshold value can be calculated by subtracting a preset error from the average energy value of the historical audio signals.

[0058] 2) When the energy of at least one predicted track signal satisfies the threshold conditions corresponding to the respective audio tracks, the audio to be processed is determined to be valid data.

[0059] That is, only when the energy of at least one predicted track signal meets the corresponding threshold conditions will the audio to be processed be marked as valid data. If the energy of at least one predicted track signal does not meet the threshold conditions of the corresponding audio track, the audio to be processed will be marked as invalid data.

[0060] Valid data is the audio to be processed that requires human audition. This data may contain the original track signal of the target audio track that needs to be annotated. In contrast, invalid data is the audio to be processed that does not require human audition. This data may not contain the original track signal of the target audio track that needs to be annotated.

[0061] Since the audio signals of different audio tracks have different energies (the energy of the audio signal is proportional to the square of the amplitude, and the energy of different audio tracks is different due to different amplitudes), for example, the audio energy of the suona track is usually higher than the first threshold, and the audio energy of the vocal track is usually higher than the second threshold, and the first threshold is higher than the second threshold (the amplitude of the suona audio is usually higher than the vocal audio). Therefore, the audio to be processed can be preliminarily screened according to the threshold corresponding to each audio track, and the predicted track signals that may belong to the audio track can be screened out, and the predicted track signals with lower energy and a high probability of not belonging to the audio track can be screened out. Subsequently, only the audio to be processed containing the predicted track signal of the audio track is manually auditioned and labeled, avoiding manual audition of a large amount of invalid data, wasting manpower costs, and improving the labeling efficiency of audio track data.

[0062] Step 230: When it is determined that the audio to be processed is valid based on the energy of the predicted track signal corresponding to at least one audio track, a track labeling result of the audio to be processed is obtained.

[0063] Exemplarily, track labeling is performed on the valid data to obtain a track labeling result, which includes the audio track type to which the original track signal belongs.

[0064] For example, you can manually audition valid data and label it as a track. After obtaining the valid data, you can manually audition the original track signal of the valid data. The original track signal is likely to contain the track signal of the target audio track that needs to be labeled. If the track signal of the target audio track is manually auditioned, it is labeled and the track labeling result is obtained. If the track signal of the target audio track is not manually auditioned, it means that the valid data does not contain the track signal of the target audio track and does not need to be labeled.

[0065] Alternatively, a track labeling model (neural network model) can be used to label the audio track to which each original track signal in the valid data belongs, thereby obtaining a track labeling result. The track labeling model is trained to output the audio track to which the original track signal belongs based on the input original track signal.

[0066] The track labeling result includes at least one original track signal for each audio track. Alternatively, the track labeling result includes the original track signal corresponding to the target audio track. For example, if the at least one original track signal includes a first original track signal, a second original track signal, and a third original track signal, the track labeling result may include: the first original track signal corresponds to the first audio track, the second original track signal corresponds to the second audio track, and the third original track signal corresponds to the third audio track. For another example, if the at least one original track signal includes the first original track signal, the second original track signal, and the third original track signal, and the requirement is to label the track signal of the second audio track, the track labeling result may include: the second original track signal corresponds to the second audio track, and the first original track signal and the third original track signal do not need to be labeled.

[0067] Step 240: Train an audio track segmentation model based on the audio to be processed and the track annotation results.

[0068] Exemplarily, when the audio to be processed is valid data and there is a track labeling result corresponding to the audio to be processed, the audio to be processed is used as sample input data, and the track labeling result is used as a sample label to train the audio track segmentation model. For example, the audio to be processed is input into the audio track segmentation model to obtain the predicted track signal corresponding to the target audio track; then, the original track signal corresponding to the target audio track in the track labeling result is obtained, the loss between the predicted track signal and the original track signal is calculated, and the audio track segmentation model is trained based on the loss. For example, the valid data includes the first original track signal, and the track labeling result includes that the first original track signal belongs to the first audio track. Then, the audio to be processed is input into the audio track segmentation model to obtain the first predicted track signal corresponding to the first audio track, and then the first loss between the first predicted track signal and the first original track signal is calculated, and the audio track segmentation model is trained based on the first loss.

[0069] Exemplarily, the audio track separation model includes a first network corresponding to a first audio track. The first network is configured to separate the first audio track from the input audio to be processed to obtain a predicted track signal for the first audio track. The track labeling result includes a first original track signal corresponding to the first audio track. The audio to be processed is used as sample input data for the first network, and the first original track signal is used as a sample label for the first network to train the audio track separation model.

[0070] This embodiment does not limit the network structure of the audio track splitting model. Any neural network model used to perform audio track splitting tasks can be applied to the audio track splitting data labeling method provided in this embodiment.

[0071] In summary, the method provided in this embodiment uses an audio track separation model to perform audio track separation on the audio to be processed, and obtains at least one predicted track signal corresponding to at least one audio track. Subsequently, the energy value of each predicted track signal is calculated, and based on the threshold condition corresponding to the audio track, it is determined whether the audio to be processed contains the track signal of the audio track that needs to be labeled. If it does, the audio to be processed is determined to be valid data, and the original track signal of the valid data is manually listened to to label the audio track. If it does not contain, it means that the audio to be processed does not contain the track signal of the audio track that needs to be labeled, and there is no need to manually listen to the original track signal of the audio to be processed, thereby improving the efficiency of track labeling. In addition, after obtaining the track labeling results, the track labeling results can be used to train the audio track separation model, and the track labeling results can be used to improve the separation accuracy of the audio track separation model, more accurately screen valid data, and form a virtuous cycle of track data labeling.

[0072] An exemplary embodiment for determining valid data is given.

[0073] Figure 3 FIG. 1 is a flow chart of a method for labeling audio track data provided by an exemplary embodiment of the present application. The method can be used for Figure 1 The terminal device or server shown. Figure 2 In the illustrated embodiment, step 220 includes step 221 .

[0074] Step 210: Calling an audio track separation model to separate at least one predicted track signal corresponding to an audio track from the audio to be processed; the audio to be processed is composed of at least one original track signal, and the audio track corresponding to the at least one original track signal is unknown.

[0075] The model structure of the audio track model can be arbitrary. Figure 4 The audio track splitting model provided by this embodiment is shown. The sub-network corresponding to the target audio track in the audio track splitting model includes a sub-band segmentation module, a time-frequency modeling module, and a masking value prediction module. When the audio track splitting model is used to separate the predicted track signals of multiple audio tracks, the audio track splitting model may include sub-networks corresponding to the multiple audio tracks, that is, the audio track splitting model includes Figure 4 Multiple subnetworks are shown.

[0076] For example, Figure 4 As shown, the processed audio X is input into the sub-band segmentation module to obtain the sub-band result Z. This sub-band segmentation result Z is then input into the time-frequency modeling module to obtain the modeling result Q. The modeling result is then input into the masking value prediction module to obtain the masking value M corresponding to the target audio track. The masking value M is multiplied by the processed audio X to obtain the predicted track signal S corresponding to the target audio track.

[0077] like Figure 5 The figure shows a subband segmentation module provided by this embodiment. The subband segmentation module includes a segmentation network, a normalization and convolution network 501, and a merging network. The audio to be processed, X, is input into the segmentation network. The segmentation network performs a short-time Fourier transform on the audio to be processed, obtaining a frequency domain signal. The frequency points of the frequency domain signal are then divided into at least one subband, obtaining a frequency domain signal for each subband. Subsequently, the frequency domain signal for each subband is input into the corresponding normalization and convolution network 501, obtaining a convolution result for each subband. The normalization and convolution network 501 includes a normalization layer and at least one convolution layer. Subsequently, the convolution result for each subband is input into the merging network, which performs data splicing in the frequency domain and merges the convolution results for each subband into a band segmentation result, Z.

[0078] like Figure 6 As shown, a time-frequency modeling module is provided in this embodiment. The time-frequency modeling module can be implemented using RNN (Recurrent Neural Network). For example, the time-frequency modeling module may include a time-domain RNN network and a frequency-domain RNN network; the band-splitting result Z is input into the time-domain RNN network according to the time-domain sequence to obtain an intermediate result Z'. The time-domain RNN network includes a normalization layer, a BLSTM (Bidirectional Long Short-Term Memory) and a convolutional network. The intermediate result Z' is input into the frequency-domain RNN network according to the frequency-domain sequence to obtain a modeling result Q. The frequency-domain RNN network includes a normalization layer, a BLSTM and a convolutional network.

[0079] like Figure 6 As shown, a masking value prediction module is provided in this embodiment. The masking value prediction module includes a segmentation network, a normalization and MLP (Multilayer Perceptron) network 502 and a merging network. The modeling result Q is input into the segmentation network, and the segmentation network divides the modeling result Q into at least one sub-band to obtain the modeling result of each sub-band. Subsequently, the modeling result of each sub-band is respectively input into the normalization and MLP network 502 corresponding to the sub-band to obtain the prediction result corresponding to each sub-band. The normalization and MLP network 502 includes a normalization layer and an MLP network. Subsequently, the prediction result corresponding to each sub-band is input into the merging network, data splicing is performed in the frequency domain, and the prediction result of each sub-band is merged into a masking value M. The masking value M is then multiplied by the audio to be processed X to obtain the first predicted track signal of the first audio track.

[0080] Step 221: When the energy of at least one predicted track signal satisfies the threshold conditions corresponding to the respective audio tracks, the audio to be processed is determined to be valid data.

[0081] Taking the first predicted track signal corresponding to the first audio track as an example, calculate the first energy value of the first predicted track signal; and calculate the sum of the energy values ​​of at least one predicted track signal; calculate the ratio of the first energy value to the sum of the energy values; when the ratio is not less than the first threshold corresponding to the first audio track, determine that the energy of the first predicted track signal meets the threshold condition corresponding to the first track.

[0082] For example, the predicted track signal includes four audio tracks: vocals, drums, bass, and other. The threshold conditions corresponding to the four audio tracks can be:

[0083] Vocals:

[0084] drum:

[0085] Bass:

[0086] other:

[0087] Among them, y v (t) is the predicted track signal of the vocal track, y d (t) is the predicted track signal of the drum track, y b (t) is the predicted orbit signal of the bass orbit, y o (t) is the predicted orbit signal of other orbits. v is the threshold for the vocal track, α d is the threshold of the drum track, α b is the threshold of the bass track, α o is the threshold for other tracks.

[0088] Step 230: When it is determined that the audio to be processed is valid based on the energy of the predicted track signal corresponding to at least one audio track, a track labeling result of the audio to be processed is obtained, where the track labeling result includes an audio track corresponding to at least one original track signal.

[0089] Step 240: Train an audio track segmentation model based on the audio to be processed and the track annotation results.

[0090] For example, Figure 8As shown, the audio to be processed 301 is input into the audio track model 302, generating four predicted track signals on four audio tracks. Energy determination 304 is then performed on the four predicted track signals. If the energy of each of the four predicted track signals meets the threshold for each audio track, the audio to be processed 301 is determined to be valid data. The valid data is then provided to a human for labeling. The results of the manual labeling can be used to train the audio track model 302 and can also be used to synthesize the audio to be processed 301.

[0091] In summary, the method provided in this embodiment uses an audio track separation model to perform audio track separation on the audio to be processed, and obtains at least one predicted track signal corresponding to at least one audio track. Subsequently, for the predicted track signal of an audio track, the ratio of the energy value of the predicted track signal to the total energy value of all predicted track signals is calculated. When the ratio is greater than the capacity threshold corresponding to the audio track, it is determined that the predicted track signal of the audio track meets the threshold condition. When at least one predicted bone carving signal meets the threshold condition, the audio to be processed is marked as valid data. Invalid data of the track signal that does not contain the target audio track is screened out, and track labeling is performed only on valid data to improve the efficiency of track labeling.

[0092] An exemplary embodiment of training an audio track segmentation model is given below.

[0093] Figure 9 FIG. 1 is a flow chart of a method for labeling audio track data provided by an exemplary embodiment of the present application. The method can be used for Figure 1 The terminal device or server shown. Figure 2 In the illustrated embodiment, step 240 of the method includes steps 241 to 245 .

[0094] Step 241: Input the audio to be processed into an audio track separation model to obtain a first number of predicted track signals.

[0095] The first number is the number of audio tracks that the audio track splitting model can separate. For example, if the audio track splitting model is used to separate track signals of three audio tracks, the first number is 3.

[0096] For another example, the audio to be processed is input into the audio track splitting model to obtain four predicted track signals on four audio tracks: vocals, drums, bass, and other.

[0097] Then, the loss between the predicted track signal and the original track signal is calculated.

[0098] For example, the audio to be processed is input into the audio track segmentation model to obtain a first number of predicted track signals, and then the loss between the first number of predicted track signals and the first number of original track signals is calculated based on the manual labeling results. The loss calculation method is as follows.

[0099] The loss function is:

[0100] Among them, L loss For loss, is the time domain loss of the audio signal of J audio tracks (ie, J is the first number), is the frequency domain loss of the audio signal of J audio tracks. J is the number of audio tracks. α is the scaling factor between time domain loss and frequency domain loss, which is a preset value.

[0101] The time domain loss function is:

[0102] Among them, y j (t) is the original track signal (label data or real data) of the j-th audio track, is the predicted track signal of the j-th audio track, and ‖‖1 is the first-order norm.

[0103] The frequency domain loss function is:

[0104]

[0105] in, is the resolution loss of the i-th resolution, and M is the total number of resolutions. is the amplitude loss of the jth audio track. is the logarithmic loss of the j audio tracks. STFT(y(t)) is the frequency domain signal obtained by short-time Fourier transform of the original track signal. is the predicted frequency domain signal obtained by performing short-time Fourier transform on the predicted orbit signal. F To take the second-order norm.

[0106] According to the calculation formula of the above loss function, it can be obtained that the loss calculation method includes the following steps 242 to 244.

[0107] Step 242: Calculate a first number of predicted trajectory signals The original track signal y of the corresponding audio track j (t) time domain loss

[0108] Exemplarily, for one of the first number of audio tracks, the first-order norm of the amplitude difference between the predicted track signal and the original track signal in the time domain is calculated. The sum of the time domain loss of an audio track is obtained. Calculate the sum of the time domain losses of the first number of audio tracks Get time domain loss

[0109] Step 243: Calculate a first number of predicted trajectory signals The original track signal y of the corresponding audio track j Frequency domain loss of (t)

[0110] Exemplarily, for one of the first number of audio tracks, the amplitude loss between the predicted track signal and the original track signal in the frequency domain is calculated. and log loss Calculate the sum of amplitude loss and logarithmic loss to get the sub-track frequency domain loss of an audio track Calculate the sum of the sub-track frequency domain losses of the first number of audio tracks to obtain the frequency domain loss

[0111] Exemplarily, the frequency domain loss of a sub-track of an audio track may be calculated as follows: performing a short-time Fourier transform on the predicted track signal according to at least one set of resolutions to obtain at least one set of predicted frequency domain signals; one set of resolutions in the at least one set of resolutions includes a window length and an offset step size; performing a short-time Fourier transform on the original track signal according to the at least one set of resolutions to obtain at least one set of frequency domain signals; for one set of resolutions in the at least one set of resolutions, calculating the amplitude loss and logarithmic loss of the predicted frequency domain signal and the frequency domain signal, and determining the sum of the amplitude loss and logarithmic loss as the resolution loss corresponding to the set of resolutions. Calculate the sum of the resolution loss of at least one set of resolutions to obtain the sub-track frequency domain loss of an audio track

[0112] The amplitude loss can be calculated by calculating the difference between the absolute value of the amplitude of the frequency domain signal and the predicted frequency domain signal to obtain the amplitude difference. Calculate the second norm of the amplitude difference and get the first second norm Calculate the second norm of the absolute value of the amplitude of the frequency domain signal and obtain the second second norm ‖|STFT(y(t))|‖ F ; Calculate the ratio of the first and second norms to get the amplitude loss

[0113] The logarithmic loss can be calculated by: calculating the common logarithm of the absolute value of the amplitude of the frequency domain signal to obtain the first logarithm log|STFT(y(t))|; calculating the common logarithm of the absolute value of the amplitude of the predicted frequency domain signal to obtain the second logarithm Calculate the first norm of the difference between the first and second pairs Get logarithmic loss

[0114] Step 244: Calculate the weighted sum of the time domain loss and the frequency domain loss The ratio of the weighted sum to the first quantity Determined as the loss value L loss .

[0115] Step 245: Train the audio track segmentation model based on the loss value.

[0116] For example, Figure 10 As shown, after the audio to be processed 301 is input into the audio track segmentation model 302, four predicted track signals of the four audio tracks are obtained, and then the loss value is calculated according to the label, and the audio track segmentation model is trained based on the loss value.

[0117] In summary, the method provided in this embodiment uses an audio track separation model to perform audio track separation on the audio to be processed, obtaining at least one predicted bone carving signal corresponding to at least one audio track. Based on the energy of the predicted track signal, the processed audio is determined to be valid data, and the valid data is manually labeled. Based on the manually labeled results, the processed audio is used as a training sample to train the audio track separation model, thereby improving the separation accuracy of the audio track separation model and more accurately screening valid data, forming a virtuous cycle of track data labeling.

[0118] It should be noted that the order of the method steps provided in the embodiments of the present application can be appropriately adjusted, and the steps can be increased or decreased accordingly according to the circumstances. Any technical personnel familiar with this technical field can easily think of the changed methods within the technical scope disclosed in this application, and they should be covered within the scope of protection of this application, so they will not be repeated here.

[0119] Figure 11 1 is a schematic diagram of a device for labeling audio track data provided by an exemplary embodiment of the present application. The device includes:

[0120] The separation module 601 is configured to call an audio track separation model to separate the audio to be processed from the audio to be processed to obtain a predicted track signal corresponding to at least one audio track; the audio to be processed is composed of at least one original track signal, and the audio track corresponding to the at least one original track signal is unknown;

[0121] A determination module 602 is configured to determine the audio to be processed as valid audio if the energy of the at least one predicted track signal meets a threshold condition;

[0122] a labeling module 603 configured to, when determining that the audio to be processed is valid based on the energy of the predicted track signal corresponding to the at least one audio track, obtain a track labeling result of the audio to be processed, the track labeling result including the audio track corresponding to the at least one original track signal;

[0123] The training module 604 is used to train the audio track segmentation model based on the audio to be processed and the track labeling result.

[0124] In an optional embodiment, the at least one predicted track signal includes a first predicted track signal corresponding to the first audio track separated by the audio track separation model;

[0125] The determination module 602 is configured to perform one of the following:

[0126] When the energy of the first predicted track signal meets a first threshold condition corresponding to the first audio track, determining the audio to be processed as the valid data;

[0127] When the energy of the at least one predicted track signal satisfies the threshold conditions corresponding to the respective audio tracks, the audio to be processed is determined as the valid data.

[0128] In an optional embodiment, the determination module 602 is configured to calculate a first energy value of the first predicted trajectory signal; and calculate a sum of energy values ​​of the at least one predicted trajectory signal;

[0129] The determination module 602 is configured to calculate a ratio of the first energy value to the sum of the energy values;

[0130] The determination module 602 is configured to determine, when the ratio is not less than a first threshold corresponding to the first audio track, whether the energy of the first predicted track signal satisfies a threshold condition corresponding to the first track.

[0131] In an optional embodiment, the audio track separation model includes a first network corresponding to a first audio track, and the first network is used to separate the input audio to be processed to obtain a predicted track signal of the first audio track;

[0132] The track marking result includes that the first original track signal in the audio to be processed corresponds to the first audio track;

[0133] The training module 604 is configured to train the audio track segmentation model by using the audio to be processed as sample input data of the first network and the first original track signal as a sample label of the first network.

[0134] In an optional embodiment, the training module 604 is configured to input the audio to be processed into the audio track separation model to obtain a first number of predicted track signals;

[0135] The training module 604 is configured to calculate a time domain loss between the first number of predicted track signals and the original track signals of the corresponding audio tracks; and calculate a frequency domain loss between the first number of predicted track signals and the original track signals of the corresponding audio tracks;

[0136] The training module 604 is configured to calculate a weighted sum of the time domain loss and the frequency domain loss; and determine a ratio of the weighted sum to the first number as a loss value;

[0137] The training module 604 is configured to train the audio track segmentation model based on the loss value.

[0138] In an optional embodiment, the training module 604 is configured to calculate, for one audio track from the first number of audio tracks, a sum of first-order norms of amplitude differences between the predicted track signal and the original track signal in the time domain to obtain a time domain loss of the audio track.

[0139] The training module 604 is configured to calculate the sum of the time domain losses of the first number of audio tracks to obtain the time domain loss.

[0140] In an optional embodiment, the training module 604 is configured to calculate, for one audio track among the first number of audio tracks, an amplitude loss and a logarithmic loss between the predicted track signal and the original track signal in the frequency domain, and calculate the sum of the amplitude loss and the logarithmic loss to obtain a per-track frequency domain loss for the audio track;

[0141] The training module 604 is configured to calculate the sum of the frequency domain losses of the first number of audio tracks to obtain the frequency domain loss.

[0142] In an optional embodiment, the training module 604 is configured to perform a short-time Fourier transform on the predicted orbit signal according to at least one set of resolutions to obtain at least one set of predicted frequency domain signals; one set of resolutions in the at least one set of resolutions includes a window length and an offset step size;

[0143] The training module 604 is configured to perform a short-time Fourier transform on the original track signal according to the at least one set of resolutions to obtain at least one set of frequency domain signals;

[0144] The training module 604 is configured to calculate, for a set of resolutions among the at least one set of resolutions, an amplitude loss and a logarithmic loss between the predicted frequency domain signal and the frequency domain signal, and determine a sum of the amplitude loss and the logarithmic loss as a resolution loss corresponding to the set of resolutions;

[0145] The training module 604 is configured to calculate the sum of the resolution losses of the at least one set of resolutions to obtain a sub-track frequency domain loss of an audio track.

[0146] In an optional embodiment, the training module 604 is configured to calculate a difference between an absolute value of an amplitude of the frequency domain signal and an absolute value of an amplitude of the predicted frequency domain signal to obtain an amplitude difference;

[0147] The training module 604 is configured to calculate the second norm of the amplitude difference to obtain a first second norm; and calculate the second norm of the absolute value of the amplitude of the frequency domain signal to obtain a second second norm;

[0148] The training module 604 is configured to calculate a ratio of the first second norm to the second second norm to obtain the amplitude loss.

[0149] In an optional embodiment, the training module 604 is configured to calculate the common logarithm of the absolute value of the amplitude of the frequency domain signal to obtain a first logarithm; and calculate the common logarithm of the absolute value of the amplitude of the predicted frequency domain signal to obtain a second logarithm;

[0150] The training module 604 is configured to calculate a first-order norm of a difference between the first logarithm and the second logarithm to obtain the logarithmic loss.

[0151] It should be noted that the audio track data annotation device provided in the above embodiment is merely illustrated by the division of the aforementioned functional modules. In actual applications, the aforementioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. Furthermore, the audio track data annotation device provided in the above embodiment and the audio track data annotation method embodiment are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0152] Embodiments of the present application further provide a computer device comprising: a processor and a memory, wherein the memory stores at least one instruction, at least one program, code set, or instruction set, and the processor loads and executes the at least one instruction, at least one program, code set, or instruction set to implement the audio track data annotation methods provided in the above-described method embodiments. The computer device may be implemented as a terminal device.

[0153] For example, Figure 12It is a structural diagram of a computer device provided by an exemplary embodiment of the present application.

[0154] Typically, the computer device 1700 includes a processor 1701 and a memory 1702 .

[0155] The processor 1701 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 1701 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), or PLA (Programmable Logic Array). The processor 1701 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 1701 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 1701 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0156] Memory 1702 may include one or more computer-readable storage media, which may be non-transitory. Memory 1702 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in memory 1702 is used to store at least one instruction, which is executed by processor 1701 to implement the audio track data labeling method provided in the method embodiment of the present application.

[0157] In some embodiments, computer device 1700 may optionally include a peripheral device interface 1703 and at least one peripheral device. Processor 1701, memory 1702, and peripheral device interface 1703 may be connected via a bus or signal lines. Each peripheral device may be connected to peripheral device interface 1703 via a bus, signal lines, or circuit boards. Specifically, the peripheral device may include at least one of a radio frequency circuit 1704, a display screen 1705, a camera assembly 1706, an audio circuit 1707, and a power supply 1708.

[0158] The peripheral device interface 1703 can be used to connect at least one I / O (Input / Output)-related peripheral device to the processor 1701 and the memory 1702. In some embodiments, the processor 1701, the memory 1702, and the peripheral device interface 1703 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 1701, the memory 1702, and the peripheral device interface 1703 can be implemented on separate chips or circuit boards, which is not limited in this embodiment of the present application.

[0159] RF circuit 1704 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. RF circuit 1704 communicates with communication networks and other communication devices via electromagnetic signals. RF circuit 1704 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. RF circuit 1704 optionally includes an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like. RF circuit 1704 can communicate with other computer devices via at least one wireless communication protocol. Such wireless communication protocols include, but are not limited to, the World Wide Web, metropolitan area networks, intranets, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and / or WiFi (Wireless Fidelity) networks. In some embodiments, RF circuit 1704 may also include circuitry related to Near Field Communication (NFC), although this application does not limit this.

[0160] The display screen 1705 is used to display a UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 1705 is a touch screen display, the display screen 1705 also has the ability to collect touch signals on the surface or above the surface of the display screen 1705. The touch signal can be input as a control signal to the processor 1701 for processing. At this time, the display screen 1705 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, there can be one display screen 1705, which is set on the front panel of the computer device 1700; in other embodiments, there can be at least two display screens 1705, which are respectively set on different surfaces of the computer device 1700 or in a folding design; in still other embodiments, the display screen 1705 can be a flexible display screen, which is set on the curved surface or folding surface of the computer device 1700. Even more, the display screen 1705 can be set as a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 1705 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0161] The camera assembly 1706 is used to capture images or videos. Optionally, the camera assembly 1706 includes a front camera and a rear camera. Typically, the front camera is set on the front panel of the computer device 1700, and the rear camera is set on the back of the computer device. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 1706 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation under different color temperatures.

[0162] The audio circuit 1707 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals that are input into the processor 1701 for processing, or input into the radio frequency circuit 1704 to achieve voice communication. For the purpose of stereo sound collection or noise reduction, there may be multiple microphones, each located in different parts of the computer device 1700. The microphone may also be an array microphone or an omnidirectional collection microphone. The speaker is used to convert electrical signals from the processor 1701 or the radio frequency circuit 1704 into sound waves. The speaker may be a traditional thin film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert electrical signals into sound waves audible to humans, but also convert electrical signals into sound waves inaudible to humans for purposes such as distance measurement. In some embodiments, the audio circuit 1707 may also include a headphone jack.

[0163] Power supply 1708 is used to power various components in computer device 1700. Power supply 1708 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 1708 includes a rechargeable battery, the rechargeable battery can be wired or wirelessly rechargeable. A wired rechargeable battery is charged via a wired line, while a wireless rechargeable battery is charged via a wireless coil. The rechargeable battery can also support fast charging technology.

[0164] In some embodiments, the computer device 1700 further includes one or more sensors 1709 , including but not limited to an acceleration sensor 1710 , a gyroscope sensor 1711 , a pressure sensor 1712 , an optical sensor 1713 , and a proximity sensor 1714 .

[0165] The accelerometer 1710 can detect the magnitude of acceleration along the three coordinate axes of the coordinate system established by the computer device 1700. For example, the accelerometer 1710 can be used to detect the components of gravity acceleration along the three coordinate axes. The processor 1701 can control the touch screen display 1705 to display the user interface in a landscape or portrait view based on the gravity acceleration signal collected by the accelerometer 1710. The accelerometer 1710 can also be used to collect game or user motion data.

[0166] The gyroscope sensor 1711 can detect the orientation and rotation angle of the computer device 1700. It can also work with the accelerometer 1710 to collect 3D motions of the user on the computer device 1700. Based on the data collected by the gyroscope sensor 1711, the processor 1701 can implement the following functions: motion sensing (e.g., changing the UI based on the user's tilt), image stabilization during shooting, game control, and inertial navigation.

[0167] The pressure sensor 1712 can be installed on the side frame of the computer device 1700 and / or below the touch screen display 1705. When the pressure sensor 1712 is installed on the side frame of the computer device 1700, it can detect the user's grip signal of the computer device 1700, and the processor 1701 can perform left and right hand recognition or shortcut operations based on the grip signal collected by the pressure sensor 1712. When the pressure sensor 1712 is installed below the touch screen display 1705, the processor 1701 controls the operational controls on the UI interface based on the user's pressure operation on the touch screen display 1705. The operational controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0168] Optical sensor 1713 is used to detect ambient light intensity. In one embodiment, processor 1701 can control the display brightness of touchscreen display 1705 based on the ambient light intensity detected by optical sensor 1713. Specifically, when the ambient light intensity is high, the display brightness of touchscreen display 1705 is increased; when the ambient light intensity is low, the display brightness of touchscreen display 1705 is decreased. In another embodiment, processor 1701 can also dynamically adjust the shooting parameters of camera assembly 1706 based on the ambient light intensity detected by optical sensor 1713.

[0169] Proximity sensor 1714, also known as a distance sensor, is typically located on the front panel of computer device 1700. Proximity sensor 1714 is used to detect the distance between the user and the front of computer device 1700. In one embodiment, when proximity sensor 1714 detects that the distance between the user and the front of computer device 1700 is gradually decreasing, processor 1701 controls touchscreen display 1705 to switch from the screen-on state to the screen-off state. When proximity sensor 1714 detects that the distance between the user and the front of computer device 1700 is gradually increasing, processor 1701 controls touchscreen display 1705 to switch from the screen-off state to the screen-on state.

[0170] Those skilled in the art will understand that Figure 12 The structure shown in the figure does not constitute a limitation on the computer device 1700, and the computer device 1700 may include more or fewer components than shown in the figure, or combine some components, or adopt a different component arrangement.

[0171] An embodiment of the present application also provides a computer-readable storage medium, which stores at least one instruction, at least one program, code set, or instruction set. When the at least one instruction, at least one program, code set, or instruction set is loaded and executed by a processor of a computer device, the audio track data labeling method provided by the above-mentioned method embodiments is implemented.

[0172] The present application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the audio track data annotation methods provided in the above-mentioned method embodiments.

[0173] Those skilled in the art will understand that all or part of the steps of implementing the above embodiments may be accomplished by hardware, or may be accomplished by a program instructing the relevant hardware, and the program may be stored in a computer-readable storage medium, and the above-mentioned readable storage medium may be a read-only memory, a disk or an optical disk, etc.

[0174] The above description is merely an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent switches, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for labeling audio track data, characterized in that: The method comprises: Calling the audio track separation model to separate the audio to be processed to obtain a predicted track signal corresponding to at least one audio track; When the energy of the at least one predicted track signal satisfies a threshold condition, determining the audio to be processed as valid audio; When it is determined that the audio to be processed is valid based on the energy of the predicted track signal corresponding to the at least one audio track, obtaining a track labeling result of the audio to be processed; Training the audio track segmentation model based on the audio to be processed and the track annotation result; The at least one predicted track signal includes a first predicted track signal corresponding to the first audio track separated by the audio track separation model; The step of determining the to-be-processed audio as valid data when the energy of the at least one predicted track signal satisfies a threshold condition comprises one of the following: When the energy of the first predicted track signal meets a first threshold condition corresponding to the first audio track, determining the audio to be processed as the valid data; When the energy of the at least one predicted track signal satisfies the threshold conditions corresponding to the respective audio tracks, the audio to be processed is determined as the valid data.

2. The method according to claim 1, characterized in that The energy of the first predicted track signal satisfies a threshold condition corresponding to the first audio track, including: calculating a first energy value of the first predicted track signal; and calculating a sum of energy values ​​of the at least one predicted track signal; calculating a ratio of the first energy value to the sum of the energy values; When the ratio is not less than a first threshold corresponding to the first audio track, it is determined that the energy of the first prediction track signal meets the threshold condition corresponding to the first audio track.

3. The method according to claim 1 or 2, characterized in that The audio track separation model includes a first network corresponding to a first audio track, wherein the first network is used to separate the input audio to be processed to obtain a predicted track signal of the first audio track; The track marking result includes that the first original track signal in the audio to be processed corresponds to the first audio track; The step of training the audio track segmentation model based on the audio to be processed and the track annotation result includes: The audio to be processed is used as sample input data of the first network, and the first original track signal is used as the sample label of the first network to train the audio track segmentation model.

4. The method according to claim 1 or 2, characterized in that The step of training the audio track segmentation model based on the audio to be processed and the track annotation result includes: Inputting the audio to be processed into the audio track division model to obtain a first number of predicted track signals; Calculating a time domain loss between the first number of predicted track signals and the original track signals of the corresponding audio tracks; and calculating a frequency domain loss between the first number of predicted track signals and the original track signals of the corresponding audio tracks; Calculating a weighted sum of the time domain loss and the frequency domain loss; and determining a ratio of the weighted sum to the first number as a loss value; The audio track segmentation model is trained based on the loss value.

5. The method according to claim 4, characterized in that The calculating the time domain loss between the first number of predicted track signals and the original track signals of the corresponding audio tracks includes: For one audio track from the first number of audio tracks, calculating the sum of the first-order norms of the amplitude differences between the predicted track signal and the original track signal in the time domain to obtain a time domain loss of the audio track; The sum of the time domain losses of the sub-tracks of the first number of audio tracks is calculated to obtain the time domain loss.

6. The method according to claim 4, characterized in that The calculating the frequency domain loss between the first number of predicted track signals and the original track signal of the corresponding audio track includes: For one audio track of the first number of audio tracks, calculating the amplitude loss and logarithmic loss in the frequency domain between the predicted track signal and the original track signal, and calculating the sum of the amplitude loss and the logarithmic loss to obtain a sub-track frequency domain loss of the audio track; The sum of the sub-track frequency domain losses of the first number of audio tracks is calculated to obtain the frequency domain loss.

7. The method according to claim 6, characterized in that The calculating the amplitude loss and logarithmic loss of the predicted track signal and the original track signal in the frequency domain, and calculating the sum of the amplitude loss and the logarithmic loss to obtain the sub-track frequency domain loss of an audio track, includes: Performing a short-time Fourier transform on the predicted trajectory signal according to at least one set of resolutions to obtain at least one set of predicted frequency domain signals; one set of resolutions in the at least one set of resolutions includes a window length and an offset step size; Performing a short-time Fourier transform on the original track signal according to the at least one set of resolutions to obtain at least one set of frequency domain signals; For one set of resolutions among the at least one set of resolutions, calculating an amplitude loss and a logarithmic loss between the predicted frequency domain signal and the frequency domain signal, and determining a sum of the amplitude loss and the logarithmic loss as a resolution loss corresponding to one set of resolutions; The sum of the resolution losses of the at least one set of resolutions is calculated to obtain a sub-track frequency domain loss of an audio track.

8. The method according to claim 7, characterized in that The calculating the amplitude loss between the predicted frequency domain signal and the frequency domain signal includes: Calculating the difference between the absolute values ​​of the amplitudes of the frequency domain signal and the predicted frequency domain signal to obtain an amplitude difference; Calculating the second norm of the amplitude difference to obtain a first second norm; and calculating the second norm of the absolute value of the amplitude of the frequency domain signal to obtain a second second norm; The ratio of the first second norm to the second second norm is calculated to obtain the amplitude loss.

9. The method according to claim 7, characterized in that The calculating the logarithmic loss between the predicted frequency domain signal and the frequency domain signal includes: Calculating the common logarithm of the absolute value of the amplitude of the frequency domain signal to obtain a first logarithm; and calculating the common logarithm of the absolute value of the amplitude of the predicted frequency domain signal to obtain a second logarithm; A first-order norm of a difference between the first logarithm and the second logarithm is calculated to obtain the logarithmic loss.

10. A device for labeling audio track data, characterized in that: The device comprises: A separation module is used to call the audio track separation model to separate the audio to be processed to obtain a predicted track signal corresponding to at least one audio track; a determination module, configured to determine the audio to be processed as valid audio if the energy of the at least one predicted track signal meets a threshold condition; a labeling module, configured to obtain a track labeling result of the audio to be processed when it is determined that the audio to be processed is valid based on the energy of the predicted track signal corresponding to the at least one audio track; A training module, configured to train the audio track segmentation model based on the audio to be processed and the track annotation result; The at least one predicted track signal includes a first predicted track signal corresponding to the first audio track separated by the audio track separation model; The step of determining the to-be-processed audio as valid data when the energy of the at least one predicted track signal satisfies a threshold condition comprises one of the following: When the energy of the first predicted track signal meets a first threshold condition corresponding to the first audio track, determining the audio to be processed as the valid data; When the energy of the at least one predicted track signal satisfies the threshold conditions corresponding to the respective audio tracks, the audio to be processed is determined as the valid data.

11. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one program, and the at least one program is loaded and executed by the processor to implement the audio track data labeling method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that The readable storage medium stores at least one program, and the at least one program is loaded and executed by a processor to implement the audio track data marking method according to any one of claims 1 to 9.

13. A computer program product, characterized in that The computer program product includes computer instructions, which are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the audio track data labeling method according to any one of claims 1 to 9.

Images