Audio track processing method, device, electronic device and storage medium

Abstract

Embodiments of the present invention provide a method, device, electronic device and storage medium for processing tracks. The track alignment method includes: determining the playback delay of an accompaniment track and the target recording progress of a recording track when starting to play the accompaniment track, wherein the recording track is used to record the singing audio when the accompaniment track is played; aligning the recording track with the accompaniment track by removing the playback delay of the accompaniment track and the portion corresponding to the target recording progress in the recording track. Track alignment is achieved simply by processing the recording track; in addition, the accuracy of track alignment is achieved, and the recording effect is improved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of computer technology, and in particular to a method, device, electronic device and storage medium for processing an audio track. Background Art

[0002] In the prior art professional recording process such as singing recording, the accompaniment playback and the vocal recording are collected separately, and are later synthesized using separate equipment.

[0003] Using a singing recording application installed in a mobile terminal can simplify all of the above processes and implement quick singing recording in a single mobile terminal. However, when a user previews the singing, he or she will still find that the recording effect is poor. Summary of the invention

[0004] In view of this, embodiments of the present invention provide a method, device, electronic device and storage medium for processing an audio track to solve or alleviate the above problems.

[0005] According to a first aspect of an embodiment of the present invention, there is provided a method for processing an audio track, comprising: determining a playback delay of an accompaniment audio track and a target recording progress of a recording audio track when starting to play the accompaniment audio track, wherein the recording audio track is used to record singing audio when the accompaniment audio track is played; and aligning the recording audio track with the accompaniment audio track by removing the portion corresponding to the playback delay of the accompaniment audio track and the target recording progress in the recording audio track.

[0006] According to a second aspect of an embodiment of the present invention, there is provided a method for processing an audio track, comprising: performing synthesis processing based on aligned recorded audio tracks and accompaniment audio tracks, wherein the recorded audio tracks and accompaniment audio tracks are obtained by processing using the audio track processing method of the first aspect; and performing post-processing on the synthesized audio track based on an audio track post-processing mode.

[0007] According to a third aspect of an embodiment of the present invention, there is provided a method for processing an audio track, comprising: starting the playback of an accompaniment audio track when a recording audio track is recorded to a target recording progress, wherein the recording audio track is used to record the singing audio when the accompaniment audio track is played; calculating the playback delay of the accompaniment audio track by playing the accompaniment audio track, so as to remove the playback delay and the target recording progress in the recording audio track, and aligning the recording audio track and the accompaniment audio track.

[0008] According to a fourth aspect of an embodiment of the present invention, a method for processing an audio track is provided, comprising: obtaining an audio file of an accompaniment audio track and an audio file of an accompaniment audio track, as well as a playback delay of the accompaniment audio track and a target recording progress of recording the audio track when starting to play the accompaniment audio track, wherein the recorded audio track is used to record the singing audio when the accompaniment audio track is played; performing audio track alignment processing by removing the playback delay of the accompaniment audio track and the portion corresponding to the target recording progress in the recorded audio track.

[0009] According to a fifth aspect of an embodiment of the present invention, there is provided a track processing device, comprising: a determination module, which determines the playback delay of an accompaniment track and a target recording progress of a recording track when starting to play the accompaniment track, wherein the recording track is used to record the singing audio when the accompaniment track is played; and a track alignment module, which aligns the recording track and the accompaniment track by removing the portion corresponding to the playback delay of the accompaniment track and the target recording progress in the recording track.

[0010] According to a sixth aspect of an embodiment of the present invention, there is provided a track processing device, comprising: a track synthesis module, which performs synthesis processing based on aligned recorded tracks and accompaniment tracks, wherein the recorded tracks and accompaniment tracks are obtained by processing using the track processing method of the first aspect; and a track post-processing module, which performs post-processing on the synthesized track based on a track post-processing mode.

[0011] According to a seventh aspect of an embodiment of the present invention, there is provided a track processing device, comprising: a playback start module, which starts the playback of an accompaniment track when the recording track is recorded to a target recording progress, wherein the recording track is used to record the singing audio when the accompaniment track is played; and a calculation module, which calculates the playback delay of the accompaniment track by playing the accompaniment track, so as to remove the playback delay and the target recording progress in the recording track, and perform track alignment between the recording track and the accompaniment track.

[0012] According to an eighth aspect of an embodiment of the present invention, there is provided a track processing device, comprising: an acquisition module, for acquiring an audio file of an accompaniment track and an audio file of an accompaniment track, as well as a playback delay of the accompaniment track and a target recording progress of the recording track when starting to play the accompaniment track, wherein the recording track is used to record the singing audio when the accompaniment track is played; an alignment processing module, for performing track alignment processing by removing the playback delay of the accompaniment track and the portion corresponding to the target recording progress in the recording track.

[0013] According to a ninth aspect of an embodiment of the present invention, there is provided an electronic device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; the memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform an operation corresponding to the method described in any one of the first to third aspects.

[0014] According to a tenth aspect of an embodiment of the present invention, a storage medium stores a computer program thereon, and when the program is executed by a processor, the method described in any one of the first to third aspects is implemented.

[0015] In the scheme of the embodiment of the present invention, since the recorded audio track has a target recording progress when the accompaniment audio track is started to be played, the audio track alignment is achieved simply by processing the recorded audio track; in addition, since the parts corresponding to the playback delay and the target recording progress reflect the excess part of the recorded audio track compared to the accompaniment audio track, the accuracy of the audio track alignment is achieved and the recording effect is improved by removing the parts corresponding to the playback delay and the target recording progress of the accompaniment audio track in the recorded audio track. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0017] Figure 1A A schematic diagram of a method for processing an audio track according to an embodiment of the present invention;

[0018] Figure 1B is a schematic diagram of a method for processing an audio track according to another embodiment of the present invention;

[0019] Figure 2A is a schematic flow chart of a method for processing an audio track according to another embodiment of the present invention;

[0020] Figure 2B is a schematic diagram of a method for processing an audio track according to another embodiment of the present invention;

[0021] Figure 2C is a schematic diagram of a method for processing an audio track according to another embodiment of the present invention;

[0022] Figure 2D is a schematic diagram of a method for processing an audio track according to another embodiment of the present invention;

[0023] Figure 3is a schematic diagram of a method for processing an audio track according to another embodiment of the present invention;

[0024] Figure 4 is a schematic diagram of a method for processing an audio track according to another embodiment of the present invention;

[0025] Figure 5 is a schematic diagram of a method for processing an audio track according to another embodiment of the present invention;

[0026] Fig. 6A is a schematic flow chart of a method for processing an audio track according to another embodiment of the present invention;

[0027] Figure 6B is a schematic flow chart of a method for processing an audio track according to another embodiment of the present invention;

[0028] Figure 6C is a schematic flow chart of a method for processing an audio track according to another embodiment of the present invention;

[0029] Figure 7 is a schematic block diagram of an audio track processing device according to another embodiment of the present invention;

[0030] Figure 8 is a schematic block diagram of an audio track processing device according to another embodiment of the present invention;

[0031] Fig.9A is a schematic block diagram of an audio track processing device according to another embodiment of the present invention;

[0032] Fig. 9B is a schematic block diagram of an audio track processing device according to another embodiment of the present invention;

[0033] Fig.10 The present invention is another embodiment of the hardware structure of an electronic device. DETAILED DESCRIPTION

[0034] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in the field based on the embodiments in the embodiments of the present invention should fall within the scope of protection of the embodiments of the present invention.

[0035] The specific implementation of the embodiment of the present invention is further described below in conjunction with the accompanying drawings of the embodiment of the present invention. Figure 1A A schematic diagram of a recording device of an audio track processing method according to an embodiment of the present invention. Figure 1B It is a schematic diagram of a playback device of an audio track processing method according to another embodiment of the present invention. Figure 1A and Figure 1B The recording device and the playback device can be part of the audio track alignment device and can be applicable to any suitable electronic device with data processing capabilities, including but not limited to: Internet of Things devices, embedded devices, smart devices, servers, mobile terminals (such as mobile phones, PADs, etc.) and PC computers, etc. Figure 1A The recording device of Figure 1B includes a human-computer interaction interface, an input wake-up module, and an acquisition drive module. Each part of the human-computer interaction interface, the input wake-up module, and the acquisition drive module will generate application layer latency, system layer latency, and drive layer latency. Figure 1B The playback device of Figure 1B includes a human-computer interaction interface, an audio transcoding module, and an output drive module. The human-computer interaction interface, the audio transcoding module, and the output drive module will also generate application layer latency, system layer latency, and drive layer latency. For example, when the recording device enables the microphone to record audio, it needs to access the hardware, and there is a first latency a between the time when recording is enabled and the time when the first frame of data is actually recorded and written to the file. When the playback device enables the accompaniment audio playback, there is a second latency b between the time when it is enabled and the time when the speaker actually starts playing.

[0036] In one example, when using an external speaker to play the accompaniment audio (corresponding to the accompaniment track), during the recording preview, the recorded track and the accompaniment track are played simultaneously. At this time, the audio of the recorded track includes the recording of the accompaniment track. In another example, when using headphones to play the accompaniment audio, the recorded track and the accompaniment track are played simultaneously. At this time, the audio of the recorded track does not include the recording of the accompaniment track.

[0037] Corresponding to the above examples, the microphone recording can be enabled immediately while the accompaniment audio playback is enabled. If a > b, then during the time from when the user hears the accompaniment sound to a - b, the microphone has not actually started collecting signals. After recording is completed, the preview effect shown to the user is that the front-end part is missing audio, and the recorded sound is faster than the accompaniment playback sound by a duration of a - b. Similarly, if a < b, when previewing, the recorded sound of the user is slower than the accompaniment playback sound by a duration of b - a, thus both resulting in a problem of poor recording effect.

[0038] Figure 2A It is a schematic flowchart of the audio track processing method according to another embodiment of the present invention. Figure 2A The audio track processing method of Figure 2A can be applicable to any suitable electronic device with data processing capabilities, including but not limited to: Internet of Things devices, embedded devices, smart devices, servers, mobile terminals (such as mobile phones, PADs, etc.) and PC computers, etc. The audio track processing method includes:

[0039] 210: Determine the playback delay of the accompaniment track and the target recording progress of the recorded track when starting to play the accompaniment track, where the recorded track is used to record the singing audio when the accompaniment track is played.

[0040] It should be understood that the accompaniment track and the accompaniment track can be set in an application, for example, in an audio track alignment application, a recording application, a singing application, a music social application, etc. The accompaniment track and the accompaniment track can be visible to the user or invisible to the user. The accompaniment track and the accompaniment track can be editable tracks or non-editable tracks. The application may include an edit mode, a preview mode, an upload mode, a save mode, etc. In the preview mode, the aligned synthetic audio track can be displayed to the user (for example, in the form of playback), and the process of track alignment is invisible to the user. In the upload mode, the aligned synthetic audio track can be uploaded to the platform (for example, the server) for sharing and playing, and the process of track alignment is invisible to the user. In the save mode, the aligned synthetic audio track can be saved to the platform (for example, the server) for storage, or it can be saved locally, and the process of track alignment is invisible to the user. In the edit mode, the above-mentioned audio track can be visible to the user, and manual editing or automatic editing can be performed. The playback delay of the accompaniment track can be initially set by the user, and the user can manually adjust the initially set playback delay based on the preview result until the desired track alignment effect is achieved. Based on the target recording progress of the recorded track, the user can also manually play the accompaniment track, and after the playback is triggered, record the target recording progress and the corresponding time; or, based on the target recording progress set by the user, automatically trigger the playback of the accompaniment track and record the corresponding time.

[0041] The play delay of the accompaniment track can be measured locally or obtained from other devices. When the client device executes the method, it can also be obtained from the server, for example, a play delay information request is sent to the server to obtain the play delay corresponding to the hardware configuration (for example, various hardware information, etc.) and / or software configuration (for example, software information including software version, etc.) of the current device. In addition, the play delay of the accompaniment track is measured locally, and the average value can be taken by multiple measurements, or the historical measurement value can be updated based on the current measurement value (for example, as the setting is used, the measurement value changes). The historical play delay of the accompaniment track can be stored in a memory, and the historical play delay can be read from the memory as the current play delay. In addition, the first play progress in the accompaniment track can be started to play, and the first play time of the play operation can be recorded. When the second play progress is reached, the second play time is recorded. The time difference between the second play time and the second play time can be calculated, and the progress duration indicated between the second play progress and the first play progress in the accompaniment track is calculated. The time difference can be made different from the progress duration to obtain the play delay. The first play time and the second play time may be system time obtained through the application, or may be time measured by an external clock, such as Internet time. The first play progress and the second play progress may be any position in the accompaniment track set in the application.

[0042] In addition, for the target recording progress of recording the audio track when starting to play the accompaniment audio track, the playing of the accompaniment audio track can be started at the first frame of the recording audio track (the first frame, that is, the moment when recording is performed), that is, the target recording progress is zero progress, in other words, the progress duration of the target recording progress is zero; the playing of the accompaniment audio track can also be started at a non-first frame of the recording audio track.

[0043] In addition, in one example, when an external speaker is used to play the accompaniment audio (corresponding to the accompaniment track), in the recording preview, the recording track and the accompaniment track are played simultaneously, and the audio of the recording track includes the recording of the accompaniment track and the human voice source.

[0044] In another example, when headphones are used for playing the accompaniment audio, the recorded audio track and the accompaniment audio track are played simultaneously, and the audio of the recorded audio track includes a recording of a human voice source.

[0045] 220: Align the recording track with the backing track by removing the playback delay of the backing track and the portion corresponding to the target recording progress in the recording track.

[0046] It should be understood that when an external speaker is used to play the accompaniment audio, the playback delay of the accompaniment audio track and the part corresponding to the target recording progress can be removed to obtain a processed recorded audio track, which is aligned with the accompaniment audio track; the recorded audio track can also be separated from the sound source to obtain a human voice source audio track, and the playback delay of the accompaniment audio track and the part corresponding to the target recording progress can be removed to obtain a processed human voice source audio track, which is aligned with the accompaniment audio track; and the playback delay of the accompaniment audio track and the part corresponding to the target recording progress can be removed to obtain a processed recorded audio track, and the sound source of the processed recorded audio track can be separated to obtain a human voice source audio track, which is aligned with the accompaniment audio track.

[0047] When headphones are used for playing the accompaniment audio, the playback delay of the accompaniment audio track and the portion corresponding to the target recording progress can be removed to obtain a processed recorded audio track, which is aligned with the accompaniment audio track.

[0048] Since the recorded audio track has a target recording progress when the accompaniment audio track is started to play, the audio track alignment is achieved by simply processing the recorded audio track; in addition, since the parts corresponding to the playback delay and the target recording progress reflect the extra part of the recorded audio track compared to the accompaniment audio track, the accuracy of the audio track alignment is achieved by removing the parts corresponding to the playback delay and the target recording progress of the accompaniment audio track in the recorded audio track.

[0049] In one example, the method may further include: performing track synthesis based on the aligned recorded audio track and accompaniment audio track. For example, the recorded audio track may record the audio information of the played accompaniment audio track and the recorded vocal information. The track synthesis may be performed using a time axis shared by the recorded audio track and the accompaniment audio track. For example, the audio frames corresponding to multiple time points on the time axis in different audio tracks may be determined to obtain a synthesized audio track.

[0050] In addition, when performing track synthesis, after the track alignment, the processing can respond to the track cooperation instruction to perform track synthesis, and can also respond to any one of the track preview instruction, the track upload instruction, and the track save instruction to perform the track alignment operation, and perform the track preview operation, the track upload operation, the track save operation, etc. It is also possible to respond to at least one of the track preview instruction, the track upload instruction, and the track save instruction, and perform the track alignment operation in sequence, and perform the corresponding operation among the track preview operation, the track upload operation, and the track save operation.

[0051] In another example, in response to a (user's) recording preview instruction, the playback delay of the accompaniment audio track and the target recording progress of the recording track when starting to play the accompaniment audio track can be determined; the recording audio track and the accompaniment audio track are aligned by removing the parts corresponding to the playback delay and the target recording progress of the accompaniment audio track in the recording audio track to obtain a preview synthesized audio track; and the preview synthesized audio track is displayed.

[0052] In another example, in response to a (user's) recording instruction, the playback delay of the accompaniment audio track and the target recording progress of the recording track when starting to play the accompaniment audio track can be determined; the recording audio track and the accompaniment audio track are aligned by removing the parts corresponding to the playback delay and the target recording progress of the accompaniment audio track in the recording audio track to obtain a composite audio track; and the composite audio track is saved.

[0053] In another implementation of the present invention, determining the playback delay of the accompaniment audio track includes: determining the startup duration corresponding to the target playback progress; and determining the playback delay of the accompaniment audio track based on the difference between the startup duration corresponding to the target playback progress and the progress duration indicated by the target playback progress.

[0054] Since the playback delay of the accompaniment track is determined according to the difference between the startup duration corresponding to the target playback progress and the progress duration indicated by the target playback progress, the accuracy of determining the playback delay is improved. In addition, the determination process is achieved without the intrusion of complex algorithms or codes. In addition, the difficulty of determining the difference in playback delay caused by the difference in different devices is also solved.

[0055] In another implementation of the present invention, determining the startup duration corresponding to the target playback progress includes: determining a first time for starting to play the accompaniment audio track and a second time corresponding to the target playback progress; and calculating the startup duration corresponding to the target playback progress based on the time difference between the second time and the first time.

[0056] Since the startup duration corresponding to the target playback progress is calculated based on the time difference between the second time and the first time, the accuracy of determining the startup duration is improved. In addition, the determination process is achieved without the intrusion of complex algorithms or codes. In addition, the difficulty of determining the difference in startup duration caused by the difference in different devices is also solved.

[0057] In another implementation of the present invention, the recording audio track and the accompaniment audio track are aligned by removing the portion of the accompaniment audio track corresponding to the playback delay and the target recording progress in the recording audio track, including: calculating the sum of the delay duration of the playback delay of the accompaniment audio track and the progress duration corresponding to the target recording progress; at the starting end of the recording audio track, removing the portion corresponding to the sum of the delay duration and the progress duration, to align the recording audio track and the accompaniment audio track.

[0058] Since the corresponding part of the sum of the delay duration and the progress duration is removed at the beginning of the recorded audio track, the recorded audio track and the accompaniment audio track are aligned, which improves the efficiency of data processing and the accuracy of the track alignment process.

[0059] In another implementation of the present invention, the method also includes: displaying a recording audio track and an accompaniment audio track in an audio track editing interface, wherein the recording audio track and the accompaniment audio track are aligned by removing the portion of the accompaniment audio track corresponding to the playback delay and the target recording progress in the recording audio track, including: responding to a track alignment instruction in the audio track editing interface, removing the portion of the accompaniment audio track corresponding to the playback delay and the target recording progress in the recording audio track; and displaying the aligned recording audio track and accompaniment audio track in the audio track editing interface.

[0060] Since users can edit audio tracks in the audio track editing interface, the operability of the audio tracks is improved.

[0061] In one example, in the track editing interface, the user can move the track, click on it, etc. For example, the user can move the recording track or the accompaniment track, and when the recording track or the accompaniment track meets the alignment relationship, the alignment relationship is displayed (for example, displayed by a dotted line). The user can perform track trimming operations based on the alignment relationship. The alignment relationship can be determined by the relative position of the part corresponding to the playback delay of the accompaniment track and the target recording progress and the current track. For example, if the relative position of the current track meets the part corresponding to the playback delay of the accompaniment track and the target recording progress (for example, the sum of the delay duration of the playback delay and the duration indicating the target recording progress), the display of the alignment relationship is triggered.

[0062] Figure 2B 2 is a schematic diagram of an audio track processing method according to another embodiment of the present invention. In this example, the description of each step is exemplary. In other examples, more or fewer steps may be included, and the order of each step may also be changed. In this example, in step 211, the user triggers any one of a recorded audio preview instruction, a recorded audio save instruction, or a recorded audio upload instruction, and sends it to the client device.

[0063] In step 212, the client device responds to one of the above instructions and performs track alignment. For example, the client device may respond to the above instructions and enter the recorded audio preview mode, or the recorded audio save mode, or the recorded audio mode accordingly, and perform track alignment.

[0064] In step 213, the client device synthesizes the audio track based on the result of the audio track alignment.

[0065] In step 214, the client device displays the processing result of the synthesized audio track corresponding to the above instructions to the user.

[0066] Figure 2CSchematic diagram of an audio track processing method according to another embodiment of the present invention. In this example, the description of each step is exemplary. In other examples, more or fewer steps may be included, and the order of each step may also be changed. As shown in the figure, in step 221, the user triggers any one of a recorded audio preview instruction, a recorded audio save instruction, or a recorded audio upload instruction, and sends it to a client device.

[0067] In step 222, the client device responds to one of the above instructions and performs track alignment. For example, the client device may respond to the above instructions and enter the recorded audio preview mode, or the recorded audio save mode, or the recorded audio mode accordingly, and perform track alignment.

[0068] In step 223, the client device sends an audio source separation request including the recorded audio track to the server device.

[0069] In step 224, the server device returns the human voice source audio track to the client device.

[0070] In step 225 , the client device synthesizes the audio tracks based on the result of the audio track alignment.

[0071] In step 226, the client device displays the processing result of the synthesized audio track corresponding to the above instructions to the user. For example, in upload mode, the aligned synthesized audio track can be uploaded to the platform server (which can be the same server as the server used for sound source separation in this example, or a different server from the server in this example) for sharing and playing, and the platform server can return the uploaded result to the client device to display or notify the user. Similarly, in the save mode in which the aligned synthesized audio track is uploaded to the platform server for saving, the platform server can return the saved result to the client device to display or notify the user.

[0072] Figure 2D Schematic diagram of an audio track processing method according to another embodiment of the present invention. In this example, the description of each step is exemplary. In other examples, more or fewer steps may be included, and the order of each step may also be changed. As shown in the figure, in step 211, the user triggers any one of a recorded audio preview instruction, a recorded audio save instruction, or a recorded audio upload instruction, and sends it to a client device.

[0073] In step 232, the client device responds to one of the above instructions and sends a sound source separation request including the recorded audio track. For example, in response to the above instructions, the client device can enter the recorded audio preview mode, the recorded audio storage mode, or the recorded audio mode accordingly, and send a sound source separation request including the recorded audio track.

[0074] In step 233, the server device returns the human voice source audio track to the client device.

[0075] In step 234, the server device performs track alignment on the human voice source track and the accompaniment track.

[0076] In step 235 , the client device synthesizes the audio tracks based on the result of the audio track alignment.

[0077] In step 236, the client device displays the processing result of the synthesized audio track corresponding to the above instructions to the user. For example, in upload mode, the aligned synthesized audio track can be uploaded to the platform server (which can be the same server as the server used for sound source separation in this example, or a different server from the server in this example) for sharing and playing, and the platform server can return the uploaded result to the client device to display or notify the user. Similarly, in the save mode in which the aligned synthesized audio track is uploaded to the platform server for saving, the platform server can return the saved result to the client device to display or notify the user.

[0078] Figure 3 Schematic diagram of a method for processing tracks of another embodiment of the present invention. In the coordinate system shown in the figure, the horizontal axis represents time and the vertical axis represents different tracks. The upper part is the recording track and the lower part is the accompaniment track. P1 is the target recording progress of the recording track, and the time point corresponding to the target recording progress is T1. P2 is the target playback progress of the accompaniment track, and the time point corresponding to the target playback progress is T2.

[0079] Figure 4 It is a schematic diagram of the audio track processing method of another embodiment of the present invention. In the coordinate system shown in the figure, the horizontal axis represents time and the vertical axis represents different audio tracks. The upper part is the recording audio track and the lower part is the accompaniment audio track. P1 is the target recording progress of the recording audio track, and the time point corresponding to the target recording progress is T1. P2 is the target playback progress of the accompaniment audio track, and the time point corresponding to the target playback progress is T2. In addition, in this example, when the target playback progress is recorded, the playback of the accompaniment audio track is started. Therefore, the duration corresponding to the shaded part is T2-T1-P2+P1 from the starting point of the self-recording audio track.

[0080] Figure 5It is a schematic diagram of the audio track processing method of another embodiment of the present invention. In the coordinate system shown in the figure, the horizontal axis represents time and the vertical axis represents different audio tracks. The upper part is the recording audio track and the lower part is the accompaniment audio track. P1 is the target recording progress of the recording audio track, and the corresponding time point when the recording starts (recording the first frame) is T1. P2 is the target playback progress of the accompaniment audio track, and the time point corresponding to the target playback progress is T2. In addition, in this example, when the recording starts, the playback of the accompaniment audio track is started. Therefore, the duration corresponding to the shaded part is T2-T1-P2 from the starting point of the recording audio track.

[0081] Fig. 6A It is a schematic flow chart of a method for processing an audio track according to another embodiment of the present invention. Fig. 6A The audio track processing method can be applied to any appropriate electronic device with data processing capabilities, including but not limited to: Internet of Things devices, embedded devices, smart devices, servers, mobile terminals (such as mobile phones, PADs, etc.) and PCs, etc. The audio track processing method includes:

[0082] 610: Perform synthesis processing based on the aligned recorded audio track and accompaniment audio track.

[0083] 620: Based on the audio track post-processing mode, post-process the synthesized audio track.

[0084] Among them, the recording track and the accompaniment track can be Figure 2A The audio track processing method is used to process it.

[0085] In another implementation of the present invention, the synthesis processing based on the aligned recorded audio track and accompaniment audio track includes: while displaying the playback object, synthesizing the recorded audio track and the accompaniment audio track to obtain a synthesized audio track, wherein the post-processing of the synthesized audio track based on the audio track post-processing mode includes: in response to the completion of the recorded audio track and the accompaniment audio track, ending the display of the playback object, and starting to display the synthesized audio track in a synthesized audio track preview mode.

[0086] Since the playback object is displayed when the audio track alignment is performed, and when the preview synthesized audio track is obtained, the display of the playback object is ended, and the preview synthesized audio track is displayed, the user's unawareness of the audio track alignment process is utilized, thereby improving the user experience of the preview.

[0087] In an example, the playback object may be a multimedia object set by a user, such as a picture object, a text object, an audio object, or a video object.

[0088] In another example, the playback object can be a presentation of a scoring effect for the audio sung by the user, for example, in response to the user's preview instruction, an interface including scoring information is generated, and the interface including scoring information is presented (switched to). In response to the switching operation of the interface, the tracks are aligned, and when the generation of the preview synthesized audio is completed, the preview interface is switched from the scoring information interface. Since the tracks are aligned in response to the switching operation of the interface, the generation of the scoring information interface is asynchronous with the track alignment and synthesis operation, reducing the processing load of the device. In addition, the above operation also improves the fluency of the user experience.

[0089] Alternatively, in response to the user's preview instruction, the tracks are aligned and a scoring information interface is generated, and the preview interface is switched from the scoring information interface. Since the tracks are aligned and a scoring information interface is generated in response to the user's preview instruction, parallel processing of track alignment and scoring information interface generation is achieved, thereby improving processing efficiency. In addition, the above operation also improves the fluency of the user experience.

[0090] In addition, when performing track synthesis, after the track alignment, the processing can respond to the track cooperation instruction to perform track synthesis, and can also respond to any one of the track preview instruction, the track upload instruction, and the track save instruction to perform the track alignment operation, and perform the track preview operation, the track upload operation, the track save operation, etc. It is also possible to respond to at least one of the track preview instruction, the track upload instruction, and the track save instruction, and perform the track alignment operation in sequence, and perform the corresponding operation among the track preview operation, the track upload operation, and the track save operation.

[0091] In another example, in response to a user recording preview instruction, the playback delay of the accompaniment audio track and the target recording progress of the recording track when starting to play the accompaniment audio track can be determined; the recording audio track and the accompaniment audio track are aligned by removing the parts corresponding to the playback delay and the target recording progress of the accompaniment audio track in the recording audio track to obtain a preview synthesized audio track; and the preview synthesized audio track is displayed.

[0092] In another example, in response to a user recording instruction, the playback delay of the accompaniment audio track and the target recording progress of the recording track when starting to play the accompaniment audio track can be determined; the recording audio track and the accompaniment audio track are aligned by removing the parts corresponding to the playback delay and the target recording progress of the accompaniment audio track in the recording audio track to obtain a composite audio track; and the composite audio track is saved.

[0093] Figure 6B It is a schematic flow chart of a method for processing an audio track according to another embodiment of the present invention. Figure 6BThe audio track processing method can be applied to any appropriate electronic device with data processing capabilities, including but not limited to: Internet of Things devices, embedded devices, smart devices, servers, mobile terminals (such as mobile phones, PADs, etc.) and PCs, etc. The audio track processing method includes:

[0094] 630: When the recording audio track is recorded to the target recording progress, the playing of the accompaniment audio track is started, wherein the recording audio track is used to record the singing audio when the accompaniment audio track is played.

[0095] 640: Calculate the playback delay of the backing track by playing the backing track so as to remove the playback delay and target recording progress in the recorded track, and align the recorded track and the backing track.

[0096] Since the recorded audio track has a target recording progress when the accompaniment audio track is started to play, the audio track alignment is achieved by simply processing the recorded audio track; in addition, since the parts corresponding to the playback delay and the target recording progress reflect the extra part of the recorded audio track compared to the accompaniment audio track, the accuracy of the audio track alignment is achieved by removing the parts corresponding to the playback delay and the target recording progress of the accompaniment audio track in the recorded audio track.

[0097] In another implementation of the present invention, when the recording audio track is recorded to the target recording progress, the playing of the accompaniment audio track is started, including: determining the first time when the recording audio track is recorded to the target recording progress; and starting the playing of the accompaniment audio track at the first time.

[0098] Since the first time when the recording track is recorded to the target recording progress is more accurate, the playing of the accompaniment track is started at the first time, and the playing delay of the accompaniment track is calculated by playing the accompaniment track, so that the accuracy of the playing delay can be determined.

[0099] In another implementation of the present invention, when the recording track is recorded to the target recording progress, the playing of the accompaniment track is started, including: when the first frame is recorded in the recording track, the playing of the accompaniment track is started.

[0100] Since the playing of the accompaniment track is started when the first frame is recorded in the recording track, when the track alignment is achieved only by processing the recording track, the length of the recording track trimming can be reduced, and the user's waiting time in the track preview can be further reduced.

[0101] In another implementation of the present invention, the playback delay of the accompaniment audio track is calculated by playing the accompaniment audio track, including: determining a second time of the target playback progress of the accompaniment audio track; and calculating the playback delay of the accompaniment audio track based on the time difference between the second time and the first time and the progress duration of the target playback progress.

[0102] Since the startup duration corresponding to the target playback progress is calculated based on the time difference between the second time and the first time, the accuracy of determining the startup duration is improved. In addition, the determination process is achieved without the intrusion of complex algorithms or codes. In addition, the difficulty of determining the difference in startup duration caused by the difference in different devices is also solved.

[0103] Figure 6C It is a schematic flow chart of a method for processing an audio track according to another embodiment of the present invention. Figure 6C The audio track processing device can be applicable to an application server, a cloud server such as a public cloud, a private cloud, a proprietary cloud or a hybrid cloud, etc. The audio track processing device includes:

[0104] 650: Obtaining the audio file of the accompaniment track and the audio file of the accompaniment track, as well as the playback delay of the accompaniment track and the target recording progress of the recording track when starting to play the accompaniment track, wherein the recording track is used to record the singing audio when playing the accompaniment track.

[0105] 660: Track alignment is performed by removing the playback delay of the backing track and the portion corresponding to the target recording progress in the recorded track.

[0106] In another implementation of the present invention, the playback delay of the accompaniment track is obtained through the second voiceprint information in the accompaniment track, and the target recording progress of the recording track when starting to play the accompaniment track is obtained through the first voiceprint information of the recording track.

[0107] In another implementation of the present invention, the method further includes: performing audio trimming processing on the first voiceprint information in the recorded audio track to obtain a recorded audio track after audio trimming, wherein the audio track alignment processing is performed by removing the playback delay of the accompaniment audio track and the portion corresponding to the target recording progress in the recorded audio track after audio trimming, including: aligning the recorded audio track after audio trimming and the accompaniment audio track by removing the playback delay of the accompaniment audio track and the portion corresponding to the target recording progress in the recorded audio track after audio trimming.

[0108] In one example, performing audio modification processing on the first voiceprint information in the recorded audio track includes: performing noise reduction processing on the first voiceprint information in the recorded audio track to obtain a recorded audio track after noise reduction.

[0109] In another example, performing audio trimming processing on the first voiceprint information in the recorded audio track includes: performing equalization processing on the first voiceprint information in the recorded audio track to obtain a recorded audio track after equalization processing.

[0110] In another example, performing audio modification processing on the first voiceprint information in the recorded audio track includes: compressing the first voiceprint information in the recorded audio track to obtain a compressed recorded audio track.

[0111] In another example, performing audio modification processing on the first voiceprint information in the recorded audio track includes: performing reverberation processing on the first voiceprint information in the recorded audio track to obtain the recorded audio track after reverberation processing.

[0112] It should be understood that the aligned audio tracks may be subjected to at least one of the above-mentioned noise reduction processing, equalization processing, compression processing and reverberation processing. The above-mentioned various processing may be parallel processing or serial processing. For serial processing, the order of the above-mentioned various processing may be arbitrary.

[0113] In another implementation of the present invention, the first voiceprint information in the recorded audio track is processed to obtain the processed recorded audio track, including: performing sound source separation on the first voiceprint information in the recorded audio track to obtain the human voice source audio track, wherein the processed recorded audio track and the accompaniment audio track are aligned by removing the playback delay of the accompaniment audio track and the portion corresponding to the target recording progress in the processed recorded audio track, including: aligning the human voice source audio track and the accompaniment audio track by removing the playback delay of the accompaniment audio track and the portion corresponding to the target recording progress in the human voice source audio track.

[0114] Figure 7 It is a schematic block diagram of a track processing device according to another embodiment of the present invention. Figure 7 The audio track processing device can be applied to any appropriate electronic device with data processing capabilities, including but not limited to: Internet of Things devices, embedded devices, smart devices, servers, mobile terminals (such as mobile phones, PADs, etc.) and PCs. The audio track processing device includes:

[0115] The determination module 710 determines the playback delay of the accompaniment audio track and the target recording progress of the recording audio track when the accompaniment audio track is started to be played, wherein the recording audio track is used to record the singing audio when the accompaniment audio track is played.

[0116] The track alignment module 720 aligns the recording track and the accompaniment track by removing the portion of the accompaniment track corresponding to the playback delay and the target recording progress from the recording track.

[0117] Since the recorded audio track has a target recording progress when the accompaniment audio track is started to play, the audio track alignment is achieved by simply processing the recorded audio track; in addition, since the parts corresponding to the playback delay and the target recording progress reflect the extra part of the recorded audio track compared to the accompaniment audio track, the accuracy of the audio track alignment is achieved by removing the parts corresponding to the playback delay and the target recording progress of the accompaniment audio track in the recorded audio track.

[0118] In another implementation of the present invention, the determination module is specifically used to: determine the startup duration corresponding to the target playback progress; determine the playback delay of the accompaniment audio track based on the difference between the startup duration corresponding to the target playback progress and the progress duration indicated by the target playback progress.

[0119] In another implementation of the present invention, the determination module is specifically used to: determine the first time for starting to play the accompaniment audio track and the second time corresponding to the target playback progress; based on the time difference between the second time and the first time, calculate the startup duration corresponding to the target playback progress.

[0120] In another implementation of the present invention, the audio track alignment module is specifically used to: calculate the sum of the delay duration of the playback delay of the accompaniment audio track and the progress duration corresponding to the target recording progress; at the starting end of the recorded audio track, remove the part corresponding to the sum of the delay duration and the progress duration, and align the recorded audio track and the accompaniment audio track.

[0121] The device of this embodiment is used to implement the corresponding methods in the aforementioned multiple method embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here. In addition, the functional implementation of each module in the device of this embodiment can refer to the description of the corresponding parts in the aforementioned method embodiments, which will not be repeated here.

[0122] Figure 8 It is a schematic block diagram of a track processing device according to another embodiment of the present invention. Figure 8 The audio track processing device can be applied to any appropriate electronic device with data processing capabilities, including but not limited to: Internet of Things devices, embedded devices, smart devices, servers, mobile terminals (such as mobile phones, PADs, etc.) and PCs. The audio track processing device includes:

[0123] The audio track synthesis module 810 performs synthesis processing according to the aligned recorded audio track and accompaniment audio track.

[0124] The audio track post-processing module 820 performs post-processing on the synthesized audio track based on the audio track post-processing mode.

[0125] Among them, the recording track and the accompaniment track can be Figure 2A The audio track processing method is used to process it.

[0126] In another implementation of the present invention, the audio track synthesis module is specifically used to: while displaying the playback object, synthesize the recorded audio track and the accompaniment audio track to obtain a synthesized audio track, wherein the audio track post-processing module is specifically used to: in response to the completion of the recorded audio track and the accompaniment audio track, end the display of the playback object, and start displaying the synthesized audio track in a synthesized audio track preview mode.

[0127] The device of this embodiment is used to implement the corresponding methods in the aforementioned multiple method embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here. In addition, the functional implementation of each module in the device of this embodiment can refer to the description of the corresponding parts in the aforementioned method embodiments, which will not be repeated here.

[0128] Fig.9A It is a schematic block diagram of a track processing device according to another embodiment of the present invention. Fig.9A The audio track processing device can be applied to any appropriate electronic device with data processing capabilities, including but not limited to: Internet of Things devices, embedded devices, smart devices, servers, mobile terminals (such as mobile phones, PADs, etc.) and PCs. The audio track processing device includes:

[0129] The play start module 910 starts the play of the accompaniment track when the recording track is recorded to the target recording progress, wherein the recording track is used to record the singing audio when the accompaniment track is played.

[0130] The calculation module 920 calculates the play delay of the accompaniment audio track by playing the accompaniment audio track, so as to remove the play delay and the target recording progress in the recording audio track, and align the recording audio track with the accompaniment audio track.

[0131] Since the recorded audio track has a target recording progress when the accompaniment audio track is started to play, the audio track alignment is achieved by simply processing the recorded audio track; in addition, since the parts corresponding to the playback delay and the target recording progress reflect the extra part of the recorded audio track compared to the accompaniment audio track, the accuracy of the audio track alignment is achieved by removing the parts corresponding to the playback delay and the target recording progress of the accompaniment audio track in the recorded audio track.

[0132] In another implementation of the present invention, the play start module is specifically used to: determine the first time when the recorded audio track is recorded to the target recording progress; and start the play of the accompaniment audio track at the first time.

[0133] In another implementation of the present invention, the play start module is specifically used to start the play of the accompaniment audio track when the first frame is recorded in the recording audio track.

[0134] In another implementation of the present invention, the calculation module is specifically used to: determine a second time of the target playback progress of the accompaniment audio track; and calculate the playback delay of the accompaniment audio track based on the time difference between the second time and the first time and the progress duration of the target playback progress.

[0135] The device of this embodiment is used to implement the corresponding methods in the aforementioned multiple method embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here. In addition, the functional implementation of each module in the device of this embodiment can refer to the description of the corresponding parts in the aforementioned method embodiments, which will not be repeated here.

[0136] Fig. 9B It is a schematic block diagram of a track processing device according to another embodiment of the present invention. Fig. 9B The audio track processing device can be applicable to an application server, a cloud server such as a public cloud, a private cloud, a proprietary cloud or a hybrid cloud, etc. The audio track processing device includes:

[0137] An acquisition module 930 acquires an audio file of an accompaniment audio track and an audio file of the accompaniment audio track, as well as a playback delay of the accompaniment audio track and a target recording progress of the recording audio track when starting to play the accompaniment audio track, wherein the recording audio track is used to record the singing audio when playing the accompaniment audio track;

[0138] The alignment processing module 940 performs track alignment processing by removing the playback delay of the accompaniment track and the part corresponding to the target recording progress in the recording track.

[0139] In another implementation of the present invention, the playback delay of the accompaniment track is obtained through the second voiceprint information in the accompaniment track, and the target recording progress of the recording track when starting to play the accompaniment track is obtained through the first voiceprint information of the recording track.

[0140] In another implementation of the present invention, the device also includes: a tuning module, which performs tuning processing on the first voiceprint information in the recorded audio track to obtain a tuned recorded audio track, wherein the alignment processing module is specifically used to align the tuned recorded audio track and the accompaniment audio track by removing the playback delay of the accompaniment audio track and the part corresponding to the target recording progress in the tuned recorded audio track.

[0141] In another implementation of the present invention, the audio correction module is specifically used to: perform sound source separation on the first voiceprint information in the recorded audio track to obtain the human voice source audio track, wherein the alignment processing module is specifically used to: align the human voice source audio track and the accompaniment audio track by removing the playback delay of the accompaniment audio track and the part corresponding to the target recording progress in the human voice source audio track.

[0142] The device of this embodiment is used to implement the corresponding methods in the aforementioned multiple method embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here. In addition, the functional implementation of each module in the device of this embodiment can refer to the description of the corresponding parts in the aforementioned method embodiments, which will not be repeated here.

[0143] Fig.10The hardware structure of the electronic device of another embodiment of the present invention is as follows; Fig.10 As shown, the hardware structure of the electronic device may include: a processor 1001, a communication interface 1002, a storage medium 1003 and a communication bus 1004;

[0144] The processor 1001, the communication interface 1002, and the storage medium 1003 communicate with each other via the communication bus 1004;

[0145] Optionally, the communication interface 1002 may be an interface of a communication module;

[0146] The processor 1001 may be specifically configured to: determine a playback delay of an accompaniment audio track and a target recording progress of a recording audio track when starting to play the accompaniment audio track, wherein the recording audio track is used to record the singing audio when the accompaniment audio track is played; align the recording audio track with the accompaniment audio track by removing a portion corresponding to the playback delay of the accompaniment audio track and the target recording progress in the recording audio track;

[0147] Alternatively, performing synthesis processing according to the aligned recorded audio track and accompaniment audio track, wherein the recorded audio track and accompaniment audio track are obtained by processing using the above-mentioned audio track processing method; performing post-processing on the synthesized audio track based on the audio track post-processing mode;

[0148] Alternatively, obtaining an audio file of an accompaniment audio track and an audio file of the accompaniment audio track, as well as a playback delay of the accompaniment audio track and a target recording progress of the recording audio track when starting to play the accompaniment audio track, wherein the recording audio track is used to record the singing audio when playing the accompaniment audio track; performing audio track alignment processing by removing the playback delay of the accompaniment audio track and the portion corresponding to the target recording progress in the recording audio track;

[0149] Alternatively, when the recording audio track is recorded to the target recording progress, the playback of the accompaniment audio track is started, wherein the recording audio track is used to record the singing audio when the accompaniment audio track is played; the playback delay of the accompaniment audio track is calculated by playing the accompaniment audio track, so as to remove the playback delay and the target recording progress in the recording audio track, and perform audio track alignment between the recording audio track and the accompaniment audio track.

[0150] The processor 1001 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It may implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0151] The storage medium 1003 can be, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (Electric Erasable Programmable Read-Only Memory, EEPROM), etc.

[0152] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a storage medium, and the computer program includes a program code configured to execute the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through a communication part, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), the above functions defined in the method of the present invention are executed. It should be noted that the storage medium described in the present invention can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more conductors, a portable computer disk, a hard disk, a random access storage medium (RAM), a read-only storage medium (ROM), an erasable programmable read-only storage medium (EPROM or flash memory), an optical fiber, a portable compact disk read-only storage medium (CD-ROM), an optical storage medium, a magnetic storage medium, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, an apparatus or a device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any storage medium other than a computer-readable storage medium, which may send, propagate, or transmit a program configured to be used by or in conjunction with an instruction execution system, an apparatus or a device. The program code contained on the storage medium can be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0153] Computer program code configured to perform the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0154] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment, or a part of a code, and the module, program segment, or a part of the code contains one or more executable instructions configured to implement the specified logical function. There are specific sequential relationships in the above-mentioned specific embodiments, but these sequential relationships are only exemplary. When the specific implementation is implemented, these steps may be less, more or the execution order may be adjusted. That is, in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0155] The modules involved in the embodiments of the present invention may be implemented by software or hardware. The names of these modules do not limit the modules themselves in some cases.

[0156] As another aspect, the present invention further provides a storage medium having a computer program stored thereon, which implements the method described in the above embodiment when executed by a processor.

[0157] As another aspect, the present invention further provides a storage medium, which may be included in the device described in the above embodiment; or it may exist independently without being assembled into the device. The above storage medium carries one or more programs, and when the above one or more programs are executed by the device, the device: determines the playback delay of the accompaniment audio track and the target recording progress of the recording audio track when starting to play the accompaniment audio track, wherein the recording audio track is used to record the singing audio when playing the accompaniment audio track; aligns the recording audio track with the accompaniment audio track by removing the playback delay of the accompaniment audio track and the part corresponding to the target recording progress in the recording audio track;

[0158] Alternatively, performing synthesis processing according to the aligned recorded audio track and accompaniment audio track, wherein the recorded audio track and accompaniment audio track are obtained by processing using the above-mentioned audio track processing method; performing post-processing on the synthesized audio track based on the audio track post-processing mode;

[0159] Alternatively, obtaining an audio file of an accompaniment audio track and an audio file of the accompaniment audio track, as well as a playback delay of the accompaniment audio track and a target recording progress of the recording audio track when starting to play the accompaniment audio track, wherein the recording audio track is used to record the singing audio when playing the accompaniment audio track; performing audio track alignment processing by removing the playback delay of the accompaniment audio track and the portion corresponding to the target recording progress in the recording audio track;

[0160] Alternatively, when the recording audio track is recorded to the target recording progress, the playback of the accompaniment audio track is started, wherein the recording audio track is used to record the singing audio when the accompaniment audio track is played; the playback delay of the accompaniment audio track is calculated by playing the accompaniment audio track, so as to remove the playback delay and the target recording progress in the recording audio track, and perform audio track alignment between the recording audio track and the accompaniment audio track.

[0161] The expressions "first", "second", "the first" or "the second" used in various embodiments of the present disclosure may modify various components regardless of order and / or importance, but these expressions do not limit the corresponding components. The above expressions are only configured for the purpose of distinguishing an element from other elements. For example, a first user device and a second user device represent different user devices, although both are user devices. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element without departing from the scope of the present disclosure.

[0162] When one element (e.g., a first element) is referred to as being "(operably or communicatively) coupled" or "(operably or communicatively) coupled to" or "connected to" another element (e.g., a second element), it is understood that the one element is directly connected to the other element or the one element is indirectly connected to the other element via yet another element (e.g., a third element). Conversely, it is understood that when an element (e.g., a first element) is referred to as being "directly connected" or "directly coupled" to another element (the second element), no element (e.g., a third element) is interposed between the two.

[0163] The above description is only a preferred embodiment of the present invention and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present invention is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present invention (but not limited to) to form a technical solution.

Claims

1. A method for processing an audio track, comprising: Determine the start duration corresponding to the target playback progress; Determining the playback delay of the accompaniment audio track according to the difference between the startup duration corresponding to the target playback progress and the progress duration indicated by the target playback progress; Determining a target recording progress of a recording track when starting to play the accompaniment track, wherein the recording track is used to record the singing audio when playing the accompaniment track; The recording audio track and the accompaniment audio track are aligned by removing the playback delay of the accompaniment audio track and the portion corresponding to the target recording progress in the recording audio track.

2. The method according to claim 1, wherein: The determining the start-up duration corresponding to the target playback progress includes: Determine a first time for starting to play the accompaniment audio track and a second time corresponding to the target playback progress; Based on the time difference between the second time and the first time, the startup duration corresponding to the target playback progress is calculated.

3. The method according to claim 1, wherein: The aligning of the recording audio track and the accompaniment audio track by removing the playback delay of the accompaniment audio track and the portion corresponding to the target recording progress in the recording audio track comprises: Calculate the sum of the delay duration of the playback delay of the accompaniment audio track and the progress duration corresponding to the target recording progress; At the beginning of the recording audio track, a portion corresponding to the sum of the delay duration and the progress duration is removed to align the recording audio track with the accompaniment audio track.

4. The method according to claim 1, wherein: The method further includes: displaying the recording audio track and the accompaniment audio track in an audio track editing interface, wherein the recording audio track and the accompaniment audio track are aligned by removing the playback delay of the accompaniment audio track and the portion corresponding to the target recording progress in the recording audio track, including: In response to the track alignment instruction in the track editing interface, an operation of removing the playback delay of the accompaniment track and the portion corresponding to the target recording progress in the recording track; In the audio track editing interface, the aligned recording audio track and accompaniment audio track are displayed.

5. A method for processing an audio track, comprising: Performing synthesis processing according to the aligned recorded audio track and accompaniment audio track, wherein the recorded audio track and accompaniment audio track are obtained by processing the audio track processing method according to any one of claims 1 to 4; Based on the audio track post-processing mode, post-process the synthesized audio track.

6. The method according to claim 5, wherein: The synthesizing process according to the aligned recorded audio track and accompaniment audio track comprises: While displaying the playback object, the recorded audio track and the accompaniment audio track are synthesized to obtain a synthesized audio track, wherein the synthesized audio track is post-processed based on the audio track post-processing mode, including: In response to the completion of the recorded audio track and the accompaniment audio track, the display of the playback object is ended, and the display of the synthesized audio track is started in the synthesized audio track preview mode.

7. A method for processing an audio track, comprising: Acquire an audio file of an accompaniment audio track and an audio file of the accompaniment audio track, as well as a playback delay of the accompaniment audio track and a target recording progress of recording the audio track when starting to play the accompaniment audio track, wherein the recording audio track is used to record the singing audio when playing the accompaniment audio track, and the playback delay of the accompaniment audio track is determined according to the difference between the startup duration corresponding to the target playback progress and the progress duration indicated by the target playback progress; Track alignment processing is performed by removing the playback delay of the accompaniment track and the portion corresponding to the target recording progress in the recording track.

8. The method according to claim 7, wherein: The playback delay of the accompaniment track is obtained through the second voiceprint information in the accompaniment track, and the target recording progress of the recording track when starting to play the accompaniment track is obtained through the first voiceprint information of the recording track.

9. The method according to claim 7, wherein: The method further comprises: Performing audio trimming processing on the first voiceprint information in the recorded audio track to obtain a recorded audio track after audio trimming, wherein the audio track alignment processing is performed by removing the playback delay of the accompaniment audio track and the portion corresponding to the target recording progress in the recorded audio track, including: The recorded audio track after the tuning process is aligned with the accompaniment audio track by removing the playback delay of the accompaniment audio track and the portion corresponding to the target recording progress in the recorded audio track after the tuning process.

10. The method according to claim 9, wherein: The performing audio trimming processing on the first voiceprint information in the recorded audio track to obtain the audio trimmed recorded audio track includes: The first voiceprint information in the recorded audio track is separated from the sound source to obtain a human voice source audio track, wherein the process of aligning the recorded audio track after the audio correction process with the accompaniment audio track by removing the playback delay of the accompaniment audio track and the portion corresponding to the target recording progress in the recorded audio track after the audio correction process comprises: The human voice source track and the accompaniment track are aligned by removing the playback delay of the accompaniment track and the portion corresponding to the target recording progress in the human voice source track.

11. A method for processing an audio track, comprising: Determine a first time when the recording track is recorded to a target recording progress, and start playing the accompaniment track at the first time, wherein the recording track is used to record the singing audio when the accompaniment track is played; Determine a second time of the target playback progress of the accompaniment audio track, and calculate the playback delay of the accompaniment audio track based on the time difference between the second time and the first time and the progress duration of the target playback progress, so as to remove the playback delay and the target recording progress in the recording audio track, and perform audio track alignment on the recording audio track and the accompaniment audio track.

12. The method according to claim 11, wherein: When the recording track is recorded to the first time of the target recording progress, starting the playing of the accompaniment track includes: When the first frame is recorded in the recording audio track, the playing of the accompaniment audio track is started.

13. A soundtrack processing device, comprising: A determination module is configured to determine a start-up duration corresponding to a target playback progress, determine a playback delay of an accompaniment audio track according to a difference between the start-up duration corresponding to the target playback progress and the progress duration indicated by the target playback progress, and determine a target recording progress of a recording audio track when starting to play the accompaniment audio track, wherein the recording audio track is used to record the singing audio when playing the accompaniment audio track; The track alignment module aligns the recording track and the accompaniment track by removing the playback delay of the accompaniment track and the portion corresponding to the target recording progress in the recording track.

14. A soundtrack processing device, comprising: A track synthesis module, which performs synthesis processing according to the aligned recording track and accompaniment track, wherein the recording track and accompaniment track are obtained by processing the track processing method according to any one of claims 1 to 4; The audio track post-processing module performs post-processing on the synthesized audio track based on the audio track post-processing mode.

15. A soundtrack processing device, comprising: An acquisition module is provided to acquire an audio file of an accompaniment audio track and an audio file of the accompaniment audio track, a playback delay of the accompaniment audio track, and a target recording progress of recording the audio track when starting to play the accompaniment audio track, wherein the recording audio track is used to record the singing audio when playing the accompaniment audio track, and the playback delay of the accompaniment audio track is determined according to the difference between the startup time corresponding to the target playback progress and the progress time indicated by the target playback progress; The alignment processing module performs track alignment processing by removing the playback delay of the accompaniment track and the part corresponding to the target recording progress in the recorded audio track.

16. A soundtrack processing device, comprising: A play start module determines the first time when the recording track is recorded to the target recording progress, and starts the play of the accompaniment track at the first time, wherein the recording track is used to record the singing audio when the accompaniment track is played; A calculation module determines a second time of the target playback progress of the accompaniment audio track, and calculates the playback delay of the accompaniment audio track based on the time difference between the second time and the first time and the progress duration of the target playback progress, so as to remove the playback delay and the target recording progress in the recording audio track, and perform audio track alignment on the recording audio track and the accompaniment audio track.

17. An electronic device, comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform an operation corresponding to the method according to any one of claims 1 to 12.

18. A storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the method according to any one of claims 1 to 12 is implemented.

Images