Methods for obtaining song start time, computer equipment, and readable storage media

Abstract

This application relates to a method, computer device, and computer-readable storage medium for obtaining the start time of a song. The method includes: acquiring a first song signal and a second song signal matching the first song signal; acquiring a second sub-song signal from the second song signal, with the start time of the second song as the starting time and a preset duration; and acquiring multiple first sub-song signals with preset durations from the first song signal; each first sub-song signal corresponds to a different start time; acquiring a target first sub-song signal based on the similarity between each first sub-song signal and the second sub-song signal, and using the start time of the target first sub-song signal as the start time of the first song. This method can improve the efficiency of obtaining the start time of a song.

Description

Technical Field

[0001] This application relates to the field of audio processing technology, and in particular to a method, apparatus, computer device, computer-readable storage medium, and computer program product for obtaining the start time of a song. Background Technology

[0002] With the development of audio processing technology, a technique has emerged that displays scrolling lyrics synchronously during song playback. This technique relies on aligning the lyrics file with the song audio to ensure the accuracy of the lyrics scrolling during song playback.

[0003] In traditional technology, aligning lyrics files with song audio relies on the consistency between the song's vocal content and the lyrics text. If the song's audio content is inconsistent with the lyrics text, such as when the song's audio content contains an opening verse, directly aligning the lyrics text will result in alignment failure. Therefore, to avoid this situation, users usually need to annotate the song's audio content to obtain the song's start time.

[0004] However, the above method of obtaining the song start time relies on manual annotation by users, so the efficiency of obtaining the song start time is low. Summary of the Invention

[0005] Therefore, it is necessary to provide a method, apparatus, computer device, computer-readable storage medium, and computer program product for obtaining the start time of a song, which can improve the efficiency of obtaining the start time of a song, in response to the above-mentioned technical problems.

[0006] Firstly, this application provides a method for obtaining the start time of a song, including:

[0007] Acquire a first song signal and a second song signal that matches the first song signal; the first song signal is the audio signal of the first song whose start time is to be acquired, and the second song signal is the song signal of the second song that has a preset association with the first song and has a set start time.

[0008] A second sub-song signal with a start time of the second song and a preset duration is obtained from the second song signal, and multiple first sub-song signals with the preset duration are obtained from the first song signal; each first sub-song signal corresponds to a different start time;

[0009] Based on the similarity between each of the first sub-song signals and the second sub-song signals, a target first sub-song signal is obtained, and the start time of the target first sub-song signal is taken as the start time of the first song.

[0010] In one embodiment, there are multiple second sub-song signals, and the duration of each second sub-song signal is a different preset length. The step of obtaining a target first sub-song signal based on the similarity between each first sub-song signal and a second sub-song signal includes: obtaining multiple current first sub-song signals with the same duration as the current second sub-song signal, and obtaining an initial similarity between each current first sub-song signal and the current second sub-song signal; the current second sub-song signal is any one of the multiple second sub-song signals; obtaining a target similarity corresponding to the current second sub-song signal based on the initial similarity corresponding to each current second sub-song signal and the duration of the current second sub-song signal; and obtaining the target first sub-song signal based on the target similarity corresponding to each current second sub-song signal.

[0011] In one embodiment, obtaining the target similarity of the current second sub-song signal based on the initial similarity levels corresponding to the current second sub-song signal and the duration of the current second sub-song signal includes: obtaining the maximum initial similarity level corresponding to the current second sub-song signal from the initial similarity levels corresponding to the current second sub-song signal; and obtaining the target similarity level corresponding to the current second sub-song signal based on the duration of the current second sub-song signal and the maximum initial similarity level corresponding to the current second sub-song signal.

[0012] In one embodiment, obtaining the target first sub-song signal based on the target similarity corresponding to each of the current second sub-song signals includes: obtaining the target second sub-song signal according to the target similarity corresponding to each of the current second sub-song signals; the target second sub-song signal is the current second sub-song signal corresponding to the maximum target similarity; and taking the current first sub-song signal with the highest initial similarity to the target second sub-song signal among the multiple current first sub-song signals corresponding to the target second sub-song signal as the target first sub-song signal.

[0013] In one embodiment, the step of obtaining a second sub-song signal from the second song signal with the start time of the second song as the starting time and a time length of a preset length, and obtaining multiple first sub-song signals with a time length of the preset length from the first song signal, includes: constructing a first time window and a second time window with a window length of the preset length; setting the starting position of the first time window at the start time of the first song signal, and setting the starting position of the second time window at the target signal time of the second song signal; the target signal time being the start time of the second song; obtaining the second sub-song signal through the second time window; shifting the first time window multiple times according to a preset offset time until the end time of the shifted time window containing the first song signal; obtaining multiple first sub-song signals through the first time window and each shifted time window; the time length of the preset offset time is less than the preset length.

[0014] In one embodiment, obtaining the first song signal and the second song signal matching the first song signal includes: performing Fourier transform processing on the first song and the second song respectively, and extracting the spectral envelope of the Fourier transform results of the first song and the second song to obtain the first song signal corresponding to the first song and the second song signal corresponding to the second song.

[0015] In one embodiment, the second song has a pre-defined lyrics file; after taking the start time of the target first sub-song signal as the start time of the first song, the method further includes: extracting a portion of the first song after its start time to obtain an extracted song corresponding to the first song; obtaining the lyrics text contained in the lyrics file of the second song; and aligning the extracted song with the lyrics text to generate a lyrics file corresponding to the first song.

[0016] Secondly, this application also provides a device for obtaining the start time of a song, comprising:

[0017] The song signal acquisition module is used to acquire a first song signal and a second song signal that matches the first song signal; the first song signal is the audio signal of the first song whose start time is to be acquired, and the second song signal is the song signal of the second song that has a preset association with the first song and has a set start time.

[0018] The sub-signal acquisition module is used to acquire a second sub-song signal from the second song signal, with the start time of the second song as the start time and the time length as a preset length, and to acquire multiple first sub-song signals with the preset time length from the first song signal; each first sub-song signal corresponds to a different start time;

[0019] The start time acquisition module is used to acquire a target first sub-song signal based on the similarity between each first sub-song signal and the second sub-song signal, and to use the start time of the target first sub-song signal as the start time of the first song.

[0020] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described in any embodiment of the first aspect.

[0021] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any embodiment of the first aspect.

[0022] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in any embodiment of the first aspect.

[0023] The aforementioned method, apparatus, computer equipment, computer-readable storage medium, and computer program product for obtaining the start time of a song obtain a first song signal of a first song whose start time is to be obtained, and a second song signal of a second song that has a preset association with the first song and is set with a start time. Then, a second sub-song signal with a start time of a preset length can be obtained from the second song signal, and multiple first sub-song signals with a preset length of time can be obtained from the first song signal. By utilizing the similarity between the first sub-song signal and the second sub-song signal, the start time of the first song can be identified. This method achieves automatic identification of the start time of the first song without requiring the user to manually mark the start time, thus improving the efficiency of obtaining the start time of a song. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a flowchart illustrating a method for obtaining the song start time in one embodiment;

[0026] Figure 2 This is a schematic diagram of the process for obtaining the target first sub-song signal in one embodiment;

[0027] Figure 3 This is a schematic diagram of the process for obtaining the second sub-song signal and the first sub-song signal in one embodiment;

[0028] Figure 4 This is a schematic diagram of the process of generating lyrics timestamps using an automatic alignment algorithm in one embodiment;

[0029] Figure 5 This is a schematic diagram of the process for generating a new qrc file in one embodiment;

[0030] Figure 6 This is a structural block diagram of a song start time acquisition device in one embodiment;

[0031] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0033] In one embodiment, such as Figure 1 As shown, a method for obtaining the start time of a song is provided. This embodiment illustrates the application of this method to a server. It is understood that this method can also be applied to a terminal, or to a system including both a terminal and a server, and is implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0034] Step S101: Obtain a first song signal and a second song signal that matches the first song signal; the first song signal is the audio signal of the first song whose start time is to be obtained, and the second song signal is the song signal of the second song that has a preset association with the first song and has a set start time.

[0035] In this context, "first song" refers to the song whose start time needs to be determined, and "first song signal" refers to the audio signal of the first song, which can be either its time-domain signal or its frequency-domain signal. "Second song" refers to a song that has a pre-defined relationship with the first song and has a set start time. For example, the first and second songs might be the same music. The first song can be processed from the second song, and this processing can involve various types of audio post-production, such as adding an intro, changing the sound quality, adjusting the speed, or performing pitch correction or mixing. The second song can also have a set start time, and its audio signal can be either its time-domain signal or its frequency-domain signal.

[0036] Specifically, after obtaining the first song whose start time needs to be obtained, the server can also obtain a second song that has a preset association with the first song and has a set start time. Then, the server can extract signals from the first song and the second song to obtain the first song signal and the second song signal.

[0037] Step S102: Obtain a second sub-song signal from the second song signal, with the start time of the second song as the starting time and the time length as a preset length; and obtain multiple first sub-song signals with a preset time length from the first song signal; each first sub-song signal corresponds to a different start time.

[0038] The second sub-song signal is a song signal from the second song signal that starts at the beginning of the song and lasts for a preset duration. For example, if the preset duration is 3 seconds, then the second sub-song signal is a song signal from the second song signal that starts at the beginning of the song and lasts for 3 seconds. The first sub-song signal, on the other hand, is a song signal extracted from the first song signal that has multiple durations of a preset length. Continuing with the preset duration of 3 seconds as an example, the first sub-song signal could be a signal from 0s to 3s in the first song signal, or a signal from 1s to 4s, or a signal from 2s to 5s, etc., and the start time of each first sub-song signal is different.

[0039] Specifically, the server can obtain a second sub-song signal from the second song signal, with the start time of the second song as the starting time and the duration as a preset length. It can also obtain multiple sub-song signals with different start times and durations as the same preset length as the second sub-song signal from the first song signal, as multiple first sub-song signals.

[0040] Step S103: Based on the similarity between each first sub-song signal and the second sub-song signal, obtain the target first sub-song signal, and use the start time of the target first sub-song signal as the start time of the first song.

[0041] The target first sub-song signal can be determined by the similarity between each first sub-song signal and the second sub-song signal. For example, the first sub-song signal with the highest similarity can be used as the target first sub-song signal. Since the first song and the second song have a pre-defined relationship, and the start time of the second sub-song signal is the start time of the song, the target first sub-song signal can be considered as the sub-song signal in the first song signal corresponding to the part of the second sub-song signal. Therefore, after the server determines the target first sub-song signal through similarity, it can use the start time of the target first sub-song signal as the start time of the first song. Thus, the start time of the target first sub-song signal can be considered as the start time of the first song.

[0042] Taking a song where the first song is obtained by adding an opening mono to the second song as an example, assuming the opening mono lasts 1 second, and the second song's starting time is 0 seconds with a preset duration of 3 seconds, then the second sub-song signal could be the 0s-3s portion of the second song signal, and the first sub-song signal could be the 0s-3s portion of the first song signal, or it could be the 1s-4s portion, or the 2s-5s portion, etc. The server can then calculate the similarity between each first sub-song signal and the second sub-song signal, and determine the first sub-song signal with the highest similarity as the target first sub-song signal. For example, if the server identifies the 1s-4s portion as having the highest similarity to the second sub-song signal, then the 1s-4s portion can be used as the target first sub-song signal, thus setting 1 second as the starting time of the first song. In this way, even if the first sub-song signal contains a 1s opening mono, the server can accurately identify the starting time of the first song as 1 second.

[0043] In the above method for obtaining the start time of a song, a first song signal of a first song whose start time is to be obtained is obtained, and a second song signal of a second song that has a preset association with the first song and has a set start time are obtained. Then, a second sub-song signal with a start time of the start time and a preset duration can be obtained from the second song signal, and multiple first sub-song signals with a preset duration can be obtained from the first song signal. By using the similarity between the first sub-song signal and the second sub-song signal, the start time of the first song can be identified. This method achieves automatic identification of the start time of the first song without requiring the user to manually mark the start time, thus improving the efficiency of obtaining the start time of a song.

[0044] In one embodiment, there are multiple second sub-song signals, and the duration of each second sub-song signal is a different preset length; for example... Figure 2 As shown, step S103 may further include:

[0045] Step S201: Obtain multiple current first sub-song signals with the same time length as the current second sub-song signal, and obtain the initial similarity between each current first sub-song signal and the current second sub-song signal; the current second sub-song signal is any one of the multiple second sub-song signals.

[0046] In this embodiment, there can be multiple second sub-song signals. Each second sub-song signal has the same start time, which is the start time of the second song, and the signal length of each second sub-song signal is different. For example, if the start time of the second song is 0s, then the multiple second sub-song signals can be signals from the 0s-3s portion of the second song signal, signals from the 0s-3.5s portion of the second song signal, or signals from the 0s-4s portion of the second song signal.

[0047] Furthermore, each second sub-song signal corresponds to multiple first sub-song signals of the same duration. For example, a second sub-song signal with a duration of 3 seconds can correspond to a signal from 0 seconds to 3 seconds in the first song signal, or a signal from 1 second to 4 seconds, or a signal from 2 seconds to 5 seconds, etc. Similarly, a second sub-song signal with a duration of 3.5 seconds can correspond to a signal from 0 seconds to 3.5 seconds in the first song signal, or a signal from 1 second to 4.5 seconds, or a signal from 2 seconds to 5.5 seconds, etc.

[0048] The current second sub-song signal refers to any one of the multiple second sub-song signals. The multiple current first sub-song signals are multiple first sub-song signals with the same time length as the current second sub-song signal. Taking the current second sub-song signal as a second sub-song signal with a signal length of 3s as an example, then the multiple current first sub-song signals refer to the first sub-song signals corresponding to 0s-3s, the first sub-song signals corresponding to 1s-4s, and the first sub-song signals corresponding to 2s-5s, and so on, among the first sub-song signals.

[0049] The initial similarity refers to the degree of similarity between each current first sub-song signal and the current second sub-song signal obtained by direct calculation. This similarity can be represented by distance, for example, by the dynamic programming distance between the song signals, or by the LCS (longest common subsequence) distance calculated from the peak points of the song signals' spectrum.

[0050] Specifically, after the server obtains the current second sub-song signal and the multiple current first sub-song signals corresponding to the current second sub-song signal, it can calculate the distance between each current first sub-song signal and the current second sub-song signal, thereby obtaining the initial similarity between each current first sub-song signal and the current second sub-song signal.

[0051] Step S202: Based on the initial similarity levels corresponding to the current second sub-song signal and the time length of the current second sub-song signal, obtain the target similarity level corresponding to the current second sub-song signal;

[0052] Step S203: Based on the target similarity corresponding to each current second sub-song signal, obtain the target first sub-song signal.

[0053] The target similarity refers to the further similarity calculated based on the initial similarity and the duration of the second sub-song signal. In this embodiment, there are multiple second sub-song signals, each corresponding to a different duration. Generally, the longer the duration of the second sub-song signal, the more song content it covers, and the greater the potential for audio modifications, thus resulting in a larger calculated distance. Conversely, if the duration of the second sub-song signal is shorter, the randomness is greater, leading to a shorter calculated distance but lower reliability (i.e., lower confidence level). Therefore, in this embodiment, a target similarity is set to comprehensively consider the relationship between the initial similarity and the duration of the current second sub-song signal, thereby further ensuring the accuracy of the target first sub-song signal identification.

[0054] Specifically, after obtaining the initial similarity between each current first sub-song signal and the current second sub-song signal, the server can use the aforementioned initial similarity and the time length of the current second sub-song signal to obtain the target similarity of the current second sub-song signal, and then use the target similarity of each current second sub-song signal to obtain the target first sub-song signal.

[0055] In this embodiment, multiple second sub-song signals can be obtained by setting different time lengths. The target similarity corresponding to each second sub-song signal can be obtained by using the time length of each second sub-song signal and combining multiple initial similarity levels of each second sub-song signal. The first sub-song signal can then be filtered using the target similarity. This method can balance the time length of the second sub-song signal and the initial similarity levels, thereby improving the accuracy of the target first sub-song signal identification.

[0056] Furthermore, step S202 may further include: obtaining the maximum initial similarity degree corresponding to the current second sub-song signal from each of the initial similarity degrees corresponding to the current second sub-song signal; and obtaining the target similarity degree corresponding to the current second sub-song signal based on the time length of the current second sub-song signal and the maximum initial similarity degree corresponding to the current second sub-song signal.

[0057] The maximum initial similarity refers to the largest initial similarity among multiple initial similarities corresponding to the current second sub-song signal. In this embodiment, the server can calculate the target similarity corresponding to the current second sub-song signal based on the maximum initial similarity corresponding to the current second sub-song signal and the time length corresponding to the current second sub-song signal.

[0058] For example, the initial similarity can be characterized by the distance between the current second sub-song signal and each current first sub-song signal. Then, the maximum initial similarity corresponding to the current second sub-song signal can be characterized by the minimum value among the distances between the current second sub-song signal and each current first sub-song signal. Let d ij To represent, the d ij This can be represented as the distance between the current second sub-song signal i and the current first sub-song signal j (assuming d). ij If the minimum distance between the second-previous sub-song signal and each current first sub-song signal is taken as the minimum distance, then the target similarity corresponding to the current second sub-song signal can be calculated using the following formula:

[0059]

[0060] Where, d ij len_win represents the minimum distance between the current second sub-song signal and each current first sub-song signal (representing the maximum initial similarity corresponding to the current second sub-song signal), while conf represents the duration of the current second sub-song signal. dis This indicates the target similarity level corresponding to the current second sub-song signal. It can be seen that the target similarity level (conf)... dis It can be calculated by combining the minimum distance between the current second sub-song signal and each current first sub-song signal, and the time length of the current second sub-song signal. This comprehensive calculation method satisfies the condition that the greater the time length and the smaller the distance (i.e., the greater the maximum initial similarity), the greater the target similarity.

[0061] Alternatively, the target similarity corresponding to the current second sub-song signal can also be obtained by calculating the ratio of the square of the current second sub-song signal's time length to the minimum distance between the current second sub-song signal and each current first sub-song signal, that is, the square of len_win and d ij The ratio between them is used as the degree of similarity between the targets.

[0062] In this embodiment, the target similarity corresponding to the current second sub-song signal can be calculated based on the time length of the current second sub-song signal and the maximum initial similarity corresponding to the current second sub-song signal. The maximum initial similarity can be characterized by the minimum distance between the current second sub-song signal and each current first sub-song signal. The time length is positively correlated with the target similarity, while the minimum distance between the current second sub-song signal and each current first sub-song signal is negatively correlated with the target similarity. This method can improve the accuracy of the target similarity calculation.

[0063] In addition, step S203 may further include: obtaining the target second sub-song signal according to the target similarity corresponding to each current second sub-song signal; the target second sub-song signal is the current second sub-song signal corresponding to the maximum target similarity; and taking the current first sub-song signal with the highest initial similarity to the target second sub-song signal among the multiple current first sub-song signals corresponding to the target second sub-song signal as the target first sub-song signal.

[0064] The target second sub-song signal is the current second sub-song signal with the highest target similarity. After obtaining the target similarity of each current second sub-song signal, the server can select the current second sub-song signal with the highest target similarity as the target second sub-song signal.

[0065] After determining the target second sub-song signal, the target first sub-song signal can be obtained by selecting the current first sub-song signal with the highest initial similarity from the multiple current first sub-song signals corresponding to the target second sub-song signal.

[0066] For example, the current second sub-song signals include: second sub-song signal 1, second sub-song signal 2, and second sub-song signal 3. The current first sub-song signal corresponding to second sub-song signal 1 includes first sub-song signal A1, first sub-song signal B1, and first sub-song signal C1; the current first sub-song signal corresponding to second sub-song signal 2 includes first sub-song signal A2, first sub-song signal B2, and first sub-song signal C2; and the current first sub-song signal corresponding to second sub-song signal 3 includes first sub-song signal A3, first sub-song signal B3, and first sub-song signal C3. After the server obtains the target similarity level corresponding to each current second sub-song signal, it can filter out the current second sub-song signal with the highest target similarity. If the target similarity level corresponding to second sub-song signal 1 is the highest, then the server can use second sub-song signal 1 as the target second sub-song signal. Then, the first sub-song signal with the highest initial similarity to the second sub-song signal 1 can be determined from the first sub-song signal A1, the first sub-song signal B1 and the first sub-song signal C1. If the first sub-song signal A1 has the highest initial similarity to the second sub-song signal 1, then the first sub-song signal A1 can be used as the target first sub-song signal.

[0067] In this embodiment, the server can determine the target second sub-song signal with the highest similarity from the current second sub-song signal. Then, it can determine the current first sub-song signal with the highest initial similarity from the multiple current first sub-song signals corresponding to the target second sub-song signal, and use it as the target first sub-song signal. This method can ensure that the target first sub-song signal fully considers the relationship between the initial similarity and the time length of the current second sub-song signal, further improving the accuracy of the target first sub-song signal selection.

[0068] In one embodiment, such as Figure 3 As shown, step S102 may further include:

[0069] Step S301: Construct a first time window and a second time window with a preset window length, and set the starting position of the first time window to the start time of the first song signal, and set the starting position of the second time window to the target signal time of the second song signal; the target signal time is the start time of the second song.

[0070] In this context, the start time of the first song signal refers to the initial signal time of the first song signal, while the target signal time of the second song signal refers to the signal time corresponding to the start time of the second song. Specifically, after receiving the first and second song signals, the server can first construct two time windows of preset length, serving as the first and second time windows respectively. Then, the starting position of the first time window can be set at the start time of the first song signal, and the starting position of the second time window can be set at the target signal time corresponding to the start time of the second song signal.

[0071] In step S302, the second sub-song signal is obtained through the second time window.

[0072] After obtaining the second time window, the server can use the second song signal contained within the second time window as the second sub-song signal. For example, if it is necessary to extract a second sub-song signal with a duration of 3 seconds, the server can first construct a second time window with a window length of 3 seconds and set the start time of this time window to the target signal time in the second song signal. Then, the sub-song signal contained in the second time window can be a second sub-song signal with a start time of 3 seconds and a duration of 3 seconds, based on the start time of the second song.

[0073] Step S303: The first time window is shifted multiple times according to a preset offset time until the end time of the shifted time window containing the first song signal is reached. Multiple first sub-song signals are obtained through the first time window and the shifted time windows. The time length of the preset offset time is less than the preset length.

[0074] The preset offset time refers to the time for shifting the first time window. The length of this preset offset time must be less than the window length of the first time window, i.e., the preset length. After obtaining the first time window, the server can collect the first song signal contained in the first time window, and can also shift the first time window multiple times according to the preset offset time until the end time of the shifted time window containing the first song signal. Thus, multiple first sub-song signals can be obtained using the first time window and the shifted time windows. For example, the first song signals contained in the first time window and the shifted time windows can be used as multiple first sub-song signals.

[0075] For example, the first time window has a length of 3 seconds, and the preset offset time can be 0.05 seconds. After obtaining the first time window, the server can first set the starting position of the first time window to the start time of the first song signal. Then, the first song signal contained in the first time window can be the first song signal from 0 seconds to 3 seconds. Then, the server can shift the first time window multiple times. The first time window obtained by the first shift can contain the first song signal from 0.05 seconds to 3.05 seconds, and the first time window obtained by the second shift can contain the first song signal from 0.1 seconds to 3.1 seconds, and so on, until the time window is shifted to the end time of the first song signal. In this way, multiple first song signals with a time length of 3 seconds can be obtained.

[0076] If it is necessary to increase the number of second sub-song signals, for example, to collect second sub-song signals corresponding to different time lengths, then a first time window and a second time window with different time lengths can be constructed again, and the above process can be repeated. For example, a first time window and a second time window with a window length of 3.5s can be constructed, and the above process can be repeated to obtain a second sub-song signal with a time length of 3.5s, as well as multiple first sub-song signals with a time length of 3.5s.

[0077] In this embodiment, the server can construct a first time window and a second time window with a preset window length, thereby obtaining the second sub-song signal through the second time window, and obtaining multiple first sub-song signals by shifting the first time window. This method can improve the acquisition efficiency of the first and second sub-song signals.

[0078] In one embodiment, step S101 may further include: in response to a song replacement request, obtaining the song to be replaced and the song to be replaced, and using the song to be replaced as the first song and the song to be replaced as the second song; performing Fourier transform processing on the first song and the second song respectively, and extracting the spectral envelope of the Fourier transform results of the first song and the second song to obtain the first song signal corresponding to the first song and the second song signal corresponding to the second song.

[0079] A song replacement request refers to a request to replace a song to be replaced with a song to be replaced. In this embodiment, the song start time acquisition method can be applied to the song replacement scenario. The song before replacement is the song to be replaced. The old song of the song to be replaced can be used as the second song, that is, the song with a pre-set start time. The new song after replacement can be used as the first song, that is, the song whose start time needs to be acquired.

[0080] For example, the second song could be version A of a song, while the first song is version B of that song. Version B could be a song with added opening remarks based on version A. Therefore, when updating a song, a song replacement request can be initiated to update the song from version A to version B.

[0081] The server can then perform Fourier transform processing on the first song and the second song respectively, and extract the spectral envelope from the Fourier transform results of the first song and the second song respectively, thereby obtaining the first song signal corresponding to the first song and the second song signal corresponding to the second song.

[0082] In this embodiment, the method for obtaining the song start time can be applied in the scenario of song replacement. By taking the song to be replaced as the second song and the replaced song as the first song, the start time of the replaced song can be obtained by using the start time of the song to be replaced. This method can improve the efficiency of obtaining the start time of the replaced song in the song replacement scenario.

[0083] Furthermore, the second song has a pre-existing lyrics file; after step S103, the process may further include: extracting a portion of the first song after the start time of the first song to obtain the extracted song corresponding to the first song; obtaining the lyrics text contained in the lyrics file based on the lyrics file of the second song; aligning the extracted song with the lyrics text to generate the lyrics file corresponding to the first song.

[0084] The lyrics file can refer to the qrc lyrics file, which is used to implement the scrolling lyrics function during the playback of the second song. Since the second song is the song before the replacement, it has a pre-existing lyrics file. In this embodiment, after the server obtains the start time of the replaced first song, it can obtain the lyrics file of the first song based on that start time.

[0085] Specifically, after the server determines the start time of the first song, it can also extract the portion of the song that starts after the first song, thus obtaining the corresponding extracted song. This method can remove any additional opening remarks that might be added when the song is replaced with the first song. At this point, it can ensure that the audio content of the extracted song is consistent with the lyrics text. Then, the lyrics text contained in the lyrics file of the second song can be extracted, and the extracted song and lyrics text can be aligned to generate the lyrics file corresponding to the first song.

[0086] In this embodiment, after obtaining the start time of the first song, the first song can be truncated. The lyrics text contained in the lyrics file of the second song can then be aligned with the truncated first song to generate the lyrics file of the first song. This method can improve the efficiency of constructing the lyrics file of the first song by eliminating the manual annotation process while ensuring the accuracy of the lyrics file of the first song.

[0087] In one embodiment, a method for detecting the starting point of a song based on local shortest dynamic programming distance is also provided. This method can be applied to scenarios where the qrc lyrics are updated after a song is replaced. The current qrc lyrics file generation process can be as follows: Figure 4 As shown, the following steps may be included:

[0088] 1. Extract acoustic features from song audio;

[0089] 2. Convert the lyrics text into a phoneme sequence. This requires a text-to-phoneme conversion dictionary. For Chinese, Pinyin can be used; for English, phonetic symbols or the International Phonetic Alphabet can be used. After obtaining the phoneme sequence, convert it into a state sequence of a Hidden Markov Model (HMM). Generally, one phoneme corresponds to three states. These states can transition to each other or to other states, and the specific transition probabilities are described by the HMM. A state is the smallest unit of word, and one frame of audio features corresponds to one state.

[0090] 3. Input the features of each frame into a Gaussian Mixture Model (GMM) or a Deep Neural Network (DNN) to obtain the probability that each frame feature belongs to each state of a Hidden Markov Model (HMM). Combine the transition probabilities and obtain the path with the highest global probability through Viterbi decoding. Align the frames and states one by one to obtain the decoding result.

[0091] 4. Convert the state into phonemes, then convert the phonemes into words to obtain the lyrics timestamp.

[0092] The above is the process of the automatic alignment algorithm. Automatic alignment requires that the audio content and the text content be consistent. However, if they are inconsistent, for example, by adding extra opening remarks at the beginning, even if Viterbi decoding can find the path with the highest probability, such paths are unreliable, and the resulting decoding results are unreliable.

[0093] Therefore, this embodiment provides a QRC lyrics update method that is applicable both when adding an opening monologue and when adding an intro dialogue. The implementation of this process is as follows: Figure 5 As shown, this allows audio replacement to be updated quickly and efficiently using algorithms without manual intervention.

[0094] The implementation of this process relies on a song start point detection method based on local shortest dynamic programming distance. First, Fourier transforms are performed on both the old and new song audio, and the spectral envelopes are extracted. Then, a sliding window is set, and the detection envelope starts from time point 0 of the new audio. The old audio uses a window of equal length to truncate its envelope, and the dynamic programming distance between the two envelopes is calculated and recorded. The window length is adjusted, and the above steps are repeated. After these steps, different window lengths and multiple sets of distances can be obtained. Since a longer window length covers more song content and the potential changes between the old and new songs are greater, a shorter distance is preferable. Conversely, a shorter window length results in a smaller difference between the two, leading to lower confidence. This embodiment calculates the ratio of the two to find the monitoring point with the highest confidence, thus obtaining the song start point. After determining the start point, the audio after the start point is automatically aligned with the original lyrics to obtain a new QRC.

[0095] Specifically, this process can be implemented through the following steps:

[0096] 1. Perform Fourier transform on both the old and new songs, extract the spectral envelope, and set a sliding window.

[0097] Assuming a window length of 3 seconds, and the sliding window of the old envelope is fixed at the starting point, calculate the shortest dynamic programming distance between the old and new envelopes, denoted as d. 11 Move the sliding window of the new envelope by 50ms each time, recalculate the distance, and repeat this process until the end of the new envelope is reached. Record this as d. 12 ,…, d 1n After one round of calculation, the length of the sliding window is changed, assuming it increases by 0.5 seconds each time. The sliding window length for the second round of calculation is 3.5 seconds. The calculation is then performed again in the same manner to obtain d. 21 ,…, d 2n By repeatedly changing the length of the sliding window and continuously moving the sliding window with the new envelope, multiple sets of distances can be obtained. Where d ij This represents the shortest dynamic programming distance between the old envelope and the j-th new envelope within the length of the i-th window.

[0098] 2. The longer the window length, the more song content it covers, and the greater the potential for changes between old and new audio, resulting in a larger distance. Conversely, if the window length is too short, there is greater randomness, and while the calculated distance is short, the confidence level is low. Therefore, this embodiment defines a metric that balances window length and the shortest dynamic programming distance, named the confidence distance, which is calculated as follows:

[0099]

[0100] Where len_win represents the duration of the window, d ijLet i represent the shortest dynamic programming distance between the old envelope and the new envelope within the length of the i-th window. Iterate through all values ​​of i and j, find the largest confidence distance, and use the corresponding window starting position as the starting point.

[0101] 3. After obtaining the starting point of the song, extract the audio after the starting point to remove the opening remarks, ensuring consistency between the audio and lyrics. Extract the lyrics from the old QRC and the extracted audio, then use the existing automatic alignment algorithm for forced alignment and output a new QRC to replace the old one.

[0102] In this embodiment, the local shortest dynamic programming distance is calculated using the spectral envelope of the new and old audio. By combining different window lengths and calculating the confidence distance, the optimal starting point for the vocals in the new audio is found. The audio is then truncated and automatically aligned, ensuring that the opening remarks do not affect the alignment algorithm. This achieves an automatic QRC replacement process, greatly reducing manpower and improving the efficiency of lyrics replacement.

[0103] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0104] Based on the same inventive concept, this application also provides a song start time acquisition device for implementing the song start time acquisition method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations of one or more song start time acquisition device embodiments provided below can be found in the limitations of the song start time acquisition method described above, and will not be repeated here.

[0105] In one embodiment, such as Figure 6 As shown, a device for obtaining the start time of a song is provided, comprising: a song signal acquisition module 601, a sub-signal acquisition module 602, and a start time acquisition module 603, wherein:

[0106] The song signal acquisition module 601 is used to acquire a first song signal and a second song signal that matches the first song signal; the first song signal is the audio signal of the first song whose start time is to be acquired, and the second song signal is the song signal of the second song that has a preset association with the first song and has a set start time.

[0107] The sub-signal acquisition module 602 is used to acquire a second sub-song signal from the second song signal, with the start time of the second song as the start time and the time length as a preset length, and to acquire multiple first sub-song signals with a preset time length from the first song signal; each first sub-song signal corresponds to a different start time;

[0108] The start time acquisition module 603 is used to acquire the target first sub-song signal based on the similarity between each first sub-song signal and the second sub-song signal, and to use the start time of the target first sub-song signal as the start time of the first song.

[0109] The modules in the aforementioned song start time acquisition device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0110] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 7 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores song signal data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When executed by the processor, the computer program implements a method for obtaining the start time of a song.

[0111] Those skilled in the art will understand that Figure 7The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0112] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0113] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0114] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0115] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0116] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0117] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0118] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for obtaining the start time of a song, characterized in that, The method includes: Acquire a first song signal and a second song signal that matches the first song signal; the first song signal is the audio signal of the first song whose start time is to be acquired, and the second song signal is the song signal of the second song that has a preset association with the first song and has a set start time. A second sub-song signal with a start time of the second song and a preset duration is obtained from the second song signal, and multiple first sub-song signals with the preset duration are obtained from the first song signal; each first sub-song signal corresponds to a different start time; there are multiple second sub-song signals, each second sub-song signal has a start time of the second song and a different preset duration; Based on the similarity between each first sub-song signal and the second sub-song signal, a target first sub-song signal is obtained, and the start time of the target first sub-song signal is used as the start time of the first song; this includes: obtaining the target first sub-song signal based on the initial similarity between each current first sub-song signal and the current second sub-song signal, and the duration of the current second sub-song signal; the current second sub-song signal is any one of a plurality of second sub-song signals; each current first sub-song signal is a first sub-song signal in the first song signal corresponding to a different start time, and the duration is the same as the duration of the current second sub-song signal.

2. The method according to claim 1, characterized in that, There are multiple second sub-song signals, and the duration of each second sub-song signal is a different preset length; The step of obtaining the target first sub-song signal based on the similarity between each of the first sub-song signals and the second sub-song signals includes: Acquire multiple current first sub-song signals with the same time length as the current second sub-song signal, and obtain the initial similarity between each current first sub-song signal and the current second sub-song signal; The current second sub-song signal is any one of a plurality of second sub-song signals; Based on the initial similarity levels corresponding to the current second sub-song signal and the time length of the current second sub-song signal, the target similarity level corresponding to the current second sub-song signal is obtained; Based on the target similarity corresponding to each of the current second sub-song signals, the target first sub-song signal is obtained.

3. The method according to claim 2, characterized in that, The step of obtaining the target similarity corresponding to the current second sub-song signal based on the initial similarity levels corresponding to the current second sub-song signal and the time length of the current second sub-song signal includes: From the initial similarity levels corresponding to the current second sub-song signal, obtain the maximum initial similarity level corresponding to the current second sub-song signal; Based on the time length of the current second sub-song signal and the maximum initial similarity corresponding to the current second sub-song signal, the target similarity corresponding to the current second sub-song signal is obtained.

4. The method according to claim 2, characterized in that, The step of obtaining the target first sub-song signal based on the target similarity corresponding to each of the current second sub-song signals includes: Based on the target similarity corresponding to each of the current second sub-song signals, the target second sub-song signal is obtained; the target second sub-song signal is the current second sub-song signal corresponding to the maximum target similarity. The current first sub-song signal with the highest initial similarity to the target second sub-song signal among the multiple current first sub-song signals corresponding to the target second sub-song signal is taken as the target first sub-song signal.

5. The method according to claim 1, characterized in that, The step of obtaining a second sub-song signal from the second song signal, with the start time of the second song as the starting time and the duration as a preset length, and obtaining multiple first sub-song signals with the preset duration from the first song signal, includes: Construct a first time window and a second time window with a window length of the preset length, and set the starting position of the first time window at the start time of the first song signal, and set the starting position of the second time window at the target signal time of the second song signal; the target signal time is the start time of the second song. The second sub-song signal is obtained through the second time window; The first time window is shifted multiple times according to a preset offset time until the end time of the shifted time window containing the first song signal is reached. Multiple first sub-song signals are obtained through the first time window and each shifted time window. The time length of the preset offset time is less than the preset length.

6. The method according to claim 1, characterized in that, The acquisition of the first song signal and the second song signal matching the first song signal includes: Fourier transform is performed on the first song and the second song respectively, and the spectral envelope of the Fourier transform results of the first song and the second song is extracted to obtain the first song signal corresponding to the first song and the second song signal corresponding to the second song.

7. The method according to claim 1, characterized in that, The second song has a pre-defined lyrics file; after using the start time of the target first sub-song signal as the start time of the first song, the method further includes: Extract the portion of the first song after the start time of the first song to obtain the extracted song corresponding to the first song; Based on the lyrics file of the second song, obtain the lyrics text contained in the lyrics file; The extracted song and the lyrics text are aligned to generate the lyrics file corresponding to the first song.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images