Song singing time acquisition method, computer equipment and readable storage medium

Abstract

The invention relates to a song singing time acquisition method, computer equipment and a computer readable storage medium. The method comprises the following steps: acquiring a first song signal and a second song signal matched with the first song signal; obtaining a second sub-song signal with the singing starting time of the second song as the starting time and the time length being the preset length from the second song signal, and obtaining a plurality of first sub-song signals with the time length being the preset length from the first song signal; each first sub-song signal corresponds to different starting time; and obtaining a target first sub-song signal according to the similarity between each first sub-song signal and the second sub-song signal, and taking the starting time of the target first sub-song signal as the singing starting time of the first song. By adopting the method, the song singing time obtaining efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the field of audio processing technology, and in particular to a method, device, computer equipment, computer-readable storage medium and computer program product for obtaining the start time of a song. Background Art

[0002] With the development of audio processing technology, a technology has emerged that synchronously displays scrolling lyrics during song playback. This technology relies on aligning the lyrics file with the song audio to ensure the accuracy of the lyrics scrolling during song playback.

[0003] In traditional technology, the alignment of lyrics files with song audio depends on the consistency between the song singing content and the lyrics text content. If the song audio content is inconsistent with the lyrics text content, for example, when the song audio content carries an opening statement, directly aligning the lyrics text will cause the alignment to fail. Therefore, in order to avoid the above situation, users are usually required to mark the song audio content to obtain the start time of the song.

[0004] However, the above method of obtaining the song start time relies on the manual labeling of the user, so the efficiency of obtaining the song start time is low. Summary of the invention

[0005] Based on this, it is necessary to provide a song start time acquisition method, device, computer equipment, computer readable storage medium and computer program product that can improve the efficiency of song start time acquisition in response to the above technical problems.

[0006] In a first aspect, the present application provides a method for obtaining a song start time, comprising:

[0007] Acquire a first song signal and a second song signal matching the first song signal; the first song signal is an audio signal of a first song whose start time is to be acquired, and the second song signal is a song signal of a second song that has a preset association relationship with the first song and is set with a start time;

[0008] Acquire a second sub-song signal having a start time of the second song and a time length of a preset length from the second song signal, and acquire a plurality of first sub-song signals having a time length of the preset length from the first song signal; each of the first sub-song signals corresponds to a different start time;

[0009] According to the similarity between each of the first sub-song signals and the second sub-song signals, a target first sub-song signal is obtained, and the start time of the target first sub-song signal is used as the start time of the first song.

[0010] In one embodiment, there are multiple second sub-song signals, and the time length of each second sub-song signal is a different preset length; the target first sub-song signal is obtained according to the similarity between each first sub-song signal and the second sub-song signal, including: obtaining multiple current first sub-song signals with the same time length as the current second sub-song signal, and obtaining the initial similarity between each current first sub-song signal and the current second sub-song signal; the current second sub-song signal is any one of the multiple second sub-song signals; according to each initial similarity corresponding to the current second sub-song signal and the time length of the current second sub-song signal, the target similarity corresponding to the current second sub-song signal is obtained; based on the target similarity corresponding to each current second sub-song signal, the target first sub-song signal is obtained.

[0011] In one of the embodiments, the target similarity degree corresponding to the current second sub-song signal is obtained based on the initial similarity degrees respectively corresponding to the current second sub-song signal and the time length of the current second sub-song signal, including: obtaining the maximum initial similarity degree corresponding to the current second sub-song signal from the initial similarity degrees respectively corresponding to the current second sub-song signal; obtaining the target similarity degree corresponding to the current second sub-song signal based on the time length of the current second sub-song signal and the maximum initial similarity degree corresponding to the current second sub-song signal.

[0012] In one of the embodiments, the target first sub-song signal is obtained based on the target similarity levels respectively corresponding to each of the current second sub-song signals, including: obtaining the target second sub-song signal according to the target similarity levels respectively corresponding to each of the current second sub-song signals; the target second sub-song signal is the current second sub-song signal corresponding to the maximum target similarity; and the current first sub-song signal having the maximum initial similarity to the target second sub-song signal among the multiple current first sub-song signals corresponding to the target second sub-song signal is used as the target first sub-song signal.

[0013] In one embodiment, the method of obtaining a second sub-song signal having a start time of the second song and a preset time length from the second song signal, and obtaining multiple first sub-song signals having a preset time length from the first song signal, includes: constructing a first time window and a second time window with a window length of the preset length, and setting the start position of the first time window at the start time of the first song signal, and setting the start position of the second time window at the target signal time of the second song signal; the target signal time is the start time of the second song; obtaining the second sub-song signal through the second time window; shifting the first time window multiple times according to a preset offset time until the end time of the shifted time window containing the first song signal, and obtaining multiple first sub-song signals through the first time window and each shifted time window; the time length of the preset offset time is less than the preset length.

[0014] In one embodiment, the obtaining of a first song signal and a second song signal matching the first song signal includes: performing Fourier transform processing on the first song and the second song respectively, and performing spectrum envelope extraction on the Fourier transform results of the first song and the second song to obtain a first song signal corresponding to the first song and a second song signal corresponding to the second song.

[0015] In one of the embodiments, the second song is pre-corresponded to a lyrics file; after taking the start time of the target first sub-song signal as the start time of the first song, the method further includes: intercepting a portion of the first song after the start time of the first song to obtain an intercepted song corresponding to the first song; according to the lyrics file of the second song, obtaining the lyrics text contained in the lyrics file; aligning the intercepted song with the lyrics text to generate a lyrics file corresponding to the first song.

[0016] In a second aspect, the present application also provides a device for obtaining a song start time, comprising:

[0017] A song signal acquisition module, used to acquire a first song signal and a second song signal matching the first song signal; the first song signal is an audio signal of a first song whose start time is to be acquired, and the second song signal is a song signal of a second song that has a preset association relationship with the first song and is set with a start time;

[0018] a sub-signal acquisition module, configured to acquire from the second song signal a second sub-song signal having a start time of the second song and a time length of a preset length, and to acquire from the first song signal a plurality of first sub-song signals having a time length of the preset length; each of the first sub-song signals corresponds to a different start time;

[0019] The start time acquisition module is used to acquire the target first sub-song signal according to the similarity between each of the first sub-song signals and the second sub-song signal, and use the start time of the target first sub-song signal as the start time of the first song.

[0020] In a third aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method described in any one of the embodiments of the first aspect when executing the computer program.

[0021] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method described in any one of the embodiments of the first aspect.

[0022] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the method described in any one of the embodiments of the first aspect.

[0023] The above-mentioned song start time acquisition method, device, computer equipment, computer-readable storage medium and computer program product obtain a first song signal of a first song whose start time is to be obtained, and a second song signal of a second song that has a preset association relationship with the first song and is set with a start time. Then, a second sub-song signal with a start time as the starting time and a time length of a preset length can be obtained from the second song signal, and multiple first sub-song signals with a preset time length are obtained from the first song signal, so as to use the similarity between the first sub-song signal and the second sub-song signal to identify the start time of the first song. In this way, automatic identification of the start time of the first song is achieved without the need for users to manually mark the start time. In this way, the efficiency of obtaining the song start time can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0025] Figure 1 A schematic diagram of a process for obtaining a song start time in one embodiment;

[0026] Figure 2 A schematic diagram of a process of obtaining a target first sub-song signal in one embodiment;

[0027] Figure 3 A schematic diagram of a process of obtaining a second sub-song signal and a first sub-song signal in one embodiment;

[0028] Figure 4 A schematic diagram of a process of generating lyrics timestamps by an automatic alignment algorithm in one embodiment;

[0029] Figure 5 A schematic diagram of a process for generating a new qrc file in an embodiment;

[0030] Figure 6 It is a structural block diagram of a device for obtaining the start time of a song in one embodiment;

[0031] Figure 7 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0033] In one embodiment, Figure 1 As shown, a method for obtaining the start time of a song is provided. This embodiment uses the method applied to a server as an example for illustration. It can be understood that the method can also be applied to a terminal, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0034] Step S101, obtaining a first song signal and a second song signal matching the first song signal; the first song signal is an audio signal of the first song whose start time is to be obtained, and the second song signal is a song signal of the second song that has a preset association with the first song and is set with a start time.

[0035] Among them, the first song refers to the song whose start time is to be obtained, that is, the song whose start time needs to be determined, and the first song signal refers to the audio signal of the first song, which can be the time domain signal of the first song or the frequency domain signal of the first song. The second song is a song that has a preset association with the first song and is set with a start time. For example, the first song and the second song are the same music, and the first song can be processed from the second song, and the processing method can be various types of audio post-production of the audio signal, such as adding an opening, changing the sound quality, changing the speed, performing sound correction or mixing, and the second song can be set with a start time. The second song signal refers to the audio signal of the second song, which can be the time domain signal of the second song or the frequency domain signal of the second song.

[0036] Specifically, after obtaining the first song whose start time needs to be obtained, the server can also obtain a second song that has a preset association with the first song and is set with a start time. Then, the server can extract signals from the first song and the second song to obtain the first song signal and the second song signal.

[0037] Step S102, obtaining a second sub-song signal with the start time of the second song as the start time and a time length of a preset length from the second song signal, and obtaining multiple first sub-song signals with a time length of a preset length from the first song signal; each first sub-song signal corresponds to a different start time.

[0038] The second sub-song signal is a song signal in the second song signal that starts at the start time and lasts for a preset length. For example, the preset length may be 3s. Then the second sub-song signal is a song signal in the second song signal that starts at the start time and lasts for 3s. The first sub-song signal is a song signal extracted from the first song signal that has multiple preset lengths. Taking the preset length of 3s as an example, the first sub-song signal may be a 0s-3s signal in the first song signal, or a 1s-4s signal, or a 2s-5s signal, etc. The start time of each first sub-song signal is different.

[0039] Specifically, the server can obtain a second sub-song signal from the second song signal with the start time of the second song as the start time and a preset length of time, and can also obtain multiple sub-song signals with different start times and a time length that is the same preset length as the second sub-song signal from the first song signal as multiple first sub-song signals.

[0040] Step S103, obtaining a target first sub-song signal according to the similarity between each first sub-song signal and the second sub-song signal, and using the start time of the target first sub-song signal as the start time of the first song.

[0041] The target first sub-song signal can be obtained by confirming the similarity between each first sub-song signal and the second sub-song signal. For example, the first sub-song signal with the greatest similarity can be used as the target first sub-song signal. Since the first song and the second song are songs with a preset association relationship, and the start time of the second sub-song signal is the start time, the target first sub-song signal can be considered to be the sub-song signal corresponding to the second sub-song signal in the first song signal. Therefore, after the server confirms the target first sub-song signal through the similarity, it can use the start time of the target first sub-song signal as the start time of the first song. Therefore, it can be considered that the start time of the target first sub-song signal is the start time of the first song.

[0042] Take the example that the first song is a song obtained by adding an opening to the second song. If the opening time is 1s, if 0s in the second song is the start time, and the preset time length is 3s, then the second sub-song signal can be the 0s-3s part of the second song signal, and the first sub-song signal can be the 0s-3s signal in the first song signal, or the 1s-4s signal or the 2s-5s signal, etc. Then the server can calculate the similarity between each first sub-song signal and the second sub-song signal, and determine the first sub-song signal with the greatest similarity as the target first sub-song signal. For example, at this time, the server can identify that the 1s-4s signal has the greatest similarity with the second sub-song signal, and then the 1s-4s signal can be used as the target first sub-song signal, so that 1s is used as the start time of the first song. In this way, even if the first sub-song signal carries a 1s opening, the server can accurately identify that the start time of the first song is 1s.

[0043] In the above-mentioned song start time acquisition method, a first song signal of the first song whose start time is to be acquired, and a second song signal of the second song which has a preset association relationship with the first song and is set with a start time are obtained. Then, a second sub-song signal with a start time as the starting time and a time length of a preset length can be obtained from the second song signal, and multiple first sub-song signals with a preset time length are acquired from the first song signal, so as to identify the start time of the first song by utilizing the similarity between the first sub-song signal and the second sub-song signal. In this way, automatic recognition of the start time of the first song is realized without the need for users to manually mark the start time. In this way, the efficiency of acquiring the song start time can be improved.

[0044] In one embodiment, there are multiple second sub-song signals, and the time lengths of the second sub-song signals are different preset lengths; Figure 2 As shown, step S103 may further include:

[0045] Step S201, obtain multiple current first sub-song signals with the same time length as the current second sub-song signal, and obtain the initial similarity between each current first sub-song signal and the current second sub-song signal; the current second sub-song signal is any one of the multiple second sub-song signals.

[0046] In this embodiment, the number of second sub-song signals can be multiple, and the starting time of each second sub-song signal is the same, which is the start time of the second song, and the signal length of each second sub-song signal is different. For example, if the start time of the second song is 0s, then the multiple second sub-song signals can be signals of the 0s-3s part of the second song signal, or signals of the 0s-3.5s part of the second song signal, or signals of the 0s-4s part of the second song signal.

[0047] Moreover, each second sub-song signal corresponds to multiple first sub-song signals with the same time length. For example, for a second sub-song signal with a signal length of 3s, it may correspond to a signal of 0s-3s in the first song signal, or a signal of 1s-4s or a signal of 2s-5s, etc.; and for a second sub-song signal with a signal length of 3.5s, it may correspond to a signal of 0s-3.5s in the first song signal, or a signal of 1s-4.5s or a signal of 2s-5.5s, etc.

[0048] The current second sub-song signal refers to any one of the multiple second sub-song signals, and the multiple current first sub-song signals are multiple first sub-song signals with the same time length as the current second sub-song signal. Taking the current second sub-song signal as a second sub-song signal with a signal length of 3s as an example, the multiple current first sub-song signals refer to the first sub-song signal corresponding to 0s-3s in the first song signal, the first sub-song signal corresponding to 1s-4s, and the first sub-song signal corresponding to 2s-5s, and so on.

[0049] The initial similarity refers to the directly calculated similarity between each current first sub-song signal and the current second sub-song signal. The similarity can be characterized by distance, for example, by the dynamic programming distance between song signals, or by the LCS (longest common subsequence) distance calculated by the spectral peak points of the song signals.

[0050] Specifically, after obtaining the current second sub-song signal and multiple current first sub-song signals corresponding to the current second sub-song signal, the server can respectively calculate the distance between each current first sub-song signal and the current second sub-song signal, thereby obtaining the initial similarity between each current first sub-song signal and the current second sub-song signal.

[0051] Step S202, obtaining a target similarity level corresponding to the current second sub-song signal according to each of the initial similarity levels corresponding to the current second sub-song signal and the time length of the current second sub-song signal;

[0052] Step S203, obtaining the target first sub-song signal based on the target similarity levels respectively corresponding to the current second sub-song signals.

[0053] The target similarity refers to the similarity further calculated based on the initial similarity and the time length of the second sub-song signal. Since there are multiple second sub-song signals in this embodiment, and they correspond to different time lengths, and generally speaking, the longer the time length of the second sub-song signal is, the more song content will be covered, and the audio change may be greater, so the calculated distance will be greater. If the time length of the second sub-song signal is smaller, the randomness is greater, and the calculated distance is shorter, but due to the greater randomness, the reliability of the result is lower, that is, the confidence is lower. Therefore, in this embodiment, the target similarity is set to comprehensively consider the relationship between the initial similarity and the time length of the current second sub-song signal, so as to further ensure the accuracy of the target first sub-song signal recognition.

[0054] Specifically, after obtaining the initial similarity between each current first sub-song signal and the current second sub-song signal, the server can use the above-mentioned initial similarities and the time length of the current second sub-song signal to obtain the target similarity of the current second sub-song signal, and thus use the target similarity of each current second sub-song signal to obtain the target first sub-song signal.

[0055] In this embodiment, multiple second sub-song signals can be obtained by setting different time lengths, and the time length of each second sub-song signal is used, combined with multiple initial similarities of each second sub-song signal, to obtain the target similarity corresponding to each second sub-song signal, so as to filter the first sub-song signal using the target similarity. In this way, the time length of the second sub-song signal and each initial similarity can be weighed, thereby improving the accuracy of identifying the target first sub-song signal.

[0056] Furthermore, step S202 may further include: obtaining the maximum initial similarity corresponding to the current second sub-song signal from the initial similarities corresponding to the current second sub-song signal; obtaining the target similarity corresponding to the current second sub-song signal based on the time length of the current second sub-song signal and the maximum initial similarity corresponding to the current second sub-song signal.

[0057] The maximum initial similarity refers to the maximum initial similarity among multiple initial similarities corresponding to the current second sub-song signal. In this embodiment, the server can calculate the target similarity corresponding to the current second sub-song signal based on the maximum initial similarity corresponding to the current second sub-song signal and the time length corresponding to the current second sub-song signal.

[0058] For example, the initial similarity can be represented by the distance between the current second sub-song signal and each current first sub-song signal, and the maximum initial similarity corresponding to the current second sub-song signal can be represented by the minimum value of the distance between the current second sub-song signal and each current first sub-song signal. ij To indicate that ij It can be expressed as the distance between the current second sub-song signal i and the current first sub-song signal j (assuming d ij is the minimum value of the distance between the previous second sub-song signal and each current first sub-song signal), then the target similarity corresponding to the current second sub-song signal can be calculated by the following calculation formula:

[0059]

[0060] Among them, d ij It is expressed as the minimum value of the distance between the current second sub-song signal and each current first sub-song signal (which can represent the maximum initial similarity corresponding to the current second sub-song signal), len_win represents the time length of the current second sub-song signal, and conf dis It indicates the target similarity degree corresponding to the current second sub-song signal. dis It can be obtained by comprehensive calculation based on the minimum value of the distance between the current second sub-song signal and each current first sub-song signal and the time length of the current second sub-song signal. This comprehensive calculation method satisfies the condition that the target similarity is greater when the time length is greater and the distance is smaller (i.e. the maximum initial similarity is greater).

[0061] In addition, the target similarity corresponding to the current second sub-song signal can also be obtained by calculating the square of the time length of the current second sub-song signal and the ratio of the minimum value of the distance between the current second sub-song signal and each current first sub-song signal, that is, the square value of len_win and d ij The ratio between them is taken as the target similarity.

[0062] In this embodiment, the target similarity corresponding to the current second sub-song signal can be calculated based on the time length of the current second sub-song signal and the maximum initial similarity corresponding to the current second sub-song signal, and the maximum initial similarity can be characterized by the minimum value of the distance between the current second sub-song signal and each current first sub-song signal, and the time length is positively correlated with the target similarity, while the minimum value of the distance between the current second sub-song signal and each current first sub-song signal is negatively correlated with the target similarity. In this way, the accuracy of the target similarity calculation can be improved.

[0063] In addition, step S203 may further include: obtaining a target second sub-song signal according to the target similarity levels corresponding to each current second sub-song signal; the target second sub-song signal is the current second sub-song signal corresponding to the maximum target similarity level; and taking the current first sub-song signal having the maximum initial similarity level with the target second sub-song signal among the multiple current first sub-song signals corresponding to the target second sub-song signal as the target first sub-song signal.

[0064] The target second sub-song signal is the current second sub-song signal with the maximum target similarity. After obtaining the target similarity of each current second sub-song signal, the server can filter out the current second sub-song signal with the maximum target similarity as the target second sub-song signal.

[0065] After the target second sub-song signal is determined, the current first sub-song signal with the greatest initial similarity can be screened out from the multiple current first sub-song signals corresponding to the target second sub-song signal, thereby obtaining the target first sub-song signal.

[0066] For example, the current second sub-song signal includes: second sub-song signal 1, second sub-song signal 2 and second sub-song signal 3, wherein the current first sub-song signal corresponding to second sub-song signal 1 includes first sub-song signal A1, first sub-song signal B1 and first sub-song signal C1, the current first sub-song signal corresponding to second sub-song signal 2 includes first sub-song signal A2, first sub-song signal B2 and first sub-song signal C2, and the current first sub-song signal corresponding to second sub-song signal 3 includes first sub-song signal A3, first sub-song signal B3 and first sub-song signal C3. After the server obtains the target similarity corresponding to each current second sub-song signal, it can filter out the current second sub-song signal with the maximum target similarity. If the target similarity corresponding to second sub-song signal 1 is the largest, the server can use second sub-song signal 1 as the target second sub-song signal. Afterwards, the first sub-song signal with the greatest initial similarity to the second sub-song signal 1 can be determined from the first sub-song signal A1, the first sub-song signal B1 and the first sub-song signal C1. If the first sub-song signal A1 has the greatest initial similarity to the second sub-song signal 1, then the first sub-song signal A1 can be used as the target first sub-song signal.

[0067] In this embodiment, the server can determine the target second sub-song signal with the greatest target similarity from the current second sub-song signal, and then determine the current first sub-song signal with the greatest initial similarity from the multiple current first sub-song signals corresponding to the target second sub-song signal as the target first sub-song signal. In this way, it can be ensured that the target first sub-song signal takes into account the relationship between the initial similarity and the time length of the current second sub-song signal, thereby further improving the accuracy of the screening of the target first sub-song signal.

[0068] In one embodiment, Figure 3 As shown, step S102 may further include:

[0069] Step S301, constructing a first time window and a second time window with a window length of a preset length, and setting the starting position of the first time window at the starting moment of the first song signal, and setting the starting position of the second time window at the target signal moment of the second song signal; the target signal moment is the start time of the second song.

[0070] Among them, the starting time of the first song signal refers to the signal starting time of the first song signal, and the target signal time of the second song signal refers to the signal time corresponding to the start time of the second song. Specifically, after obtaining the first song signal and the second song signal, the server can first construct two time windows with a preset window length, as the first time window and the second time window. Afterwards, the starting position of the first time window can be set at the starting time of the first song signal, and the starting position of the second time window can be set at the target signal time corresponding to the start time of the second song in the second song signal.

[0071] Step S302, obtaining a second sub-song signal through a second time window.

[0072] After obtaining the second time window, the server can use the second song signal contained in the second time window as the second sub-song signal. For example, if it is necessary to intercept a second sub-song signal with a time length of 3 seconds, the server can first construct a second time window with a window length of 3 seconds, and set the starting time of the time window to the target signal time in the second song signal, then the sub-song signal contained in the second time window can be a second sub-song signal with a time length of 3 seconds and a start time of the start time of the second song.

[0073] Step S303, shift the first time window multiple times according to the preset offset time until the end moment of the first song signal in the shifted time window, and obtain multiple first sub-song signals through the first time window and each shifted time window; the time length of the preset offset time is less than the preset length.

[0074] The preset offset time refers to the time for the first time window to be shifted. The time length of the preset offset time must be smaller than the window length of the first time window, that is, the preset length. After obtaining the first time window, the server can collect the first song signal contained in the first time window, and can also shift the first time window multiple times according to the preset offset time until the end time of the first song signal is included in the shifted time window, thereby utilizing the first time window and each shifted time window to obtain multiple first sub-song signals. For example, the first song signals respectively contained in the first time window and each shifted time window can be used as multiple first sub-song signals.

[0075] For example, the window length of the first time window is 3s, and the preset offset time can be 0.05s. After obtaining the first time window, the server can first set the starting position of the first time window to the starting time of the first song signal. Then the first song signal contained in the first time window can be the first song signal of 0s-3s. After that, the server can shift the first time window multiple times. The first song signal contained in the time window obtained by the first shift can be the first song signal of 0.05s-3.05s, and the first song signal contained in the time window obtained by the second shift can be the first song signal of 0.1s-3.1s, and so on, until the time window is shifted to the end time of the first song signal. In this way, multiple first song signals with a time length of 3s can be obtained.

[0076] Afterwards, if the number of second sub-song signals needs to be increased, for example, second sub-song signals corresponding to different time lengths need to be collected, the first time window and the second time window with different time lengths can be constructed again, and the above process can be repeated. For example, the first time window and the second time window with a window length of 3.5s can be constructed, and the above process can be repeated to obtain a second sub-song signal with a time length of 3.5s, and multiple first sub-song signals with a time length of 3.5s.

[0077] In this embodiment, the server can construct a first time window and a second time window with a preset window length, thereby obtaining a second sub-song signal through the second time window, and obtain multiple first sub-song signals by translating the first time window. This method can improve the efficiency of obtaining the first sub-song signal and the second sub-song signal.

[0078] In one embodiment, step S101 may further include: in response to a song replacement request, obtaining the song to be replaced and the song after replacement, and using the song after replacement as the first song, and using the song to be replaced as the second song; performing Fourier transform processing on the first song and the second song respectively, and performing spectrum envelope extraction on the Fourier transform results of the first song and the second song to obtain a first song signal corresponding to the first song, and a second song signal corresponding to the second song.

[0079] A song replacement request refers to a request for replacing the song to be replaced with the song after replacement. In this embodiment, the song start time acquisition method can be applied to the song replacement scenario, wherein the song before replacement is the song to be replaced, and the old song of the song to be replaced can be used as the second song, that is, a song with a pre-set start time, and the new song after replacement can be used as the first song, that is, the song for which the start time needs to be obtained.

[0080] For example, the second song may be version A of a certain song, and the first song may correspond to version B of the song, where version B of the song may be a song that adds an opening statement based on version A. Therefore, when updating the song version, a song replacement request may be initiated to update the song from version A to version B.

[0081] Afterwards, the server can perform Fourier transform processing on the first song and the second song respectively, and perform spectrum envelope extraction on the Fourier transform results of the first song and the second song, so as to obtain a first song signal corresponding to the first song and a second song signal corresponding to the second song.

[0082] In this embodiment, the method for obtaining the song start time can be applied in the scenario of song replacement. By taking the song to be replaced as the second song and the replaced song as the first song, the start time of the song to be replaced can be used to obtain the start time of the replaced song. In this way, the efficiency of obtaining the start time of the replaced song in the song replacement scenario can be improved.

[0083] Furthermore, the second song has a corresponding lyrics file in advance; after step S103, it can also include: intercepting a part of the first song after the start time of the first song to obtain the intercepted song corresponding to the first song; according to the lyrics file of the second song, obtaining the lyrics text contained in the lyrics file; aligning the intercepted song with the lyrics text to generate a lyrics file corresponding to the first song.

[0084] The lyrics file may refer to a qrc lyrics file, which is used to implement the scrolling lyrics function during the playback of the second song. Since the second song is the song before the song is replaced, it has a corresponding lyrics file in advance. After the server of this embodiment obtains the start time of the first song after the replacement, it can obtain the lyrics file of the first song based on the start time.

[0085] Specifically, after the server determines the start time of the first song, it can also intercept the part of the song that is located after the start time of the first song to obtain the corresponding intercepted song. In this way, the opening remarks that may be added when the song is replaced with the first song can be removed. At this time, it can be ensured that the audio content of the intercepted song is consistent with the lyrics text content. Afterwards, the lyrics file of the second song can be used to extract the lyrics text contained in the lyrics file, and the intercepted song and the lyrics text are aligned to generate the lyrics file corresponding to the first song.

[0086] In this embodiment, after obtaining the start time of the first song, the first song can be truncated, so that the lyrics text contained in the lyrics file of the second song can be used to align with the truncated first song to generate the lyrics file of the first song. In this way, the manual labeling process can be omitted while ensuring the accuracy of the lyrics file of the first song, thereby improving the efficiency of constructing the lyrics file of the first song.

[0087] In one embodiment, a song start point detection method based on the local shortest dynamic programming distance is also provided, which can be applied in the application scenario of updating the qrc lyrics after the song is replaced. The generation process of the current qrc lyrics file can be as follows: Figure 4 As shown, the following steps may be included:

[0088] 1. Extract acoustic features from song audio;

[0089] 2. Convert the lyrics text into a phoneme sequence. Here, a text-to-phoneme conversion dictionary is needed. For Chinese, pinyin can be used, and for English, phonetic symbols, international phonetic symbols, etc. can be used. After obtaining the phoneme sequence, convert the phoneme sequence into a state sequence of a hidden Markov model. Generally speaking, one phoneme corresponds to three states, which can be self-transferred or transferred to other states. The specific transition probability is described by the hidden Markov model (HMM). The state is the smallest granularity of a word, and one frame of audio features corresponds to one state.

[0090] 3. Input each frame feature into a Gaussian mixture model (GMM) or a deep neural network (DNN) to obtain the probability that each frame feature belongs to each state of the hidden Markov model (HMM). Combined with the transition probability, the path with the highest global probability is obtained through Viterbi decoding. The frames and states are aligned one by one to obtain the decoding result.

[0091] 4. Convert the state into phonemes, and then convert the phonemes into words to get the lyrics timestamp.

[0092] The above is the process of the automatic alignment algorithm. Automatic alignment needs to ensure that the audio content and the text content are consistent. However, if they are inconsistent, such as adding an additional opening statement at the beginning, even if Viterbi decoding can find the maximum probability path, such a path is untrustworthy and the decoding result is unreliable.

[0093] Therefore, in this embodiment, a qrc lyrics update method applicable to both the case of not adding opening remarks and adding opening narration is provided. The implementation of this process can be as follows: Figure 5 As shown, audio replacement can effectively and quickly utilize the algorithm to update qrc without human intervention.

[0094] Among them, the implementation of this process depends on the song start point detection method based on the local shortest dynamic programming distance. First, the old song audio and the new song audio are Fourier transformed respectively, and the spectrum envelope is extracted. Then, a sliding window is set to detect the envelope from the new audio time point 0, and the old audio uses an equal-length window to intercept the envelope, and the dynamic programming distance between the two envelopes is counted and recorded. Adjust the window length and repeat the above steps again. After the above steps, different window lengths and multiple groups of distances can be obtained. Since the longer the window length, the more song content is covered, the greater the possible changes between the new and old songs, the shorter the distance, and the shorter the window length, the smaller the difference between the two, and the lower the confidence. This embodiment calculates the ratio of the two to find the monitoring point with the highest confidence and obtains the song start point. After determining the start point, the audio after the start point is intercepted and the original lyrics are automatically aligned to obtain a new qrc.

[0095] The process can be specifically implemented through the following steps:

[0096] 1. Perform Fourier transform on the new and old songs respectively, extract the spectrum envelope, and set the sliding window.

[0097] Assume that the window length is 3 seconds, the sliding window of the old envelope is fixed at the starting point, and calculate the shortest dynamic programming distance between the old envelope and the new envelope, denoted as d 11 , move the sliding window of the new envelope, move 50ms each time, calculate the distance again, and repeat this process until it reaches the end of the new envelope, which is recorded as d 12 , …, d 1n After one round of calculation is completed, the length of the sliding window is changed. Assuming that it is increased by 0.5 seconds each time, the length of the sliding window for the second round of calculation is 3.5 seconds. Calculate again in the above way to get d 21 , …, d 2n By changing the length of the sliding window multiple times and continuously moving the sliding window of the new envelope, multiple sets of distances can be obtained. ij represents the shortest dynamic programming distance between the old envelope and the jth new envelope within the i-th window length.

[0098] 2. The longer the window length is, the more songs are covered, and the greater the change between the old and new audios may be, so the distance is larger. If the window length is too short, the randomness is greater, and the calculated distance is short but the confidence is low. Therefore, this embodiment defines an indicator that weighs the window length and the shortest dynamic programming distance, named confidence distance, and the calculation method is as follows:

[0099]

[0100] Where len_win represents the length of the window, d ijIt represents the shortest dynamic programming distance between the old envelope and the jth new envelope within the i-th window length. Traverse all values ​​of i and j to find the maximum confidence distance, and the corresponding window start position is the starting point.

[0101] 3. After getting the start point of the song, intercept the audio after the start point of the song, and then remove the opening part to make the audio and lyrics consistent. Extract the lyrics and the intercepted audio from the old qrc, and then use the automatic alignment algorithm in the existing solution to force alignment, and output the new qrc to replace the old qrc.

[0102] Through this embodiment, the spectral envelope of the new and old audio is used to calculate the local shortest dynamic programming distance. At the same time, different window lengths are combined and the confidence distance is calculated to find the best vocal starting point of the new audio, and then the audio is intercepted and automatically aligned, so that the opening remarks will not affect the alignment algorithm, and the process of automatic replacement of qrc is realized, which greatly reduces manpower and thereby improves the efficiency of lyrics replacement.

[0103] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0104] Based on the same inventive concept, the embodiment of the present application also provides a song start time acquisition device for implementing the song start time acquisition method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more song start time acquisition device embodiments provided below can refer to the limitations of the song start time acquisition method above, and will not be repeated here.

[0105] In one embodiment, Figure 6 As shown, a device for obtaining the start time of a song is provided, comprising: a song signal obtaining module 601, a sub-signal obtaining module 602 and a song start time obtaining module 603, wherein:

[0106] The song signal acquisition module 601 is used to acquire a first song signal and a second song signal matching the first song signal; the first song signal is an audio signal of the first song whose start time is to be acquired, and the second song signal is a song signal of the second song which has a preset association relationship with the first song and is set with a start time;

[0107] The sub-signal acquisition module 602 is used to acquire a second sub-song signal having a start time of the second song and a time length of a preset length from the second song signal, and acquire a plurality of first sub-song signals having a time length of a preset length from the first song signal; each first sub-song signal corresponds to a different start time;

[0108] The start time acquisition module 603 is used to acquire the target first sub-song signal according to the similarity between each first sub-song signal and the second sub-song signal, and use the start time of the target first sub-song signal as the start time of the first song.

[0109] Each module in the above-mentioned song start time acquisition device can be implemented in whole or in part by software, hardware and their combination. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0110] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 7 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store song signal data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for obtaining the start time of a song is implemented.

[0111] Those skilled in the art will understand that Figure 7The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0112] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0113] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0114] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0115] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0116] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0117] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0118] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for obtaining the start time of a song, characterized in that: The method comprises: Acquire a first song signal and a second song signal matching the first song signal; the first song signal is an audio signal of a first song whose start time is to be acquired, and the second song signal is a song signal of a second song that has a preset association relationship with the first song and is set with a start time; Acquire a second sub-song signal having a start time of the second song and a time length of a preset length from the second song signal, and acquire a plurality of first sub-song signals having a time length of the preset length from the first song signal; each of the first sub-song signals corresponds to a different start time; According to the similarity between each of the first sub-song signals and the second sub-song signals, a target first sub-song signal is obtained, and the start time of the target first sub-song signal is used as the start time of the first song.

2. The method according to claim 1, characterized in that There are multiple second sub-song signals, and the time lengths of the second sub-song signals are different preset lengths; The step of acquiring a target first sub-song signal according to the similarity between each of the first sub-song signals and the second sub-song signal comprises: Acquire multiple current first sub-song signals having the same time length as the current second sub-song signal, and acquire the initial similarity between each of the current first sub-song signals and the current second sub-song signal; The current second sub-song signal is any one of a plurality of second sub-song signals; According to the initial similarity levels respectively corresponding to the current second sub-song signal and the time length of the current second sub-song signal, obtaining the target similarity level corresponding to the current second sub-song signal; Based on the target similarity levels respectively corresponding to the current second sub-song signals, the target first sub-song signal is obtained.

3. The method according to claim 2, characterized in that The step of obtaining the target similarity level corresponding to the current second sub-song signal according to the initial similarity levels respectively corresponding to the current second sub-song signal and the time length of the current second sub-song signal comprises: From the initial similarity levels respectively corresponding to the current second sub-song signal, obtaining the maximum initial similarity level corresponding to the current second sub-song signal; According to the time length of the current second sub-song signal and the maximum initial similarity corresponding to the current second sub-song signal, the target similarity corresponding to the current second sub-song signal is obtained.

4. The method according to claim 2, characterized in that The step of acquiring the target first sub-song signal based on the target similarity levels respectively corresponding to the current second sub-song signals comprises: According to the target similarity levels respectively corresponding to the current second sub-song signals, a target second sub-song signal is obtained; the target second sub-song signal is the current second sub-song signal corresponding to the maximum target similarity level; Among the multiple current first sub-song signals corresponding to the target second sub-song signal, the current first sub-song signal having the greatest initial similarity to the target second sub-song signal is used as the target first sub-song signal.

5. The method according to claim 1, characterized in that: The method of obtaining a second sub-song signal having a start time of the second song and a time length of a preset length from the second song signal, and obtaining a plurality of first sub-song signals having a time length of the preset length from the first song signal, comprises: Constructing a first time window and a second time window with a window length of the preset length, and setting the starting position of the first time window at the starting moment of the first song signal, and setting the starting position of the second time window at the target signal moment of the second song signal; the target signal moment is the start moment of the second song; Obtaining the second sub-song signal through the second time window; The first time window is shifted multiple times according to a preset offset time until the end moment of the first song signal in the shifted time window, and multiple first sub-song signals are obtained through the first time window and each shifted time window; the time length of the preset offset time is less than the preset length.

6. The method according to claim 1, characterized in that The step of acquiring a first song signal and a second song signal matching the first song signal includes: The first song and the second song are respectively subjected to Fourier transform processing, and spectrum envelope extraction is performed on the Fourier transform results of the first song and the second song to obtain a first song signal corresponding to the first song and a second song signal corresponding to the second song.

7. The method according to claim 1, characterized in that The second song is pre-associated with a lyrics file; after the start time of the target first sub-song signal is used as the start time of the first song, the method further includes: intercepting a portion of the first song after the start time of the first song to obtain an intercepted song corresponding to the first song; According to the lyrics file of the second song, obtaining the lyrics text contained in the lyrics file; The intercepted song is aligned with the lyrics text to generate a lyrics file corresponding to the first song.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images