Audio replacement method and terminal

Abstract

The present invention discloses an audio replacement method and terminal, including the steps of: obtaining a first lossy audio and a first lossless audio; calculating and obtaining a linear regression function between the first lossy audio and the first lossless audio based on a DTW algorithm; calculating and obtaining a loss function value according to the linear regression function; judging whether the versions of the first lossy audio and the first lossless audio are the same according to the loss function value, and if the judgment result is that the versions are the same, replacing the first lossy audio with the first lossless audio according to the linear regression function. Since the present invention does not require manual matching, it can effectively reduce the labor cost of matching lossy audio and lossless audio and improve the speed of matching alignment; at the same time, through the DTW algorithm, the error caused by the time axis discrepancy when calculating the corresponding relationship between the two audio sequences is reduced, thereby improving the accuracy of audio matching, that is, providing a low-cost, fast and high-precision audio replacement method.

Description

Technical Field

[0001] The present invention relates to the technical field of audio processing, and in particular to an audio replacement method and a terminal. Background Art

[0002] In the audio resources of the existing music library, due to reasons such as collection, transmission, storage, and compression, the audio resources of the existing music library are all subjected to lossy compression to varying degrees in order to reduce the file size, thereby increasing the transmission speed and saving storage space. With the development of current communication technology, the upgrade of audio equipment, and the growing demand of consumers for lossless audio, it is necessary to upgrade the lossy audio in the existing music library to lossless audio.

[0003] However, song files include video tracks and audio tracks. In order to replace lossy audio tracks with lossless audio tracks, it is necessary to ensure that the audio and video of the song files are synchronized and match the subtitle files. Since the sources of lossy audio and lossless audio are not necessarily the same, there are two problems:

[0004] 1. The lossy audio is the same as the lossless audio version, but there is a time difference between the audio content before and after, which is just the difference in displacement on the time axis;

[0005] 2. The lossy audio and lossless audio versions are different (such as the concert version and the MV version), different sentences have different speeds, and the audio on the timeline has speed expansion and contraction.

[0006] Therefore, traditional audio matching solutions require manual browsing of audio files to determine whether the versions are consistent and whether the time points are synchronized. However, manual matching has the problem of long matching time and certain human errors in time point synchronization judgment. At the same time, there are subtle differences in the playback speeds of different audios, which are difficult to detect manually, but there will be large errors when matching audio with subtitles and rating files. Summary of the invention

[0007] The technical problem to be solved by the present invention is to provide an audio replacement method and a terminal, which can realize automatic replacement of lossy audio and lossless audio by automatically matching lossy audio and lossless audio of the same version.

[0008] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0009] An audio replacement method comprises the steps of:

[0010] Obtaining a first lossy audio and a first lossless audio;

[0011] Calculate and obtain a linear regression function between the first lossy audio and the first lossless audio based on the DTW algorithm;

[0012] Calculate and obtain a loss function value according to the linear regression function;

[0013] Determine whether the versions of the first lossy audio and the first lossless audio are the same according to the loss function value; if the judgment result is that the versions are the same, replace the first lossy audio with the first lossless audio according to the linear regression function.

[0014] In order to solve the above technical problems, another technical solution adopted by the present invention is:

[0015] An audio replacement terminal comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, each step of the above-mentioned audio replacement method is implemented.

[0016] The beneficial effects of the present invention are: an audio replacement method and terminal, which calculates and obtains the linear regression function between lossy audio and lossless audio through the DTW algorithm, calculates and obtains the loss function value based on the linear regression function, judges whether the versions of lossy audio and lossless audio are the same based on the loss function value, and when the judgment result is that the versions are the same, replaces the first lossy audio with the first lossless audio according to the linear regression function to achieve automatic replacement of lossy audio and lossless audio. Therefore, since there is no need for manual matching, the labor cost of matching lossy audio and lossless audio can be effectively reduced, and the speed of matching alignment can be improved; at the same time, the DTW algorithm is used to reduce the error caused by the time axis discrepancy when calculating the corresponding relationship between the two audio sequences, thereby improving the accuracy of audio matching, that is, a low-cost, fast and high-precision audio replacement method is provided. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A schematic diagram of a flow chart of an audio replacement method according to an embodiment of the present invention;

[0018] Figure 2 A time axis comparison connection diagram of lossy audio and lossless audio involved in an embodiment of the present invention;

[0019] Figure 3 for Figure 2 Similar path sequence diagram for lossy and lossless audio in ;

[0020] Figure 4 Another timeline comparison connection diagram of lossy audio and lossless audio involved in an embodiment of the present invention;

[0021] Figure 5 for Figure 4 Similar path sequence diagram for lossy and lossless audio in ;

[0022] Figure 6The present invention is a schematic diagram of the structure of an audio replacement terminal according to an embodiment of the present invention.

[0023] Description of labels:

[0024] 1. An audio replacement terminal; 2. A processor; 3. A memory. DETAILED DESCRIPTION

[0025] In order to explain the technical content, achieved objectives and effects of the present invention in detail, the following is an explanation in conjunction with the implementation modes and the accompanying drawings.

[0026] Please refer to Figures 1 to 5 , an audio replacement method, comprising the steps of:

[0027] Obtaining a first lossy audio and a first lossless audio;

[0028] Calculate and obtain a linear regression function between the first lossy audio and the first lossless audio based on the DTW algorithm;

[0029] Calculate and obtain a loss function value according to the linear regression function;

[0030] Determine whether the versions of the first lossy audio and the first lossless audio are the same according to the loss function value; if the judgment result is that the versions are the same, replace the first lossy audio with the first lossless audio according to the linear regression function.

[0031] From the above description, it can be seen that the beneficial effects of the present invention are: calculating and obtaining the linear regression function between the lossy audio and the lossless audio through the DTW algorithm, calculating and obtaining the loss function value based on the linear regression function, judging whether the versions of the lossy audio and the lossless audio are the same based on the loss function value, and when the judgment result is that the versions are the same, replacing the first lossy audio with the first lossless audio according to the linear regression function to achieve automatic replacement of the lossy audio and the lossless audio. Therefore, since there is no need for manual matching, the labor cost of matching the lossy audio and the lossless audio can be effectively reduced, and the speed of matching alignment can be improved; at the same time, through the DTW algorithm, the error caused by the time axis discrepancy when calculating the corresponding relationship between the two audio sequences can be reduced, thereby improving the accuracy of audio matching, that is, providing a low-cost, fast and high-precision audio replacement method.

[0032] Furthermore, the step of calculating and obtaining the loss function value according to the linear regression function specifically includes the following steps:

[0033] The square of the difference between the actual ordinate value formed by the first lossy audio and the first lossless audio at the same time point and the theoretical ordinate value obtained according to the linear regression function is accumulated to obtain the loss function value.

[0034] From the above description, it can be seen that the loss function value is obtained by taking the square of the difference between the actual ordinate values ​​of the lossy audio and the lossless audio and the theoretical ordinate values ​​obtained according to the linear regression function, so as to simultaneously determine whether the versions of the lossy audio and the lossless audio are the same and whether there is a scaling difference in speed based on the obtained loss function value, so as to facilitate subsequent audio correction.

[0035] Further, judging whether the versions of the first lossy audio and the first lossless audio are the same according to the loss function value specifically includes the following steps:

[0036] Determine whether the loss function value is greater than a loss threshold; if so, the judgment result of the first lossy audio and the first lossless audio is that the versions are different; otherwise, the judgment result of the first lossy audio and the first lossless audio is that the versions are the same.

[0037] From the above description, we can know that if the loss function value is less than the loss threshold, it means that there is a certain linear relationship between the two audios. Therefore, the lossy audio and the lossless audio are judged to be the same version, and there is only an offset on the time axis; if the loss function value is greater than the loss threshold, it means that the lossy audio and the lossless audio are not only offset on the time axis, but also have a scaling difference in speed, so the two audio versions are not the same. Therefore, by setting the loss threshold, it is possible to effectively and quickly determine whether the two audio versions are the same.

[0038] Further, the replacing the first lossy audio with the first lossless audio according to the linear regression function specifically includes the following steps:

[0039] Using a constant in the linear regression function as an offset time between the first lossy audio and the first lossless audio on the time axis;

[0040] The first lossy audio is replaced by the first lossless audio after the offset time is corrected.

[0041] From the above description, it can be seen that the offset time of the lossy audio and lossless audio of the same version on the time axis is obtained through the constant in the linear regression function, so that the lossy audio and lossless audio are replaced after correcting the offset time to ensure the synchronization of sound and picture of the song file and the synchronization matching of the subtitle file.

[0042] Further, when the time of the first lossless audio is used as the ordinate of the linear regression function and the time of the first lossy audio is used as the abscissa of the linear regression function, the correction of the first lossless audio specifically includes the following steps:

[0043] If the offset time is greater than 0, the first lossless audio is supplemented with a blank segment with a duration equal to the offset time. If the offset time is equal to 0, the first lossless audio is retained. If the offset time is less than 0, the beginning of the first lossless audio is cut off with a segment with a duration equal to the offset time.

[0044] From the above description, it can be seen that the beginning of the lossless audio is cut or a blank segment is added according to the offset time of the audio to correct the offset time between the lossy audio and the lossless audio on the timeline.

[0045] Furthermore, after obtaining the first lossy audio and the first lossless audio, the method further includes the following steps:

[0046] Unifying the sampling rates of the first lossy audio and the first lossless audio;

[0047] Extracting the first lossy audio and the first lossless audio;

[0048] The step of calculating and obtaining a linear regression function between the first lossy audio and the first lossless audio based on the DTW algorithm specifically includes the following steps:

[0049] A linear regression function between the first lossy audio and the first lossless audio is obtained based on a DTW algorithm and according to the human voice audio of the first lossy audio and the first lossless audio.

[0050] From the above description, it can be seen that by preprocessing the sampling rates of the first lossy audio and the first lossless audio to be judged to be unified and the vocal audio extraction to avoid the interference of audio quality and audio background music on audio matching alignment, the matching accuracy of the lossy audio and the lossless audio is guaranteed.

[0051] Furthermore, after extracting the first lossy audio and the first lossless audio from the human voice audio, the method further includes the following steps:

[0052] Extracting chroma features of the first lossy audio and the first lossless audio;

[0053] The step of calculating and obtaining a linear regression function between the first lossy audio and the first lossless audio based on the DTW algorithm specifically includes the following steps:

[0054] A linear regression function between the first lossy audio and the first lossless audio is obtained by calculating based on the DTW algorithm and according to the chromaticity features of the human voice audio of the first lossy audio and the first lossless audio.

[0055] From the above description, it can be seen that since the chromaticity feature is a general term for the chromaticity vector and the chromaticity spectrum, it only considers the frequency of the music. Therefore, using the chromaticity features of the human voice audio of the two audios to calculate the linear regression function of the two has the advantages of low computational difficulty and fast computational speed.

[0056] Furthermore, the calculating based on the DTW algorithm and obtaining the linear regression function between the first lossy audio and the first lossless audio specifically includes the following steps:

[0057] A similar path sequence between the first lossy audio and the first lossless audio is calculated and obtained based on a DTW algorithm, and a linear regression function between the first lossy audio and the first lossless audio is calculated and obtained according to the similar path sequence.

[0058] From the above description, we can see that the DTW dynamic time warping algorithm can be used to measure the similarity of two audio sequences. Even if there are differences in time and speed, the optimal correspondence between the two audio sequences can be found in the end to reduce the error caused by the difference in time axes when calculating the correspondence between the two audio sequences.

[0059] Furthermore, the calculating and obtaining the linear regression function between the first lossy audio and the first lossless audio according to the similar path sequence specifically includes the following steps:

[0060] Points whose horizontal axis or vertical axis is 0 in the similar path sequence are discarded, and a linear regression function between the first lossy audio and the first lossless audio is calculated and obtained according to other points in the similar path sequence except the discarded points.

[0061] From the above description, it can be seen that the accuracy of the linear regression function is ensured by discarding the points where the horizontal axis or the vertical axis in the similar path sequence is 0 and then calculating the linear regression function.

[0062] Please refer to Figure 6 , an audio replacement terminal includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements each step of the above-mentioned audio replacement method when executing the computer program.

[0063] The above-mentioned audio replacement method and terminal of the present invention can be applied to various music libraries that need to replace lossy audio with corresponding lossless audio scenes, such as KTV (Karaoke TV, a place that provides karaoke audio-visual equipment and sight-singing space), bars or external wall display screens, etc., which need to display song files. The following is an explanation through specific implementation methods:

[0064] Please refer to Figures 1 to 5 , Embodiment 1 of the present invention is:

[0065] An audio replacement method comprises the steps of:

[0066] S1. Obtain a first lossy audio and a first lossless audio;

[0067] Among them, for the lossy audio to be replaced, one by one matching is performed in a given lossless audio database to obtain the same lossless audio version as the lossy audio. In this embodiment, it is assumed that the lossy audio to be replaced is the first lossy audio, and the lossless audio currently undergoing matching judgment is the first lossless audio. After the files of the first lossy audio and the first lossless audio are obtained, in order to avoid interference of audio quality and audio background music on audio matching alignment, the first lossy audio a and the first lossless audio b are preprocessed:

[0068] S11, unifying the sampling rates of the first lossy audio and the first lossless audio;

[0069] In this embodiment, the sampling rates of the first lossy audio a and the first lossless audio b are unified to 44.1 kHz.

[0070] S12, extracting the first lossy audio and the first lossless audio;

[0071] In this embodiment, the spleeter open source library is used to extract the vocal audios a1 and b1 of two audio segments a and b.

[0072] S13, extracting chromaticity features of the human voice audio of the first lossy audio and the first lossless audio;

[0073] In this embodiment, the chroma_stft function of librosa is used to extract the chroma features of a1 and b1, namely, the chroma features, which are C_a1 and C_b1 respectively. The chroma feature is a general term for the chroma vector and the chroma spectrum. The advantage of using this feature is that it only considers the frequency of music, and the calculation difficulty is low and the speed is fast.

[0074] S2. Calculate and obtain a linear regression function between the first lossy audio and the first lossless audio based on the DTW algorithm;

[0075] In this embodiment, based on step S13, in this embodiment, a linear regression function between the first lossy audio and the first lossless audio is calculated according to the chromaticity characteristics of the human voice audio of the first lossy audio and the first lossless audio, specifically:

[0076] S21. Calculate and obtain a similar path sequence between the first lossy audio and the first lossless audio based on a DTW algorithm;

[0077] like Figure 2This is a comparison diagram of the lossy audio and lossless audio of the same version on the timeline. Figure 3 Based on Figure 2 The obtained similar path sequence is Figure 3 In the figure, the vertical axis corresponds to the time point of high-quality lossless audio, and the horizontal axis corresponds to the time point of lossy audio of the original video. Figure 3 In this red line, the distance between the chrominance vectors is very small. This is the correspondence between the same places in the two audio segments, that is, the similar path sequence.

[0078] And as Figure 4 This is a comparison diagram of different versions of lossy audio and lossless audio on the timeline. Figure 5 Based on Figure 4 The obtained similar path sequence has different versions, so the sentence speed of lossy audio and lossless audio is different, and the audio on the timeline also has speed scaling.

[0079] S22: discard the points whose horizontal axis or vertical axis is 0 in the similar path sequence, and calculate and obtain a linear regression function between the first lossy audio and the first lossless audio according to other points in the similar path sequence except the discarded points.

[0080] That is, discard the points where x=0 or y=0 in the similar path sequence, and calculate the linear regression function based on the similar path function

[0081] S3, calculate and obtain the loss function value according to the linear regression function;

[0082] In this embodiment, step S3 specifically includes the following steps:

[0083] The square of the difference between the actual ordinate value formed by the first lossy audio and the first lossless audio at the same time point and the theoretical ordinate value obtained according to the linear regression function is accumulated to obtain the loss function value.

[0084] Corresponding to the above linear regression function Then the loss function value Where y is the actual ordinate value, is the theoretical ordinate value.

[0085] S4. Determine whether the versions of the first lossy audio and the first lossless audio are the same according to the loss function value. If the judgment result is that the versions are the same, replace the first lossy audio with the first lossless audio according to the linear regression function.

[0086] In this embodiment, step S4 specifically includes:

[0087] S41. Determine whether the loss function value is greater than the loss threshold. If so, the judgment result of the first lossy audio and the first lossless audio is that the versions are different. Otherwise, the judgment result of the first lossy audio and the first lossless audio is that the versions are the same.

[0088] In this embodiment, the loss threshold is 1.2, and in other equivalent embodiments, it can be set accordingly according to the specific audio length or the accuracy requirement. Figure 3 and Figure 5 In these two cases, the following two judgment results can be obtained:

[0089] (1) When corresponding Figure 5 When the value of the Loss function exceeds the threshold of 1.2, it is determined that the lossy audio is different from the lossless audio version. Not only is there a time offset, but the speed is also different. In this case, the lossless audio cannot replace the lossy audio.

[0090] (2) When corresponding Figure 3 When the value of the Loss function is less than 1.2, it is determined that the lossy audio is the same as the lossless audio version, and there is only a time offset. In this case, the lossless audio can replace the lossy audio;

[0091] S42, using a constant in the linear regression function as an offset time between the first lossy audio and the first lossless audio on the time axis;

[0092] The above linear regression function The constant β is used as the offset time between the first lossy audio and the first lossless audio on the time axis.

[0093] S43: Replace the first lossy audio with the first lossless audio after the offset time is corrected.

[0094] Among them, the correction of the first lossless audio according to different values ​​of the constant β is divided into the following three cases:

[0095] (1) If the offset time β is greater than 0, the first lossless audio is supplemented with a blank segment with a duration equal to the offset time β.

[0096] (2) If the offset time β is equal to 0, the first lossless audio is maintained.

[0097] (3) If the offset time β is less than 0, a segment with a duration equal to the offset time β is cut from the beginning of the first lossless audio.

[0098] In this way, the first lossless audio after the offset time is corrected replaces the first lossy audio, ensuring the synchronization of sound and picture of the replaced song file and the synchronization matching of the subtitle file.

[0099] Please refer to Figure 6 , Embodiment 2 of the present invention is:

[0100] An audio replacement terminal 1 includes a memory 3, a processor 2, and a computer program stored in the memory 3 and executable on the processor 2. When the processor 2 executes the computer program, the steps of the first embodiment are implemented.

[0101] In summary, the audio replacement method and terminal provided by the present invention have the following advantages:

[0102] (1) Preprocessing the sampling rates of the first lossy audio and the first lossless audio to be judged to be uniform and extracting the human voice audio to avoid the interference of audio quality and audio background music on audio matching alignment; using the DTW algorithm to reduce the error caused by the time axis difference when calculating the corresponding relationship between the two audio sequences; correcting the lossless audio by the constant in the linear regression function to ensure the synchronization of the sound and picture of the song file and the synchronization of the subtitle file; thereby improving the accuracy of audio matching;

[0103] (2) Since there is no need for manual matching, the labor cost of matching lossy audio and lossless audio can be effectively reduced, and the speed of matching alignment can be improved;

[0104] (3) Use the loss function value to effectively and quickly determine whether the two audio versions are the same;

[0105] This provides an audio replacement method with lower cost, faster speed and higher matching accuracy.

[0106] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent transformations made using the contents of the present invention's specification and drawings, or directly or indirectly applied in related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. An audio replacement method, characterized in that: Includes steps: Obtaining a first lossy audio and a first lossless audio; Calculating and obtaining a linear regression function between the first lossy audio and the first lossless audio based on a DTW algorithm: calculating and obtaining a similar path sequence between the first lossy audio and the first lossless audio based on the DTW algorithm, and calculating and obtaining a linear regression function between the first lossy audio and the first lossless audio according to the similar path sequence: discarding points whose horizontal axis or vertical axis is 0 in the similar path sequence, and calculating and obtaining a linear regression function between the first lossy audio and the first lossless audio according to other points in the similar path sequence except the discarded points; Calculate and obtain a loss function value according to the linear regression function; Determine whether the versions of the first lossy audio and the first lossless audio are the same according to the loss function value; if the judgment result is that the versions are the same, replace the first lossy audio with the first lossless audio according to the linear regression function: use the constant in the linear regression function as the offset time between the first lossy audio and the first lossless audio on the time axis; and replace the first lossy audio with the first lossless audio after correcting the offset time.

2. The audio replacement method according to claim 1, characterized in that: The calculation and obtaining of the loss function value according to the linear regression function specifically comprises the following steps: The square of the difference between the actual ordinate value formed by the first lossy audio and the first lossless audio at the same time point and the theoretical ordinate value obtained according to the linear regression function is accumulated to obtain the loss function value.

3. The audio replacement method according to claim 1, characterized in that: The determining, according to the loss function value, whether the versions of the first lossy audio and the first lossless audio are the same specifically comprises the following steps: Determine whether the loss function value is greater than a loss threshold; if so, the judgment result of the first lossy audio and the first lossless audio is that the versions are different; otherwise, the judgment result of the first lossy audio and the first lossless audio is that the versions are the same.

4. The audio replacement method according to claim 1, characterized in that: When the time of the first lossless audio is used as the ordinate of the linear regression function and the time of the first lossy audio is used as the abscissa of the linear regression function, the correction of the first lossless audio specifically includes the following steps: If the offset time is greater than 0, the first lossless audio is supplemented with a blank segment with a duration equal to the offset time. If the offset time is equal to 0, the first lossless audio is retained. If the offset time is less than 0, the beginning of the first lossless audio is cut off with a segment with a duration equal to the offset time.

5. An audio replacement method according to any one of claims 1 to 4, characterized in that: After obtaining the first lossy audio and the first lossless audio, the following steps are also included: Unifying the sampling rates of the first lossy audio and the first lossless audio; Extracting the first lossy audio and the first lossless audio; The step of calculating and obtaining a linear regression function between the first lossy audio and the first lossless audio based on the DTW algorithm specifically includes the following steps: A linear regression function between the first lossy audio and the first lossless audio is obtained based on a DTW algorithm and according to the human voice audio of the first lossy audio and the first lossless audio.

6. The audio replacement method according to claim 5, characterized in that: After extracting the first lossy audio and the first lossless audio, the following steps are also included: Extracting chroma features of the first lossy audio and the first lossless audio; The step of calculating and obtaining a linear regression function between the first lossy audio and the first lossless audio based on the DTW algorithm specifically includes the following steps: A linear regression function between the first lossy audio and the first lossless audio is obtained by calculating based on the DTW algorithm and according to the chromaticity features of the human voice audio of the first lossy audio and the first lossless audio.

7. An audio replacement terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, each step of the audio replacement method described in any one of claims 1 to 6 is implemented.

Images