Audio detection method, device, terminal and storage medium

Abstract

The embodiment of the present application provides an audio detection method, device, terminal and storage medium, wherein the method includes: the terminal processes the audio signal of the audio to be detected, obtains the audio fingerprint sequence of the audio to be detected, the audio fingerprint sequence includes the audio fingerprint of each signal frame in the audio signal, and the audio fingerprint is used to identify the digital features of the corresponding signal frame; the terminal calculates the autocorrelation function of the audio to be detected according to the audio fingerprint sequence; the terminal detects whether the audio to be detected is content-duplicate audio according to the peak value of the autocorrelation function. Through the embodiment of the present application, content-duplicate audio can be detected according to the peak value of the autocorrelation function, so that audio detection can be completed quickly and accurately.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to an audio detection method, device, terminal and storage medium. Background Art

[0002] At present, the detection of normal audio and content-duplicate audio is mainly achieved by using a neural network model. When using this method for detection, it is necessary to first annotate the normal audio and content-duplicate audio data in the database, and then use the neural network model to train the annotated data to obtain a classification training model, and finally input the audio to be tested into the trained classification model for detection to obtain the type of audio to be detected. It can be seen that the neural network training classification model needs to rely on a large amount of annotated data. When the amount of data increases, the annotation cost and annotation errors will increase accordingly. In addition, the amount of calculation required by the neural network model is large, which leads to a decrease in the detection speed and the accuracy of the detection results. Summary of the invention

[0003] The embodiments of the present application provide an audio detection method, device, terminal, and storage medium, which can improve the detection accuracy and efficiency of duplicate audio content.

[0004] On the one hand, an embodiment of the present application provides an audio detection method, the method comprising:

[0005] The terminal processes the audio signal of the audio to be detected to obtain an audio fingerprint sequence of the audio to be detected, where the audio fingerprint sequence includes an audio fingerprint of each signal frame in the audio signal, and the audio fingerprint is used to identify a digital feature of the corresponding signal frame;

[0006] The terminal calculates the autocorrelation function of the audio to be detected according to the audio fingerprint sequence;

[0007] The terminal detects whether the audio to be detected is audio with repeated content according to the peak value of the autocorrelation function.

[0008] Optionally, the terminal processes an audio signal of the audio to be detected to obtain an audio fingerprint sequence of the audio to be detected, including:

[0009] The terminal performs frame division and windowing processing on the audio signal of the audio to be detected to obtain multiple time domain signal frames of the audio to be detected;

[0010] The terminal performs Fourier transform on the multiple time domain signal frames to obtain multiple frequency domain signal frames;

[0011] The terminal determines the audio fingerprint sequence of the audio to be detected according to the sub-band energy difference of the sub-band in each frequency domain signal frame.

[0012] Optionally, the terminal calculates the autocorrelation function of the audio to be detected according to the audio fingerprint sequence, including:

[0013] The terminal translates the audio fingerprint sequence to obtain a translated audio fingerprint sequence;

[0014] The terminal calculates the autocorrelation function of the audio to be detected according to the audio fingerprint sequence and the shifted audio fingerprint sequence.

[0015] Optionally, the terminal calculates the autocorrelation function of the audio to be detected according to the audio fingerprint sequence and the shifted audio fingerprint sequence, including:

[0016] The terminal multiplies the audio fingerprint of each signal frame in the audio fingerprint sequence X(n) with the audio fingerprint of the corresponding signal frame in the translated audio fingerprint sequence X(n+τ), and then accumulates the multiplication operation to obtain the autocorrelation function of the audio to be detected;

[0017] The τ represents a translation sequence point of the audio fingerprint sequence.

[0018] Optionally, the terminal detects whether the audio to be detected is audio with repeated content according to a peak value of the autocorrelation function, including:

[0019] The terminal determines the number of peaks corresponding to m thresholds according to the peak value of the autocorrelation function, wherein the m thresholds are thresholds μ1 to μ m , the number of peaks corresponding to each threshold is the number of peaks in the peaks of the autocorrelation function that are greater than the threshold, and the thresholds μ1 to μ m The size of is reduced successively, and m is an integer greater than 1;

[0020] The terminal determines whether the audio to be detected is content-repeated audio according to the number of peaks corresponding to the m thresholds respectively.

[0021] Optionally, the terminal determines whether the audio to be detected is audio with repeated content according to the number of peaks corresponding to the m thresholds respectively, including:

[0022] If the number of peaks corresponding to the first threshold is greater than 1, the terminal determines whether the audio to be detected is content-repeated audio according to the number of peaks corresponding to the m thresholds respectively; the first threshold is one of the m thresholds;

[0023] If the number of peaks corresponding to the first threshold is equal to 1, it is determined that the audio to be detected is not audio with repeated content.

[0024] Optionally, the terminal determines whether the audio to be detected is audio with repeated content according to the number of peaks corresponding to the m thresholds respectively, including:

[0025] If the difference between the numbers of peaks corresponding to two adjacent thresholds among the m thresholds is less than a preset value, the terminal determines that the audio to be detected is audio with repeated content;

[0026] If the difference between the numbers of peaks corresponding to any two adjacent thresholds among the m thresholds is not less than a preset value, the terminal determines that the audio to be detected is not content-repeated audio.

[0027] Optionally, if the audio to be detected is the content repeated audio, the method further includes:

[0028] The terminal filters the audio to be detected from multiple audios to be recommended to obtain the filtered audio to be recommended.

[0029] On the one hand, an embodiment of the present application provides an audio detection device, the device comprising:

[0030] A processing module, configured for the terminal to process an audio signal of the audio to be detected, and obtain an audio fingerprint sequence of the audio to be detected, wherein the audio fingerprint sequence includes an audio fingerprint of each signal frame in the audio signal, and the audio fingerprint is used to identify a digital feature of a corresponding signal frame;

[0031] A calculation module, used for the terminal to calculate the autocorrelation function of the audio to be detected according to the audio fingerprint sequence;

[0032] The detection module is used for the terminal to detect whether the audio to be detected is audio with repeated content according to the peak value of the autocorrelation function.

[0033] On the one hand, an embodiment of the present application provides a terminal, comprising: a processor, a communication interface and a memory, wherein the processor, the communication interface and the memory are interconnected, wherein the memory stores an executable program code, and the processor is used to call the executable program code to execute the method provided by the embodiment of the present application.

[0034] Accordingly, the present application also provides a computer-readable storage medium, in which instructions are stored, and when the computer-readable storage medium is run on a computer, the computer executes the method provided in the embodiment of the present application.

[0035] Accordingly, the embodiment of the present application further provides a computer program product or a computer program, wherein the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method provided in the embodiment of the present application.

[0036] In the embodiment of the present application, the terminal processes the audio signal of the audio to be detected, obtains the audio fingerprint sequence of the audio to be detected, calculates the autocorrelation function of the audio to be detected according to the audio fingerprint sequence, and then detects whether the audio to be detected is a content-duplicate audio according to the peak value of the autocorrelation function. It can be seen that the embodiment of the present application not only does not need to rely on labeled data, but also can quickly and accurately detect content-duplicate audio. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0038] Figure 1 is a flowchart of an audio detection method shown in an embodiment of the present application;

[0039] Figure 2 is a flow chart of another audio detection method shown in an embodiment of the present application;

[0040] Figure 3 is a flowchart of another audio detection method shown in an embodiment of the present application;

[0041] Figure 4a , Figure 4b , Figure 4c is a schematic diagram of the autocorrelation function of some songs shown in the embodiment of the present application;

[0042] Figure 5 is a schematic diagram of an audio detection device shown in an embodiment of the present application;

[0043] Figure 6 It is a schematic diagram of the structure of a terminal shown in an embodiment of the present application. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0045] At present, the detection of duplicate audio content mainly uses neural network models. This method relies on a large amount of labeled data and requires continuous training of the neural network model to obtain a classification model. Finally, the audio to be detected is input into the classification model for detection. However, when the amount of data increases, its labeling cost and labeling errors will increase accordingly, and the neural network model requires a large amount of calculation.

[0046] Based on this, the embodiments of the present application provide an audio detection method, device, terminal and storage medium to detect content-duplicate audio more accurately and quickly. In this method, the terminal processes the audio signal of the audio to be detected to obtain the audio fingerprint sequence of the audio to be detected; the terminal calculates the autocorrelation function of the audio to be detected based on the audio fingerprint sequence; the terminal detects whether the audio to be detected is content-duplicate audio based on the peak value of the autocorrelation function. It can be seen that the present application does not need to rely on labeled data and can improve the detection accuracy and efficiency of content-duplicate audio.

[0047] It should be noted that the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, a smart car, etc., but is not limited to these.

[0048] Audio may include but is not limited to sound data in the form of songs.

[0049] To facilitate understanding of the embodiments of the present application, the specific implementation of the audio detection method is described in detail below.

[0050] See also Figure 1 , Figure 1 A flowchart of an audio detection method provided in an embodiment of the present application is shown in FIG.

[0051] S101: The terminal processes an audio signal of an audio to be detected to obtain an audio fingerprint sequence of the audio to be detected.

[0052] The audio fingerprint sequence includes the audio fingerprint of each signal frame in the audio signal, and the audio fingerprint is used to identify the digital features of the corresponding signal frame.

[0053] In an optional implementation, the terminal processes an audio signal of the audio to be detected to obtain an audio fingerprint sequence of the audio to be detected, including: the terminal performs frame windowing processing on the audio signal of the audio to be detected to obtain multiple time domain signal frames of the audio to be detected; the terminal performs Fourier transform on the multiple time domain signal frames to obtain multiple frequency domain signal frames; the terminal determines the audio fingerprint sequence of the audio to be detected based on the sub-band energy difference of the sub-band in each frequency domain signal frame.

[0054] Among them, the terminal determines the audio fingerprint sequence of the audio to be detected according to the subband energy difference of the subband in each frequency domain signal frame, including: the terminal divides each frequency domain signal frame into subbands to obtain the starting frequency of each subband in each frequency domain signal frame; obtains the subband energy difference of each subband in each frequency domain signal frame according to the starting frequency of each subband in each frequency domain signal frame; determines to obtain the audio fingerprint sequence of the audio to be detected according to the subband energy difference of each subband in multiple frequency domain signal frames.

[0055] In this implementation, the terminal performs frame windowing processing on the audio signal of the audio to be detected to obtain multiple time domain signal frames of the audio to be detected, which may include: dividing the audio signal with a sampling rate of a preset frequency into time domain signal frames of a preset length, and overlapping adjacent frames by a certain length; adding a Hanning window to each time domain signal frame for smoothing to prevent spectrum leakage. For example, assuming that the audio to be detected is a song, when the terminal performs frame windowing processing on the audio signal of the song to be detected, the audio signal with a sampling rate of A kHz can be divided into time domain signal frames with a length of B ms, and adjacent frames overlap C ms, and a Hanning window is added to each time domain signal frame for smoothing. For example, the sampling rate A kHz can be 8kHz, the length of the signal frame B ms can be 320ms, and the overlap between frames C ms can be 248ms.

[0056] In this implementation, s(t) represents a time domain signal frame, S n (k) represents the frequency domain signal frame, where k represents the frequency point index of the signal frame in the frequency domain, and n represents the frame index of the frequency domain signal frame in the frequency domain.

[0057] In this implementation, the terminal divides each frequency domain signal frame into subbands to obtain the starting frequency of each subband in each frequency domain signal frame, which may include: dividing each frequency domain signal frame into I non-overlapping subbands according to a certain distance; obtaining the starting frequency of each subband. For example, the terminal may divide each frequency domain signal frame into I non-overlapping subbands according to a logarithmic distance, and then obtain the starting frequency f(i) of each subband, where i is an integer greater than 1 and less than 1. The calculation formula of f(i) is as follows (1), where F max is the upper frequency limit, F min is the lower limit frequency, and I is the number of sub-band divisions. For example, I is 33.

[0058]

[0059] In this implementation, the terminal obtains the subband energy difference of each subband in each frequency domain signal frame according to the starting frequency of each subband in each frequency domain signal frame, including: obtaining the subband energy of each subband in each frequency domain signal frame according to the starting frequency of each subband in each frequency domain signal frame; obtaining the subband energy difference of each subband in each frequency domain signal frame according to the subband energy of each subband in each frequency domain signal frame. For example, the terminal can obtain the energy E of the i-th subband in the frequency domain signal frame according to the starting frequency of the i-th subband n (i), then according to the sub-band energy E n (i) Obtain the subband energy difference D of the i-th subband in the frequency domain signal frame n (i), E n (i) and D n The calculation formulas of (i) are as follows: (2) and (3), where n represents the frame index of the frequency domain signal frame in the frequency domain, that is, the nth frame frequency domain signal:

[0060]

[0061] D n (i) = E n (i)-E n (i+1)-(E n-1 (i)-E n-1 (i+1)) (3)

[0062] In this implementation, the terminal determines the audio fingerprint sequence of the audio to be detected based on the subband energy difference of each subband in multiple frequency domain signal frames, including: mapping the subband energy difference of each subband in each frequency domain signal frame into a sub-fingerprint; obtaining the audio fingerprint of each frequency domain signal frame based on the sub-fingerprint; and determining the audio fingerprint sequence of the audio to be detected based on the audio fingerprint of each frequency domain signal frame.

[0063] Each sub-fingerprint corresponds to a fingerprint value, and the audio fingerprint of each frequency domain signal frame includes multiple sub-fingerprints. Optionally, the sub-fingerprint can be obtained through a hash function, so the sub-fingerprint can also be called a hash fingerprint.

[0064] For example, the terminal can calculate the subband energy difference D in equation (3) n (i) Map it into a sub-fingerprint and obtain the i-th fingerprint value F in the frequency domain signal frame n (i), F n The calculation formula of (i) is as follows:

[0065]

[0066] Then according to the sub-fingerprint F n (i) Obtain the audio fingerprint of each frequency domain signal frame, which can be recorded as x j , where j is a positive integer and j∈[1,n]; x j It is composed of Y (e.g., 32) bit sub-fingerprints, that is, represented by Y (e.g., 32) bit 0 / 1 vectors. Finally, the audio fingerprint sequence X(n) of the audio to be detected is determined, where X(n) = [x1, x2, …, x n ].

[0067] Assuming that the audio to be detected is a song, in one case, after the song to be detected is processed by the above process, the audio fingerprints of different parts such as the prelude, verse, chorus, interlude, and outro in the obtained audio fingerprint sequence are different. In another case, after the song to be detected is processed by the above process, there will be a large number of repeated audio fingerprints in different parts of the obtained audio fingerprint sequence. In another case, for a song to be detected with repeated lyrics of the verse or chorus, after being processed by the above process, the audio fingerprints of different parts of the audio fingerprint sequence of the song to be detected are also different. It can be seen that this implementation method can identify the digital features of different parts of the audio to be detected, which is then helpful to detect whether the audio to be detected is audio with repeated content.

[0068] S102: The terminal calculates the autocorrelation function of the audio to be detected according to the audio fingerprint sequence.

[0069] In an optional implementation, the terminal calculates the autocorrelation function of the audio to be detected according to the audio fingerprint sequence of the audio to be detected, including: the terminal translates the audio fingerprint sequence to obtain the translated audio fingerprint sequence; the terminal calculates the autocorrelation function of the audio to be detected according to the audio fingerprint sequence and the translated audio fingerprint sequence.

[0070] For example, assuming that the audio fingerprint sequence of the audio to be detected is X(n), the terminal can obtain the translated audio fingerprint sequence X(n+τ) by translating X(n), where τ represents the translation sequence point. Based on X(n) and X(n+τ), the terminal can calculate the autocorrelation function r of the audio to be detected. x In an optional implementation, the audio fingerprint of each signal frame in the audio fingerprint sequence X(n) is multiplied and then accumulated with the audio fingerprint of the corresponding signal frame in the translated audio fingerprint sequence X(n+τ) to obtain the autocorrelation function r of the audio to be detected. x (τ). For example, as shown in formula (5):

[0071]

[0072] Wherein, N means that X(n) has N sequence points.

[0073] S103: The terminal detects whether the audio to be detected is audio with repeated content according to the peak value of the autocorrelation function.

[0074] If different parts of the audio fingerprint sequence of the audio to be detected have different audio fingerprints, then in formula (5), the autocorrelation function will only produce a peak value when τ=0 (that is, the audio fingerprint sequence of the audio to be detected completely overlaps with itself); if different parts of the audio fingerprint sequence of the audio to be detected have a large number of repeated audio fingerprints, the autocorrelation function of the audio to be detected will produce multiple peaks.

[0075] Based on this, in an optional implementation, the terminal in S103 detects whether the audio to be detected is audio with repeated content according to the peak value of the autocorrelation function, including: the terminal determines the number of peaks corresponding to m thresholds according to the peak value of the autocorrelation function; the terminal determines whether the audio to be detected is audio with repeated content according to the number of peaks corresponding to the m thresholds. The audio with repeated content is audio formed by continuously splicing the same audio segment.

[0076] In another optional implementation, in S103, the terminal detects whether the audio to be detected is audio with repeated content according to the peak value of the autocorrelation function, including: the terminal determines the number of peaks corresponding to m thresholds according to the peak value of the autocorrelation function; when the number of peaks corresponding to the first threshold is greater than 1, the terminal determines whether the audio to be detected is audio with repeated content according to the number of peaks corresponding to the m thresholds. The first threshold is one of the m thresholds.

[0077] In the above embodiment, the m thresholds are respectively threshold μ1 to threshold μ m , m is an integer greater than 1, the threshold μ1 to the threshold μ m can be values ​​greater than 0 and less than 1 respectively, and the m thresholds can be pre-set or set based on the accuracy of detection. The number of peaks corresponding to each threshold is the number of peaks in the autocorrelation function that are greater than the threshold. In an optional embodiment, the thresholds μ1 to μ m The sizes of are successively reduced. In another optional implementation, the thresholds μ1 to μ m It should be noted that in the above two optional specific implementations, the terminal can determine whether the audio to be detected is a content-duplicate audio based on the number of peaks corresponding to the m thresholds. The specific implementation method can be to determine whether the audio to be detected is a content-duplicate audio based on the difference in the number of peaks of adjacent thresholds. For details, please refer to Figure 3 Related instructions for step S305 in .

[0078] It can be seen that in the embodiment of the present application, the terminal obtains the audio fingerprint sequence of the audio to be detected, calculates the corresponding autocorrelation function according to the audio fingerprint sequence, and detects whether the audio to be detected is content-duplicate audio according to the peak value of the autocorrelation function. This method does not need to rely on labeled data and can quickly and accurately detect content-duplicate audio.

[0079] In the audio detection method provided in the embodiment of the present application, when recommending multiple audios, the terminal can filter out audios with duplicate contents from the multiple audios to be recommended before recommending. Figure 2 , Figure 2 A flowchart of another audio detection method provided in an embodiment of the present application. The method described in the embodiment of the present application includes steps S201-S205.

[0080] S201: The terminal processes an audio signal of each audio among multiple audios to be recommended to obtain an audio fingerprint sequence of the audio.

[0081] S202: The terminal calculates the autocorrelation function of each audio according to the audio fingerprint sequence of each audio.

[0082] S203: The terminal detects whether there is any audio with duplicate content in the multiple audios according to the peak value of the autocorrelation function of each audio. If there is any audio with duplicate content, the terminal executes step S204; if there is no audio with duplicate content, the terminal executes step S205.

[0083] S204: The terminal filters audios with duplicate contents from multiple audios to be recommended, and outputs the filtered audios to be recommended.

[0084] S205: The terminal outputs the audio to be recommended.

[0085] In an optional implementation, the specific processes of steps S201, S202, and S203 can be respectively referred to the descriptions of S101, S102, and S103 above, and will not be repeated here.

[0086] It can be seen that the embodiment of the present application detects whether there is audio with duplicate content in the multiple audios to be recommended through the peak value of the autocorrelation function of each audio among the multiple audios to be recommended. Therefore, when recommending audio to the user, the audio with duplicate content can be first filtered out from the multiple audios to be recommended, and then the recommendation is made, thereby improving the user experience.

[0087] See also Figure 3 , Figure 3 A flowchart of another audio detection method provided in an embodiment of the present application is shown in FIG.

[0088] S301: The terminal processes the audio signal of the audio to be detected to obtain an audio fingerprint sequence of the audio to be detected.

[0089] S302: The terminal calculates the autocorrelation function of the audio to be detected according to the audio fingerprint sequence.

[0090] S303: The terminal determines the number of peaks corresponding to m different thresholds according to the peak value of the autocorrelation function.

[0091] Among them, the m thresholds are threshold μ1 to threshold μ m , m is an integer greater than 1, and the number of peaks corresponding to each threshold is the number of peaks in the autocorrelation function that are greater than the threshold. m The sizes of are successively reduced. In another optional implementation, the thresholds μ1 to μ m The sizes increase successively.

[0092] S304. The terminal determines whether the number of peaks corresponding to the first threshold is greater than 1 or equal to 1. When the number of peaks corresponding to the first threshold is greater than 1, the terminal executes step S305; when the number of peaks corresponding to the first threshold is equal to 1, the terminal executes step S307.

[0093] Among them, the first threshold is one of the m thresholds. Optionally, the first threshold can be a relatively larger threshold among the m thresholds. If the number of peaks corresponding to the first threshold is equal to 1, then for other thresholds greater than the first threshold, the corresponding number of peaks will be equal to 1, indicating that the peak corresponding to the first threshold can only be generated by the autocorrelation of the audio to be detected and itself, and cannot be generated by a large number of repeated audio fingerprint fragments in the audio to be detected, so step S307 is executed. If the number of peaks corresponding to the first threshold is greater than 1, it means that the number of peaks corresponding to each threshold needs to be further analyzed to determine whether it is audio with repeated content, so step S305 needs to be executed.

[0094] S305. The terminal determines whether the differences between the numbers of peaks corresponding to two adjacent thresholds among the m thresholds are all less than a preset value. If all are less than the preset value, the terminal executes step S306; if there is a difference that is not less than the preset value, the terminal executes step S307.

[0095] Among them, all two adjacent thresholds among the m thresholds include the following combinations: μ1 and μ2, μ2 and μ3, ..., μ m-1 and μ m The preset value may be pre-set or may be set based on the accuracy of the detection.

[0096] That is to say, for the difference between the peak numbers corresponding to adjacent thresholds in the m thresholds, if all the differences are less than the preset value, it not only indicates that there are a large number of repeated audio fingerprint segments in the audio to be detected, but also that the repetition pattern is relatively consistent, that is, this situation will occur in the audio to be detected obtained by repeatedly splicing a certain segment, so step S306 is executed. If there is a difference greater than the preset value, it means that there are a large number of repeated audio fingerprint segments in the audio to be detected, but the repetition pattern is inconsistent, so it is very likely to be an audio structure design, such as the repetition of the main song or chorus lyrics, so step S307 is executed.

[0097] S306: The terminal determines that the audio to be detected is audio with repeated content.

[0098] S307: The terminal determines that the audio to be detected is not audio with repeated content.

[0099] In an optional implementation, the specific processes of steps S301 and S302 can refer to the descriptions of S101 and S102 above, respectively, and will not be repeated here.

[0100] In an optional implementation, the terminal may directly execute step S305 after executing step S303.

[0101] The above steps are described below by way of examples.

[0102] For example, assuming that the first threshold is μ2. The terminal processes the audio signal of the song 1 to be detected to obtain the audio fingerprint sequence of the song 1 to be detected, and then calculates the autocorrelation function of the song 1 to be detected according to the audio fingerprint sequence of the song 1 to be detected, as shown in Figure 4a As shown, Figure 4a In the example, the number of peaks corresponding to the first threshold μ2 is determined based on the peak value of the autocorrelation function, that is, the number of peaks whose peak values ​​of the autocorrelation function are greater than the first threshold μ2 is 1, so it can be determined that the song 1 to be detected is not a song with duplicate content.

[0103] For another example, assume that m is 3, the first threshold is μ2, and the preset value is 1. The terminal processes the audio signal of the song 2 to be detected to obtain the audio fingerprint sequence of the song 2 to be detected, and then calculates the autocorrelation function of the song 2 to be detected according to the audio fingerprint sequence of the song 2 to be detected as follows: Figure 4b As shown, Figure 4b In the above, three different thresholds μ1, μ2, and μ3 are determined according to the peak value of the autocorrelation function. m The corresponding peak numbers are 1, 3, and 19 respectively. Then it is found that the peak number corresponding to the first threshold μ2 is 3, and 3 is greater than 1. Therefore, for the song 2 to be detected, all two adjacent thresholds μ1 and μ2, μ2 and μ mThe difference in the number of peaks corresponding to μ1 and μ2 is 3-1=2, and the difference in the number of peaks corresponding to μ2 and μ m The difference between the corresponding peak numbers is 19-3=16, and both 2 and 16 are not less than the preset value (ie, 1), so it can be determined that the song 2 to be detected is not a song with duplicate content.

[0104] For another example, assume that m is 3, the first threshold is μ2, and the preset value is 1. The terminal processes the audio signal of the song 3 to be detected, and can obtain the audio fingerprint sequence of the song 3 to be detected. Then, the audio fingerprint sequence of the song 3 to be detected is calculated to obtain the autocorrelation function of the song 3 to be detected, as shown in FIG. Figure 4c As shown, Figure 4c In the above, three different thresholds μ1, μ2, and μ3 are determined according to the peak value of the autocorrelation function. m The corresponding peak numbers are 5, 5, and 5 respectively. Then it is found that the peak number corresponding to the first threshold μ2 is 5, and 5 is greater than 1. Therefore, for the song 3 to be detected, all two adjacent thresholds μ1 and μ2, μ2 and μ m The difference in the number of peaks corresponding to μ1 and μ2 is 5-5=0, and the difference in the number of peaks corresponding to μ2 and μ m The difference between the corresponding peak numbers is 5-5=0, and 0 is smaller than the preset value (ie, 1), so it can be determined that the song 3 to be detected is a song with repeated content.

[0105] It can be seen that in the embodiment of the present application, the terminal obtains the audio fingerprint sequence of the song to be detected, calculates the corresponding autocorrelation function according to the audio fingerprint sequence, and detects whether the song to be detected is a song with repeated content according to the peak value of the autocorrelation function. This method does not need to rely on labeled data, and can quickly detect songs with repeated content. At the same time, according to the characteristics of the peak value of the autocorrelation function of songs with repeated content, it effectively reduces the false detection rate of certain normal songs with partially repeated music structures (such as DJ dance music, European and American songs, etc.). Please refer to Figure 5 , Figure 5 Schematic diagram of an audio detection device shown in an embodiment of the present application. The audio detection device described in this embodiment may include the following parts:

[0106] The processing module 501 is used for the terminal to process the audio signal of the audio to be detected, and obtain an audio fingerprint sequence of the audio to be detected, wherein the audio fingerprint sequence includes an audio fingerprint of each signal frame in the audio signal, and the audio fingerprint is used to identify a digital feature of the corresponding signal frame;

[0107] A calculation module 502, used for the terminal to calculate the autocorrelation function of the audio to be detected according to the audio fingerprint sequence;

[0108] The detection module 503 is used for the terminal to detect whether the audio to be detected is audio with repeated content according to the peak value of the autocorrelation function.

[0109] In an optional implementation, the processing module 501 is specifically used for the terminal to perform frame windowing processing on the audio signal of the audio to be detected to obtain multiple time domain signal frames of the audio to be detected; the terminal performs Fourier transform on the multiple time domain signal frames to obtain multiple frequency domain signal frames; the terminal determines the audio fingerprint sequence of the audio to be detected based on the sub-band energy difference of the sub-band in each frequency domain signal frame.

[0110] In an optional implementation, the calculation module 502 is specifically configured to: the terminal translates the audio fingerprint sequence to obtain the translated audio fingerprint sequence; and the terminal calculates the autocorrelation function of the audio to be detected according to the audio fingerprint sequence and the translated audio fingerprint sequence.

[0111] In an optional implementation, the calculation module 502 is specifically used for the terminal to multiply the audio fingerprint of each signal frame in the audio fingerprint sequence X(n) with the audio fingerprint of the corresponding signal frame in the translated audio fingerprint sequence X(n+τ) and then accumulate them to obtain the autocorrelation function of the audio to be detected; wherein τ represents the translated sequence point of the audio fingerprint sequence.

[0112] In an optional implementation, the detection module 503 is specifically used for the terminal to determine the number of peaks corresponding to the m thresholds according to the peak value of the autocorrelation function; the terminal determines whether the audio to be detected is content-repeated audio according to the number of peaks corresponding to the m thresholds. m , m is an integer greater than 1, the number of peaks corresponding to each threshold is the number of peaks greater than the threshold among the peaks of the autocorrelation function, and the threshold μ1 to the threshold μ m The sizes decrease in sequence.

[0113] In an optional implementation, the detection module 503 is specifically used to, if the number of peaks corresponding to the first threshold is greater than 1, the terminal determines whether the audio to be detected is content-duplicate audio based on the number of peaks corresponding to m thresholds respectively; the first threshold is one of the m thresholds; if the number of peaks corresponding to the first threshold is equal to 1, it is determined that the audio to be detected is not content-duplicate audio.

[0114] In an optional implementation, the detection module 503 is specifically used to, if the difference in the number of peaks corresponding to all two adjacent thresholds among the m thresholds is less than a preset value, the terminal determines that the audio to be detected is content-duplicate audio; if the difference in the number of peaks corresponding to any two adjacent thresholds among the m thresholds is not less than a preset value, the terminal determines that the audio to be detected is not content-duplicate audio.

[0115] The audio detection device described in this embodiment may further include:

[0116] The filtering module 504 is used for, if the audio to be detected is audio with repeated content, the terminal filters the audio to be detected from the multiple audios to be recommended to obtain the filtered audio to be recommended.

[0117] It can be understood that the specific implementation of each module in the audio detection device described in the embodiment of the present application and the beneficial effects that can be achieved can be referred to the description of the aforementioned related embodiments, and will not be repeated here.

[0118] See also Figure 6 , Figure 6 601, a user interface 602, a communication interface 603, and a memory 604. The processor 601, the user interface 602, the communication interface 603, and the memory 604 may be connected via a bus or other means, and the embodiment of the present application takes the connection via a bus as an example.

[0119] Among them, the processor 601 (or CPU (Central Processing Unit)) is the computing core and control core of the terminal, which can parse various instructions in the terminal and process various data of the terminal. For example, the CPU can be used to parse the power on and off instructions sent by the user to the terminal, and control the terminal to perform power on and off operations; for another example, the CPU can transmit various interactive data between the internal structures of the terminal, etc. The user interface 602 is a medium for realizing interaction and information exchange between the user and the terminal. Its specific embodiment can include a display screen (Display) for output and a keyboard (Keyboard) for input, etc. It should be noted that the keyboard here can be a physical keyboard, a touch screen virtual keyboard, or a keyboard that combines a physical and a touch screen virtual. The communication interface 603 can optionally include a standard wired interface, a wireless interface (such as Wi-Fi, a mobile communication interface, etc.), which is controlled by the processor 601 for sending and receiving data. The memory 604 (Memory) is a memory device in the terminal, which is used to store programs and data. It can be understood that the memory 604 here can include the built-in memory of the terminal, and of course, it can also include the extended memory supported by the terminal. The memory 604 provides a storage space, which stores the operating system of the terminal, including but not limited to: Android system, iOS system, Windows Phone system, etc., which is not limited in this application.

[0120] In the embodiment of the present application, the processor 601 performs the following operations by running the executable program code in the memory 604:

[0121] Processing the audio signal of the audio to be detected to obtain an audio fingerprint sequence of the audio to be detected; wherein the audio fingerprint sequence includes the audio fingerprint of each signal frame in the audio signal, and the audio fingerprint is used to identify the digital features of the corresponding signal frame;

[0122] Calculate the autocorrelation function of the audio to be detected according to the audio fingerprint sequence;

[0123] According to the peak value of the autocorrelation function, it is detected whether the audio to be detected is audio with repeated content.

[0124] In an optional implementation manner, when the processor 601 processes the audio signal of the song to be detected and obtains the audio fingerprint sequence of the song to be detected, it is specifically used to:

[0125] The terminal performs frame division and windowing processing on the audio signal of the audio to be detected to obtain multiple time domain signal frames of the audio to be detected;

[0126] The terminal performs Fourier transform on the multiple time domain signal frames to obtain multiple frequency domain signal frames;

[0127] The terminal determines the audio fingerprint sequence of the audio to be detected according to the sub-band energy difference of the sub-band in each frequency domain signal frame.

[0128] In an optional implementation manner, when the processor 601 calculates the autocorrelation function of the audio to be detected according to the audio fingerprint sequence, it is specifically configured to:

[0129] Shift the audio fingerprint sequence to obtain a shifted audio fingerprint sequence;

[0130] According to the audio fingerprint sequence and the shifted audio fingerprint sequence, the autocorrelation function of the audio to be detected is calculated.

[0131] In an optional implementation manner, when the processor 601 calculates the autocorrelation function of the audio to be detected according to the audio fingerprint sequence and the shifted audio fingerprint sequence, it is specifically configured to:

[0132] The audio fingerprint of each signal frame in the audio fingerprint sequence X(n) is multiplied and then accumulated with the audio fingerprint of the corresponding signal frame in the translated audio fingerprint sequence X(n+τ) to obtain the autocorrelation function of the audio to be detected; wherein τ represents the translated sequence point of the audio fingerprint sequence. In an optional implementation, when the processor 601 detects whether the audio to be detected is content-repeated audio according to the peak value of the autocorrelation function, it is specifically used to:

[0133] According to the peak value of the autocorrelation function, the number of peaks corresponding to the m thresholds is determined; wherein the m thresholds are threshold μ1 to threshold μ m, the number of peaks corresponding to each threshold is the number of peaks in the autocorrelation function that are greater than the threshold, and m is an integer greater than 1;

[0134] Whether the audio to be detected is audio with repeated content is determined according to the number of peaks corresponding to the m thresholds.

[0135] In an optional implementation manner, when the processor 601 determines whether the audio to be detected is audio with repeated content according to the number of peaks corresponding to the m thresholds, the processor 601 is specifically configured to:

[0136] If the number of peaks corresponding to the first threshold is greater than 1, determining whether the audio to be detected is audio with repeated content according to the number of peaks corresponding to the m thresholds respectively; the first threshold is one of the m thresholds;

[0137] If the number of peaks corresponding to the first threshold is equal to 1, it is determined that the audio to be detected is not audio with repeated content.

[0138] In an optional implementation manner, when the processor 601 determines whether the audio to be detected is audio with repeated content according to the number of peaks corresponding to the m thresholds, the processor 601 is specifically configured to:

[0139] If the difference between the numbers of peaks corresponding to two adjacent thresholds among all the m thresholds is less than a preset value, it is determined that the audio to be detected is audio with repeated content;

[0140] If the difference between the numbers of peaks corresponding to any two adjacent thresholds among the m thresholds is not less than a preset value, it is determined that the audio to be detected is not audio with repeated content.

[0141] In an optional implementation manner, the processor 601 is further configured to, if the audio to be detected is audio with repeated content, filter the audio to be detected from the multiple audios to be recommended to obtain the filtered audio to be recommended.

[0142] In a specific implementation, the processor 601, user interface 602, communication interface 603 and memory 604 described in the embodiments of the present application can execute the implementation method of the terminal described in the audio detection method provided in the embodiments of the present application, and can also execute the implementation method described in the audio detection device provided in the embodiments of the present application, which will not be repeated here.

[0143] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. The computer program includes program instructions. When the program instructions are executed by a processor, the audio detection method provided by the embodiment of the present application is implemented. For details, please refer to the implementation methods provided in the above steps, which will not be repeated here.

[0144] The embodiment of the present application also provides a computer program product or a computer program, wherein the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the terminal reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the terminal executes the method as described in the embodiment of the present application. The specific implementation method can be referred to the above description, which will not be repeated here.

[0145] It should be noted that, for the above-mentioned various method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0146] A person skilled in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0147] The above disclosure is only part of the embodiments of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. An audio detection method, characterized in that: The method comprises: The terminal processes the audio signal of the audio to be detected to obtain an audio fingerprint sequence of the audio to be detected, where the audio fingerprint sequence includes an audio fingerprint of each signal frame in the audio signal, and the audio fingerprint is used to identify a digital feature of the corresponding signal frame; The terminal calculates the autocorrelation function of the audio to be detected according to the audio fingerprint sequence; The terminal detects whether the audio to be detected is audio with repeated content according to the peak value of the autocorrelation function.

2. The method according to claim 1, characterized in that The terminal processes the audio signal of the audio to be detected to obtain the audio fingerprint sequence of the audio to be detected, including: The terminal performs frame division and windowing processing on the audio signal of the audio to be detected to obtain multiple time domain signal frames of the audio to be detected; The terminal performs Fourier transform on the multiple time domain signal frames to obtain multiple frequency domain signal frames; The terminal determines the audio fingerprint sequence of the audio to be detected according to the sub-band energy differences of the sub-bands in the multiple frequency domain signal frames.

3. The method according to claim 1, characterized in that The terminal calculates the autocorrelation function of the audio to be detected according to the audio fingerprint sequence, including: The terminal translates the audio fingerprint sequence to obtain a translated audio fingerprint sequence; The terminal calculates the autocorrelation function of the audio to be detected according to the audio fingerprint sequence and the shifted audio fingerprint sequence.

4. The method according to claim 3, characterized in that: The terminal calculates the autocorrelation function of the audio to be detected according to the audio fingerprint sequence and the shifted audio fingerprint sequence, including: The terminal multiplies the audio fingerprint of each signal frame in the audio fingerprint sequence X(n) with the audio fingerprint of the corresponding signal frame in the translated audio fingerprint sequence X(n+τ), and then accumulates the multiplication operation to obtain the autocorrelation function of the audio to be detected; The τ represents a translation sequence point of the audio fingerprint sequence.

5. The method according to claim 1, characterized in that The terminal detects, according to the peak value of the autocorrelation function, whether the audio to be detected is audio with repeated content, including: The terminal determines the number of peaks corresponding to m thresholds according to the peak value of the autocorrelation function, wherein the m thresholds are thresholds μ1 to μ m , the number of peaks corresponding to each threshold is the number of peaks greater than the threshold among the peaks of the autocorrelation function, and the threshold μ1 to the threshold μ m The size of is reduced successively, and m is an integer greater than 1; The terminal determines whether the audio to be detected is content-repeated audio according to the number of peaks corresponding to the m thresholds respectively.

6. The method according to claim 5, characterized in that The terminal determines, according to the number of peaks corresponding to the m thresholds, whether the audio to be detected is audio with repeated content, including: If the number of peaks corresponding to the first threshold is greater than 1, the terminal determines whether the audio to be detected is content-repeated audio according to the number of peaks corresponding to the m thresholds respectively; the first threshold is one of the m thresholds; If the number of peaks corresponding to the first threshold is equal to 1, it is determined that the audio to be detected is not audio with repeated content.

7. The method according to claim 5 or 6, characterized in that: The terminal determines, according to the number of peaks corresponding to the m thresholds, whether the audio to be detected is audio with repeated content, including: If the difference between the numbers of peaks corresponding to two adjacent thresholds among the m thresholds is less than a preset value, the terminal determines that the audio to be detected is audio with repeated content; If the difference between the numbers of peaks corresponding to any two adjacent thresholds among the m thresholds is not less than a preset value, the terminal determines that the audio to be detected is not content-repeated audio.

8. The method according to claim 1, characterized in that: If the audio to be detected is the content repeated audio, the method further includes: The terminal filters the audio to be detected from multiple audios to be recommended to obtain the filtered audio to be recommended.

9. A terminal, characterized in that: include: A processor, a communication interface and a memory, wherein the processor, the communication interface and the memory are connected to each other, wherein the memory stores an executable program code, and the processor is used to call the executable program code to execute the method as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor is caused to perform the method according to any one of claims 1 to 8.

Images