Audio fingerprinting method, apparatus, terminal device, and storage medium

Abstract

The application discloses an audio fingerprint identification method and device, a terminal device and a storage medium. The audio fingerprint identification method comprises the following steps: acquiring full audio data; performing frame transformation on the full audio data to obtain each frequency spectrum data after frame division; selecting two-dimensional frequency band data meeting a preset target frequency threshold from the frequency spectrum data; performing dimension reduction on the frequency band data to obtain one-dimensional audio fingerprints of the full audio data; and performing audio fingerprint identification based on a local sensitive hash calculation improved Philips audio fingerprint identification method, that is, selecting the full audio data to obtain corresponding frequency spectrum data, selecting frequency band data meeting a preset frequency, and then performing dimension reduction on the frequency band data to obtain audio fingerprints, so that the technical problems of low accuracy and large data volume of audio fingerprint identification can be solved, the accuracy of audio fingerprint identification is improved, the data volume of the audio fingerprint is reduced, and the audio fingerprint identification method can be effectively used in the fields of audio fingerprint storage and audio retrieval deduplication.

Description

Technical Field

[0001] This application relates to the field of audio data processing technology, and in particular to an audio fingerprint recognition method, apparatus, terminal device, and storage medium. Background Technology

[0002] Fingerprint systems have a long history and have received much attention in both academia and industry, for example, in areas such as deduplication of massive samples and retrieval of similar content.

[0003] Because the Philips audio fingerprinting method is a classic and robust audio fingerprinting method, it has been widely used to this day.

[0004] However, the Philips audio fingerprinting method has the following drawbacks: First, the Philips audio fingerprinting method only selects a 3-second data segment from the audio for calculation, which makes the segment fingerprint unable to represent the entire audio segment, resulting in low accuracy of audio fingerprinting; Second, the recognized audio fingerprint is a two-dimensional array with a large number of 8192 (256*32), that is, the large amount of audio fingerprint data makes it difficult to store and retrieve audio fingerprints. Summary of the Invention

[0005] The main objective of this application is to provide an audio fingerprint recognition method, apparatus, terminal device, and storage medium, aiming to solve the technical problems of low accuracy and large data volume in audio fingerprint recognition, improve the accuracy of audio fingerprint recognition, reduce the data volume of audio fingerprints, and thus effectively apply them to audio fingerprint storage and audio retrieval deduplication fields.

[0006] To achieve the above objectives, this application provides an audio fingerprint recognition method, the audio fingerprint recognition method comprising:

[0007] Obtain full audio data;

[0008] The full audio data is subjected to frame-by-frame transformation to obtain the spectrum data of each frame;

[0009] From the spectrum data, select two-dimensional frequency band data that meets the preset target frequency threshold;

[0010] The frequency band data is subjected to dimensionality reduction to obtain a one-dimensional audio fingerprint of the full audio data.

[0011] Optionally, the step of performing dimensionality reduction on the frequency band data to obtain a one-dimensional audio fingerprint of the full audio data includes:

[0012] The frequency band data is subjected to overlapping framing and energy value calculation to obtain a two-dimensional fingerprint matrix;

[0013] The fingerprint matrix is ​​transformed to obtain the audio fingerprint.

[0014] Optionally, the step of performing overlapping framing and energy value calculation on the frequency band data to obtain a two-dimensional fingerprint matrix includes:

[0015] The frequency band data is subjected to overlapping framing to select non-overlapping frequency band data;

[0016] The corresponding energy values ​​are obtained by calculating the non-overlapping frequency band data respectively;

[0017] The fingerprint matrix is ​​obtained by calculating the energy value and the non-overlapping frequency band data according to a preset formula.

[0018] Optionally, the step of transforming the fingerprint matrix to obtain the audio fingerprint includes:

[0019] The fingerprint matrix is ​​summed vertically to obtain a one-dimensional weight matrix;

[0020] The audio fingerprint is obtained by calculating the weight matrix according to the preset symbol function.

[0021] Optionally, before the step of selecting two-dimensional frequency band data that meets a preset target frequency threshold from the spectrum data, the method further includes:

[0022] Obtain the initial frequency threshold;

[0023] The initial frequency threshold is optimized based on preset sensitivity conditions to obtain the target frequency threshold.

[0024] Optionally, the step of performing frame-by-frame transformation on the full audio data to obtain the framed spectral data includes:

[0025] The full audio data is divided into overlapping frames to obtain the audio data of each frame of the full audio data.

[0026] The audio data is subjected to short-time Fourier transform to obtain the corresponding spectrum data.

[0027] Optionally, the step of performing overlapping frame segmentation on the full audio data to obtain the audio data of each frame of the full audio data includes:

[0028] Determine the computational elements of the full audio data, wherein the computational elements include one or more of the total length of the full audio data, the frame rate, and the fingerprint length of the audio fingerprint;

[0029] Based on the aforementioned computational elements, the full audio data is subjected to overlapping frame segmentation to obtain the audio data of each frame of the full audio data.

[0030] This application also proposes an audio fingerprint recognition device, which includes:

[0031] The audio acquisition module is used to acquire full audio data;

[0032] The audio framing module is used to perform framing transformation on the full audio data to obtain the spectral data after framing.

[0033] The frequency band selection module is used to select two-dimensional frequency band data that meets a preset target frequency threshold from the spectrum data;

[0034] The fingerprint extraction module is used to perform dimensionality reduction on the frequency band data to obtain a one-dimensional audio fingerprint of the full audio data.

[0035] This application also proposes a terminal device, which includes a memory, a processor, and an audio fingerprint recognition program stored in the memory and executable on the processor. When the audio fingerprint recognition program is executed by the processor, it implements the steps of the audio fingerprint recognition method as described above.

[0036] This application also proposes a computer-readable storage medium storing an audio fingerprint recognition program, which, when executed by a processor, implements the steps of the audio fingerprint recognition method as described above.

[0037] The audio fingerprinting method, apparatus, terminal device, and storage medium proposed in this application acquire full audio data; perform frame-by-frame transformation on the full audio data to obtain each spectral data after frame division; select two-dimensional frequency band data that meets a preset target frequency threshold from the spectral data; perform dimensionality reduction on the frequency band data to obtain a one-dimensional audio fingerprint of the full audio data; and perform audio fingerprinting based on an improved Philips audio fingerprinting method using locality-sensitive hashing. That is, by selecting full audio data to obtain corresponding spectral data and selecting frequency band data that meets a preset frequency, and then performing dimensionality reduction on the frequency band data to obtain the audio fingerprint, the technical problems of low accuracy and large data volume in audio fingerprinting can be solved, thereby improving the accuracy of audio fingerprinting and reducing the data volume of audio fingerprints, and thus effectively used in audio fingerprint storage and audio retrieval deduplication fields. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of the functional modules of the terminal device to which the audio fingerprint recognition device of this application belongs;

[0039] Figure 2 This is a flowchart illustrating a first exemplary embodiment of the audio fingerprint recognition method of this application;

[0040] Figure 3 for Figure 2 A schematic diagram of the specific process of step S230 in the embodiment;

[0041] Figure 4 for Figure 2 A detailed flowchart of step S240 in the embodiment;

[0042] Figure 5 This is a schematic diagram of the overall process in the embodiments of this application.

[0043] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0044] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0045] The main solution of this application embodiment is as follows: acquiring full audio data; performing frame-by-frame transformation on the full audio data to obtain each spectrum data after frame division; selecting two-dimensional frequency band data that meets a preset target frequency threshold from the spectrum data; performing dimensionality reduction operation on the frequency band data to obtain a one-dimensional audio fingerprint of the full audio data; and performing audio fingerprint recognition based on the improved Philips audio fingerprint recognition method using locality-sensitive hashing calculation. That is, by selecting full audio data to obtain the corresponding spectrum data and selecting frequency band data that meets a preset frequency, and then performing dimensionality reduction operation on the frequency band data to obtain the audio fingerprint, the technical problems of low accuracy and large data volume in audio fingerprint recognition can be solved, thereby improving the accuracy of audio fingerprint recognition and reducing the data volume of audio fingerprints, and thus effectively used in audio fingerprint storage and audio retrieval deduplication fields.

[0046] Technical terms used in the embodiments of this application:

[0047] SimHash, a traditional hash algorithm, only maps the original content to a signature value as evenly and randomly as possible, essentially functioning as a pseudo-random number generation algorithm. If two signatures generated by a traditional hash algorithm are unequal, it provides no further information than indicating that the original content is different. This is because even if the original content differs by only one byte, the resulting signatures can be significantly different. Therefore, traditional hashing cannot measure the similarity of the original content at the signature level. SimHash, however, is a locality-sensitive hashing algorithm, and the hash signatures it generates can, to some extent, characterize the similarity of the original content.

[0048] The Philips audio fingerprint recognition method, along with its workflow, is described below:

[0049] A segment of audio is cropped, and 3 seconds of audio is selected as the basis for fingerprint calculation. The selected full audio is framed with overlap using a Hamming window. Each frame is about 0.37 seconds long, and the next frame overlaps with the previous frame by about 31 / 32. That is, the window moves backward by about 0.0116 seconds (0.37 / 32) each time. Therefore, about 257 frames (n+1) of data are collected from the 3-second audio. Fourier transform is applied to each frame of data, and its spectrum (absolute value / frame length) is calculated. The obtained spectrum is selected from the frequency band of 300Hz-2000Hz and framed. The non-overlapping data is evenly divided into 33 frames (m+1) of data. The energy value E of each frame of data is calculated, that is, its sum of squares is calculated. The corresponding fingerprint matrix F is calculated according to the following formula (1). Finally, the Philips audio fingerprint recognition method obtains a 256*32 0-1 matrix to represent the fingerprint of the corresponding audio. When calculating the similarity between two audio segments, the corresponding fingerprint bit error rate (BER) can be calculated to determine whether the two audio segments are similar.

[0050]

[0051] Bit Error Rate (BER), or Bit Error Probability, is the number of bits that are altered in a digital transmission data stream due to noise, interference, distortion, or bit synchronization errors. It is a percentage expressed as a percentage of the total number of bits transmitted over a given time interval.

[0052] The Hamming window is a type of cosine window, also known as an improved raised cosine window. Both the Hamming and Hanning windows are cosine windows, differing only in their weighting coefficients. The weighting coefficients of the Hamming window result in smaller sidelobes. Analysis shows that the first sidelobe of the Hamming window attenuates to -42 dB. The spectrum of the Hamming window is also synthesized from the spectra of three rectangular time windows, but its sidelobe attenuation rate is 20 dB / (10 oct), which is slower than that of the Hanning window.

[0053] A hash algorithm transforms a piece of data into a unique identifier, which is closely related to every byte of the original data. A key characteristic of hash algorithms is that it's very difficult to find a reverse pattern. Hash algorithms are a broad category, encompassing not only algorithms but also a set of principles. They can improve storage efficiency, enhance data retrieval, and even be used for digital signatures to ensure data security. Therefore, hash algorithms are widely used in internet applications. While called an algorithm, hash algorithms are more accurately described as a set of principles. There is no fixed formula for hash algorithms; any algorithm that adheres to the principles of hashing can be considered a hash algorithm.

[0054] The embodiments of this application take into account the following disadvantages of the Philips audio fingerprint recognition method: First, the Philips audio fingerprint recognition method only selects a 3-second data segment from the audio for calculation, which makes the segment fingerprint unable to represent the entire audio segment, resulting in low accuracy of audio fingerprint recognition; Second, the recognized audio fingerprint is a two-dimensional array with a number of up to 8192 (256*32), which is a large amount of data, making it difficult to store and retrieve audio fingerprints.

[0055] Therefore, the embodiments of this application, starting from the practical problems of audio fingerprint recognition, select complete Philips audio and combine the dimensionality reduction capability of locality-sensitive hashing (Simhash) to design an improved Philips audio fingerprint hash calculation method based on locality-sensitive hashing. This solves the technical problems of low accuracy and large amount of stored data in audio fingerprint recognition, improves the accuracy of audio fingerprint recognition, reduces the amount of audio fingerprint data, and can be effectively used in audio fingerprint storage and audio retrieval deduplication fields.

[0056] Specifically, refer to Figure 1 , Figure 1 This is a functional module diagram of the terminal device to which the audio fingerprint recognition device of this application belongs. The audio fingerprint recognition device can be a device capable of audio fingerprint recognition, independent of the terminal device, and can be implemented on the terminal device in hardware or software form. The terminal device can be a smart mobile terminal with data processing capabilities, such as a mobile phone or tablet computer, or a fixed terminal device or server with data processing capabilities.

[0057] In this embodiment, the terminal device to which the audio fingerprint recognition device belongs includes at least an output module 110, a processor 120, a memory 130, and a communication module 140.

[0058] The memory 130 stores the operating system and the audio fingerprint recognition program. The audio fingerprint recognition device can acquire full audio data; perform frame transformation on the full audio data to obtain the segmented spectrum data; select two-dimensional frequency band data that meets the preset target frequency threshold from the spectrum data; and perform dimensionality reduction on the frequency band data to obtain a one-dimensional audio fingerprint of the full audio data, etc., and store these information in the memory 130. The output module 110 can be a display screen, etc. The communication module 140 can include a WIFI module, a mobile communication module, and a Bluetooth module, etc., and communicates with external devices or servers through the communication module 140.

[0059] When the audio fingerprint recognition program in memory 130 is executed by the processor, it performs the following steps:

[0060] Obtain full audio data;

[0061] The full audio data is subjected to frame-by-frame transformation to obtain the spectrum data of each frame;

[0062] From the spectrum data, select two-dimensional frequency band data that meets the preset target frequency threshold;

[0063] The frequency band data is subjected to dimensionality reduction to obtain a one-dimensional audio fingerprint of the full audio data.

[0064] Furthermore, when the audio fingerprint recognition program in memory 130 is executed by the processor, it also performs the following steps:

[0065] The frequency band data is subjected to overlapping framing and energy value calculation to obtain a two-dimensional fingerprint matrix;

[0066] The fingerprint matrix is ​​transformed to obtain the audio fingerprint.

[0067] Furthermore, when the audio fingerprint recognition program in memory 130 is executed by the processor, it also performs the following steps:

[0068] The frequency band data is subjected to overlapping framing to select non-overlapping frequency band data;

[0069] The corresponding energy values ​​are obtained by calculating the non-overlapping frequency band data respectively;

[0070] The fingerprint matrix is ​​obtained by calculating the energy value and the non-overlapping frequency band data according to a preset formula.

[0071] Furthermore, when the audio fingerprint recognition program in memory 130 is executed by the processor, it also performs the following steps:

[0072] The fingerprint matrix is ​​summed vertically to obtain a one-dimensional weight matrix;

[0073] The audio fingerprint is obtained by calculating the weight matrix according to the preset symbol function.

[0074] Furthermore, when the audio fingerprint recognition program in memory 130 is executed by the processor, it also performs the following steps:

[0075] Obtain the initial frequency threshold;

[0076] The initial frequency threshold is optimized based on preset sensitivity conditions to obtain the target frequency threshold.

[0077] Furthermore, when the audio fingerprint recognition program in memory 130 is executed by the processor, it also performs the following steps:

[0078] The full audio data is divided into overlapping frames to obtain the audio data of each frame of the full audio data.

[0079] The audio data is subjected to short-time Fourier transform to obtain the corresponding spectrum data.

[0080] Furthermore, when the audio fingerprint recognition program in memory 130 is executed by the processor, it also performs the following steps:

[0081] Determine the computational elements of the full audio data, wherein the computational elements include one or more of the total length of the full audio data, the frame rate, and the fingerprint length of the audio fingerprint;

[0082] Based on the aforementioned computational elements, the full audio data is subjected to overlapping frame segmentation to obtain the audio data of each frame of the full audio data.

[0083] This embodiment, through the above-described scheme, specifically involves acquiring full audio data; performing frame-by-frame transformation on the full audio data to obtain segmented spectrum data; selecting two-dimensional frequency band data that meets a preset target frequency threshold from the spectrum data; performing dimensionality reduction on the frequency band data to obtain a one-dimensional audio fingerprint of the full audio data; and performing audio fingerprint recognition based on an improved Philips audio fingerprinting method using locality-sensitive hashing. Specifically, by selecting full audio data to obtain corresponding spectrum data and selecting frequency band data that meets a preset frequency, and then performing dimensionality reduction on the frequency band data to obtain the audio fingerprint, this approach solves the technical problems of low accuracy and large data volume in audio fingerprint recognition, improves the accuracy of audio fingerprint recognition, reduces the data volume of audio fingerprints, and is thus effectively used in fields such as audio fingerprint storage and audio retrieval deduplication.

[0084] Based on, but not limited to, the terminal device architecture described above, this application proposes method embodiments.

[0085] Reference Figure 2 , Figure 2 This is a flowchart illustrating a first exemplary embodiment of the audio fingerprint recognition method of this application. The audio fingerprint recognition method includes:

[0086] Step S210: Obtain full audio data;

[0087] The subject executing the method in this embodiment can be an audio fingerprint recognition device, an audio fingerprint recognition terminal device, or a server. This embodiment takes an audio fingerprint recognition device as an example. The audio fingerprint recognition device can be integrated into terminal devices such as smartphones and tablets with data processing functions.

[0088] This embodiment primarily focuses on the identification of audio data, especially Philips audio data. Starting from the practical problems of audio fingerprint recognition, this embodiment selects complete Philips audio data and combines the dimensionality reduction capabilities of Locality Sensitive Hashing (Simhash) to design an improved Philips audio fingerprint hashing calculation method. This addresses the technical issues of low accuracy and large data volume in audio fingerprint recognition, improving accuracy and reducing data size, thus effectively enabling its application in audio fingerprint storage and audio retrieval deduplication.

[0089] This embodiment takes into account that, since the Philips audio fingerprinting method only selects a 3-second data segment from the audio for calculation, the segment fingerprint cannot represent the entire audio segment, resulting in low accuracy of audio fingerprinting. Therefore, by improving the Philips audio fingerprint hashing, specifically by selecting audio samples from segment audio to complete audio, the audio samples can cover the entire audio, thus improving the accuracy of audio fingerprinting.

[0090] In addition, by selecting full-audio data, the comprehensiveness and integration of the frequency band data selection in step S230 can be improved.

[0091] Step S220: Perform frame-by-frame transformation on the full audio data to obtain the framed spectrum data.

[0092] Specifically, the Hamming window is used to overlap and frame the selected full audio data to obtain the audio data of each frame after the full audio data is framed, which can solve the problem of partial signal interception in the Hamming window; the short-time Fourier transform is performed on each frame of audio data to obtain the spectrum data of the full audio, which can analyze the non-stationary signals in the full audio data, and then select the corresponding frequency band data for each spectrum data to improve the accuracy of audio fingerprint recognition.

[0093] Alternatively, the specific implementation steps can be as follows:

[0094] Overlapping frames are performed on the full audio data to obtain the audio data of each frame; short-time Fourier transform is performed on each frame of audio data to obtain the corresponding spectrum data.

[0095] Step S230: Select two-dimensional frequency band data that meets the preset target frequency threshold from the spectrum data;

[0096] This embodiment takes into account that in the traditional Philips audio fingerprint recognition method, only the frequency band of 300Hz-2000Hz is selected for calculation, but it does not fully cover the frequency band that the human ear is most sensitive to.

[0097] Therefore, this embodiment sets sensitive conditions and optimizes the initial frequency threshold of 300Hz-2000Hz according to the sensitive conditions to select frequency bands that meet the sensitive conditions from each spectrum data, which can basically cover the most sensitive frequency band of the human ear.

[0098] Optionally, before selecting two-dimensional frequency band data that meets a preset target frequency threshold from the spectrum data in step S230, the method further includes:

[0099] Obtain an initial frequency threshold; optimize the initial frequency threshold according to preset sensitivity conditions to obtain the target frequency threshold.

[0100] Specifically, based on the scheme of this application, this embodiment starts from the sensitive frequency band rules of the human ear in the real world, constructs a real-world sensitive condition, and verifies the effectiveness of the audio fingerprint recognition method proposed in this application under this sensitive condition. Finally, the accuracy and data volume of the audio fingerprints identified by the method of this application are significantly improved.

[0101] In this application embodiment, it is preferred to extend the frequency threshold in the sensitive conditions to 300Hz-3000Hz. In other embodiments, the sensitive conditions may include other frequency thresholds. That is, by selecting the frequency band data in the 300Hz-3000Hz range from the obtained spectrum data (expanding the original 300Hz-2000Hz to 300Hz-3000Hz), the sensitive frequency band of the human ear can be basically covered.

[0102] Step S240: Perform a dimensionality reduction operation on the frequency band data to obtain a one-dimensional audio fingerprint of the full audio data.

[0103] Specifically, when calculating the similarity between two audio samples, an improved Philips audio fingerprinting method based on Locality Sensitive Hashing (LSH) is used to obtain one-dimensional audio fingerprints for each sample. The bit error rate (BER) of each fingerprint is then calculated to determine whether the two audio segments are similar. For example, fingerprint hashing is performed on 500 collected audio samples, and 50 randomly selected audio samples are downsampled and included in the calculation. Based on the calculation results, 50 randomly selected audio samples with the same pronunciation but different sampling rates can be accurately identified.

[0104] This embodiment, through the above-described scheme, specifically involves acquiring full audio data; performing frame-by-frame transformation on the full audio data to obtain segmented spectrum data; selecting two-dimensional frequency band data that meets a preset target frequency threshold from the spectrum data; performing dimensionality reduction on the frequency band data to obtain a one-dimensional audio fingerprint of the full audio data; and performing audio fingerprint recognition based on an improved Philips audio fingerprinting method using locality-sensitive hashing. Specifically, by selecting full audio data to obtain corresponding spectrum data and selecting frequency band data that meets a preset frequency, and then performing dimensionality reduction on the frequency band data to obtain the audio fingerprint, this approach solves the technical problems of low accuracy and large data volume in audio fingerprint recognition, improves the accuracy of audio fingerprint recognition, reduces the data volume of audio fingerprints, and is thus effectively used in fields such as audio fingerprint storage and audio retrieval deduplication.

[0105] Reference Figure 3 , Figure 3 for Figure 2 A detailed flowchart of step S230 in the embodiment. Based on the above... Figure 2 In the embodiment shown, step S220 involves performing frame-by-frame transformation on the full audio data to obtain the framed spectral data, including:

[0106] Step S310: Overlap frame segmentation is performed on the full audio data to obtain the audio data of each frame of the full audio data;

[0107] This embodiment takes into account that audio signals are time-varying, with minimal characteristic changes within a short time range, and are therefore treated as steady-state signals; however, beyond this short time range, the audio signal will change. Therefore, by reframing the full audio data using a Hamming window, the problem of partially intercepted signals within the Hamming window can be resolved.

[0108] Further, in step S310, the full audio data is subjected to overlapping frame segmentation to obtain audio data of each frame of the full audio data, including:

[0109] Step S311: Determine the calculation elements of the full audio data, wherein the calculation elements include one or more of the total length of the full audio data, the frame rate, and the fingerprint length of the audio fingerprint;

[0110] Specifically, computational elements are used to calculate the full audio data to obtain an audio fingerprint of the full audio data. Using one or more computational elements can improve the accuracy of audio fingerprint recognition. The specific values ​​of the computational elements can be determined by historical data or by the user based on actual conditions. In this embodiment, full audio data A is selected for fingerprint calculation, with a total length of N, a frame rate of F samples per second, and a fingerprint hash length L is determined.

[0111] Step S312: Based on the calculation elements, the full audio data is divided into overlapping frames to obtain the audio data of each frame of the full audio data.

[0112] Specifically, the selected full audio is framed using a Hamming window, with each frame being L × 0.01 seconds long. Let T be the length of each frame's audio data. Then:

[0113] T = 0.01 × L × F

[0114] When the Hamming window moves across the full audio data, it maintains an overlap of (L-1) / L between the previous and subsequent frames to ensure that the subsequent frame is offset by only 0.01 seconds from the previous frame. The formula for calculating the Hamming window, W(n), is:

[0115] W(n)=0.54-0.46×cos(2×π×n / (T-1)),0≤n≤T-1

[0116] That is, the audio data H after frame division is:

[0117] H(f, n) = A(n) × W(n)

[0118] Where H(f,n) represents the value of the nth audio data in the fth frame, and 0≤f≤N / T, 0≤n≤T-1.

[0119] Step S320: Perform short-time Fourier transform on the audio data to obtain the corresponding spectrum data.

[0120] Specifically, a short-time Fourier transform is applied to each frame of data, and the result is V:

[0121] V(f)=fft(H(f))

[0122] Where fft is the short-time Fourier transform, and H(f) represents the f-th frame.

[0123] Calculate the spectral data P (absolute value / frame length) of the full audio data A:

[0124] P(f,i)=abs(V(f,i)) / T

[0125] Where abs is the absolute value calculation, and P(f,i) is the value of the i-th element in the f-th frame.

[0126] This embodiment, through the above-described scheme, specifically determines the computational elements of the full audio data. These computational elements include one or more of the total length of the full audio data, the frame rate, and the fingerprint length of the audio fingerprint. Based on these computational elements, the full audio data is divided into overlapping frames to obtain each frame of audio data. Short-time Fourier transforms are then performed on each frame of audio data to obtain the corresponding spectral data. By determining the computational elements of the full audio data, obtaining each frame of audio data, and performing short-time Fourier transforms, the accuracy of audio fingerprint recognition can be improved.

[0127] Reference Figure 4 , Figure 4 for Figure 2 A schematic flowchart of step S240 in this embodiment. This embodiment is based on the above. Figure 2 as well as Figure 3 In the embodiment shown, step S240 involves performing a dimensionality reduction operation on the frequency band data to obtain a one-dimensional audio fingerprint of the full audio data, including:

[0128] Step S410: Overlapping framing and energy value calculation are performed on the frequency band data to obtain a two-dimensional fingerprint matrix;

[0129] This embodiment takes into account that the Philips audio fingerprint recognition method calculates a large number of fingerprints in a two-dimensional array of 8192 (256*32), resulting in a large data volume that is unsuitable for storage and retrieval. Therefore, this embodiment incorporates the concept of Locality Sensitive Hashing (Simhash), widely used in text processing, and introduces the calculation method of Simhash into audio fingerprint hash calculation. This facilitates storage and retrieval, improving deduplication and retrieval efficiency in practical data deduplication and retrieval.

[0130] Further, in step S410, overlapping framing and energy value calculation are performed on the frequency band data to obtain a two-dimensional fingerprint matrix, including:

[0131] Step S411: Perform overlapping framing on the frequency band data to select non-overlapping frequency band data;

[0132] Specifically, the data S(f) is divided into frames, as described in step S312 above, and will not be repeated here. Then, S(f) is divided into L+1 frames of data without overlap. Let Q be the result of S(f) after framing, that is, Q(f,m) represents the m-th frame result after S(f) is divided into frames.

[0133] Step S412: Calculate the corresponding energy values ​​for the non-overlapping frequency band data respectively;

[0134] Specifically, the energy value E of each frame of data is calculated separately, that is, the sum of its squares is obtained:

[0135]

[0136] Where n represents the amount of data in each frame after S(f) is divided into frames, 0≤f≤N / T, 0≤m≤L+1;

[0137] Step S413: Calculate the energy value and the non-overlapping frequency band data according to a preset formula to obtain the fingerprint matrix.

[0138] The corresponding fingerprint matrix Fp is calculated using the following formula:

[0139]

[0140] Step S420: Convert the fingerprint matrix to obtain the audio fingerprint.

[0141] This application takes into account that fingerprint matrices are two-dimensional matrices, which are not easy to store and retrieve. Therefore, by converting the two-dimensional fingerprint matrix into a one-dimensional matrix, and then obtaining the corresponding one-dimensional audio fingerprint, the technical problem of large data volume of audio fingerprints can be solved.

[0142] Further, step S420, converting the fingerprint matrix to obtain the audio fingerprint, includes:

[0143] Step S421: Sum the fingerprint matrix vertically to obtain a one-dimensional weight matrix;

[0144] Specifically, the weight matrix W is updated based on the fingerprint matrix Fp. That is, by summing the fingerprint matrix vertically, the two-dimensional fingerprint matrix Fp is transformed into a one-dimensional weight matrix W.

[0145]

[0146] Step S422: Calculate the weight matrix according to the preset symbol function to obtain the audio fingerprint.

[0147] Specifically, the weight matrix W is calculated using a sign function to obtain an audio fingerprint of length L:

[0148]

[0149] Here, Hash represents a hash function.

[0150] This embodiment, through the above-described scheme, specifically involves overlapping and framing the frequency band data to select non-overlapping frequency band data; calculating the corresponding energy values ​​for each of the non-overlapping frequency band data; calculating the fingerprint matrix using the energy values ​​and the non-overlapping frequency band data according to a preset formula; summing the fingerprint matrix vertically to obtain a one-dimensional weight matrix; and calculating the weight matrix according to a preset sign function to obtain the audio fingerprint. By calculating the non-overlapping frequency band data to obtain a two-dimensional fingerprint matrix, and then converting the two-dimensional fingerprint matrix into a one-dimensional audio fingerprint, the technical problem of large data volume in audio fingerprints can be solved, thereby improving the deduplication and retrieval efficiency of audio fingerprint hashing in practical data deduplication and retrieval.

[0151] Reference Figure 5 , Figure 5 This is a schematic diagram of the overall process in the embodiments of this application, such as... Figure 5 As shown, the specific steps may include:

[0152] ① Select full audio data A to participate in fingerprint calculation with a length of N, a frame rate of F samples per second, and determine the fingerprint hash length L;

[0153] ② Using a Hamming window, the selected audio is divided into overlapping frames, each frame being L × 0.01 seconds long. Let the data length of each frame be T. Then:

[0154] T = 0.01 × L × F

[0155] When the Hamming window moves across the audio, it maintains an overlap of (L-1) / L between the previous and subsequent frames to ensure that the subsequent frame is offset by only 0.01 seconds from the previous frame. The formula for calculating the Hamming window is as follows:

[0156] W(n)=0.54-0.46×cos(2×π×n / (T-1)),0≤n≤T-1

[0157] That is, the result H after framing is:

[0158] H(f, n) = A(n) × W(n)

[0159] Where H(f,n) represents the value of the nth data in the fth frame, and 0≤f≤N / T, 0≤n≤T-1;

[0160] ③ Apply a Fourier transform to each frame of data, and let the result be V. Then:

[0161] V(f)=fft(H(f))

[0162] Where fft is the short-time Fourier transform, H(f) represents the f-th frame, and its spectrum P (absolute value / frame length) is calculated:

[0163] P(f,i)=abs(V(f,i)) / T

[0164] Where abs is the absolute value calculation, and P(f,i) is the value of the i-th element in the f-th frame;

[0165] ④ Select data S(f) of the obtained spectrum P(f) with a frequency in the 300Hz-3000Hz band (expanding the original 300Hz-2000Hz to 300Hz-3000Hz);

[0166] ⑤ Perform a frame division operation on the data S(f), as in 2), and divide it into L+1 frames of data without overlap. Let the result of S(f) after frame division be Q, that is, Q(f,m) represents the m-th frame result after S(f) is framed.

[0167] ⑥ Calculate the energy value E for each frame of data, i.e., find its sum of squares:

[0168]

[0169] Where n represents the amount of data in each frame after S(f) is divided into frames, and 0≤f≤N / T, 0≤m≤L+1;

[0170] ⑦ According to the following formula:

[0171]

[0172] Calculate the corresponding fingerprint matrix Fp;

[0173] ⑧ Update the weight matrix W according to the Fp matrix:

[0174]

[0175] ⑨ Calculate the W matrix using the sign function to obtain the L-bit hash fingerprint:

[0176]

[0177] Furthermore, this application also proposes an audio fingerprint recognition device, which includes:

[0178] The audio acquisition module is used to acquire full audio data;

[0179] The audio framing module is used to perform framing transformation on the full audio data to obtain the spectral data after framing.

[0180] The frequency band selection module is used to select two-dimensional frequency band data that meets a preset target frequency threshold from the spectrum data;

[0181] The fingerprint extraction module is used to perform dimensionality reduction on the frequency band data to obtain a one-dimensional audio fingerprint of the full audio data.

[0182] The principle and implementation process of audio fingerprint recognition in this embodiment are described in the above embodiments and will not be repeated here.

[0183] Furthermore, this application also proposes a terminal device, which includes a memory, a processor, and an audio fingerprint recognition program stored in the memory and executable on the processor. When the audio fingerprint recognition program is executed by the processor, it implements the steps of the audio fingerprint recognition method as described above.

[0184] Since this audio fingerprint recognition program employs all the technical solutions of all the aforementioned embodiments when executed by the processor, it possesses at least all the beneficial effects brought about by all the technical solutions of all the aforementioned embodiments, which will not be elaborated upon here.

[0185] Furthermore, embodiments of this application also propose a computer-readable storage medium storing an audio fingerprint recognition program, which, when executed by a processor, implements the steps of the audio fingerprint recognition method as described above.

[0186] Since this audio fingerprint recognition program employs all the technical solutions of all the aforementioned embodiments when executed by the processor, it possesses at least all the beneficial effects brought about by all the technical solutions of all the aforementioned embodiments, which will not be elaborated upon here.

[0187] Compared to existing technologies, the audio fingerprinting method, apparatus, terminal device, and storage medium proposed in this application acquire full audio data; perform frame-by-frame transformation on the full audio data to obtain each spectral data after frame division; select two-dimensional frequency band data that meets a preset target frequency threshold from the spectral data; perform dimensionality reduction on the frequency band data to obtain a one-dimensional audio fingerprint of the full audio data; and improve the Philips audio fingerprinting method based on locality-sensitive hashing for audio fingerprinting. That is, by selecting full audio data to obtain corresponding spectral data and selecting frequency band data that meets a preset frequency, and then performing dimensionality reduction on the frequency band data to obtain the audio fingerprint, the technical problems of low accuracy and large data volume in audio fingerprinting can be solved, thereby improving the accuracy of audio fingerprinting and reducing the data volume of audio fingerprints, and thus effectively used in audio fingerprint storage and audio retrieval deduplication fields.

[0188] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0189] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0190] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, controlled terminal, or network device, etc.) to execute the methods of each embodiment of this application.

[0191] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. An audio fingerprint recognition method, characterized in that, The audio fingerprint recognition method includes the following steps: Obtain full audio data; The full audio data is subjected to frame-by-frame transformation to obtain the spectrum data of each frame; From the spectrum data, select two-dimensional frequency band data that meets the preset target frequency threshold; The frequency band data is subjected to dimensionality reduction to obtain a one-dimensional audio fingerprint of the full audio data; The step of performing dimensionality reduction on the frequency band data to obtain a one-dimensional audio fingerprint of the full audio data includes: The frequency band data is subjected to overlapping framing and energy value calculation to obtain a two-dimensional fingerprint matrix; The fingerprint matrix is ​​transformed to obtain the audio fingerprint.

2. The audio fingerprint recognition method as described in claim 1, characterized in that, The steps of performing overlapping framing and energy value calculation on the frequency band data to obtain a two-dimensional fingerprint matrix include: The frequency band data is subjected to overlapping framing to select non-overlapping frequency band data; The corresponding energy values ​​are obtained by calculating the non-overlapping frequency band data respectively; The fingerprint matrix is ​​obtained by calculating the energy value and the non-overlapping frequency band data according to a preset formula.

3. The audio fingerprint recognition method as described in claim 1, characterized in that, The step of converting the fingerprint matrix to obtain the audio fingerprint includes: The fingerprint matrix is ​​summed vertically to obtain a one-dimensional weight matrix; The audio fingerprint is obtained by calculating the weight matrix according to the preset symbol function.

4. The audio fingerprint recognition method as described in claim 1, characterized in that, Before the step of selecting two-dimensional frequency band data that meets the preset target frequency threshold from the spectrum data, the method further includes: Obtain the initial frequency threshold; The initial frequency threshold is optimized based on preset sensitivity conditions to obtain the target frequency threshold.

5. The audio fingerprint recognition method as described in claim 1, characterized in that, The step of performing frame-by-frame transformation on the full audio data to obtain the framed spectrum data includes: The full audio data is divided into overlapping frames to obtain the audio data of each frame of the full audio data. The audio data is subjected to short-time Fourier transform to obtain the corresponding spectrum data.

6. The audio fingerprint recognition method as described in claim 5, characterized in that, The step of performing overlapping frame segmentation on the full audio data to obtain the audio data of each frame of the full audio data includes: Determine the computational elements of the full audio data, wherein the computational elements include one or more of the total length of the full audio data, the frame rate, and the fingerprint length of the audio fingerprint; Based on the aforementioned computational elements, the full audio data is subjected to overlapping frame segmentation to obtain the audio data of each frame of the full audio data.

7. An audio fingerprint recognition device, characterized in that, The audio fingerprint recognition device includes: The audio acquisition module is used to acquire full audio data; The audio framing module is used to perform framing transformation on the full audio data to obtain the spectral data after framing. The frequency band selection module is used to select two-dimensional frequency band data that meets a preset target frequency threshold from the spectrum data; The fingerprint extraction module is used to perform dimensionality reduction on the frequency band data to obtain a one-dimensional audio fingerprint of the full audio data. The step of performing dimensionality reduction on the frequency band data to obtain a one-dimensional audio fingerprint of the full audio data includes: The frequency band data is subjected to overlapping framing and energy value calculation to obtain a two-dimensional fingerprint matrix; The fingerprint matrix is ​​transformed to obtain the audio fingerprint.

8. A terminal device, characterized in that, The terminal device includes a memory, a processor, and an audio fingerprint recognition program stored in the memory and executable on the processor. When the audio fingerprint recognition program is executed by the processor, it implements the steps of the audio fingerprint recognition method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores an audio fingerprint recognition program, which, when executed by a processor, implements the steps of the audio fingerprint recognition method as described in any one of claims 1-6.

Images