Audio fingerprinting system based on spatio-temporal self-attention

Abstract

The application discloses an audio fingerprint identification system based on space-time self-attention, and relates to the technical field of audio information processing, and specifically comprises the following modules: an audio preprocessing module, a space-time decomposition module, a feature extraction module, a feature fusion module, a space-time reconstruction module and a fingerprint identification module; the application carries out preprocessing on the audio data to be identified through the audio preprocessing module; the space-time decomposition module decomposes the preprocessed audio data into a plurality of sub-fingerprint segments; the feature extraction module extracts features from each sub-fingerprint segment; the feature fusion module fuses the features of the plurality of sub-fingerprint segments; the space-time reconstruction module reconstructs the fused fingerprint signals in space-time by using a space-time self-attention mechanism; and the fingerprint calculation module calculates the audio fingerprint based on the reconstructed signals and carries out matching search; the accuracy of audio fingerprint identification is improved, and the problems of inaccurate identification and low search efficiency when processing complex audio data are effectively solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of audio information processing, in particular to an audio fingerprint identification system based on spatiotemporal self-attention. BACKGROUND

[0002] With the wide application of digital audio information, audio fingerprint identification, as an efficient, lossless and robust identity and content identification technology, has gradually become a key tool in the fields of music copyright protection, content retrieval and security monitoring. In order to improve the robustness and applicability of identification, recent research has developed from single time domain or frequency domain feature extraction to multi-modal, multi-scale and spatiotemporal joint modeling.

[0003] The prior art has the following disadvantages:

[0004] (1) The pre-processing and noise reduction strategy relies on static fixed parameters, which is difficult to adapt to different shooting environments and channel differences, resulting in loss of fingerprint information in low signal-to-noise ratio scenarios;

[0005] (2) The segmentation of sub-fingerprint segments is still limited in time alignment and dynamic distortion processing, and the traditional dynamic time warping has limited effect in extreme time stretching or compression;

[0006] (3) Feature extraction is mainly offline frame-level calculation, which lacks real-time performance, and lacks adaptive ability for identification and removal of special audio such as background music and prompt tone;

[0007] (4) The weight allocation in the fingerprint fusion stage relies on manual or simple attention weights, which is difficult to fully reflect the relative importance of different sub-fingerprints in identification, and is easy to cause information redundancy or important information to be weakened;

[0008] (5) The robustness of the reconstruction and matching stage needs to be improved, and the existing method has large fluctuations in identification rate in strong noise, high proportion of silent segments, and low similarity scenarios, and the scalability and retrieval efficiency of the existing fingerprint library still need to be further optimized.

[0009] The above information disclosed in the background section is only used to strengthen the understanding of the background of the present disclosure, and therefore it can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0010] The purpose of the present application is to provide an audio fingerprint identification system based on spatiotemporal self-attention to solve the problems in the background art.

[0011] In order to achieve the above object, the present application provides the following technical scheme: an audio fingerprint identification system based on space-time self-attention, specifically comprising the following modules: an audio preprocessing module, a space-time decomposition module, a feature extraction module, a feature fusion module, a space-time reconstruction module, and a fingerprint identification module;

[0012] The audio preprocessing module: pre-processes the audio data to be identified;

[0013] The space-time decomposition module: decomposes the pre-processed audio data into a plurality of sub-fingerprint segments;

[0014] The feature extraction module: extracts features from each sub-fingerprint segment, including a frequency domain transformation unit, a Bark domain transformation unit, an energy calculation unit, and a self-attention mechanism unit;

[0015] The feature fusion module: fuses the features of the plurality of sub-fingerprint segments, including a splicing fusion unit and a weighted fusion unit;

[0016] The space-time reconstruction module: reconstructs the fused fingerprint signal in space-time using a space-time self-attention mechanism, including a time reconstruction unit, a frequency reconstruction unit, and a signal reconstruction unit;

[0017] The fingerprint identification module: calculates the audio fingerprint based on the reconstructed signal, including a Hamming distance calculation unit, a matching degree calculation unit, and a sorting and retrieval unit.

[0018] As a preferred scheme of the audio fingerprint identification system based on space-time self-attention according to the present application, wherein:

[0019] Receiving audio data to be identified;

[0020] For each received channel, in each sliding time window, calculating an estimated value of the direct current component of the audio data of the current channel;

[0021] For the audio data of the current time point of each received channel, performing a mean removal operation based on the corresponding estimated value of the direct current component, and dynamically updating the data set at different time points through a sliding window;

[0022] Judging and removing the silent segment of the audio data based on the periodic change of phonemes;

[0023] Identifying and removing special audio data, including background music and prompt tone, through spectral analysis;

[0024] Retaining the audio components between 300Hz and 8000Hz through a band-pass filter;

[0025] Estimating the power spectrum of the noise from the silent segment, and subtracting the noise spectrum from the audio data;

[0026] The audio data was resampled to the standard sampling rate of 44.1kHz.

[0027] As a preferred embodiment of the audio fingerprint recognition system based on spatiotemporal self-attention described in this invention, wherein:

[0028] The preprocessed audio data is divided into 3-second sub-fingerprint segments by decomposing it over a fixed time period, with an overlap rate of 50% between adjacent segments.

[0029] As a preferred embodiment of the audio fingerprint recognition system based on spatiotemporal self-attention described in this invention, wherein:

[0030] Frequency domain transformation unit: The window length of the Hanning window is set to 512 points, and its frame shift is 256 points;

[0031] The audio data is divided into frames of 512 points each according to the frame shift, with the starting point advancing by 256 points each time.

[0032] A sliding Hanning window is applied to the sub-segments, Fourier coefficients are calculated for each window, and the feature vectors of several windows are combined to extract frequency domain features.

[0033] Bark domain transform unit: maps frequency domain features to the Bark frequency scale and divides them into 24 critical frequency bands;

[0034] Energy calculation unit: Calculates the energy for each frequency band of the Bark domain features and plots the energy-time curve, which reflects the energy distribution of the audio signal at different time points.

[0035] Self-attention mechanism unit: Based on the energy-time curve, the frequency domain features at different time points are weighted through the self-attention mechanism, and the attention weights are calculated based on the query matrix, key matrix, and value matrix.

[0036] As a preferred embodiment of the audio fingerprint recognition system based on spatiotemporal self-attention described in this invention, wherein:

[0037] The splicing and fusion unit directly splices the sub-fingerprint fragments in chronological order;

[0038] Weighted fusion unit: The sub-fingerprint segments are weighted and averaged according to the attention weights, and important segments are given higher weights.

[0039] As a preferred embodiment of the audio fingerprint recognition system based on spatiotemporal self-attention described in this invention, wherein:

[0040] Time reconstruction unit: Adjust the time position of each sub-fingerprint segment according to the weight coefficient to form a new time reference. This process is achieved by dynamic time warping algorithm, which can effectively align audio segments with time distortion.

[0041] a frequency reconstruction unit: inverse transform the frequency domain features based on the new time reference to obtain a time domain reconstructed signal;

[0042] a signal reconstruction unit: inverse transform the time domain reconstructed signal in the Bark domain to restore the original frequency domain characteristics to obtain a final reconstructed signal.

[0043] As a preferred scheme of the audio fingerprint recognition system based on space-time self-attention according to the application, wherein:

[0044] a Hamming distance calculation unit: convert the reconstructed signal into a binary fingerprint sequence to measure the similarity between fingerprints through Hamming distance;

[0045] During the conversion, the reconstructed signal is wavelet transformed to extract energy features, a fingerprint database is constructed, and a fast search algorithm is used to find the best matching item in the fingerprint database;

[0046] a matching degree calculation unit: calculate the matching degree score according to the Hamming distance, and judge the recognition result according to the matching degree threshold, the threshold is set to 16, and when the matching degree is lower than the threshold, it is determined as successful recognition;

[0047] a sorting and searching unit: sort all audio data to be identified in descending order according to the matching degree score, and store the Hash mapping in the database using a local sensitive Hash algorithm.

[0048] On the other hand, the application provides a computer device comprising a memory and a processor, the memory storing a computer program, wherein: the computer program is executed by the processor to realize the steps of the audio fingerprint recognition system based on space-time self-attention according to the application.

[0049] On the other hand, the application provides a computer readable storage medium having a computer program stored thereon, wherein: the computer program is executed by the processor to realize the steps of the audio fingerprint recognition system based on space-time self-attention according to the application.

[0050] In the above technical solution, the application provides the technical effects and advantages:

[0051] 1. By decomposing the fingerprint into several sub-fingerprint segments and splicing or weighted fusion according to the time dimension, a robust fingerprint against time sequence disturbance is formed, effectively solving the problem of inaccurate classification and recognition of existing technologies in processing complex and variable urban audio data, effectively identifying and processing videos with too short or cyclic playback, and improving the comprehensiveness and accuracy of audio fingerprint recognition.

[0052] 2. Through the energy-oriented spatio-temporal attention mechanism, high-energy regions are dynamically selected according to signal energy to apply attention, low-information regions are ignored, the efficiency of audio fingerprint retrieval is improved, the database volume and query time are reduced, and the demand for fast retrieval is met.

[0053] 3. The spatio-temporal self-attention mechanism can effectively handle various complex timing disturbance situations, and significantly improves the robustness of audio fingerprint identification. BRIEF DESCRIPTION OF DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments described in the present application, and other drawings can also be obtained by those skilled in the art based on these drawings.

[0055] Figure 1 The method flowchart of the audio fingerprint identification system based on the spatio-temporal self-attention of the present application.

[0056] Figure 2 The module schematic diagram of the audio fingerprint identification system based on the spatio-temporal self-attention of the present application. DETAILED DESCRIPTION

[0057] Example implementations will now be described more fully with reference to the accompanying drawings. Example implementations may, however, be implemented in many different forms and should not be construed as limited to the examples set forth herein; rather, these example implementations are provided so that this disclosure will be thorough and complete, and will fully convey the scope of example implementations to those skilled in the art.

[0058] Embodiment 1, with reference to Figure 1 and Figure 2 , the first embodiment of the present application provides an audio fingerprint identification system based on spatio-temporal self-attention, which specifically includes the following modules: an audio preprocessing module, a spatio-temporal decomposition module, a feature extraction module, a feature fusion module, a spatio-temporal reconstruction module, and a fingerprint identification module;

[0059] Audio pre-processing module: pre-process the audio data to be identified;

[0060] Receiving audio data to be identified;

[0061] For each received channel, in each sliding time window, calculate the estimated value of the direct current component of the audio data of the current channel;

[0062] For the audio data of the current time point of each received channel, based on the estimated value of the corresponding direct current component, perform a mean value operation, and dynamically update the data set at different time points through a sliding window;

[0063] Periodic variation of phonemes is used to determine and remove the silent section of audio data;

[0064] Special audio data, including background music and prompt tone, is identified and removed through spectral analysis;

[0065] A band-pass filter is used to retain audio components between 300 Hz and 8000 Hz;

[0066] The power spectrum of noise is estimated according to the silent section, and the noise spectrum is subtracted from the audio data;

[0067] The audio data is resampled to the standard sampling rate of 44.1 kHz.

[0068] Temporal-spatial decomposition module: the preprocessed audio data is decomposed into several sub-fingerprint segments;

[0069] The preprocessed audio data is equally divided into 3-second sub-fingerprint segments by fixed-time-length decomposition, and the overlap rate between adjacent segments is set to 50%.

[0070] Feature extraction module: feature extraction is performed on each sub-fingerprint segment, including frequency domain transformation unit, Bark domain transformation unit, energy calculation unit and self-attention mechanism unit;

[0071] Frequency domain transformation unit: the window length of the Hanning window is set to 512 points, and the frame shift is 256 points;

[0072] The audio data is divided into frames with a length of 512 points according to the frame shift, and the starting point advances 256 points each time;

[0073] The sliding Hanning window is applied to the sub-segment, and the Fourier coefficient is calculated for each window. The frequency domain features are extracted by combining the feature vectors of several windows;

[0074] Bark domain transformation unit: the frequency domain features are mapped to the Bark frequency scale and divided into 24 critical frequency bands;

[0075] Energy calculation unit: the energy of each frequency band of the Bark domain features is calculated, and the energy-time curve is drawn to reflect the energy distribution of the audio signal at different time points;

[0076] Self-attention mechanism unit: based on the energy-time curve, the frequency domain features at different time points are weighted through the self-attention mechanism, and the attention weight is calculated according to the query matrix, key matrix and value matrix.

[0077] Feature fusion module: the features of multiple sub-fingerprint segments are fused, including splicing fusion unit and weighted fusion unit;

[0078] Splicing fusion unit: the sub-fingerprint segments are spliced directly in time point order;

[0079] Weighted fusion unit: weighted average of sub-fingerprint segments according to attention weights, giving important segments higher weights.

[0080] Spacetime reconstruction module: spacetime self-attention mechanism is used to reconstruct the fused fingerprint signal, including time reconstruction unit, frequency reconstruction unit and signal reconstruction unit;

[0081] Time reconstruction unit: adjust the time position of each sub-fingerprint segment according to the weight coefficient to form a new time reference. This process is realized by dynamic time warping algorithm, which can effectively align the audio segments with time distortion;

[0082] Frequency reconstruction unit: inverse transform of frequency domain features based on the new time reference to obtain time domain reconstruction signal;

[0083] Signal reconstruction unit: inverse Bark domain transform of time domain reconstruction signal to restore the original frequency domain characteristics to obtain the final reconstruction signal.

[0084] Fingerprint identification module: calculate audio fingerprint based on reconstruction signal, including hamming distance calculation unit, matching degree calculation unit and sorting retrieval unit;

[0085] Hamming distance calculation unit: convert the reconstruction signal into a binary fingerprint sequence and measure the similarity between fingerprints by Hamming distance;

[0086] During the conversion, wavelet transform is performed on the reconstruction signal to extract energy features, construct a fingerprint database, and find the best matching item in the fingerprint database through a fast search algorithm;

[0087] Matching degree calculation unit: calculate matching degree score according to Hamming distance, and judge the recognition result according to matching degree threshold. The threshold is set to 16. When the matching degree is lower than this threshold, it is determined as successful recognition;

[0088] Sorting retrieval unit: sort all audio data to be identified in descending order according to matching degree score, and store in the database by Hash mapping using local sensitive Hash algorithm.

[0089] Embodiment 2, as a second embodiment of the present application, provides a spatio-temporal self-attention-based audio fingerprinting system, which receives audio data to be identified, supports input of audio files in multiple formats, including but not limited to WAV, MP3, and AAC formats, and is equipped with a real-time audio acquisition function to directly acquire environmental audio signals through a microphone device. The audio preprocessing module performs noise reduction processing on the audio signal, uses an adaptive filtering algorithm to eliminate background noise, and improves the signal-to-noise ratio of the signal. Then, the audio signal is resampled to convert audio of different sampling rates to a standard sampling rate of 44.1 kHz, and the long audio is cut into a plurality of overlapping short time segments by a space-time decomposition module, each segment being 3 seconds long, and adjacent segments having an overlap rate of 50%.

[0090] In a preferred embodiment, the feature extraction module further includes a multi-scale feature fusion unit that simultaneously extracts local detail features and global structure features of the audio through a multi-scale convolutional neural network, and fuses features of different scales through a feature pyramid structure to enhance the expression ability of the features.

[0091] The self-attention mechanism unit is used to capture the long-range dependence of the frequency domain features in the time and frequency dimensions, and the double-flow self-attention mechanism is used to process the feature dependence in the time dimension and the frequency dimension, respectively. By calculating the correlation between different time frames, the time sequence pattern of the audio is captured.

[0092] By calculating the correlation between different frequency channels, the frequency spectrum structure of the audio is captured. The outputs of the two self-attention units are integrated through an adaptive fusion mechanism to form a spatio-temporal joint representation.

[0093] In a preferred embodiment, the self-attention mechanism unit adopts a multi-head attention mechanism, and 8 attention heads are used to calculate the attention weights of different subspaces in parallel, with each attention head having a dimension of 64. The outputs of the multi-head attention are combined into the original dimension through linear transformation.

[0094] In a preferred embodiment, the fingerprinting module further includes a fingerprint generation unit for converting the enhanced features into compact audio fingerprints. A binary fingerprint is converted from the continuous features through a binarization unit, an adaptive threshold method is used for feature binarization, the binary features are grouped and encoded into integer hash values through a block encoding unit, each block has a size of 8 bits, adjacent blocks overlap by 4 bits, the hash values are organized into a time sequence fingerprint sequence through a fingerprint organization unit, and timestamp information is added to form a complete binary fingerprint sequence representation.

[0095] In a preferred embodiment, the fingerprint generation further includes fingerprint enhancement, by decomposing the fingerprint into several sub-fingerprint segments, and splicing and fusing in time dimension, to form a robust fingerprint against timing disturbance. For each 3-second audio segment, a 256-bit binary fingerprint is generated, and the time interval between adjacent fingerprints is 1.5 seconds.

[0096] The retrieval matching module is used to retrieve the results matched with the input audio fingerprint in the fingerprint database. Based on the multi-level retrieval strategy, the candidate matching items are quickly screened through the locality-sensitive hashing algorithm, and the Hamming distance between the fingerprint sequences is calculated through the exact matching algorithm to determine the best matching result.

[0097] The retrieval matching module further includes a time alignment unit, which realizes accurate positioning of the audio segment by analyzing the timestamp information of the fingerprint sequence.

[0098] In a preferred embodiment, the retrieval matching module adopts a distributed index structure, divides the fingerprint database into multiple sub-indices, and performs retrieval operations in parallel, which significantly improves the retrieval efficiency; the supported fingerprint database size can reach 100 million fingerprint records, and the average retrieval time is less than 100 milliseconds.

[0099] The present application also includes an audio fingerprint database for storing pre-extracted audio fingerprints and related metadata;

[0100] The database adopts a hierarchical storage structure, including a fingerprint index layer, a fingerprint data layer and a metadata layer; the fingerprint index layer stores the hash index of the fingerprint, which is used for fast retrieval;

[0101] The fingerprint data layer stores the complete fingerprint sequence;

[0102] The metadata layer stores the description information related to the audio.

[0103] Experiments show that under various complex environments and audio deformation conditions (including background noise, audio editing, speed change, pitch change, etc.), high recognition accuracy can be maintained, and the average recognition accuracy is more than 95%, which is about 15% higher than that of the traditional audio fingerprint recognition system. At the same time, the average retrieval time of the system is less than 100 milliseconds, which meets the real-time recognition requirements.

[0104] Embodiment 3 is a third embodiment of the present application, which provides an audio fingerprint recognition system based on spatio-temporal self-attention. The audio fingerprint retrieval adopts an energy-oriented spatio-temporal attention mechanism, and the specific implementation is as follows: the audio fingerprint retrieval method includes the following steps:

[0105] Step one: Receive the audio data to be searched, sample and frame the audio data, set the sampling rate to 44.1 kHz, the frame length to 512 sampling points, and the frame shift to 256 sampling points. Then perform a short-time Fourier transform on each frame of signal to obtain the frequency domain features. Convert the frequency domain features of the audio data into a time-frequency spectrogram, where the horizontal axis represents time and the vertical axis represents frequency, and the values on the spectrogram represent the energy of the corresponding time-frequency point.

[0106] Step two: Energy analysis and region division. Perform energy analysis on the time-frequency spectrogram to calculate the energy value of each time-frequency point. The energy calculation uses the square of the signal amplitude as the energy indicator. According to the energy distribution, divide the time-frequency spectrogram into high-energy regions and low-energy regions. The division criterion is based on the energy threshold, which can be dynamically adjusted, and the initial setting is 1.5 times the average energy of the time-frequency spectrogram.

[0107] Step three: For the identified high-energy regions, apply a spatio-temporal attention mechanism for feature enhancement. The spatio-temporal attention mechanism specifically includes two branches: time dimension attention and frequency dimension attention. The time dimension attention calculates the correlation between different time frames, and the frequency dimension attention calculates the correlation between different frequency bins. For low-energy regions, reduce their weight in the feature extraction process or completely ignore them to reduce computational complexity.

[0108] Step four: Audio fingerprint index construction and storage. Perform quantization processing on the extracted audio fingerprint feature vector, map the feature vector to a hash space using the Locality-Sensitive Hashing (LSH) algorithm, and generate a binary fingerprint code. Construct an inverted index structure to organize similar fingerprint codes together, thereby speeding up the subsequent retrieval process. The index structure uses a tree structure for storage, and each leaf node contains a set of audio fingerprints with similar features.

[0109] Step five: For the query audio, perform steps one to three to obtain the fingerprint features of the query audio. Use the constructed index structure to search for the most similar candidates to the query fingerprint in the database. The similarity calculation uses the Hamming distance, and the distance threshold is set to 16, i.e., fingerprints with a Hamming distance less than or equal to 16 are considered to be matched. Finally, perform timing verification on the candidate matching items to confirm the final matching result. Through the energy-oriented spatio-temporal attention mechanism, the system focuses on processing regions with large amounts of information in the audio signal, while simplifying or ignoring regions with less information. Experiments show that compared with traditional audio fingerprint retrieval methods, the system reduces the database storage space by 35% and shortens the retrieval time by 42% while maintaining the same recognition accuracy (about 98.5%). In particular, in a noisy environment (signal-to-noise ratio of 5 dB), the recognition accuracy of this method can still be maintained at more than 92%, while the traditional method decreases to about 85%.

[0110] In another preferred embodiment, the weight distribution of the spatio-temporal attention mechanism adopts a nonlinear function mapping, so that the energy and the attention weight present a logarithmic relationship, rather than a simple linear relationship. This nonlinear mapping can better handle the energy dynamic range in the audio signal, improve the feature extraction capability in the medium energy area, and thus further improve the recognition accuracy while maintaining the retrieval efficiency.

[0111] The above merely describes certain exemplary embodiments of the present application by way of illustration, and it is needless to say that those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present application. Therefore, the above drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present application.

Claims

1. An audio fingerprint recognition system based on spatiotemporal self-attention, characterized in that, Specifically, it includes the following modules: audio preprocessing module, spatiotemporal decomposition module, feature extraction module, feature fusion module, spatiotemporal reconstruction module, and fingerprint recognition module; Audio preprocessing module: preprocesses the audio data to be recognized; Spatiotemporal decomposition module: decomposes the preprocessed audio data into several sub-fingerprint segments; Feature extraction module: performs feature extraction for each sub-fingerprint segment, including frequency domain transformation unit, Bark domain transformation unit, energy calculation unit and self-attention mechanism unit; The frequency domain transformation unit applies a sliding Hanning window to the sub-segments, calculates Fourier coefficients for each window, and combines the feature vectors of several windows to extract frequency domain features. The Bark domain transform unit maps the frequency domain features to the Bark frequency scale and divides them into 24 critical frequency bands. The energy calculation unit calculates the energy for each frequency band of the Bark domain feature and plots an energy-time curve, which reflects the energy distribution of the audio signal at different time points. The self-attention mechanism unit is based on the energy-time curve and uses a self-attention mechanism to weight the frequency domain features at different time points, and calculates the attention weights based on the query matrix, key matrix, and value matrix. Feature fusion module: fuses features from multiple sub-fingerprint fragments, including a splicing fusion unit and a weighted fusion unit; Spatiotemporal reconstruction module: The spatiotemporal self-attention mechanism is used to reconstruct the fused fingerprint signal in a spatiotemporal manner, including a time reconstruction unit, a frequency reconstruction unit and a signal reconstruction unit; The time reconstruction unit adjusts the time position of each sub-fingerprint segment according to the weighting coefficient to form a new time reference; The frequency reconstruction unit performs an inverse transformation on the frequency domain features based on a new time reference to obtain a time-domain reconstructed signal. The signal reconstruction unit performs an inverse Bark domain transform on the time-domain reconstructed signal to restore the original frequency domain characteristics and obtain the final reconstructed signal. Fingerprint recognition module: Calculates audio fingerprints based on reconstructed signals, including a Hamming distance calculation unit, a matching degree calculation unit, and a sorting and retrieval unit.

2. The audio fingerprint recognition system based on spatiotemporal self-attention according to claim 1, characterized in that: Receive audio data to be identified; For each receiving channel, within each sliding time window, calculate an estimate of the DC component of the audio data for the current channel; For the audio data at the current time point of each receiving channel, a mean-reduction operation is performed based on the estimated value of the corresponding DC component, and the data set is dynamically updated at different time points through a sliding window; Based on the periodic changes of phonemes, identify and remove silent segments from audio data; Identify and remove special audio data, including background music and prompts, through spectrum analysis; Preserve audio components between 300Hz and 8000Hz by using a bandpass filter; The power spectrum of the noise is estimated based on the silent segment, and the noise spectrum is subtracted from the audio data; The audio data was resampled to the standard sampling rate of 44.1kHz.

3. The audio fingerprint recognition system based on spatiotemporal self-attention according to claim 1, characterized in that: Hamming distance calculation unit: converts the reconstructed signal into a binary fingerprint sequence, and measures the similarity between fingerprints using Hamming distance; During the conversion process, wavelet transform is performed on the reconstructed signal to extract energy features, a fingerprint database is constructed, and the best match is found in the fingerprint database using a fast search algorithm. Matching score calculation unit: Calculates the matching score based on Hamming distance, and judges the recognition result based on the matching score threshold. The threshold is set to 16. When the matching score is lower than this threshold, it is judged as a successful recognition. The sorting and retrieval unit sorts all audio data to be identified in descending order according to their matching scores and stores them in the database using a locality-sensitive hashing algorithm.

4. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements a module of the audio fingerprint recognition system based on spatiotemporal self-attention as described in any one of claims 1 to 3.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the module of the audio fingerprint recognition system based on spatiotemporal self-attention as described in any one of claims 1 to 3.

Images