Audio processing access control method based on multi-factor audio features

Abstract

The application relates to an audio processing access control method based on multi-factor audio features, which comprises the following steps: in the process of recording, the unique number of a recording device is spliced with a current recording time stamp to form a frequency domain watermark which cannot be perceived by human ears, the frequency domain watermark is embedded into an audio signal to obtain an audio file; audio spectrum features, an audio duration and a frequency domain watermark are combined in a preset sequence, and a unique authentication code corresponding to the audio file is calculated through a built-in key through a hash message authentication code; the unique authentication code is checked, if the checking is successful, the audio file can be normally opened; the method has strong anti-counterfeiting capability, the frequency domain watermark embedding has no perceptual influence on sound quality, the recording habit of a user is not changed, the authentication process is automatically completed, the authentication can be completed without relying on a network or a third-party server, use change caused by network instability is avoided, and security risks of exposing the audio file to cloud verification are eliminated.

Description

Technical Field

[0001] This invention relates to the technical field of vehicle audio processing access control methods, and in particular to an audio processing access control method based on multi-factor audio features. Background Technology

[0002] Locally deployed audio processing models face the risk of unauthorized access or misuse: unauthorized users may bypass existing usage restrictions, use the model to process unauthorized audio, or even extract model parameters, leading to intellectual property leaks. Traditional protection measures mainly include software license management (such as serial numbers and dongles) and network authentication (such as online license verification), but these methods either increase the user's additional hardware burden or require the model to be constantly connected to the network, affecting ease of use. Currently, there is a lack of an effective solution for access control that directly utilizes the characteristics of audio itself for audio processing models.

[0003] Digital watermarking technology, as a common means of copyright protection and data integrity verification, has been applied to some extent in the audio field. For example, existing technologies propose embedding imperceptible watermarks into speech signals to achieve copyright marking or tamper detection. However, these existing technologies mostly focus on post-event verification or source tracing, rather than real-time control of access permissions. For instance, CN115035903B, with an authorization announcement date of December 6, 2022, discloses a method for injecting physical speech watermarks, a speech source tracing method, and a device. This method watermarks recorded speech by playing specific ultrasonic signals in the environment, thus providing a basis for future source tracing. Another example is CN118053435B, with an authorization announcement date of September 10, 2024, which discloses an audio anti-tampering method, device, equipment, and medium based on digital watermarking technology. This method dynamically generates digital watermarks based on communication content and user identity and inserts them into the audio stream to ensure that the transmitted speech has not been tampered with.

[0004] However, these technical solutions are used for tracking illegal recordings and preventing tampering of audio transmissions, respectively. They still have the following shortcomings in terms of function and purpose: they do not involve real-time authentication and control of audio file execution through the multi-factor characteristics of the audio itself.

[0005] Therefore, in this patent application, the applicant has carefully researched an audio processing access control method based on multi-factor audio features to solve the above problems. Summary of the Invention

[0006] The present invention addresses the shortcomings of the existing technology by providing an audio processing access control method based on multi-factor audio features. This method has strong anti-spoofing capabilities, and the frequency domain watermark embedding has no perceptible impact on sound quality, does not change the user's recording habits, and the authentication process is completed automatically without relying on the network or third-party servers. This avoids inconsistencies in usage caused by network instability and eliminates the security risks of exposing audio files to the cloud for verification.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] An audio processing access control method based on multi-factor audio features includes the following steps:

[0009] (1) Watermark embedding steps: Determine the target audio to be embedded with watermark by the audio acquisition module, and obtain the device ID of the audio acquisition module and the current timestamp of the audio acquisition module before its acquisition.

[0010] Combine the device ID and the current timestamp to obtain the original watermark information;

[0011] The preset key K is used as the encryption key for the HMAC operation of the Hash Message Authentication Code, and the secure hash algorithm SHA is used as the hash function for encryption in the HMAC operation. The original watermark information is then subjected to HMAC operation to obtain the first watermark code.

[0012] The first watermark code is mapped to a pseudo-random perturbation bit stream, and then quantization exponential modulation or spread spectrum coding is performed in the preset frequency band of the audio through discrete cosine transform or discrete wavelet transform to obtain the target audio with embedded watermark.

[0013] (2) Audio watermark extraction steps:

[0014] Obtain the second watermark code, spectral feature vector, and audio duration from the target audio with the embedded watermark; combine the second watermark code, spectral feature vector, and audio duration in a predetermined order to obtain the authentication information;

[0015] The preset key K is used as the encryption key for the HMAC operation of the Hash Message Authentication Code, and the secure hash algorithm SHA is used as the hash function for encryption in the HMAC operation. The HMAC operation is performed on the information to be authenticated to obtain the authentication code.

[0016] (3) Authentication and verification steps before processing the target audio:

[0017] Determine whether the device ID corresponding to the second watermark code is in the authorized list and whether the timestamp is within the allowed time period. If either is not present, authentication fails and the target audio cannot be accessed or processed. If both are present, compare the authentication code with the reference value when the first watermark code was generated. If they match, the target audio can be accessed and processed; otherwise, authentication fails and the target audio cannot be accessed or processed.

[0018] As a preferred embodiment, in step (2), the spectral feature vector includes at least one of the following: Mel frequency cepstral coefficient, chromaticity feature, spectral contrast, harmonic-noise ratio, zero crossover rate, and energy entropy.

[0019] As a preferred embodiment, obtaining the Mel frequency cepstral coefficients in the spectral feature vector in step (2) includes:

[0020] Based on the target audio with embedded watermark, the energy envelope spectrum corresponding to the target audio is obtained according to the Mel formula. The energy envelope spectrum is then input into the Mel filter bank to obtain the Mel frequency scale.

[0021] By performing logarithmic and discrete cosine transforms on the Mel frequency scale, the Mel frequency cepstral coefficients can be obtained and used as the spectral feature vector of the target audio with embedded watermark.

[0022] As a preferred embodiment, obtaining the chromaticity features in the spectral feature vector in step (2) includes:

[0023] Based on the target audio with embedded watermark, a short-time Fourier transform is first performed on it, and then the frequency is mapped to 12 half-tone chroma dimensions to obtain a 12-dimensional chroma feature vector, which is then used as the spectral feature vector of the target audio with embedded watermark.

[0024] As a preferred embodiment, obtaining the spectral contrast in the spectral feature vector in step (2) includes:

[0025] Based on the target audio with embedded watermark, a short-time Fourier transform is first performed on it. Then, based on the result of the short-time Fourier transform, the peak and valley values ​​of each frequency band are calculated, and the spectral energy comparison of each sub-band is calculated. This yields an N-dimensional spectral contrast vector, which is used as the spectral feature vector of the target audio with embedded watermark.

[0026] As a preferred option, in step (1), a watermark is redundantly embedded in each bit of the audio over three consecutive frames in a preset frequency band.

[0027] Compared with existing technologies, this invention has significant advantages and beneficial effects. Specifically, it mainly forms a unique authentication code by binding multiple factors such as frequency domain watermark, audio spectrum features, and audio duration, which greatly increases the difficulty of audio forgery and has strong anti-forgery capabilities, making multi-factor authentication more rigorous and reliable. Moreover, the embedding of frequency domain watermark has no perceptible impact on sound quality and does not change users' recording habits. The authentication process is completed automatically, and legitimate users can hardly feel the existence of security measures. It does not affect the normal audio processing flow and real-time performance. In particular, authentication can be completed without relying on the network or third-party servers, avoiding inconvenience caused by network instability, and eliminating the security risks of exposing audio files to the cloud for verification.

[0028] Secondly, this control method can be applied to the protection of various local audio files, and is not limited to noise reduction models. As long as each unique number and key is burned into different recording devices, it can be extended to audio file applications such as speech recognition and speech synthesis, thereby forming a universal audio file protection and access control.

[0029] To more clearly illustrate the structural features and effects of the present invention, the following detailed description is provided in conjunction with specific embodiments. Detailed Implementation

[0030] The present invention will be further described below with reference to specific embodiments.

[0031] An audio processing access control method based on multi-factor audio features includes the following steps:

[0032] (1) Watermark embedding steps: Determine the target audio to be embedded with watermark by the audio acquisition module, and obtain the device ID of the audio acquisition module and the current timestamp of the audio acquisition module before its acquisition.

[0033] The original watermark information is obtained by combining the device ID and the current timestamp. For example, the microphone's device ID is DEV12345, and the current timestamp is 20250617T224500.

[0034] The original watermark information W0 is: W0 = DEV12345||20250617T224500.

[0035] The preset key K is used as the encryption key for the HMAC (Hash Message Authentication Code) operation, and the secure hash algorithm SHA is used as the hash function for encryption in the HMAC operation. The original watermark information is then subjected to the HMAC operation to obtain the first watermark code.

[0036] HMAC is a key-related hash message authentication code. HMAC operations utilize a hash algorithm, taking a key and a message as input, and generating a message digest as output. Defining an HMAC requires a hash function for encryption and an encryption key. Specifically, in this embodiment, a preset key K can be used as the encryption key for the HMAC operation, SHA can be used as the hash function for encryption, and the original watermark information W0 can be used to perform the HMAC operation to obtain the authentication code.

[0037] SHA is a family of cryptographic hash functions and a secure hash algorithm certified by the Federal Information Processing Standards (FIPS). SHA is an algorithm that can compute a fixed-length string (also known as a message digest) corresponding to a digital message, and different input messages will produce different strings with a high probability. The reason the SHA algorithm is called "secure" is mainly based on the following two points:

[0038] First, it is theoretically very difficult to deduce the original input message from the message digest;

[0039] Secondly, it is theoretically very difficult to map two different sets of messages to the same message digest. Any change to the input message has a high probability of causing the generated message digests to be very different.

[0040] The SHA family includes SHA-1, SHA-224, SHA-256, SHA-384, SHA-512, and SHA-3. Designed by the National Security Agency (NSA) and published by the National Institute of Standards and Technology (NIST), they are U.S. government standards. SHA-224, SHA-256, SHA-384, and SHA-512 are sometimes collectively referred to as SHA-2.

[0041] To balance security and computational cost, this embodiment preferably uses a SHA-2-type algorithm as the hash function for HMAC operations. Therefore, the first watermark code W = HMAC_SHA256(K, W0).

[0042] The first watermark code is mapped to a pseudo-random perturbation bitstream, and then quantization exponential modulation or spread spectrum coding is performed in a preset frequency band of the audio using discrete cosine transform or discrete wavelet transform to obtain the target audio with embedded watermark. Quantization exponential modulation (QIM) or spread spectrum coding (DSSS) is used to enhance robustness and compression resistance. Furthermore, spread spectrum coding (DSSS) also achieves higher resistance to pruning.

[0043] In this embodiment, in step (1), a watermark is redundantly embedded in each bit of the audio within a preset frequency band across three consecutive frames. To reduce the bit error rate, this embodiment preferably uses forward error correction coding such as BCH(127,64) or LDPC. Preferably, the preset frequency band is a band from 3 kHz to 5 kHz.

[0044] The sound acquisition module can be a recording device or a separate device, such as the microphone of a headset or a separate microphone. In this embodiment, the sound acquisition module is an authorized microphone (i.e., a microphone with a device ID), and the entire process of step (1) is implemented through the embedded module inside the authorized microphone.

[0045] In this embodiment, the watermark embedding step is implemented in the authorized microphone. The preset key K is a shared symmetric encryption key written into the authorized microphone and the verification system. Its preferred length is 256 bits. The preset key K is generated by a high-entropy pseudo-random number and stored in hexadecimal or binary form, for example: K = "9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08". The preset key K is generated through a Hardware Security Module (HSM) or a Cryptographically Secure Random Number Generator (CSPRNG) compliant with the NIST SP 800-90A standard, ensuring uniqueness and unpredictability. The preset key K is written into the authorized microphone's secure chip or Trusted Execution Environment (TEE) and then written into the verification system through a secure channel, ensuring encrypted protection during transmission.

[0046] The pre-defined key K is not disclosed to users, cannot be exported, and is only accessed in the following two places:

[0047] 1. Used on the recording device to generate the first watermark code;

[0048] 2. In the verification system, it is used to recalculate the authentication code.

[0049] Therefore, even if a user copies an audio file, without a pre-set key K, a valid authentication code cannot be forged, thus ensuring system security.

[0050] (2) Audio watermark extraction steps:

[0051] The second watermark code, spectral feature vector, and audio duration are obtained from the target audio with the embedded watermark; the second watermark code, spectral feature vector, and audio duration are combined in a predetermined order to obtain the authentication information; in this embodiment, the second watermark code is defined as W1, the spectral feature vector is F, and the audio duration is D, where D=6s.

[0052] The preset key K is used as the encryption key for the HMAC operation, and the secure hash algorithm SHA is used as the hash function for encryption in the HMAC operation. The HMAC operation is performed on the information to be authenticated to obtain the authentication code.

[0053] In step (2), the spectral feature vector includes at least one of the following: Mel frequency cepstral coefficient, chromaticity feature, spectral contrast, harmonic-noise ratio, zero crossover rate, and energy entropy.

[0054] In this embodiment, obtaining the Mel frequency cepstral coefficients of the spectral feature vector in step (2) includes:

[0055] Based on the target audio with embedded watermark, the energy envelope spectrum corresponding to the target audio is obtained according to the Mel formula. The energy envelope spectrum is then input into the Mel filter bank to obtain the Mel frequency scale.

[0056] The Mel frequency scale is subjected to logarithmic transformation and discrete cosine transformation to obtain Mel frequency cepstral coefficients (MFCC); the Mel frequency cepstral coefficients serve as the spectral feature vector of the target audio with embedded watermark.

[0057] In this embodiment, F = [0.91, 0.67, 0.12, -0.05, 0.33, 0.44, -0.11, 0.08, 0.21, -0.02, 0.19, 0.05, -0.09, 0.13, 0.27, -0.04], and a hash operation is performed on F.

[0058] That is, hash(F) =

[0059] 8efdc1de402cc9a3cda95a7bd1bf8d8d8f321cd54ce40bd1709dae7c00ef3e04.

[0060] The following describes the steps for obtaining chromaticity features, spectral contrast, harmonic-to-noise ratio, zero crossover rate, and energy entropy:

[0061] A. The chromaticity features obtained in the spectral feature vector in step (2) include:

[0062] Based on the target audio with embedded watermark, a short-time Fourier transform is first performed on it, and then the frequency is mapped to 12 half-tone chroma dimensions to obtain a 12-dimensional chroma feature vector, which is then used as the spectral feature vector of the target audio with embedded watermark.

[0063] B. Obtaining the spectral contrast in the spectral feature vector in step (2) includes:

[0064] Based on the target audio with embedded watermark, a short-time Fourier transform is first performed on it. Then, based on the result of the short-time Fourier transform, the peak and valley values ​​of each frequency band are calculated, and the spectral energy comparison of each sub-band is calculated. This yields an N-dimensional spectral contrast vector, which is used as the spectral feature vector of the target audio with embedded watermark.

[0065] C. Obtaining the harmonic-noise ratio in the spectral feature vector in step (2) includes:

[0066] Based on the target audio with an embedded watermark, a short-time Fourier transform is first performed on it, and finally the peak harmonic ratio or autocorrelation function is calculated.

[0067] D. Obtaining the zero-crossing rate in the spectral feature vector in step (2) includes:

[0068] Based on the target audio with embedded watermark, a short-time Fourier transform is first performed on it, and finally the zero-crossing points of the audio waveform are statistically obtained.

[0069] E. Obtaining the energy entropy in the spectral feature vector in step (2) includes:

[0070] Based on the target audio with embedded watermark, a short-time Fourier transform is first performed on it, and finally the entropy is calculated based on the spectral energy distribution.

[0071] In this embodiment, the authentication code is defined as AuthCode.

[0072] AuthCode = HMAC_SHA256(K, W1 + hash(F) + D), then substitute the corresponding values ​​above, i.e., AuthCode =

[0073] 1b91f4683682bc1af79d973ce3d1d3deae5eff2f525ff4beb90630dd8332b157.

[0074] (3) Authentication and verification steps before processing the target audio:

[0075] The system checks whether the device ID corresponding to the second watermark is on the authorized list and whether the timestamp is within the allowed time period. If either is missing, authentication fails, and the target audio cannot be accessed or processed. Otherwise, the authentication code is compared with the reference value used when the first watermark was generated. If they match, the target audio can be accessed and processed; otherwise, authentication fails, and the target audio cannot be accessed or processed. Access and processing can refer to decrypting and loading the target audio into memory and performing noise reduction processing.

[0076] It should be noted that steps (2) and (3) are both implemented by a computer program, i.e., the verification system, executed by the computer's processor. Both steps are completed in sub-seconds, without affecting the user experience. After each processing step, the audio is unloaded from memory and re-encrypted and saved, ensuring that each processing of the target audio requires new authentication. For authentication failures, security policies can be set to improve reliability. For example, if authentication fails multiple times, an alarm can be triggered or the target audio can be temporarily locked to prevent malicious attacks. For audio that has timed out, re-recording can be requested to ensure real-time recording. If the recording device is damaged or lost, access to the recording device can be revoked by updating the authorization list or key. In this embodiment, the preset key K of the device can be replaced to improve long-term security.

[0077] The key design feature of this invention is that it forms a unique authentication code by binding multiple factors, including frequency domain watermark, audio spectrum features, and audio duration. This greatly increases the difficulty of audio forgery and provides strong anti-forgery capabilities, making multi-factor authentication more rigorous and reliable. Moreover, the embedding of the frequency domain watermark has no perceptible impact on sound quality and does not change the user's recording habits. The authentication process is completed automatically, and legitimate users are almost unaware of the security measures. It does not affect the normal audio processing flow and real-time performance. In particular, authentication can be completed without relying on the network or third-party servers, avoiding inconvenience caused by network instability. At the same time, it eliminates the security risks of exposing audio files to the cloud for verification.

[0078] Secondly, this control method can be applied to the protection of various local audio files, and is not limited to noise reduction models. As long as each unique number and key is burned into different recording devices, it can be extended to audio file applications such as speech recognition and speech synthesis, thereby forming a universal audio file protection and access control.

[0079] The above description is merely a preferred embodiment of the present invention and does not constitute any limitation on the technical scope of the present invention. Therefore, any minor modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention shall still fall within the scope of the technical solution of the present invention.

Claims

1. An audio processing access control method based on multi-factor audio features, characterized in that: Includes the following steps: (1) Watermark embedding steps: Determine the target audio to be embedded with watermark by the audio acquisition module, and obtain the device ID of the audio acquisition module and the current timestamp of the audio acquisition module before its acquisition. Combine the device ID and the current timestamp to obtain the original watermark information; The preset key K is used as the encryption key for the HMAC operation of the Hash Message Authentication Code, and the secure hash algorithm SHA is used as the hash function for encryption in the HMAC operation. The original watermark information is then subjected to HMAC operation to obtain the first watermark code. The first watermark code is mapped to a pseudo-random perturbation bit stream, and then quantization exponential modulation or spread spectrum coding is performed in the preset frequency band of the audio through discrete cosine transform or discrete wavelet transform to obtain the target audio with embedded watermark. (2) Audio watermark extraction steps: Obtain the second watermark code, spectral feature vector, and audio duration from the target audio with the embedded watermark; combine the second watermark code, spectral feature vector, and audio duration in a predetermined order to obtain the authentication information; The preset key K is used as the encryption key for the HMAC operation of the Hash Message Authentication Code, and the secure hash algorithm SHA is used as the hash function for encryption in the HMAC operation. The HMAC operation is performed on the information to be authenticated to obtain the authentication code. (3) Authentication and verification steps before processing the target audio: Determine whether the device ID corresponding to the second watermark code is in the authorized list and whether the timestamp is within the allowed time period. If either is not present, authentication fails and the target audio cannot be accessed or processed. If both are present, compare the authentication code with the reference value when the first watermark code was generated. If they match, the target audio can be accessed and processed; otherwise, authentication fails and the target audio cannot be accessed or processed.

2. The audio processing access control method based on multi-factor audio features according to claim 1, characterized in that: In step (2), the spectral feature vector includes at least one of the following: Mel frequency cepstral coefficients, chromaticity features, spectral contrast, harmonic-noise ratio, zero crossover rate, and energy entropy.

3. The audio processing access control method based on multi-factor audio features according to claim 2, characterized in that: The Mel frequency cepstral coefficients obtained in step (2) from the spectral feature vector include: Based on the target audio with an embedded watermark, the energy envelope spectrum corresponding to the target audio is obtained according to the Mel formula. The energy envelope spectrum is then input into the Mel filter bank to obtain the Mel frequency scale. By performing logarithmic and discrete cosine transforms on the Mel frequency scale, the Mel frequency cepstral coefficients can be obtained and used as the spectral feature vector of the target audio with embedded watermark.

4. The audio processing access control method based on multi-factor audio features according to claim 2, characterized in that: The chromaticity features obtained in step (2) from the spectral feature vector include: Based on the target audio with embedded watermark, a short-time Fourier transform is first performed on it, and then the frequency is mapped to 12 half-tone chroma dimensions to obtain a 12-dimensional chroma feature vector, which is then used as the spectral feature vector of the target audio with embedded watermark.

5. The audio processing access control method based on multi-factor audio features according to claim 2, characterized in that: Obtaining the spectral contrast in the spectral feature vector in step (2) includes: Based on the target audio with embedded watermark, a short-time Fourier transform is first performed on it. Then, based on the result of the short-time Fourier transform, the peak and valley values ​​of each frequency band are calculated, and the spectral energy comparison of each sub-band is calculated. This yields an N-dimensional spectral contrast vector, which is used as the spectral feature vector of the target audio with embedded watermark.

6. The audio processing access control method based on multi-factor audio features according to claim 1, characterized in that: In step (1), a watermark is redundantly embedded in each bit of the audio over three consecutive frames in a preset frequency band.