Audio processing methods, apparatus, devices and storage media

Abstract

This disclosure provides an audio processing method, apparatus, device, and storage medium, relating to the field of data processing technology. The method includes: performing time-frequency conversion processing on initial time-domain data of a single frame of audio to be processed, obtaining initial frequency-domain data of the single frame of audio to be processed, the initial frequency-domain data including information on the correspondence between frequency and amplitude; obtaining low-frequency frequency-domain data of the single frame of audio to be processed based on the information on the correspondence between frequency and amplitude in the initial frequency-domain data; obtaining an encryption parameter sequence of the single frame of audio to be processed based on the low-frequency frequency-domain data; obtaining denoised time-domain data of the single frame of audio to be processed based on the low-frequency frequency-domain data and the initial time-domain data; and encrypting the denoised time-domain data using the encryption parameter sequence to obtain encrypted audio data of the single frame of audio to be processed. This method improves the convenience and flexibility of generating true random numbers.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and more specifically, to an audio processing method, apparatus, device, and readable storage medium. Background Technology

[0002] Data encryption refers to the process of transforming plaintext into ciphertext using encryption algorithms and keys, and then restoring the plaintext using decryption algorithms and keys. Data encryption is one of the most reliable methods for protecting information in computer systems. It is particularly important in Internet of Things (IoT) applications.

[0003] Some related technologies use algorithms to generate random numbers for encryption. These generated random numbers are usually pseudo-random sequences, and the results are predictable, making them easy to crack. Other related technologies generate truly random numbers based on uncertain sources, such as circuit noise, air noise, and light source noise. Generating truly random numbers based on these sources often requires complex circuits or equipment, thus limiting the application of these truly random number generators.

[0004] As mentioned above, improving the convenience and flexibility of generating truly random numbers to enhance the reliability of encryption has become an urgent problem to be solved.

[0005] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] The purpose of this disclosure is to provide an audio processing method, apparatus, device, and readable storage medium that at least to some extent overcomes the problem of poor convenience and flexibility in generating truly random numbers due to the fact that related technologies generate truly random numbers based on uncertain sources.

[0007] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0008] According to one aspect of this disclosure, an audio processing method is provided, comprising: performing time-frequency conversion processing on initial time-domain data of a single frame of audio to be processed to obtain initial frequency-domain data of the single frame of audio to be processed, the initial frequency-domain data including information on the correspondence between frequency and amplitude; obtaining low-frequency frequency-domain data of the single frame of audio to be processed based on the information on the correspondence between frequency and amplitude in the initial frequency-domain data; obtaining an encryption parameter sequence of the single frame of audio to be processed based on the low-frequency frequency-domain data; obtaining noise-reduced time-domain data of the single frame of audio to be processed based on the low-frequency frequency-domain data and the initial time-domain data; and encrypting the noise-reduced time-domain data using the encryption parameter sequence to obtain encrypted audio data of the single frame of audio to be processed.

[0009] According to one embodiment of this disclosure, obtaining the low-frequency domain data of the single-frame audio to be processed based on the frequency-amplitude correspondence information in the initial frequency domain data includes: acquiring the data of frequencies in the initial frequency domain data whose amplitude is lower than a preset amplitude threshold as the low-frequency domain data.

[0010] According to one embodiment of this disclosure, obtaining the encryption parameter sequence of the single-frame audio to be processed based on the low-frequency domain data includes: using the amplitude values ​​corresponding to each frequency in the low-frequency domain data as a random number sequence, and mapping the random number sequence to the encryption parameter sequence through a nonlinear function.

[0011] According to one embodiment of this disclosure, the nonlinear function is an S-shaped growth curve function.

[0012] According to one embodiment of this disclosure, obtaining the denoised time-domain data of the single-frame audio to be processed based on the low-frequency frequency domain data and the initial time-domain data includes: performing the inverse time-frequency conversion process on the low-frequency frequency domain data to obtain the noise time-domain data of the single-frame audio to be processed; and removing the noise time-domain data from the initial time-domain data of the single-frame audio to be processed to obtain the denoised time-domain data of the single-frame audio to be processed.

[0013] According to one embodiment of this disclosure, encrypting the denoised time-domain data using the encryption parameter sequence to obtain encrypted audio data of the single-frame audio to be processed includes: obtaining the average amplitude of the denoised time-domain data; multiplying each encryption parameter in the encryption parameter sequence by the average amplitude to obtain a scrambling parameter sequence; and inserting the scrambling parameter sequence into the denoised time-domain data using an audio scrambling algorithm to obtain encrypted audio data of the single-frame audio to be processed.

[0014] According to one embodiment of this disclosure, the method further includes: acquiring audio data to be processed; performing frame-by-frame windowing processing on the audio data to be processed to obtain initial temporal data of multiple frames of single-frame audio to be processed.

[0015] According to another aspect of this disclosure, an audio processing apparatus is provided, comprising: a time-frequency conversion module, configured to perform time-frequency conversion processing on initial time-domain data of a single frame of audio to be processed, thereby obtaining initial frequency-domain data of the single frame of audio to be processed, the initial frequency-domain data including information on the correspondence between frequency and amplitude; a low-frequency frequency-domain data acquisition module, configured to obtain low-frequency frequency-domain data of the single frame of audio to be processed based on the information on the correspondence between frequency and amplitude in the initial frequency-domain data; an encryption parameter sequence acquisition module, configured to obtain an encryption parameter sequence of the single frame of audio to be processed based on the low-frequency frequency-domain data; a noise reduction time-domain data acquisition module, configured to obtain noise reduction time-domain data of the single frame of audio to be processed based on the low-frequency frequency-domain data and the initial time-domain data; and a disordered order module, configured to encrypt the noise reduction time-domain data using the encryption parameter sequence, thereby obtaining encrypted audio data of the single frame of audio to be processed.

[0016] According to an embodiment of this disclosure, the low-frequency domain data acquisition module is further configured to: acquire data of frequencies in the initial frequency domain data whose amplitude is lower than a preset amplitude threshold as the low-frequency domain data.

[0017] According to an embodiment of this disclosure, the encryption parameter sequence acquisition module is further configured to: take the amplitude corresponding to each frequency in the low-frequency domain data as a random number sequence, and map the random number sequence to the encryption parameter sequence through a nonlinear function.

[0018] According to one embodiment of this disclosure, the nonlinear function is an S-shaped growth curve function.

[0019] According to an embodiment of this disclosure, the noise reduction time-domain data acquisition module includes: a noise time-domain data acquisition module, used to perform the inverse time-frequency conversion processing on the low-frequency frequency domain data to obtain the noise time-domain data of the single frame audio to be processed; and a noise time-domain data removal module, used to remove the noise time-domain data from the initial time-domain data of the single frame audio to be processed to obtain the noise-reduced time-domain data of the single frame audio to be processed.

[0020] According to an embodiment of this disclosure, the scrambling module is further configured to: obtain the average amplitude of the denoised time-domain data; multiply each encryption parameter in the encryption parameter sequence by the average amplitude to obtain a scrambling parameter sequence; and insert the scrambling parameter sequence into the denoised time-domain data using an audio scrambling algorithm to obtain the encrypted audio data of the single frame audio to be processed.

[0021] According to one embodiment of this disclosure, the apparatus further includes: a frame-segmentation and windowing module, configured to: acquire audio data to be processed; perform frame-segmentation and windowing processing on the audio data to be processed to obtain initial temporal data of multiple frames of single-frame audio to be processed.

[0022] According to another aspect of this disclosure, an apparatus is provided, comprising: a memory, a processor, and executable instructions stored in the memory and executable in the processor, wherein the processor, when executing the executable instructions, implements any of the methods described above.

[0023] According to another aspect of this disclosure, a computer-readable storage medium is provided that stores computer-executable instructions thereon, which, when executed by a processor, implement any of the methods described above.

[0024] The audio processing method provided in the embodiments of this disclosure performs time-frequency conversion processing on the initial time-domain data of the single frame audio to be processed to obtain the initial frequency-domain data of the single frame audio to be processed. Then, based on the information of the frequency-amplitude correspondence in the initial frequency-domain data, the low-frequency frequency-domain data of the single frame audio to be processed is obtained. Then, based on the low-frequency frequency-domain data, the encryption parameter sequence of the single frame audio to be processed is obtained. Based on the low-frequency frequency-domain data and the initial time-domain data, the noise-reduced time-domain data of the single frame audio to be processed is obtained. Then, the noise-reduced time-domain data is encrypted using the encryption parameter sequence to obtain the encrypted audio data of the single frame audio to be processed. This method can generate true random numbers using the noise data in the audio to be processed and then encrypt the noise-reduced audio to be processed, improving the convenience and flexibility of generating true random numbers, thereby improving the reliability of audio encryption.

[0025] It should be understood that the above general description and the following detailed description are merely exemplary and do not limit this disclosure. Attached Figure Description

[0026] The above and other objects, features and advantages of this disclosure will become more apparent from a detailed description of exemplary embodiments thereof with reference to the accompanying drawings.

[0027] Figure 1 A schematic diagram of a system structure according to an embodiment of this disclosure is shown.

[0028] Figure 2 A flowchart of an audio processing method according to an embodiment of this disclosure is shown.

[0029] Figure 3 A flowchart of an audio framing processing method according to an embodiment of this disclosure is shown.

[0030] Figure 4 This is a time-domain waveform diagram of a speech signal according to an exemplary embodiment.

[0031] Figure 5 It is based on Figure 4 The diagram shows a waveform of a single-frame speech signal.

[0032] Figure 6 It is based on Figure 5 The waveform of a single frame of audio signal after windowing is shown.

[0033] Figure 7 It is based on Figure 6 The image shows a windowed spectrum of a single-frame speech signal.

[0034] Figure 8 It shows Figure 2 The step S208 shown is a schematic diagram of the processing procedure in one embodiment.

[0035] Figure 9 It shows Figure 2 The step S210 shown is a schematic diagram of the processing procedure in one embodiment.

[0036] Figure 10 It is based on Figures 2 to 9 The diagram illustrates a process for generating encrypted voice data with true random numbers.

[0037] Figure 11 A block diagram of an audio processing apparatus according to an embodiment of the present disclosure is shown.

[0038] Figure 12 A block diagram of another audio processing apparatus according to an embodiment of the present disclosure is shown.

[0039] Figure 13 A schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0040] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted.

[0041] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced with one or more of the specific details omitted, or other methods, apparatuses, steps, etc., can be employed. In other instances, well-known structures, methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0042] Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this disclosure, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified. The symbol " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0043] In this disclosure, unless otherwise expressly specified and limited, the term "connection" and similar terms should be interpreted broadly, for example, it can refer to an electrical connection or the ability to communicate with each other; it can refer to a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms in this disclosure according to the specific circumstances.

[0044] As mentioned above, some related technologies use algorithms to generate random numbers for encryption. These generated random numbers are often pseudo-random sequences. For example, the `time()` function is used to change the seed, thus obtaining different pseudo-random number sequences. However, the results returned by `time()` follow a predictable pattern, making encryption with it easily crackable. Other related technologies rely on truly random numbers to generate random keys. However, the generation of truly random numbers depends on uncertain sources, such as circuit noise, air noise, and light source noise. Generating truly random numbers based on these sources often requires complex circuits or equipment. From a cost and convenience perspective, the application of these truly random number generators is limited.

[0045] Therefore, this disclosure provides an audio processing method that performs time-frequency conversion on the initial time-domain data of the single-frame audio to be processed to obtain the initial frequency-domain data of the single-frame audio to be processed. Then, based on the frequency-amplitude correspondence information in the initial frequency-domain data, the low-frequency frequency-domain data of the single-frame audio to be processed is obtained. Then, based on the low-frequency frequency-domain data, the encryption parameter sequence of the single-frame audio to be processed is obtained. Based on the low-frequency frequency-domain data and the initial time-domain data, the noise-reduced time-domain data of the single-frame audio to be processed is obtained. Then, the noise-reduced time-domain data is encrypted using the encryption parameter sequence to obtain the encrypted audio data of the single-frame audio to be processed. This method enables the generation of truly random numbers from the noise data in the audio to be processed and then uses these numbers to encrypt the noise-reduced audio, improving the convenience and flexibility of generating truly random numbers and thus improving the reliability of audio encryption.

[0046] Figure 1 An exemplary system architecture 10 is shown that can be applied to the audio processing methods or audio processing apparatus disclosed herein.

[0047] like Figure 1As shown, system architecture 10 may include terminal device 102, network 104, and server 106. Terminal device 102 may be various electronic devices with a display screen and supporting input and output, including but not limited to smartphones, tablets, laptops, desktop computers, wearable devices, virtual reality devices, smart microphones, voice recorders, smart speakers, etc. Network 104 is used as a medium to provide a communication link between terminal device 102 and server 106. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc. Server 106 may be a server or server cluster providing various services, such as a backend processing server, database server, etc.

[0048] Users can use terminal device 102 to interact with server 106 via network 104 to receive or send data. For example, a user can download a single frame of audio to be processed from server 106 to terminal device 102 via network 104, and then obtain the low-frequency domain data of the single frame of audio through processing software on terminal device 102. Alternatively, a user can operate on terminal device 102 to send encrypted audio data to server 106 for storage via network 104.

[0049] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0050] Figure 2 This is a flowchart illustrating an audio processing method according to an exemplary embodiment. Figure 2 The method shown can be applied, for example, to the server side of the above system, or to the terminal devices in the above system.

[0051] refer to Figure 2 The method 20 provided in this embodiment may include the following steps.

[0052] In step S202, the initial time-domain data of the single frame audio to be processed is subjected to time-frequency conversion processing to obtain the initial frequency-domain data of the single frame audio to be processed. The initial frequency-domain data includes information on the correspondence between frequency and amplitude.

[0053] In some embodiments, audio data to be encrypted can be obtained first, for example, voice data acquired by the audio acquisition module of an apparatus implementing the method provided in the embodiments of this disclosure. Then, the obtained audio data to be encrypted is subjected to frame-by-frame windowing processing, and each single frame of audio data is processed separately. For a detailed implementation of the frame-by-frame windowing processing of the audio data to be encrypted, please refer to... Figures 3 to 6 .

[0054] In some embodiments, time-frequency conversion processing may be performed on time-domain data by means of a Fourier transform method. For example, time-frequency conversion may be performed on the framed and windowed audio data x[n] by means of a Discrete Fourier Transform (DFT) method:

[0055]

[0056] In Equation (1), i, k ∈ [0, N - 1], and both i and k are integers. The time-frequency conversion by the discrete Fourier function decomposes the waveform in the time domain into the superposition of multiple sine waves of different frequencies. In Equation (1), N is the number of sine waves obtained by decomposition, which can be set according to actual needs. For example, it can be set to 100, 200, etc., or it can also be set to 512.

[0057] Figure 7 is based on Figure 6 A spectrogram after windowing a single-frame speech signal is shown. As Figure 7 shown, the single-frame speech signal in Figure 6 is subjected to DFT conversion and divided into N = 512 sine waves. Figure 7 The spectrogram showing the variation of the amplitudes of these 512 sine waves with frequency is shown in Figure 7 . Among them, it is shown that the frequency distribution of the 512 sine waves is between 0 - 8000 Hz. Each peak or valley value on the vertical axis represents the amplitude corresponding to a sine wave frequency. Connecting the amplitudes of the 512 frequency points obtains the curve in

[0058] In step S204, the low-frequency domain data of the single-frame audio to be processed is obtained according to the information on the correspondence between frequency and amplitude in the initial frequency domain data.

[0059] In some embodiments, after taking the absolute value of the amplitude on the spectrum of the initial frequency domain data, an amplitude threshold T may be set. The audio of the high-amplitude frequency part higher than the threshold T is retained as the noise reduction (frequency domain) data, and the amplitudes of the low-frequency domain data of the low-amplitude frequency part lower than the threshold T are used as a set of true random number sequences as the output (m1, m2,... m p ), where p is a positive integer and p < N (for example, N = 512), and these are used as random noise data.

[0060] In some embodiments, different amplitude thresholds T may be set according to factors such as the sound source and environment of the audio data.

[0061] In step S206, an encryption parameter sequence for the single-frame audio to be processed is obtained according to the low-frequency domain data.

[0062] In some embodiments, the amplitude values ​​corresponding to each frequency in the low-frequency domain data can be used as a random number sequence, and then the random number sequence can be mapped to an encrypted parameter sequence through a nonlinear function.

[0063] In some embodiments, the nonlinear function can be, for example, a sigmoid function. For instance, a truly random sequence of amplitudes (m1, m2, ... m) corresponding to each frequency in low-frequency domain data can be used. p After being mapped by the nonlinear function Sigmoid:

[0064]

[0065] Equation (2) can map the true random number sequence generated from the low-frequency domain data of a single frame of audio data to a sequence (L1, L2, ..., L) between [0, 1]. p ), which serves as the sequence of encryption parameters for the single frame of audio to be processed.

[0066] In other embodiments, the nonlinear function may be, for example, the tanh function.

[0067] In step S208, the noise reduction time domain data of the single frame audio to be processed is obtained based on the low-frequency frequency domain data and the initial time domain data.

[0068] In some embodiments, after converting the low-frequency frequency domain data from the frequency domain to the time domain to obtain noisy time domain data, the noisy time domain data can be removed from the initial time domain data of the single frame audio to be processed, thereby obtaining the denoised time domain data of the single frame audio to be processed. Specific implementation methods can be found in [reference needed]. Figure 8 .

[0069] In other embodiments, the low-frequency data in the initial frequency domain data of the single frame audio to be processed can be removed, and the remaining frequency domain data can be converted from the frequency domain to the time domain to obtain the noise-reduced time domain data of the single frame audio to be processed.

[0070] In step S210, the noise reduction time-domain data is encrypted using the encryption parameter sequence to obtain the encrypted audio data of the single frame audio to be processed.

[0071] In some embodiments, the encryption parameter sequence obtained from the low-frequency domain data in step S206 can be used to perform out-of-order encryption on the noise-reduced time-domain data obtained in step S210 to obtain the encrypted audio data of the single frame audio to be processed. Specific implementation details can be found in [reference needed]. Figure 9 .

[0072] In some embodiments, the data of each frame of the audio to be processed can be processed according to steps S202 to S210, and then the encrypted audio data of each frame can be merged to obtain encrypted audio data.

[0073] According to the audio processing method provided in this disclosure, the initial time-domain data of the single frame audio to be processed is subjected to time-frequency conversion processing to obtain the initial frequency-domain data of the single frame audio to be processed. Then, the low-frequency frequency-domain data of the single frame audio to be processed is obtained according to the information of the frequency-amplitude correspondence in the initial frequency-domain data. Then, the encryption parameter sequence of the single frame audio to be processed is obtained according to the low-frequency frequency-domain data. The noise-reduced time-domain data of the single frame audio to be processed is obtained according to the low-frequency frequency-domain data and the initial time-domain data. Then, the noise-reduced time-domain data is encrypted using the encryption parameter sequence to obtain the encrypted audio data of the single frame audio to be processed. This method can realize the encryption of the noise-reduced audio to be processed by generating true random numbers using the noise data in the audio to be processed, thereby improving the convenience and flexibility of generating true random numbers and improving the reliability of audio encryption.

[0074] In IoT application scenarios, both identification terminals and the cloud require a high level of encryption. According to the audio processing method provided in this disclosure, a true random number sequence can be obtained from the acquired random audio data through software, which can be used as a seed for encryption processing, and has high practicality. Moreover, the true random number sequence obtained in the noise reduction process can also prepare data for subsequent applications such as speech recognition.

[0075] Figure 3 This is a flowchart illustrating an audio framing method according to an exemplary embodiment. Figure 3 The method shown can be applied, for example, to the server side of the above system, or to the terminal devices in the above system. Figure 3 The method shown can be performed in Figure 2 Before step S202 in the process.

[0076] refer to Figure 3 The method 30 provided in this embodiment may include the following steps.

[0077] In step S302, the audio data to be processed is acquired.

[0078] In some embodiments, the audio data to be processed may be voice signal data to be encrypted. For example, a segment of dual-channel PCM (Pulse Code Modulation) format audio data x'(n) with a sampling frequency of 16kHz can be acquired by an audio acquisition module, where n represents time and x'(n) represents the audio vibration amplitude value (simply referred to as amplitude) that changes with time n.

[0079] Figure 4 This is a time-domain waveform diagram of a speech signal according to an exemplary embodiment. For example... Figure 4 As shown, in Figure 4Among them, if x'(n) is used to represent the voice signal to be encrypted, then 0 ≤ n < 20s, -1 < x'(n) < 1, and the amplitude value can be obtained by normalizing the sampled PCM data.

[0080] In step S304, frame windowing processing is performed on the audio data to be processed, and the initial time-domain data of multiple frames of single-frame audio to be processed is obtained.

[0081] In some embodiments, frame windowing processing can be performed on the audio data to be processed, and the frame length and frame step size can be set according to the actual situation. For example, the frame length can be set to 20ms, 30ms, 50ms, etc., and the step size can be set to 10ms, 20ms, 40ms, etc. For example, for Figure 4 the voice signal audio data x'(n) in, 32ms audio can be set as one frame, framed at a step size of 20ms, with an overlap of (32 - 20 =) 12ms between adjacent frames, and a compensation frame is added at the last frame, that is, the last frame is (32 + 32 =) 64ms of audio, which is used to retain the boundary information of the audio.

[0082] Figure 5 is based on Figure 4 A single-frame voice signal waveform diagram shown. As Figure 5 shown, Figure 5 is Figure 4 the voice signal waveform diagram of the 50th frame in, where 512 sampling points are randomly sampled from the 32ms audio of the 50th frame, then Figure 5 the unit of the horizontal axis time in is (32 / 512 =) 0.0625ms, then the moving step size for this sampled number of frames is (20 / 0.0625 =) 320, and the amplitude value of the voice signal of the 50th frame is between -0.5 and 0.5.

[0083] In some embodiments, Hamming window can be used for windowing processing. For example, the window function can be

[0084]

[0085] In formula (3), a and b are the starting and ending time points of the single-frame audio data on the time axis (horizontal axis). Taking Figure 5 the single-frame voice signal of as an example, a + 32ms = b, and a increases by 20ms each time. The expression for performing frame windowing processing on the audio data x'(n) using formula (3) can be:

[0086] x[n] = x′(n)W Hamming (a - b) (4)

[0087] In formula (4), x[n] is the audio data after frame windowing processing of the audio data x'(n).

[0088] Figure 6 It is based on Figure 5 The waveform of a single-frame audio signal after windowing is shown. Figure 6 As shown, for Figure 5 After the single-frame speech signal in the text is processed by adding a Hamming window according to equation (4), the amplitude waveform of the signal becomes smoother.

[0089] Figure 8 It shows Figure 2 The step S208 shown is a schematic diagram of the processing procedure in one embodiment. (See attached diagram.) Figure 8 As shown in the present embodiment, step S208 may further include the following steps.

[0090] Step S802: Perform inverse time-frequency conversion on the low-frequency domain data to obtain the noise time-domain data of the single frame audio to be processed.

[0091] In some embodiments, the low-frequency domain data obtained in step S204 can be subjected to inverse time-frequency transformation processing, such as inverse Discrete Fourier Transform (IDFT), to obtain the noise signal time-domain dataset V. s .

[0092] Step S804: Remove the noise time-domain data from the initial time-domain data of the single frame audio to be processed to obtain the noise-reduced time-domain data of the single frame audio to be processed.

[0093] In some embodiments, the noise time-domain data in the initial time-domain data of the single frame audio to be processed can be removed to obtain the remaining noise-reduced audio dataset {V} as the effective speech signal.

[0094] According to the method provided in the embodiments of this disclosure, after removing the low-amplitude frequency components in the collected audio data as noise audio data and then encrypting them, higher quality encrypted audio data can be obtained.

[0095] Figure 9 It shows Figure 2 The step S210 shown is a schematic diagram of the processing procedure in one embodiment. (See attached diagram.) Figure 9 As shown in the present embodiment, step S210 may further include the following steps.

[0096] Step S902: Obtain the average amplitude of the denoised time-domain data.

[0097] Step S904: Multiply each encryption parameter in the encryption parameter sequence by the average amplitude to obtain the scrambling parameter sequence.

[0098] Step S906: Insert the scrambling parameter sequence into the noise reduction time domain data using an audio scrambling algorithm to obtain the encrypted audio data of the single frame audio to be processed.

[0099] In some embodiments, for example, the generated true random number sequence (L1, L2, ..., L...) can be used. p As an input parameter in the encrypted audio data generation process, the average amplitude of the obtained noise-reduced frequency dataset {V} is multiplied to obtain the scrambling parameter sequence. Then, an algorithm capable of scrambling speech data is used to insert the scrambling parameter sequence back into the noise-reduced frequency dataset {V}, resulting in the encrypted audio dataset {V}. sec}

[0100] Figure 10 It is based on Figures 2 to 9 This diagram illustrates a process for generating encrypted voice data with truly random numbers. Figure 10 As shown, the input speech data 10002 is first preprocessed (S1002), including frame-by-frame windowing (S10022) to obtain single-frame audio data. Then, each single-frame audio data is subjected to discrete Fourier transform (S10024). For the frequency domain data of each single-frame audio data, the absolute value of the amplitude of the single-frame frequency domain data is taken (S1004), and the sampling points with amplitudes lower than a preset threshold are taken as noise data (S1006). Nonlinear processing is performed through the Sigmoid function (S1008) to obtain a true random number sequence. The noise data obtained in step S1006 in the input speech data 1002 is removed (S1010). The noise-reduced data is then encrypted in random order through the true random number sequence obtained in S1008 (S1012) to obtain single-frame encrypted data. Finally, the encrypted data of each frame is merged into encrypted speech data 10006 for output.

[0101] Figure 11 This is a block diagram illustrating an audio processing apparatus according to an exemplary embodiment. Figure 11 The device shown can be applied, for example, to the server side of the above system, or to the terminal device of the above system.

[0102] refer to Figure 11 The apparatus 110 provided in this embodiment may include a time-frequency conversion module 1102, a low-frequency domain data acquisition module 1104, an encryption parameter sequence acquisition module 1106, a noise reduction time domain data acquisition module 1108, and a disordered module 1110.

[0103] The time-frequency conversion module 1102 can be used to perform time-frequency conversion processing on the initial time-domain data of the single frame audio to be processed, and obtain the initial frequency-domain data of the single frame audio to be processed. The initial frequency-domain data includes information on the correspondence between frequency and amplitude.

[0104] The low-frequency domain data acquisition module 1104 can be used to obtain the low-frequency domain data of the single frame audio to be processed based on the information of the frequency-amplitude correspondence in the initial frequency domain data.

[0105] The encryption parameter sequence acquisition module 1106 can be used to obtain the encryption parameter sequence of the single frame audio to be processed based on low-frequency domain data.

[0106] The noise reduction time domain data acquisition module 1108 can be used to obtain the noise reduction time domain data of the single frame audio to be processed based on the low frequency domain data and the initial time domain data.

[0107] The out-of-order module 1110 can be used to encrypt the noise reduction time-domain data using the encryption parameter sequence to obtain the encrypted audio data of the single frame audio to be processed.

[0108] Figure 12 This is a block diagram illustrating another audio processing apparatus according to an exemplary embodiment. Figure 12 The device shown can be applied, for example, to the server side of the above system, or to the terminal device of the above system.

[0109] refer to Figure 12 The apparatus 120 provided in this embodiment may include a frame-segmentation and windowing module 1201, a time-frequency conversion module 1202, a low-frequency domain data acquisition module 1204, an encryption parameter sequence acquisition module 1206, a noise reduction time domain data acquisition module 1208, and a disordered module 1210. The noise reduction time domain data acquisition module 1208 may include a noise time domain data acquisition module 12082 and a noise time domain data removal module 12084.

[0110] The frame-segmentation and windowing module 1201 can be used to: acquire audio data to be processed; perform frame-segmentation and windowing processing on the audio data to be processed to obtain the initial temporal domain data of multiple frames of single-frame audio to be processed.

[0111] The time-frequency conversion module 1202 can be used to perform time-frequency conversion processing on the initial time-domain data of the single frame audio to be processed, and obtain the initial frequency-domain data of the single frame audio to be processed. The initial frequency-domain data includes information on the correspondence between frequency and amplitude.

[0112] The low-frequency domain data acquisition module 1204 can be used to obtain the low-frequency domain data of the single frame audio to be processed based on the information of the frequency-amplitude correspondence in the initial frequency domain data.

[0113] The low-frequency domain data acquisition module 1204 can also be used to acquire low-frequency domain data as the frequency data whose amplitude is lower than the preset amplitude threshold in the initial frequency domain data.

[0114] The encryption parameter sequence acquisition module 1206 can be used to obtain the encryption parameter sequence of the single frame audio to be processed based on low-frequency domain data.

[0115] The encryption parameter sequence acquisition module 1206 can also be used to take the amplitude corresponding to each frequency in the low-frequency domain data as a random number sequence, and map the random number sequence to an encryption parameter sequence through a nonlinear function.

[0116] Nonlinear functions can be S-shaped growth curve functions.

[0117] The noise reduction time-domain data acquisition module 1208 can be used to obtain the noise reduction time-domain data of the single frame audio to be processed based on the low-frequency frequency domain data and the initial time-domain data.

[0118] The noise time-domain data acquisition module 12082 can be used to perform inverse time-frequency conversion processing on low-frequency frequency domain data to obtain noise time-domain data of the single frame audio to be processed.

[0119] The noise time-domain data removal module 12084 can be used to remove noise time-domain data from the initial time-domain data of the single frame audio to be processed, and obtain the noise-reduced time-domain data of the single frame audio to be processed.

[0120] The out-of-order module 1210 can be used to encrypt the noise reduction time-domain data using the encryption parameter sequence to obtain the encrypted audio data of the single frame audio to be processed.

[0121] The scrambling module 1210 can also be used to: obtain the average amplitude of the denoised time-domain data; multiply each encryption parameter in the encryption parameter sequence by the average amplitude to obtain the scrambling parameter sequence; and insert the scrambling parameter sequence into the denoised time-domain data using an audio scrambling algorithm to obtain the encrypted audio data of the single frame audio to be processed.

[0122] The specific implementation of each module in the device provided in this embodiment can be referred to the content of the above method, and will not be repeated here.

[0123] Figure 13 A schematic diagram of the structure of an electronic device according to an embodiment of this disclosure is shown. It should be noted that... Figure 13 The devices shown are merely examples of computer systems and should not be construed as limiting the functionality and scope of use of the embodiments disclosed herein.

[0124] like Figure 13As shown, device 1300 includes a central processing unit (CPU) 1301, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 1302 or a program loaded from storage portion 1308 into random access memory (RAM) 1303. The RAM 1303 also stores various programs and data required for the operation of device 1300. CPU 1301, ROM 1302, and RAM 1303 are interconnected via bus 1304. Input / output (I / O) interface 1305 is also connected to bus 1304.

[0125] The following components are connected to I / O interface 1305: an input section 1306 including a keyboard, mouse, etc.; an output section 1307 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1308 including a hard disk, etc.; and a communication section 1309 including a network interface card such as a LAN card, modem, etc. The communication section 1309 performs communication processing via a network such as the Internet. A drive 1310 is also connected to I / O interface 1305 as needed. Removable media 1311, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 1310 as needed so that computer programs read from them can be installed into storage section 1308 as needed.

[0126] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1309, and / or installed from removable medium 1311. When the computer program is executed by central processing unit (CPU) 1301, it performs the functions defined above in the system of this disclosure.

[0127] It should be noted that the computer-readable medium disclosed herein may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0128] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0129] The modules described in the embodiments of this disclosure can be implemented in software or hardware. The described modules can also be housed in a processor; for example, a processor may be described as including a time-frequency conversion module, a low-frequency domain data acquisition module, an encryption parameter sequence acquisition module, a noise reduction time-domain data acquisition module, and a reordering module. The names of these modules do not necessarily limit the module itself; for example, the time-frequency conversion module may also be described as "a module that converts time-domain data into frequency-domain data."

[0130] In another aspect, this disclosure also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more programs, which, when executed by the device, cause the device to include:

[0131] The initial time-domain data of the single-frame audio to be processed is subjected to time-frequency conversion to obtain the initial frequency-domain data of the single-frame audio to be processed. The initial frequency-domain data includes information on the correspondence between frequency and amplitude. Based on the information on the correspondence between frequency and amplitude in the initial frequency-domain data, the low-frequency frequency-domain data of the single-frame audio to be processed is obtained. Based on the low-frequency frequency-domain data, the encryption parameter sequence of the single-frame audio to be processed is obtained. Based on the low-frequency frequency-domain data and the initial time-domain data, the noise-reduced time-domain data of the single-frame audio to be processed is obtained. The noise-reduced time-domain data is encrypted using the encryption parameter sequence to obtain the encrypted audio data of the single-frame audio to be processed.

[0132] Exemplary embodiments of this disclosure have been specifically shown and described above. It should be understood that this disclosure is not limited to the detailed structures, arrangements, or implementations described herein; rather, this disclosure is intended to cover various modifications and equivalent arrangements contained within the spirit and scope of the appended claims.

Claims

1. An audio processing method, characterized in that, include: The initial time-domain data of the single frame audio to be processed is subjected to time-frequency conversion to obtain the initial frequency-domain data of the single frame audio to be processed. The initial frequency-domain data includes information on the correspondence between frequency and amplitude. The low-amplitude frequency domain data of the single frame audio to be processed is obtained based on the information of the frequency-amplitude correspondence in the initial frequency domain data; The encryption parameter sequence of the single frame audio to be processed is obtained based on the low amplitude frequency domain data; The noise reduction time domain data of the single frame audio to be processed is obtained based on the low amplitude frequency domain data and the initial time domain data; The noise reduction time-domain data is encrypted using the encryption parameter sequence to obtain the encrypted audio data of the single frame audio to be processed.

2. The method according to claim 1, characterized in that, The low-amplitude frequency domain data of the single-frame audio to be processed is obtained based on the frequency-amplitude correspondence information in the initial frequency domain data, including: The low-amplitude frequency domain data refers to the frequency data whose amplitude is lower than a preset amplitude threshold in the initial frequency domain data.

3. The method according to claim 1 or 2, characterized in that, The encryption parameter sequence of the single-frame audio to be processed is obtained based on the low-amplitude frequency domain data, including: The amplitude values ​​corresponding to each frequency in the low-amplitude frequency domain data are used as a random number sequence, and the random number sequence is mapped to the encryption parameter sequence through a nonlinear function.

4. The method according to claim 3, characterized in that, The nonlinear function is an S-shaped growth curve function.

5. The method according to claim 1, wherein The noise reduction temporal data of the single frame audio to be processed is obtained based on the low-amplitude frequency domain data and the initial time domain data, including: The low-amplitude frequency domain data is subjected to the inverse time-frequency conversion process to obtain the noise time domain data of the single frame audio to be processed; The noise time-domain data is removed from the initial time-domain data of the single frame audio to be processed to obtain the noise-reduced time-domain data of the single frame audio to be processed.

6. The method according to claim 1, characterized in that, The denoised time-domain data is encrypted using the encryption parameter sequence to obtain the encrypted audio data of the single frame audio to be processed, including: Obtain the average amplitude of the denoised time-domain data; Multiply each encryption parameter in the encryption parameter sequence by the average amplitude to obtain the scrambling parameter sequence; The scrambling parameter sequence is inserted into the noise reduction time domain data using an audio scrambling algorithm to obtain the encrypted audio data of the single frame audio to be processed.

7. The method according to claim 2 or 5, characterized in that, Also includes: Acquire the audio data to be processed; The audio data to be processed is subjected to frame-by-frame windowing to obtain the initial temporal domain data of multiple frames of single-frame audio to be processed.

8. An audio processing apparatus, characterized in that, include: The time-frequency conversion module is used to perform time-frequency conversion processing on the initial time-domain data of the single frame audio to be processed, so as to obtain the initial frequency-domain data of the single frame audio to be processed. The initial frequency-domain data includes information on the correspondence between frequency and amplitude. The low-amplitude frequency domain data acquisition module is used to obtain the low-amplitude frequency domain data of the single-frame audio to be processed based on the information of the frequency-amplitude correspondence in the initial frequency domain data. An encryption parameter sequence acquisition module is used to obtain the encryption parameter sequence of the single frame audio to be processed based on the low amplitude frequency domain data. The noise reduction time-domain data acquisition module is used to obtain the noise reduction time-domain data of the single frame audio to be processed based on the low amplitude frequency domain data and the initial time-domain data. The out-of-order module is used to encrypt the noise reduction time-domain data using the encryption parameter sequence to obtain the encrypted audio data of the single frame audio to be processed.

9. An apparatus comprising: A memory, a processor, and executable instructions stored in the memory and executable in the processor, characterized in that the processor, when executing the executable instructions, implements the method as described in any one of claims 1-7.

10. A computer-readable storage medium having computer-executable instructions stored thereon, characterized in that, When the executable instructions are executed by the processor, they implement the method as described in any one of claims 1-7.

Images