Method for encrypting and transmitting dysarthria voice based on biological characteristics

Abstract

The invention discloses a dysarthria voice encryption transmission method based on biological characteristics. The method comprises the following steps: acquiring dysarthria voice data and a biological characteristic pattern of a user; converting the dysarthria voice data into a phase spectrum spectrogram and an amplitude spectrum spectrogram; encoding the biological characteristic graph into a biological characteristic code, and inputting the biological characteristic code into a pulse neural network for pulse processing to generate a weighted key graph; equipotential weighting operation is carried out on the phase spectrum spectrogram through the weighted key graph, spectrogram data extraction is carried out on the amplitude spectrum spectrogram and the phase spectrum spectrogram, and a speech spectrum watermark graph is obtained through synthesis; performing AI voice processing through the dysarthria voice data to obtain a false voice spectrogram; performing wavelet decomposition embedding on the speech spectrum watermark and the false speech spectrum to obtain a plurality of wavelet components; aES encryption processing is carried out through a plurality of wavelet components to obtain inverse transformation voice, fusion of personal voice features and biological features of a user is realized, voice encryption transmission of the user is completed, and thus personal information of the user is guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of speech processing, and in particular to a method for encrypting and transmitting dysarthric speech based on biological features. Background Art

[0002] Dysarthria is a pronunciation and movement disorder caused by damage to the central nervous system. This speech dysfunction affects the individual vocal tract and vocal cords, thereby affecting the patient's language expression ability and speech clarity, causing extremely serious troubles to the patient's daily communication. With the continuous development of speech recognition technology, speech recognition technology for various types of dysarthria patients has become increasingly mature. This technology can convert the abnormal pronunciation of dysarthria patients into normal pronunciation, providing these patients with better communication opportunities and platforms, allowing them to communicate smoothly with normal people. However, in these synthetic speech technologies, the pronunciation is automatically synthesized with the help of AI and does not have the characteristic performance of the original sound. Combined with the recent deep fakes and various fake video incidents, this has caused a series of security risks. When criminals use AI-generated voices to commit fraud or other illegal and criminal activities, it will seriously damage the legitimate rights and interests of patients. AI-generated voices are sometimes difficult to distinguish whether they are real voices or AI synthesis. The entire sound clip does not carry any personal characteristics, and there is a certain risk of abuse. For example, criminals may forge voices for illegal activities such as fraud and spreading false information, which poses a serious threat to the information security of the imitated person. Summary of the invention

[0003] The present invention provides a biometric-based method for encrypting and transmitting dysarthric speech, which is used to solve the technical problem that the speech synthesized by the existing speech synthesis technology for users with dysarthria is easily abused, causing hidden dangers to the information security of users.

[0004] The present invention provides a first aspect to provide a method for encrypting and transmitting dysarthric speech based on biometrics, the method comprising:

[0005] Obtaining the user's dysarthric speech data and biometric features;

[0006] Transforming the dysarthric speech data into a phase spectrum spectrogram and an amplitude spectrum spectrogram;

[0007] Encoding the biometric image to obtain a biometric code, inputting the biometric code into a pulse neural network for pulse processing, and generating a weighted key image;

[0008] Performing equal weighting operation on the phase spectrum spectrogram through the weighted key graph, extracting spectrum data from the amplitude spectrum spectrogram and the phase spectrum spectrogram after the equal weighting operation, and synthesizing to obtain a spectrum watermark image;

[0009] Performing AI speech processing on the dysarthria speech data to obtain a false speech spectrogram;

[0010] Performing wavelet decomposition and embedding on the spectrum watermark image and the pseudo-speech spectrum image to obtain multiple wavelet components;

[0011] AES encryption is performed on multiple wavelet components to obtain the inverse transformed speech.

[0012] Specifically, the step of transforming the dysarthric speech data into a phase spectrum spectrogram and an amplitude spectrum spectrogram includes:

[0013] Performing denoising on the dysarthric speech data;

[0014] By performing window processing on the denoised dysarthric speech data, a multi-frame dysarthric speech data is obtained;

[0015] Perform spectrogram conversion on each frame of dysarthria speech data through short-time Fourier transform to generate a pathological speech spectrogram;

[0016] The pathological speech spectrogram is separated and transformed to obtain a phase spectrum spectrogram and an amplitude spectrum spectrogram.

[0017] Specifically, the step of encoding the biometric feature graph to obtain a biometric feature code, inputting the biometric feature code into a spiking neural network for spiking processing, and generating a weighted key graph includes:

[0018] Converting the biological characteristic image into a grayscale image, and performing binarization processing on the grayscale image to obtain a biological binary image;

[0019] Performing discrete coding operation on the biological binary image to obtain a biological characteristic coding signal;

[0020] The biometric encoding signal is input into a pulse neural network for pulse processing, and a pulse signal is output to form a weighted key graph.

[0021] Specifically, the step of performing an equal weighting operation on the phase spectrum spectrogram through a weighted key graph, extracting spectrum data from the amplitude spectrum spectrogram and the phase spectrum spectrogram after the equal weighting operation, and synthesizing the spectrum watermark image includes:

[0022] Performing an equal weighting operation on each pixel block in the phase spectrum graph by using a weighted key graph;

[0023] Extracting spectrum data in the amplitude spectrum spectrogram and the phase spectrum spectrogram after equal weighting operation to obtain phase spectrum data and amplitude spectrum data;

[0024] Combining the extracted phase spectrum data and amplitude spectrum data in the frequency domain to obtain a frequency domain complex image;

[0025] The frequency domain complex image is transformed into a spectral watermark image using inverse Fourier transform.

[0026] Specifically, the step of performing AI speech processing on the dysarthria speech data to obtain a false speech spectrogram includes:

[0027] Based on a preset speech syllable database, the pronunciation feature recognition is performed on the dysarthria speech data to obtain pronunciation feature text data;

[0028] Performing AI speech generation processing on the pronunciation feature text data to obtain fake speech audio data;

[0029] Performing window processing on the fake voice audio data to obtain multiple frames of fake voice audio data;

[0030] Each frame of fake speech audio data is converted into a spectrogram through short-time Fourier transform to generate a fake speech spectrogram.

[0031] Specifically, the step of performing wavelet decomposition and embedding on the spectrogram watermark image and the pseudo-speech spectrogram to obtain a plurality of wavelet components includes:

[0032] Arnold algorithm is used to perform a scrambling operation on the spectral watermark image, and Harr wavelet is used to perform a first-level wavelet decomposition on the scrambled spectral watermark image to obtain multiple detail sub-images and an approximation sub-image at a first-level resolution;

[0033] Downsampling the pseudo speech spectrogram is performed, and Harr wavelet is used to perform three-level wavelet decomposition on the downsampled pseudo speech spectrogram to obtain a plurality of detail sub-graphs and an approximation sub-graph at different resolution levels;

[0034] Combined with the weighted key graph, each subgraph obtained after the first-level wavelet decomposition is embedded into the corresponding subgraph obtained after the third-level wavelet decomposition, and multiple wavelet components are obtained.

[0035] Specifically, the step of combining the weighted key graph, embedding each subgraph obtained after the first-level wavelet decomposition into the corresponding subgraph obtained after the third-level wavelet decomposition, and obtaining multiple wavelet components includes:

[0036] Divide the weighted key graph into multiple weighted key subgraphs, calculate the mean of all pixels in each weighted key subgraph, and use the mean as the embedding coefficient;

[0037] Based on different embedding coefficients, each sub-image obtained after the first-level wavelet decomposition is embedded into the corresponding sub-image obtained after the third-level wavelet decomposition to obtain multiple wavelet components;

[0038] The above embedding process is expressed as:

[0039]

[0040] Where: Represents multiple wavelet components obtained after embedding; Represents the three detail sub-images obtained after the three-level wavelet decomposition. represents the approximation subgraph obtained after three-level wavelet decomposition; Represents the three detail sub-images obtained after the first-level wavelet decomposition, represents the approximation subgraph obtained after the first-level wavelet decomposition; represents the embedding coefficient.

[0041] Specifically, the step of performing AES encryption processing on multiple wavelet components to obtain inverse transformed speech includes:

[0042] Perform AES encryption on the low-frequency part of each wavelet component;

[0043] Performing inverse wavelet transform on all encrypted wavelet components to synthesize and obtain a synthesized encrypted sound spectrum;

[0044] The synthesized encrypted sound spectrum is converted by a vocoder to obtain an inverse transformed speech.

[0045] The second aspect of the present invention further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of any of the above-mentioned methods for encrypting and transmitting dysarthric speech are implemented.

[0046] The third aspect of the present invention further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of any of the above-mentioned methods for encrypting and transmitting dysarthric speech are implemented.

[0047] It can be seen from the above technical solutions that the present invention has the following advantages:

[0048] The present invention provides a biometric-based encrypted transmission method for dysarthric speech, the method comprising: acquiring dysarthric speech data and a biometric map of a user; transforming the dysarthric speech data into a phase spectrum spectrogram and an amplitude spectrum spectrogram; encoding the biometric map to obtain a biometric code, inputting the biometric code into a pulse neural network for pulse processing, and generating a weighted key map; performing an equal-weighting operation on the phase spectrum spectrogram through the weighted key map, extracting spectrogram data from the amplitude spectrum spectrogram and the phase spectrum spectrogram after the equal-weighting operation, and synthesizing to obtain a spectrogram watermark map; performing AI speech processing on the dysarthric speech data to obtain a false speech spectrogram; performing wavelet decomposition and embedding on the spectrogram watermark map and the false speech spectrogram to obtain multiple wavelet components; performing AES encryption processing on the multiple wavelet components to obtain an inverse transformed speech.

[0049] In the present invention, the biometric information of the user is pulse-encoded by a pulse neural network, a weighted key graph of the biometrics is constructed, and the weighted key graph is used to weight the dysarthria speech representation information, thereby obtaining a spectrogram watermark graph that integrates the dysarthria audio features and the biometric information, so that the synthesized speech carries personal characteristics to enhance the protection of the user's speech feature information; the important information is further encrypted by wavelet transform and AES encryption algorithm, and the spectrogram watermark graph is embedded into the pseudo speech spectrogram generated by AI, so that the user's pathological speech can realize personal information encryption during the generation and conversion and transmission process, so that the inverse transformed speech has robust characteristics and security performance during the transmission process, thereby solving the technical problem that the speech synthesized by the existing speech synthesis technology for users with dysarthria is easily abused and causes information security risks to users. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0051] Figure 1 A flowchart of a method for encrypting and transmitting dysarthric speech based on biometric features provided by an embodiment of the present invention;

[0052] Figure 2 A data processing flow chart of a biometric-based dysarthric speech encryption transmission method provided by an embodiment of the present invention;

[0053] Figure 3 This is an example diagram of the process of generating a weighted key graph provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The embodiment of the present invention provides a method for encrypting and transmitting dysarthric speech based on biometric features, which is used to solve the technical problem that the speech synthesized by the existing speech synthesis technology for users with dysarthria is easily abused, causing potential information security risks for users.

[0055] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] See also Figure 1 and Figure 2 The first aspect of the present invention provides a method for encrypting and transmitting dysarthric speech based on biometric features, the method comprising:

[0057] Step 101, obtaining the user's dysarthric speech data and biometric feature map.

[0058] In this step, the dysarthric speech data (pathological speech data) and biometric map of the patient with dysarthria are obtained to extract the pronunciation characteristics of the patient and synthesize the sound with the biometric information, so as to protect the patient's personal information. The dysarthric speech data may include syllable data such as single vowels, compound vowels, consonants and nasal vowels; and the biometric map may be an image with the patient's fingerprint data collected.

[0059] Step 102, transforming the dysarthric speech data into a phase spectrum spectrogram and an amplitude spectrum spectrogram.

[0060] Specifically, the dysarthric speech data is first denoised; then, the denoised dysarthric speech data is windowed to obtain multiple frames of dysarthric speech data; then, each frame of dysarthric speech data is spectrogram-converted by short-time Fourier transform (STFT) to generate a pathological speech spectrogram; finally, the pathological speech spectrogram is separated and transformed to obtain a phase spectrum spectrogram and an amplitude spectrum spectrogram.

[0061] It should be noted that the dysarthric speech data is a continuous time series, and the frequency domain information of the signal needs to be extracted through short-time Fourier transform; since the Fourier transform assumes that the signal is stationary, it is necessary to use a window function (Hamming window) to divide the dysarthric speech data signal into short-time frames. For example, each frame is set to a length of 20ms and an overlap of 50% (for example, each frame is 25ms and the overlap is 12.5ms). This overlap can help capture more subtle changes. At the same time, dividing it into short time frames and applying a window function can reduce edge effects.

[0062] Subsequently, a short-time Fourier transform is applied to each frame to convert the signal from the time domain to the frequency domain to obtain the spectrum of the frame; the frequency components of each frame are analyzed by short-time Fourier transform, and its power spectrum density is calculated. The power value is converted to a logarithmic scale to adapt to the nonlinear perception of sound intensity by the human ear. Finally, color mapping is used to convert the numerical value of the power spectrum into images of different colors to form a clear pathological speech spectrogram; in the pathological speech spectrogram: the horizontal axis represents time, the vertical axis represents frequency, and the color depth shows the sound intensity.

[0063] It should be noted that the power spectrum refers to the power contained in the different frequency components of the signal itself. By examining the power spectrum of a signal, we can determine which frequencies exist in the signal and their relative strengths. Taking the square of the STFT result |X(t,f)|^2, it represents the power of each frequency component. STFT calculates the frequency component X of the signal at each moment.

[0064] Finally, in order to facilitate the separation and transformation of the pathological speech spectrogram, the pathological speech spectrogram can be adjusted to a size of 224x224 pixels, and then the pathological speech spectrogram can be separated into two parts, a phase spectrum spectrogram and an amplitude spectrum spectrogram, using Fourier transform.

[0065] Step 103, encoding the biometric image to obtain a biometric code, inputting the biometric code into a pulse neuron for pulse processing, and generating a weighted key image.

[0066] This step specifically includes:

[0067] Sub-step 1031, converting the biological characteristic image into a grayscale image, and performing binarization processing on the grayscale image to obtain a biological binary image.

[0068] In this sub-step, grayscale and binarization processing is performed based on the obtained biological characteristic image to obtain a set of black and white stripe images (biological binary images).

[0069] Grayscale refers to the process of converting a color image (usually an RGB image) into a grayscale image. In a grayscale image, the value of each pixel represents the brightness of the pixel and no longer contains color information. A grayscale image usually has only a single channel (i.e., an intensity value), so it is simpler and easier to process than a color image. In the RGB color space, each pixel of a color image consists of three channels (red R, green G, blue B). The grayscale process usually combines the values ​​of these three channels into a single grayscale value through the following weighted formula:

[0070] Gray=0.2989×R+0.5870×G+0.1140×B

[0071] In the above weighted formula, 0.2989, 0.5870 and 0.1140 represent weighted coefficients, which represent the sensitivity of the human eye to different colors. Among them, the green channel has the greatest impact on brightness, followed by red, and blue has the least. Each pixel value in a grayscale image is usually between 0 and 255 (for 8-bit grayscale images), representing different brightness from black to white. The grayscale image only reflects the brightness information of the object, and does not contain color information.

[0072] Among them, binarization refers to the process of converting a grayscale image into an image containing only two colors (usually black and white). In biological binary images, the value of each pixel is either black or white, and is usually used to extract significant features or objects in the image. The core of binarization is to select a threshold and then compare each pixel value of the image with the threshold. The binarized image has only two pixel values ​​(0 and 255), corresponding to black and white respectively. In biological binary images, black usually represents the background or irrelevant areas, and white represents the target or object of interest.

[0073] Sub-step 1032, performing discrete coding operation on the biological binary image to obtain a biological characteristic coding signal.

[0074] It can be understood that the discrete coding operation is to map the white and black in the biological binary image into digital codes such as 0 and 1 respectively. By expressing the pixel relationship through this mapping relationship, a biological binary image is transformed into a 01 code map. Stretching the 01 code map can be regarded as a series of spike sequences, similar to discrete non-continuous sequence signals.

[0075] Sub-step 1033, input the biometric encoding signal into the pulse neural network for pulse processing, and output the pulse signal to form a weighted key graph.

[0076] Among them, spiking neural networks (SNN) are a type of neural network that simulates the dynamic behavior of biological neural networks, in which neurons transmit information through pulses (spikes). In SNN, the transmission of information not only depends on the activation value of neurons, but also through the pulses (or spikes) emitted by neurons. This model is closer to the working mode of biological brains than traditional artificial neural networks (ANN), especially in the processing of time series information and dynamic events.

[0077] Spike processing refers to the process of converting input signals into neuron spikes in SNN. Since spiking neural networks process discrete spikes rather than continuous activation values, a coding strategy must be used to map input information to the spike activity of neurons.

[0078] In this step, the biometric encoding signal is continuously input into the neurons of the SNN to simulate the human electrical signal triggering mechanism. 0 is regarded as no stimulation, and 1 is regarded as being stimulated and responding. Under continuous stimulation, the trigger condition threshold will be met under continuous triggering. At this time, the neuron will respond and output a pulse signal, which indicates that the data can meet the discrimination condition set in this example at this moment. Then, based on the pulse signal result output as a point, a unique biometric information key is constructed to obtain a weighted key graph.

[0079] More specifically, the biological binary image is further optimized into a unique 01 sequence, similar to convolution processing. At the input of each pixel, the pulse neural network will determine whether a trigger response will be caused to express the pixel at the moment. This expression makes each pixel in the image become 0 and a pulse trigger representation. In this example, as shown in the figure, it is possible to consider setting the first pulse after 0 to a fixed value d1, and then continuous pulse outputs appear, and reweighting is used to form d2=d1(1+D*d1), where D represents the feedforward influence weight, so that a complete digital code can be formed instead of pixel representation, thereby obtaining the required weighted key map.

[0080] Step 104, perform an equal weighting operation on the phase spectrum spectrogram through the weighted key map, extract spectrum data from the amplitude spectrum spectrogram and the phase spectrum spectrogram after the equal weighting operation, and synthesize to obtain a spectrum watermark map.

[0081] This step specifically includes:

[0082] Sub-step 1041, performing an equal weighting operation on each pixel block in the phase spectrum graph through the weighted key graph.

[0083] It should be noted that each image has a fixed value in each pixel block, and this embodiment performs an equal weighting operation on each pixel block in the phase spectrum spectrogram according to the weighted key map extracted from the biometric information;

[0084] Among them, the following formula is used for equal weighting operation:

[0085]

[0086] Where: The pixel data after weighting in the nth pixel block; Represents the original pixel data in the nth pixel block, represents the current weighted value, m represents the weighted weight, represents the weighted influence of the previous bit, and k represents the influence attenuation factor, where 0 <k<1。

[0087] Sub-step 1042, extracting the spectrum data in the amplitude spectrum spectrogram and the phase spectrum spectrogram after the equal weighting operation to obtain the phase spectrum data And amplitude spectrum data M(u,v).

[0088] Sub-step 1043, combining the extracted phase spectrum data and amplitude spectrum data in the frequency domain to obtain a frequency domain complex image F(u,v).

[0089] Sub-step 1044, using inverse Fourier transform to transform the frequency domain complex image into a spectral watermark image W. The spectral watermark image W carries biometric information.

[0090] Step 105, performing AI speech processing on the dysarthria speech data to obtain a false speech spectrogram.

[0091] This step specifically includes:

[0092] Sub-step 1051 , based on a preset speech syllable database, pronunciation feature recognition is performed on the dysarthria speech data to obtain pronunciation feature text data.

[0093] The process of constructing the preset speech syllable database includes: in an environment without background noise, recording the speech data of normal people and patients with articulation disorders (who have undergone a comprehensive medical assessment), including vowels, compound vowels, consonants and nasal vowels; classifying the audio according to the syllable category represented by the audio, taking the audio files that meet the sound quality standards, and uniformly cutting them to one second in length for storage.

[0094] This embodiment trains an intelligent network recognition model through a preset speech syllable database, and uses the trained intelligent network recognition model to recognize the pronunciation features of the dysarthria speech data. Then, the pronunciation features of the patient are recognized according to the CNN binary classification algorithm, and the patient's pronunciation is converted into the normal pronunciation of a normal user. Finally, the converted normal pronunciation is converted into a corresponding text to obtain pronunciation feature text data.

[0095] Sub-step 1052, performing AI speech generation processing on the pronunciation feature text data to obtain fake speech audio data.

[0096] In this embodiment, various AI-generated speech such as text to speech can be used to generate corresponding audio representations to obtain fake speech audio data.

[0097] Sub-step 1053, performing window processing on the fake speech audio data to obtain multiple frames of fake speech audio data.

[0098] It is understandable that the window processing here is consistent with the processing in step 102, and will not be described in detail here.

[0099] Sub-step 1054, performing spectrogram conversion on each frame of false speech audio data by short-time Fourier transform to generate a false speech spectrogram S'.

[0100] It can be understood that the process of converting the audio data into a spectrogram by short-time Fourier transform here is consistent with the conversion process in step 102, and will not be described in detail here.

[0101] Step 106, performing wavelet decomposition and embedding on the spectral watermark image and the pseudo-speech spectrogram to obtain multiple wavelet components.

[0102] This step specifically includes:

[0103] Sub-step 1061, using the Arnold algorithm to perform a scrambling operation on the spectral watermark image W, and using the Harr wavelet to perform a first-level wavelet decomposition on the scrambled spectral watermark image W' to obtain multiple detail sub-images at a first-level resolution. and an approximation subgraph .

[0104] The Arnold algorithm usually means mapping the pixel coordinates of an image to another position through a specific linear transformation, resulting in a change in the pixel distribution of the image; the basic idea is to rotate, displace, and perform other operations on the image within a matrix framework, so that each transformation causes a nonlinear change in the structure of the image; through this mapping, the original image gradually becomes unrecognizable, but after several transformations, if the same transformation is applied again, it will return to the original image.

[0105] Suppose there is an image of size N×N, where the position of each pixel can be represented by a two-dimensional coordinate, namely (x, y), where x, y∈{0,1,2,...,N−1}.

[0106] The Arnold transformation can be defined by the following formula:

[0107]

[0108] Among them, (x′, y′) is the new position after Arnold transformation, (x, y) represents the original position, and N is the size of the image; Arnold transformation is to "distort" or "transform" the position of each pixel in the image to achieve image confusion. The matrix in the formula is called the Arnold matrix, which is the core of mapping pixels from one position to another.

[0109] Among them, wavelet transform is a commonly used signal processing technology, which is widely used in image encryption, compression, denoising and other fields. It can provide more efficient time-frequency analysis than traditional Fourier transform by decomposing the image into sub-bands of different scales. Wavelet transform is based on wavelet function, which decomposes the image into multiple frequency bands by recursively decomposing the image, usually including the following parts:

[0110] Low-frequency part: contains most of the information of the image, representing the rough structure or details of the image; high-frequency part: captures the details of the image, such as edges, textures, etc.; The key advantage of wavelet transform is that it can switch between spatial domain and frequency domain, allowing images to be analyzed at different scales.

[0111] The wavelet transform is used to perform a first-level decomposition of the image, dividing it into a low-frequency part (LL) and a high-frequency part (LH, HL, HH). Specifically, the LL (low-frequency) part mainly contains the rough outline of the image; the LH (low-high frequency), HL (high-low frequency), and HH (high-high frequency) parts contain detailed information of the image, such as edges, textures, etc.

[0112] Sub-step 1062, down-sampling processing is performed on the pseudo speech spectrogram, and Harr wavelet is used to perform three-level wavelet decomposition on the down-sampled pseudo speech spectrogram to obtain multiple detail sub-graphs at different resolution levels and an approximation sub-graph.

[0113] In this sub-step, the pseudo language spectrogram S' is downsampled to reduce the image size; then the downsampled pseudo speech spectrogram is decomposed by Harr wavelet three-level decomposition to obtain multiple detail sub-images at different resolution levels. and an approximation subgraph .

[0114] Downsampling is usually referred to as the process of reducing the resolution or size of an image. It reduces the amount of data by reducing the pixel density in the image. The most common methods are maximum sampling or average sampling. For example, when downsampling an image, each 2x2 or 3x3 pixel block may be averaged to obtain a smaller image, and maximum pooling is used to take the maximum pixel value in the area as the new pixel.

[0115] Among them, wavelet multi-level decomposition can further extract the detailed information of the image by performing wavelet transform on the low-frequency part (LL) again, thereby achieving a deeper decomposition and obtaining more frequency bands.

[0116] Sub-step 1063 , combining the weighted key graph, embedding each sub-graph obtained after the first-level wavelet decomposition into the corresponding sub-graph obtained after the third-level wavelet decomposition, to obtain multiple wavelet components.

[0117] Specifically, the weighted key graph is divided into four weighted key sub-graphs, the mean of all pixels in each weighted key sub-graph is calculated, and the mean is used as the embedding coefficient; based on different embedding coefficients, each sub-graph obtained after the first-level wavelet decomposition is embedded into the corresponding sub-graph obtained after the third-level wavelet decomposition to obtain multiple wavelet components; in the embedding process, the same-frequency merging method is used to generate new wavelet components.

[0118] The above embedding process is expressed as:

[0119]

[0120] Where: Represents multiple wavelet components obtained after embedding; Represents the three detail sub-images obtained after the three-level wavelet decomposition. represents the approximation subgraph obtained after three-level wavelet decomposition; Represents the three detail sub-images obtained after the first-level wavelet decomposition, represents the approximation subgraph obtained after the first-level wavelet decomposition; represents the embedding coefficient.

[0121] Step 107, performing AES encryption processing on multiple wavelet components to obtain inverse transformed speech.

[0122] Specifically, firstly, the low-frequency part of each wavelet component is encrypted by AES; then, all encrypted wavelet components are subjected to Harr wavelet inverse transform to synthesize the synthesized encrypted sound spectrum; finally, the synthesized encrypted sound spectrum is transformed by a vocoder to obtain the inverse transformed speech. The vocoder is a speech synthesis tool, such as the HiFi-GAN vocoder.

[0123] Among them, AES is a group-based symmetric key encryption algorithm, which means that it uses the same key for encryption and decryption operations; the AES encryption algorithm uses group encryption and is usually used to encrypt fixed-size data blocks, which are 128 bits (16 bytes). The AES algorithm supports three different key lengths: 128, 192, and 256 bits, corresponding to different encryption security levels.

[0124] The present invention can transform the patient's dysarthria speech data into inverse transformed speech, which can accurately express the user's emotions for the patient's daily communication. The inverse transformed speech finally outputted can have robust characteristics and security performance during the transmission process, thereby protecting the user's legitimate rights and interests and preventing criminals from using the user's audio data to commit fraud or other illegal activities.

[0125] The present invention has the following advantages:

[0126] 1. Using pulse neural network to extract the user's biometric digital watermark key can achieve the mixing of personal biometric information and unique key, and realize more covert information function transmission.

[0127] 2. A complete set of digital watermark insertion technology for spoken word audio has been constructed, using spectrogram and biometric watermark insertion to achieve encrypted transmission of personal information, thereby ensuring the security of personal information for patients with articulation disorders using speech generators.

[0128] 3. Obtain the user's pronunciation characteristics based on the dysarthria speech data, and combine the user's biological characteristics to generate natural speech, thereby accurately expressing the user's emotions.

[0129] The second aspect of the present invention further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of any of the above-mentioned methods for encrypting and transmitting dysarthric speech are implemented.

[0130] The third aspect of the present invention further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of any of the above-mentioned methods for encrypting and transmitting dysarthric speech are implemented.

[0131] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0132] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0133] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0134] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0135] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0136] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A biometric-based method for dysarthric speech encryption and transmission, characterized in that: The method comprises: Obtaining the user's dysarthric speech data and biometric features; Transforming the dysarthric speech data into a phase spectrum spectrogram and an amplitude spectrum spectrogram; Encoding the biometric image to obtain a biometric code, inputting the biometric code into a pulse neural network for pulse processing, and generating a weighted key image; Performing equal weighting operation on the phase spectrum spectrogram through the weighted key graph, extracting spectrum data from the amplitude spectrum spectrogram and the phase spectrum spectrogram after the equal weighting operation, and synthesizing to obtain a spectrum watermark image; Performing AI speech processing on the dysarthria speech data to obtain a false speech spectrogram; Performing wavelet decomposition and embedding on the spectrum watermark image and the pseudo-speech spectrum image to obtain multiple wavelet components; AES encryption is performed on multiple wavelet components to obtain the inverse transformed speech.

2. The method for encrypting and transmitting dysarthric speech according to claim 1, characterized in that: The step of transforming the dysarthric speech data into a phase spectrum spectrogram and an amplitude spectrum spectrogram comprises: Performing denoising on the dysarthric speech data; By performing window processing on the denoised dysarthric speech data, a multi-frame dysarthric speech data is obtained; Perform spectrogram conversion on each frame of dysarthria speech data through short-time Fourier transform to generate a pathological speech spectrogram; The pathological speech spectrogram is separated and transformed to obtain a phase spectrum spectrogram and an amplitude spectrum spectrogram.

3. The method for encrypting and transmitting dysarthric speech according to claim 1, characterized in that: The step of encoding the biometric characteristic graph to obtain a biometric characteristic code, inputting the biometric characteristic code into a pulse neural network for pulse processing, and generating a weighted key graph comprises: Converting the biological characteristic image into a grayscale image, and performing binarization processing on the grayscale image to obtain a biological binary image; Performing discrete coding operation on the biological binary image to obtain a biological characteristic coding signal; The biometric encoding signal is input into a pulse neural network for pulse processing, and a pulse signal is output to form a weighted key graph.

4. The method for encrypting and transmitting dysarthric speech according to claim 1, characterized in that: The step of performing an equal weighting operation on the phase spectrum spectrogram through a weighted key graph, extracting spectrum data from the amplitude spectrum spectrogram and the phase spectrum spectrogram after the equal weighting operation, and synthesizing to obtain a spectrum watermark graph comprises: Performing an equal weighting operation on each pixel block in the phase spectrum graph by using a weighted key graph; Extracting spectrum data in the amplitude spectrum spectrogram and the phase spectrum spectrogram after equal weighting operation to obtain phase spectrum data and amplitude spectrum data; Combining the extracted phase spectrum data and amplitude spectrum data in the frequency domain to obtain a frequency domain complex image; The frequency domain complex image is transformed into a spectral watermark image using inverse Fourier transform.

5. The method for encrypting and transmitting dysarthric speech according to claim 1, characterized in that: The step of performing AI speech processing on the dysarthria speech data to obtain a false speech spectrogram comprises: Based on a preset speech syllable database, the pronunciation feature recognition is performed on the dysarthria speech data to obtain pronunciation feature text data; Performing AI speech generation processing on the pronunciation feature text data to obtain fake speech audio data; Performing window processing on the fake voice audio data to obtain multiple frames of fake voice audio data; Each frame of fake speech audio data is converted into a spectrogram through short-time Fourier transform to generate a fake speech spectrogram.

6. The method for encrypting and transmitting dysarthric speech according to claim 1, characterized in that: The step of performing wavelet decomposition and embedding on the spectrogram watermark image and the pseudo speech spectrogram to obtain a plurality of wavelet components comprises: Arnold algorithm is used to perform a scrambling operation on the spectral watermark image, and Harr wavelet is used to perform a first-level wavelet decomposition on the scrambled spectral watermark image to obtain multiple detail sub-images and an approximation sub-image at a first-level resolution; Downsampling the pseudo speech spectrogram is performed, and Harr wavelet is used to perform three-level wavelet decomposition on the downsampled pseudo speech spectrogram to obtain a plurality of detail sub-graphs and an approximation sub-graph at different resolution levels; Combined with the weighted key graph, each subgraph obtained after the first-level wavelet decomposition is embedded into the corresponding subgraph obtained after the third-level wavelet decomposition, and multiple wavelet components are obtained.

7. The method for encrypting and transmitting dysarthric speech according to claim 6, characterized in that: The step of combining the weighted key graph and embedding each subgraph obtained after the first-level wavelet decomposition into the corresponding subgraph obtained after the third-level wavelet decomposition to obtain multiple wavelet components includes: Divide the weighted key graph into multiple weighted key subgraphs, calculate the mean of all pixels in each weighted key subgraph, and use the mean as the embedding coefficient; Based on different embedding coefficients, each sub-image obtained after the first-level wavelet decomposition is embedded into the corresponding sub-image obtained after the third-level wavelet decomposition to obtain multiple wavelet components; The above embedding process is expressed as: Where: Represents multiple wavelet components obtained after embedding; Represents the three detail sub-images obtained after the three-level wavelet decomposition. represents the approximation subgraph obtained after three-level wavelet decomposition; Represents the three detail sub-images obtained after the first-level wavelet decomposition, represents the approximation subgraph obtained after the first-level wavelet decomposition; represents the embedding coefficient.

8. The method for encrypting and transmitting dysarthric speech according to claim 1, characterized in that: The step of performing AES encryption processing on multiple wavelet components to obtain inverse transformed speech comprises: Perform AES encryption on the low-frequency part of each wavelet component; Performing inverse wavelet transform on all encrypted wavelet components to synthesize and obtain a synthesized encrypted sound spectrum; The synthesized encrypted sound spectrum is converted by a vocoder to obtain an inverse transformed speech.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method for encrypting and transmitting dysarthric speech as described in any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for encrypting and transmitting dysarthric speech as described in any one of claims 1 to 8 are implemented.

Images