A method for encryption transmission of dysarthria speech based on biometrics

Abstract

The application discloses a dysarthria speech encryption transmission method based on biological characteristics, which comprises the following steps: acquiring dysarthria speech data and biological characteristic maps of a user; transforming the dysarthria speech data into a phase spectrum spectrogram and an amplitude spectrum spectrogram; encoding the biological characteristic maps into biological characteristic codes and inputting the codes into a pulse neural network for pulse processing to generate a weighted key map; performing equal-weighted operation on the phase spectrum spectrogram through the weighted key map, performing spectrum data extraction on the amplitude spectrum spectrogram and the phase spectrum spectrogram, and synthesizing a speech spectrum watermark map; performing AI speech processing on the dysarthria speech data to obtain a paralog spectrum map; performing wavelet decomposition embedding on the speech spectrum watermark map and the paralog spectrum map to obtain a plurality of wavelet components; and performing AES encryption processing on the plurality of wavelet components to obtain inverse transformation speech, so as to realize the fusion of the personal sound characteristics and the biological characteristics of the user, complete the sound encryption transmission of the user, and thus guarantee the personal information of the user.

Description

Technical Field

[0001] The present invention relates to the technical field of speech processing, and in particular to a biometric-based method for encrypting and transmitting dysarthric speech. Background Art

[0002] Dysarthria is a speech and movement disorder caused by damage to the central nervous system. This speech dysfunction affects the vocal tract and vocal cords, hindering the patient's ability to express themselves and speech clarity, causing significant difficulties in daily communication. With the continuous development of speech recognition technology, speech recognition technology for patients with various types of dysarthria is becoming increasingly mature. This technology can convert the abnormal pronunciation of dysarthria patients into normal pronunciation, providing these patients with better communication opportunities and platforms, allowing them to communicate smoothly with normal people. However, in these synthetic speech technologies, the pronunciation is automatically synthesized using AI and lacks the characteristic expression of the original voice. Combined with the recent emergence of deepfakes and various fake video incidents, this raises a series of security risks. When criminals use AI-generated speech to commit fraud or other illegal activities, it seriously undermines the legitimate rights and interests of patients. AI-generated speech is sometimes difficult to distinguish between real voices and AI synthesis, and the entire sound clip lacks any personal characteristics, posing a certain risk of misuse. For example, criminals may forge voices for fraud, dissemination of false information, and other illegal activities, posing a serious threat to the information security of those being imitated. Summary of the Invention

[0003] The present invention provides a biometric-based method for encrypting and transmitting dysarthric speech, which is used to solve the technical problem that the speech synthesized by the existing speech synthesis technology used for users with dysarthria is easily abused, causing potential information security risks for users.

[0004] The present invention provides a first aspect of a method for encrypting and transmitting dysarthric speech based on biometric features, the method comprising:

[0005] Obtaining the user's dysarthric speech data and biometric features;

[0006] Performing transformation processing on the dysarthric speech data to transform it into a phase spectrum spectrogram and an amplitude spectrum spectrogram;

[0007] Encoding the biometric image to obtain a biometric code, inputting the biometric code into a spiking neural network for spiking processing to generate a weighted key image;

[0008] Performing an equal-weighted operation on the phase spectrum spectrogram through a weighted key graph, extracting spectrum data from the amplitude spectrum spectrogram and the phase spectrum spectrogram after the equal-weighted operation, and synthesizing them to obtain a spectrum watermark image;

[0009] Performing AI speech processing on the dysarthric speech data to obtain a false speech spectrogram;

[0010] Performing wavelet decomposition and embedding on the spectrum watermark image and the false speech spectrum image to obtain multiple wavelet components;

[0011] AES encryption is performed on multiple wavelet components to obtain the inverse transformed speech.

[0012] Specifically, the step of transforming the dysarthric speech data into a phase spectrum spectrogram and an amplitude spectrum spectrogram includes:

[0013] performing denoising processing on the dysarthric speech data;

[0014] By performing window processing on the denoised dysarthric speech data, a multi-frame dysarthric speech data is obtained;

[0015] Perform spectrogram conversion on each frame of dysarthric speech data through short-time Fourier transform to generate a pathological speech spectrogram;

[0016] The pathological speech spectrogram is separated and transformed to obtain a phase spectrum spectrogram and an amplitude spectrum spectrogram.

[0017] Specifically, the step of encoding the biometric image to obtain a biometric code, inputting the biometric code into a spiking neural network for spiking processing, and generating a weighted key image includes:

[0018] Converting the biological characteristic image into a grayscale image and performing binarization processing on the grayscale image to obtain a biological binary image;

[0019] Performing discrete coding operation on the biological binary image to obtain a biological characteristic coding signal;

[0020] The biometric encoding signal is input into a pulse neural network for pulse processing, and a pulse signal is output to form a weighted key graph.

[0021] Specifically, the step of performing an equal-weighted operation on the phase spectrum spectrogram through a weighted key graph, extracting spectrum data from the amplitude spectrum spectrogram and the phase spectrum spectrogram after the equal-weighted operation, and synthesizing the spectrum watermark image includes:

[0022] Performing an equal weighting operation on each pixel block in the phase spectrum graph through a weighted key graph;

[0023] Extracting spectrum data from the amplitude spectrum spectrogram and the phase spectrum spectrogram after the equal weighting operation to obtain phase spectrum data and amplitude spectrum data;

[0024] Combining the extracted phase spectrum data and amplitude spectrum data in the frequency domain to obtain a frequency domain complex image;

[0025] The frequency domain complex image is transformed into a spectral watermark image using inverse Fourier transform.

[0026] Specifically, the step of performing AI speech processing on the dysarthric speech data to obtain a false speech spectrogram includes:

[0027] Based on a preset speech syllable database, pronunciation feature recognition is performed on the dysarthric speech data to obtain pronunciation feature text data;

[0028] Performing AI speech generation processing on the pronunciation feature text data to obtain fake speech audio data;

[0029] Performing window processing on the fake voice audio data to obtain multiple frames of fake voice audio data;

[0030] Each frame of fake speech audio data is converted into a spectrogram through short-time Fourier transform to generate a fake speech spectrogram.

[0031] Specifically, the step of performing wavelet decomposition and embedding on the spectral watermark image and the false speech spectrogram to obtain multiple wavelet components includes:

[0032] The spectral watermark image is scrambled using the Arnold algorithm, and the scrambled spectral watermark image is decomposed by a first-level wavelet using the Harr wavelet to obtain a plurality of detail sub-images and an approximation sub-image at a first-level resolution;

[0033] Downsampling the pseudo speech spectrogram is performed, and Harr wavelet is used to perform three-level wavelet decomposition on the downsampled pseudo speech spectrogram to obtain a plurality of detail subgraphs and an approximation subgraph at different resolution levels;

[0034] Combined with the weighted key graph, each subgraph obtained after the first-level wavelet decomposition is embedded into the corresponding subgraph obtained after the third-level wavelet decomposition to obtain multiple wavelet components.

[0035] Specifically, the step of combining the weighted key graph and embedding each subgraph obtained after the first-level wavelet decomposition into the corresponding subgraph obtained after the third-level wavelet decomposition to obtain multiple wavelet components includes:

[0036] Divide the weighted key graph into multiple weighted key subgraphs, calculate the mean of all pixels in each weighted key subgraph, and use the mean as the embedding coefficient;

[0037] Based on different embedding coefficients, each sub-image obtained after the first-level wavelet decomposition is embedded into the corresponding sub-image obtained after the third-level wavelet decomposition to obtain multiple wavelet components;

[0038] The embedding process is expressed as:

[0039]

[0040] In the formula: represents a plurality of wavelet components obtained after embedding; represents three detail subgraphs obtained after three-level wavelet decomposition, represents an approximation subgraph obtained after three-level wavelet decomposition; represents three detail subgraphs obtained after one-level wavelet decomposition, represents an approximation subgraph obtained after one-level wavelet decomposition; represents embedding coefficients.

[0041] Specifically, the step of performing AES encryption processing on the plurality of wavelet components to obtain inverse-transform speech includes:

[0042] AES encryption is performed on the low-frequency part in each wavelet component;

[0043] All encrypted wavelet components are inverse wavelet transformed to synthesize an encrypted sound spectrum;

[0044] The encrypted sound spectrum is converted by a vocoder to obtain inverse-transform speech.

[0045] The second aspect of the present application also provides a computer device including a memory and a processor, the memory stores a computer program, and the processor implements the steps of the speech encryption transmission method for dysarthria according to any one of the above aspects when executing the computer program.

[0046] The third aspect of the present application also provides a computer-readable storage medium having a computer program stored thereon, and the computer program implements the steps of the speech encryption transmission method for dysarthria according to any one of the above aspects when executed by a processor.

[0047] As can be seen from the above technical solutions, the present application has the following advantages:

[0048] The present invention provides a biometric-based method for encrypting and transmitting dysarthric speech, which includes: acquiring dysarthric speech data and a biometric map of a user; transforming the dysarthric speech data into a phase spectrum spectrogram and an amplitude spectrum spectrogram; encoding the biometric map to obtain a biometric code, inputting the biometric code into a pulse neural network for pulse processing, and generating a weighted key map; performing an equal-weighted operation on the phase spectrum spectrogram through the weighted key map, extracting spectrogram data from the amplitude spectrum spectrogram and the phase spectrum spectrogram after the equal-weighted operation, and synthesizing them to obtain a spectrogram watermark map; performing AI speech processing on the dysarthric speech data to obtain a false speech spectrogram; performing wavelet decomposition and embedding on the spectrogram watermark map and the false speech spectrogram to obtain multiple wavelet components; and performing AES encryption processing on the multiple wavelet components to obtain inverse transformed speech.

[0049] In the present invention, the user's biometric information is pulse-encoded through a pulse neural network to construct a weighted key graph of the biometrics, and the weighted key graph is used to achieve weighting of the dysarthria speech representation information, thereby obtaining a spectral watermark graph that integrates the dysarthria audio features and biometric information, so that the synthesized speech carries personal characteristics to enhance the protection of the user's speech feature information; the important information is further encrypted through wavelet transform and AES encryption algorithm, and the spectral watermark graph is embedded into the AI-generated fake speech spectrogram, so that the user's pathological speech can realize personal information encryption during the generation and conversion transmission process, so that the inverse transformed speech has robust characteristics and security performance during the transmission process, thereby solving the technical problem that the speech synthesized by the existing speech synthesis technology for users with dysarthria is easily abused and causes information security risks to users. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0051] Figure 1 A flowchart of a method for encrypting and transmitting dysarthric speech based on biometric features provided by an embodiment of the present invention;

[0052] Figure 2 A data processing flow chart of a biometric-based method for dysarthric speech encryption and transmission provided by an embodiment of the present invention;

[0053] Figure 3 This is an example diagram of the process of generating a weighted key graph provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The embodiment of the present application provides a speech encryption transmission method based on a biological feature for dysarthria, which is used for solving the technical problem that the speech synthesized by the speech synthesis technology used by a user suffering from dysarthria is easy to be misused to cause information security risks of the user.

[0055] In order to make the technical problems, features and advantages of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application. Obviously, the following described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the present application.

[0056] Please refer to Figure 1 and Figure 2 , the first aspect of the present application provides a speech encryption transmission method based on a biological feature for dysarthria, the method comprising:

[0057] Step 101, acquiring dysarthria speech data and biological feature map of a user.

[0058] In this step, the dysarthria speech data (pathological speech data) and the biological feature map of the patient suffering from dysarthria are acquired to extract the pronunciation features and biological feature information of the patient to synthesize the sound for protecting the personal information of the patient. The dysarthria speech data can include monophthong, diphthong, consonant and nasal vowel syllable data; and the biological feature map can be an image collecting the fingerprint data of the patient.

[0059] Step 102, transforming the dysarthria speech data to generate a phase spectrum spectrogram and an amplitude spectrum spectrogram.

[0060] Specifically, first, the dysarthria speech data is denoised; then, the dysarthria speech data after the denoising is windowed to obtain multiple frames of dysarthria speech data; then, the short-time Fourier transform (STFT) is performed on each frame of dysarthria speech data to generate a pathological speech spectrogram; finally, the pathological speech spectrogram is separated and transformed to obtain a phase spectrum spectrogram and an amplitude spectrum spectrogram.

[0061] It's important to note that dysarthric speech data is a continuous time series, requiring a short-time Fourier transform (SFT) to extract the signal's frequency domain information. Because the Fourier transform assumes stationary signals, a window function (Hamming window) is used to segment the dysarthric speech data into short time frames. For example, each frame is set to 20ms in length, with a 50% overlap (for example, 25ms per frame with a 12.5ms overlap). This overlap helps capture more subtle variations. Furthermore, segmenting the data into short time frames and applying a window function can reduce edge artifacts.

[0062] Subsequently, a short-time Fourier transform (SFT) is applied to each frame to convert the signal from the time domain to the frequency domain, yielding the frame's spectrum. The SFT analyzes the frequency components of each frame and calculates its power spectral density. The power values ​​are converted to a logarithmic scale to accommodate the human ear's nonlinear perception of sound intensity. Finally, color mapping is used to convert the power spectrum values ​​into images of varying colors, forming a clear spectrogram of pathological speech. In this spectrogram, the horizontal axis represents time, the vertical axis represents frequency, and the color depth indicates sound intensity.

[0063] It's important to note that the power spectrum refers to the power contained in the different frequency components of a signal. By examining a signal's power spectrum, we can determine which frequencies are present and their relative strengths. Taking the square of the STFT result, |X(t,f)|^2, represents the power of each frequency component. The STFT calculates the frequency component X of the signal at each moment.

[0064] Finally, in order to facilitate the separation and transformation of the pathological speech spectrogram, the pathological speech spectrogram can be adjusted to 224x224 pixels, and then the pathological speech spectrogram can be separated into two parts: phase spectrum spectrogram and amplitude spectrum spectrogram using Fourier transform.

[0065] Step 103 , encoding the biometric image to obtain a biometric code, inputting the biometric code into a pulse neuron for pulse processing to generate a weighted key image.

[0066] This step specifically includes:

[0067] Sub-step 1031 : converting the biological feature image into a grayscale image, and performing binarization processing on the grayscale image to obtain a biological binary image.

[0068] In this sub-step, grayscale and binarization processing is performed on the obtained biological characteristic image to obtain a set of black and white stripe images (biological binary images).

[0069] Gray-scale refers to the process of converting a color image (usually an RGB image) into a grayscale image. In a grayscale image, each pixel's value represents the brightness of that pixel without containing color information. Grayscale images usually have only a single channel (i.e., one intensity value), making them simpler and easier to process than color images. In the RGB color space, each pixel of a color image consists of three channels (red R, green G, blue B), and the process of grayscale conversion usually combines the values of these three channels into a single grayscale value through the following weighted formula:

[0070] Gray = 0.2989 x R + 0.5870 x G + 0.1140 x B

[0071] The 0.2989, 0.5870, and 0.1140 in the above weighted formula represent the weighting coefficients, representing the sensitivity of the human eye to different colors; among them, the green channel has the greatest impact on brightness, followed by red, and blue has the smallest. Each pixel value in a grayscale image is usually between 0 and 255 (for an 8-bit grayscale image), representing different brightness from black to white. The grayscale image only reflects the brightness information of the object and does not contain color information.

[0072] Binaryzation refers to the process of converting a grayscale image into an image containing only two colors (usually black and white). In a binary image, each pixel value is either black or white, and is usually used to extract significant features or objects in the image. The core of binaryzation is to select a threshold value (Threshold) and then compare each pixel value of the image with the threshold value. The binaryzation image has only two pixel values (0 and 255), corresponding to black and white, respectively. In a binary image, black usually represents the background or irrelevant area, and white represents the target or object of interest.

[0073] Sub-step 1032, performing a discrete coding operation on the binary image to obtain a biometric feature encoding signal.

[0074] It can be understood that the discrete coding operation is to map the white and black in the binary image into the digital codes 0 and 1 respectively, and to express the pixel relationship through this mapping relationship, so that a binary image is changed into a 01 code image. Stretching the 01 code image can be regarded as a series of spike sequences, similar to a discrete and discontinuous sequence signal.

[0075] Sub-step 1033, inputting the biometric feature encoding signal into a spiking neural network for pulse processing, and outputting a pulse signal to form a weighted key image.

[0076] Spiking Neural Networks (SNNs) are a type of neural network that simulates the dynamic behavior of biological neural networks, in which neurons transmit information through pulses (spikes). In SNNs, information transmission relies not only on the activation value of neurons but also on the pulses (or spikes) emitted by neurons. This model is closer to the workings of the biological brain than traditional artificial neural networks (ANNs), especially in processing time series information and dynamic events.

[0077] Spike processing is the process of converting input signals into neuronal spikes in SNNs. Since spiking neural networks process discrete spikes rather than continuous activation values, an encoding strategy must be used to map input information to neuronal spike activity.

[0078] In this step, the biometric encoding signal is continuously input into the SNN neurons to simulate the human electrical signal triggering mechanism. 0 represents no stimulation, and 1 represents stimulation and response. Under continuous stimulation and sustained triggering, the trigger condition threshold is met. At this point, the neuron responds with a pulse signal, indicating that the data has met the discrimination criteria set in this example. A unique biometric key is then constructed based on the pulse signal output, resulting in a weighted key graph.

[0079] More specifically, the biological binary image is further optimized into a unique 01 sequence, similar to convolution processing. At each pixel input, the spiking neural network determines whether a trigger response will be triggered to represent the pixel at that moment. This expression transforms each pixel in the image into a 0 and a pulse trigger representation. In this example, as shown in the figure, it is possible to consider setting the first pulse after 0 to a fixed value d1, and then continuously outputting pulses, using reweighting to form d2=d1(1+D*d1), where D represents the feedforward influence weight. This can form a complete digital code instead of a pixel representation, thereby obtaining the required weighted key map.

[0080] Step 104: perform an equal-weighting operation on the phase spectrum spectrogram through the weighted key graph, extract spectrum data from the amplitude spectrum spectrogram and the phase spectrum spectrogram after the equal-weighting operation, and synthesize them to obtain a spectrum watermark graph.

[0081] This step specifically includes:

[0082] Sub-step 1041 , performing an equal weighting operation on each pixel block in the phase spectrum graph using the weighted key graph.

[0083] It should be noted that each image has a fixed value at each pixel block, and the embodiment performs equal weighting operation on each pixel block in the phase spectrum graph according to the weighted key graph extracted from the biometric information.

[0084] The equal weighting operation is performed by using the following formula:

[0085]

[0086] In the formula: The pixel data after weighting in the nth pixel block; The original pixel data in the nth pixel block is represented by The current weighting value is represented by m, and the weighting weight is represented by m. The weighting influence of the previous bit is represented by k, and the influence attenuation factor is represented by k, where 0 < k < 1.

[0087] Sub-step 1042, extracting the spectrum data in the amplitude spectrum graph and the phase spectrum graph after the equal weighting operation, obtaining the phase spectrum data and the amplitude spectrum data M(u, v).

[0088] Sub-step 1043, combining the extracted phase spectrum data and amplitude spectrum data in the frequency domain to obtain the frequency domain complex image F(u, v).

[0089] Sub-step 1044, using inverse Fourier transform to transform the frequency domain complex image into a spectrogram watermark W. The spectrogram watermark W carries biometric information.

[0090] Step 105, performing AI speech processing on the dysarthria speech data to obtain a paralanguage spectrogram.

[0091] This step specifically includes:

[0092] Sub-step 1051, based on a preset speech syllable database, performing pronunciation feature recognition on the dysarthria speech data to obtain pronunciation feature text data.

[0093] The construction process of the preset speech syllable database includes: in a background noise-free environment, sequentially recording speech data of normal people and dysarthria patients (who have undergone comprehensive medical assessment), including monophthong, diphthong, consonant and nasal vowel; according to the syllable category represented by the audio, classify, take the audio files that meet the sound quality standard, and uniformly crop to one second in length for storage.

[0094] This embodiment trains an intelligent network recognition model through a preset speech syllable database, and uses the trained intelligent network recognition model to recognize the pronunciation features of the dysarthria speech data. Then, the pronunciation features of the patient are recognized based on the CNN binary classification algorithm, and the patient's pronunciation is converted into the normal pronunciation of a normal user. Finally, the converted normal pronunciation is converted into corresponding text to obtain pronunciation feature text data.

[0095] Sub-step 1052, performing AI speech generation processing on the pronunciation feature text data to obtain fake speech audio data.

[0096] This embodiment can use various AI-generated speech methods such as text to speech to generate corresponding audio representations to obtain fake speech audio data.

[0097] Sub-step 1053, performing window processing on the false speech audio data to obtain multiple frames of false speech audio data.

[0098] It is understandable that the windowing process here is consistent with the process in step 102 and will not be described in detail here.

[0099] Sub-step 1054 , performing spectrogram conversion on each frame of fake speech audio data by short-time Fourier transform to generate a fake speech spectrogram S′.

[0100] It can be understood that the process of converting the audio data into a spectrogram by short-time Fourier transform here is consistent with the conversion process in step 102, and will not be described in detail here.

[0101] Step 106: perform wavelet decomposition and embedding on the spectral watermark image and the pseudo-speech spectrogram to obtain multiple wavelet components.

[0102] This step specifically includes:

[0103] Sub-step 1061, using the Arnold algorithm to perform a scrambling operation on the spectral watermark image W, and using Harr wavelet to perform a first-level wavelet decomposition on the scrambled spectral watermark image W' to obtain multiple detail sub-images at the first-level resolution. and an approximating subgraph .

[0104] The Arnold algorithm usually means mapping the pixel coordinates of an image to another position through a specific linear transformation, resulting in a change in the pixel distribution of the image; its basic idea is to rotate, displace and other operations on the image within a matrix framework, so that each transformation causes a nonlinear change in the structure of the image; through this mapping, the original image gradually becomes unrecognizable, but after several transformations, if the same transformation is applied again, it will return to the original image.

[0105] Suppose there is an image of size N×N, where the position of each pixel can be represented by a two-dimensional coordinate, namely (x, y), where x, y∈{0,1,2,...,N−1}.

[0106] The Arnold transform can be defined by the following formula:

[0107]

[0108] Here, (x′, y′) is the new position after the Arnold transformation, (x, y) represents the original position, and N is the image size. The Arnold transformation "distorts" or "transforms" the position of each pixel in the image, thereby blurring the image. The matrix in this formula is called the Arnold matrix, and it is the core of mapping pixels from one position to another.

[0109] Wavelet transform is a commonly used signal processing technology, widely used in image encryption, compression, denoising and other fields. By decomposing an image into sub-bands of different scales, it can provide more efficient time-frequency analysis than the traditional Fourier transform. Wavelet transform is based on wavelet functions and recursively decomposes the image into multiple frequency bands, which usually include the following parts:

[0110] Low-frequency part: contains most of the information of the image and represents the rough structure or details of the image; high-frequency part: captures the details of the image, such as edges, textures, etc.; The key advantage of wavelet transform is that it can switch between spatial domain and frequency domain, allowing images to be analyzed at different scales.

[0111] Wavelet transform is used to perform a first-level decomposition of the image, dividing it into low-frequency components (LL) and high-frequency components (LH, HL, HH). Specifically, the LL (low-frequency) component mainly contains the rough outline of the image, while the LH (low-high frequency), HL (high-low frequency), and HH (high-high frequency) components contain detailed image information such as edges and textures.

[0112] In sub-step 1062, the pseudo speech spectrogram is down-sampled and the down-sampled pseudo speech spectrogram is decomposed into three levels of wavelet using Harr wavelet to obtain a plurality of detail sub-graphs at different resolution levels and an approximation sub-graph.

[0113] In this sub-step, the pseudo-speech spectrogram S' is downsampled to reduce the image size; then the downsampled pseudo-speech spectrogram is decomposed by Harr wavelet three-level decomposition to obtain multiple detail sub-images at different resolution levels. and an approximating subgraph .

[0114] Downsampling is typically described as the process of reducing the image resolution or size. It reduces the amount of data by lowering the pixel density within the image. The most common methods are maximum sampling or average sampling. For example, when downsampling an image, each 2x2 or 3x3 pixel block might be averaged to produce a smaller image. Max pooling is then used to take the maximum pixel value within that area as the new pixel.

[0115] Among them, wavelet multi-level decomposition further extracts the detailed information of the image by performing wavelet transform on the low-frequency part (LL) again, thereby achieving deeper decomposition and obtaining more frequency bands.

[0116] Sub-step 1063 , combining the weighted key graph, embedding each sub-graph obtained after the first-level wavelet decomposition into the corresponding sub-graph obtained after the third-level wavelet decomposition, to obtain multiple wavelet components.

[0117] Specifically, the weighted key graph is divided into four weighted key subgraphs, the mean of all pixels in each weighted key subgraph is calculated, and the mean is used as the embedding coefficient; based on different embedding coefficients, each subgraph obtained after the first-level wavelet decomposition is embedded into the corresponding subgraph obtained after the third-level wavelet decomposition to obtain multiple wavelet components; in the embedding process, the same-frequency merging method is used to generate new wavelet components.

[0118] The above embedding process is expressed as:

[0119]

[0120] Where: Represents multiple wavelet components obtained after embedding; Represents the three detail sub-graphs obtained after the three-level wavelet decomposition, represents the approximate subgraph obtained after three-level wavelet decomposition; Represents the three detail sub-graphs obtained after the first-level wavelet decomposition, represents the approximate subgraph obtained after the first-level wavelet decomposition; represents the embedding coefficient.

[0121] Step 107: Perform AES encryption processing on the multiple wavelet components to obtain inverse transformed speech.

[0122] Specifically, the low-frequency portion of each wavelet component is first encrypted using AES. All encrypted wavelet components are then subjected to an inverse Harr wavelet transform to produce a synthesized encrypted spectrum. Finally, the synthesized encrypted spectrum is converted using a vocoder, a speech synthesis tool such as the HiFi-GAN vocoder, to produce the inverse transformed speech.

[0123] AES is a block-based symmetric key encryption algorithm, meaning it uses the same key for encryption and decryption. It employs block ciphers and is typically used to encrypt fixed-size data blocks, typically 128 bits (16 bytes). The AES algorithm supports three different key lengths: 128, 192, and 256 bits, corresponding to different encryption security levels.

[0124] The present invention can transform the patient's dysarthria speech data into inverse transformed speech, which can accurately express the user's emotions in the patient's daily communication. The final output inverse transformed speech can have robust characteristics and security performance during the transmission process, protecting the user's legitimate rights and interests and preventing criminals from using the user's audio data to commit fraud or other illegal activities.

[0125] The present invention has the following advantages:

[0126] 1. Using pulse neural network to extract the user's biometric digital watermark key can achieve the mixing of personal biometric information and unique key, realizing more covert information function transmission.

[0127] 2. A complete set of audio digital watermark insertion technology for spoken words has been constructed, using spectrogram and biometric watermark insertion to achieve encrypted transmission of personal information, thereby ensuring the security of personal information for patients with articulation disorders using speech generators.

[0128] 3. The user's pronunciation characteristics are obtained based on the dysarthric speech data, and the user's biometric characteristics are combined to generate natural speech to accurately express the user's emotions.

[0129] A second aspect of the present invention further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any of the above-described methods for encrypting and transmitting dysarthric speech when executing the computer program.

[0130] The third aspect of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above-mentioned methods for encrypting and transmitting dysarthric speech.

[0131] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0132] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0133] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0134] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0135] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0136] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions described in the above embodiments can still be modified, or some of the technical features thereof can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A biometric-based method for encrypting and transmitting dysarthric speech, characterized in that: The method comprises: Obtaining the user's dysarthric speech data and biometric features; Performing transformation processing on the dysarthric speech data to transform it into a phase spectrum spectrogram and an amplitude spectrum spectrogram; Encoding the biometric image to obtain a biometric code, inputting the biometric code into a spiking neural network for spiking processing to generate a weighted key image; Performing an equal-weighted operation on the phase spectrum spectrogram through a weighted key graph, extracting spectrum data from the amplitude spectrum spectrogram and the phase spectrum spectrogram after the equal-weighted operation, and synthesizing them to obtain a spectrum watermark image; Performing AI speech processing on the dysarthric speech data to obtain a false speech spectrogram; Performing wavelet decomposition and embedding on the spectrum watermark image and the false speech spectrum image to obtain multiple wavelet components; AES encryption is performed on multiple wavelet components to obtain the inverse transformed speech.

2. The method for encrypting and transmitting dysarthric speech according to claim 1, wherein: The step of transforming the dysarthric speech data into a phase spectrum spectrogram and an amplitude spectrum spectrogram comprises: performing denoising processing on the dysarthric speech data; By performing window processing on the denoised dysarthric speech data, a multi-frame dysarthric speech data is obtained; Perform spectrogram conversion on each frame of dysarthric speech data through short-time Fourier transform to generate a pathological speech spectrogram; The pathological speech spectrogram is separated and transformed to obtain a phase spectrum spectrogram and an amplitude spectrum spectrogram.

3. The method for encrypting and transmitting dysarthric speech according to claim 1, wherein: The step of encoding the biometric image to obtain a biometric code, inputting the biometric code into a spiking neural network for spiking processing, and generating a weighted key image comprises: Converting the biological characteristic image into a grayscale image and performing binarization processing on the grayscale image to obtain a biological binary image; Performing discrete coding operation on the biological binary image to obtain a biological characteristic coding signal; The biometric encoding signal is input into a pulse neural network for pulse processing, and a pulse signal is output to form a weighted key graph.

4. The method for encrypting and transmitting dysarthric speech according to claim 1, wherein: The step of performing an equal-weighted operation on the phase spectrum spectrogram through a weighted key graph, extracting spectrum data from the amplitude spectrum spectrogram and the phase spectrum spectrogram after the equal-weighted operation, and synthesizing the spectrum watermark image comprises: Performing an equal weighting operation on each pixel block in the phase spectrum graph through a weighted key graph; Extracting spectrum data from the amplitude spectrum spectrogram and the phase spectrum spectrogram after the equal weighting operation to obtain phase spectrum data and amplitude spectrum data; Combining the extracted phase spectrum data and amplitude spectrum data in the frequency domain to obtain a frequency domain complex image; The frequency domain complex image is transformed into a spectral watermark image using inverse Fourier transform.

5. The method for encrypting and transmitting dysarthric speech according to claim 1, wherein: The step of performing AI speech processing on the dysarthric speech data to obtain a false speech spectrogram includes: Based on a preset speech syllable database, pronunciation feature recognition is performed on the dysarthric speech data to obtain pronunciation feature text data; Performing AI speech generation processing on the pronunciation feature text data to obtain fake speech audio data; Performing window processing on the fake voice audio data to obtain multiple frames of fake voice audio data; Each frame of fake speech audio data is converted into a spectrogram through short-time Fourier transform to generate a fake speech spectrogram.

6. The method for encrypting and transmitting dysarthric speech according to claim 1, wherein: The step of performing wavelet decomposition and embedding on the spectral watermark image and the false speech spectrogram to obtain multiple wavelet components includes: The spectral watermark image is scrambled using the Arnold algorithm, and the scrambled spectral watermark image is decomposed by a first-level wavelet using the Harr wavelet to obtain a plurality of detail sub-images and an approximation sub-image at a first-level resolution; Downsampling the pseudo speech spectrogram is performed, and Harr wavelet is used to perform three-level wavelet decomposition on the downsampled pseudo speech spectrogram to obtain a plurality of detail subgraphs and an approximation subgraph at different resolution levels; Combined with the weighted key graph, each subgraph obtained after the first-level wavelet decomposition is embedded into the corresponding subgraph obtained after the third-level wavelet decomposition to obtain multiple wavelet components.

7. The method for encrypting and transmitting dysarthric speech according to claim 6, characterized in that: The step of combining the weighted key graph and embedding each subgraph obtained after the first-level wavelet decomposition into the corresponding subgraph obtained after the third-level wavelet decomposition to obtain multiple wavelet components includes: Divide the weighted key graph into multiple weighted key subgraphs, calculate the mean of all pixels in each weighted key subgraph, and use the mean as the embedding coefficient; Based on different embedding coefficients, each sub-image obtained after the first-level wavelet decomposition is embedded into the corresponding sub-image obtained after the third-level wavelet decomposition to obtain multiple wavelet components; The above embedding process is expressed as: Where: Represents multiple wavelet components obtained after embedding; Represents the three detail sub-graphs obtained after the three-level wavelet decomposition, represents the approximate subgraph obtained after three-level wavelet decomposition; Represents the three detail sub-graphs obtained after the first-level wavelet decomposition, represents the approximate subgraph obtained after the first-level wavelet decomposition; represents the embedding coefficient.

8. The method for encrypting and transmitting dysarthric speech according to claim 1, wherein: The step of performing AES encryption processing on multiple wavelet components to obtain inverse transformed speech includes: Perform AES encryption on the low-frequency part of each wavelet component; Perform inverse wavelet transform on all encrypted wavelet components to synthesize and obtain a synthetic encrypted sound spectrum; The synthesized encrypted speech spectrum is converted by a vocoder to obtain an inverse transformed speech.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method for encrypting and transmitting dysarthric speech according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for encrypting and transmitting dysarthric speech according to any one of claims 1 to 8 are implemented.

Images