A Side-Channel Secure Communication Method Based on Ultrasonic Waves and Gyroscopes
By employing a side-channel secure communication method using ultrasound and gyroscopes, leveraging the hardware characteristics of gyroscopes and machine learning decoders, the risk of eavesdropping in IoT device communication is resolved. This enables secure and convenient word-level communication, supporting covert one-to-one and one-to-many communication with high accuracy and low latency.
Patent Information
- Application Number
- CN202311079117.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-08-24
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2043-08-24
AI Technical Summary
Existing IoT device communication solutions are vulnerable to eavesdropping during network sharing, lack sufficient user authentication and encryption, and are difficult to establish covert communication channels to defend against eavesdropping.
A side-channel secure communication method based on ultrasound and gyroscope is adopted. By collecting the hardware characteristics of the receiver's gyroscope, the information is encoded and decoded using a machine learning decoder to achieve ultrasonic modulation and demodulation, establish a covert communication frequency band, and support one-to-one and one-to-many secure communication.
It enables secure, convenient, and real-time word-level communication between IoT devices, with an accuracy rate of 95.5% and a communication latency of less than 0.63 seconds. It effectively prevents eavesdropping, supports instant communication, and expands vocabulary.
Smart Images

Figure CN117119445B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of secure wireless communication, and more particularly to a side-channel secure communication method based on ultrasound and a gyroscope. Background Technology
[0002] Internet of Things (IoT) devices have been widely adopted across various industries. To enable IoT devices to learn from each other and share information, a connected IoT network is essential. Existing commonly used IoT networks (such as Wi-Fi, Bluetooth, and NFC near-field communication technologies) are primarily designed for compatible IoT devices, and these communication solutions require the addition of specific equipment, increasing costs.
[0003] Existing highly sensitive information communication solutions primarily rely on user authentication and encryption. However, when shared information over a network is at risk of being eavesdropped on, receiver identification or user authentication is insufficient. Recent research indicates that eavesdroppers can infer events occurring on smart home devices, such as turning a light bulb on or off, and ultimately deduce user activity from encrypted data.
[0004] Therefore, how to establish one-to-one or one-to-many covert communication channels to defend against eavesdropping is a problem that needs to be solved.
[0005] In view of this, the present invention is hereby proposed. Summary of the Invention
[0006] The purpose of this invention is to provide a side-channel secure communication method based on ultrasound and gyroscope, which can prevent attackers from stealing data in IoT near-field communication, realize secure, convenient and real-time word-level IoT device communication, and thus solve the above-mentioned technical problems existing in the prior art.
[0007] The objective of this invention is achieved through the following technical solution:
[0008] A side-channel secure communication method based on ultrasound and gyroscope, comprising:
[0009] Step S1: The sender collects the gyroscope hardware characteristics of each receiver, including the resonant frequency and resonant response band; and distributes the machine learning-based decoder to each receiver.
[0010] In step S2, the sender converts the word information to be sent into speech audio, encodes it to obtain an encoded signal, selects the communication frequency band based on the gyroscope hardware characteristics of the target receiver, performs ultrasonic modulation, and then sends the encoded signal to the target receiver.
[0011] Step S3: After receiving the encoded signal sent by the sender, the target receiver uses its own machine learning-based decoder to decode the encoded signal through signal preprocessing and signal recognition to obtain the word information corresponding to each signal segment.
[0012] Compared with existing technologies, the side-channel secure communication method based on ultrasound and gyroscope provided by this invention has the following advantages:
[0013] This invention encodes audio signals with a base frequency range of 20Hz to 20,000Hz corresponding to the words to be sent into a complex frequency range of 10Hz. Machine learning techniques are then used to perform real-time semantic decoding of the received signals. This invention addresses the challenge of eavesdropping when multiple attackers are involved. Extensive evaluations show that this invention can achieve word-level communication with an accuracy of 95.5%, while the accuracy against non-target targets drops to 1.69%. The communication latency is less than 0.63 seconds.
[0014] The specific advantages are summarized as follows:
[0015] (1) This invention explores the possibility of creating a secret communication channel between a loudspeaker and a target gyroscope, from which it is difficult for an eavesdropper to discern complete information. This invention supports both one-to-one and one-to-many secret communication.
[0016] (2) In order to achieve real-time capability, this invention studies word-level ultrasonic communication to gyroscope, which has not been disclosed in previous studies.
[0017] (3) This invention is highly robust and supports real-time communication with IoT devices equipped with gyroscopes. The decoding module does not require the collection of expensive training data, but is conditioned on synthetic data, which helps to expand the vocabulary.
[0018] (4) The present invention has a good word recognition accuracy, with an average word recognition accuracy of 95.5%. Without retraining the decoding model, the recognition accuracy of new words (outside the training dataset) can reach at least 90%. Attached Figure Description
[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0020] Figure 1 A flowchart of a side-channel secure communication method based on ultrasound and gyroscope provided in an embodiment of the present invention.
[0021] Figure 2 A detailed flowchart of the side-channel secure communication method based on ultrasound and gyroscope provided in this embodiment of the invention.
[0022] Figure 3 A schematic diagram illustrating the working principle of the gyroscope involved in the side-channel secure communication method provided in this embodiment of the invention.
[0023] Figure 4 The flowchart illustrates the encoding and decoding processes in the side-channel secure communication method provided in this embodiment of the invention. Detailed Implementation
[0024] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the specific content of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments, which do not constitute a limitation of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.
[0025] First, the following explanations are provided for the terms that may be used in this article:
[0026] The term "and / or" means that either or both can be achieved simultaneously. For example, X and / or Y means that it includes both "X" or "Y" as well as the three cases of "X and Y".
[0027] The terms “including,” “comprising,” “containing,” “having,” or other similar semantic descriptions should be interpreted as non-exclusive inclusion. For example, “including a technical feature element (such as raw material, component, ingredient, carrier, dosage form, material, size, part, component, mechanism, device, step, process, method, reaction conditions, processing conditions, parameter, algorithm, signal, data, product or article of manufacture, etc.)” should be interpreted as including not only the expressly listed technical feature element, but also other technical feature elements that are not expressly listed and are well-known in the art.
[0028] The term "composed of" excludes any technical features not expressly listed. When used in a claim, it closes the claim to exclude all technical features other than those expressly listed, except for associated conventional impurities. If the term appears only in a clause of a claim, it limits the claim to the elements expressly listed in that clause; elements recited in other clauses are not excluded from the overall claim.
[0029] Unless otherwise explicitly specified or limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of the above terms in this document according to the specific circumstances.
[0030] When concentration, temperature, pressure, size, or other parameters are expressed as numerical ranges, such ranges should be understood to specifically disclose all ranges formed by any pairing of upper limits, lower limits, or preferred values within that range, regardless of whether the range is explicitly stated; for example, if the numerical range "2 to 8" is stated, then that range should be interpreted to include ranges such as "2 to 7", "2 to 6", "5 to 7", "3 to 4 and 6 to 7", "3 to 5 and 7", "2 and 5 to 7", etc. Unless otherwise stated, the numerical ranges described herein include both their endpoints and all integers and fractions within that range.
[0031] The following is a detailed description of the side-channel secure communication method based on ultrasound and gyroscope provided by this invention. Contents not described in detail in the embodiments of this invention are prior art known to those skilled in the art. Where specific conditions are not specified in the embodiments of this invention, they should be performed according to conventional conditions in the art or conditions recommended by the manufacturer. Where the manufacturers of reagents or instruments used in the embodiments of this invention are not specified, they are all conventional products that can be purchased commercially.
[0032] like Figure 1 As shown, this embodiment of the invention provides a side-channel secure communication method based on ultrasound and gyroscope, including:
[0033] Step S1: The sender collects the gyroscope hardware characteristics of each receiver, including the resonant frequency and resonant response band; and distributes the machine learning-based decoder to each receiver.
[0034] In step S2, the sender converts the word information to be sent into speech audio, encodes it to obtain an encoded signal, selects the communication frequency band based on the gyroscope hardware characteristics of the target receiver, performs ultrasonic modulation, and then sends the encoded signal to the target receiver.
[0035] Step S3: After receiving the encoded signal sent by the sender, the target receiver uses its own machine learning-based decoder to decode the encoded signal through signal preprocessing and signal recognition to obtain the word information corresponding to each signal segment.
[0036] In step S1 of the above method, the sending party collects the gyroscope hardware characteristics of each receiving party in the following manner, wherein the gyroscope hardware characteristics include: resonant frequency and resonant response frequency band, including:
[0037] The transmitting direction sends an ultrasonic signal with a linear modulation frequency to each receiver, so that the gyroscope of each receiver can calculate the frequency response and obtain the resonant frequency and resonant response frequency band of each receiver's gyroscope.
[0038] The resonant frequency is defined as the frequency of the ultrasonic signal received by the gyroscope when the amplitude of the ultrasonic signal exceeds twice the signal strength at rest, but is less than 0.5 Hz.
[0039] The resonant response frequency band is the frequency at which the ultrasonic signal received by the gyroscope has an amplitude that is twice as strong as when it is stationary. The frequency band in which the ultrasonic signal frequency is located at this time is determined to be the resonant response frequency band.
[0040] In step S2 of the above method, the sender converts the word information to be sent into speech audio and encodes it to obtain an encoded signal in the following manner:
[0041] The sender converts the word information to be sent into an audio signal using TTS, extracts the time-frequency ridge of the pitch of each word from the spectrogram of the audio signal, scales the time-frequency ridge proportionally to the frequency band of 1000-1010Hz to obtain a scaled spectrogram, and converts the scaled spectrogram into an audio signal using the Griffin-Lim algorithm as the encoded signal.
[0042] In step S2 of the above method, the sender selects the communication frequency band based on the gyroscope hardware characteristics of the target receiver and performs ultrasonic modulation before sending the coded signal to the target receiver in the following manner:
[0043] Based on the resonant frequency and resonant response band of the gyroscope hardware characteristics of each receiver, the corresponding communication band is selected for the target receiver. Only the device corresponding to the target receiver can receive the complete coded signal sent by the sender. Then, the coded signal is modulated from 1000 to 1010 Hz onto the selected communication band to obtain an ultrasonic signal. The coded signal is then sent to the target receiver through the ultrasonic signal.
[0044] In step S2 of the above method, the corresponding communication frequency band for the target receiver is selected based on the resonant frequency and resonant response frequency band of the gyroscope hardware characteristics of the target receiver in the following manner:
[0045] Suppose the resonant response frequency band of the target receiver's gyroscope is f1~f2, and the resonant frequency fr is in the middle of the resonant response frequency band f1~f2. After removing the range fr-10~fr+10 from the resonant response frequency band, the corresponding communication frequency band is selected according to the following conditions.
[0046] If the resonant response frequency bands of the gyroscope hardware characteristics of the target receiver and the non-target receiver do not overlap, then the resonant response frequency band of the target receiver is selected as the communication frequency band, that is, the communication frequency band in the range of f1~fr-10 and fr+10~f2.
[0047] If the resonant response frequency bands of the gyroscope hardware characteristics of the target receiver and the non-target receiver overlap by no more than 50%, then the resonant response frequency band not covered by the target receiver is selected as the communication frequency band. That is, if the coverage range is in the range of fr to f2, then the range of f1 to fr-10 is selected as the communication frequency band; if the coverage range is in the range of f1 to fr, then the range of fr+10 to f2 is selected as the communication frequency band.
[0048] If the resonant response frequency bands of the gyroscope hardware characteristics of the target receiver and the non-target receiver overlap by more than 50%, then the resonant frequency of the non-target receiver is selected as the communication frequency band. Let the resonant frequency of the non-target receiver be fr2, then the communication frequency band is selected as fr2-5 to fr2+5.
[0049] In step S2 of the above method, amplitude modulation is used to multiply the encoded signal by a high-frequency carrier to modulate the encoded signal from 1000 to 1010 Hz onto the selected communication frequency band to obtain an ultrasonic signal.
[0050] In step S3 of the above method, the target receiver uses its own machine learning-based decoder to perform signal preprocessing in the following manner:
[0051] The machine learning-based decoder segments the received encoded signal according to the variance to obtain multiple signal segments. Each signal segment corresponds to a single word. Each signal segment is filtered and subjected to short-time Fourier transform to obtain the spectrogram of each signal segment. The spectrogram is then normalized to obtain the sample signal of a single word.
[0052] The target receiver uses its own machine learning-based decoder to identify the signal in the following ways:
[0053] If a user uses existing words in the system to communicate, after receiving a sample signal, the system uses a pre-trained long short-term memory recurrent neural network model to identify the sample signal of a single word and obtain the word information corresponding to the sample signal.
[0054] or,
[0055] If a user uses new words for communication, after receiving a sample signal, the feature vector of the sample signal is extracted by a pre-trained triplet network model and compared with the feature vectors in the pre-established word feature library. The word corresponding to the feature vector with the smallest distance is taken as the recognized word information.
[0056] The word feature library is established in the following way: a 32-dimensional feature vector is extracted from the sample signal of each word using a pre-trained triplet network model to form the word feature library, and the feature vectors of newly generated words are added to the word feature library to expand the word feature library.
[0057] In step S3 of the above method, the long short-term memory recurrent neural network model consists of 4 layers of long short-term neural networks and 3 fully connected layers. Each of the 4 layers of long short-term neural networks has 512 long short-term memory neural network units. The neurons in the 3 fully connected layers are 512-1000-500-100 respectively. The 4 layers of long short-term neural networks will output a 512-dimensional feature vector. The 512-dimensional feature vector is input into the 3 fully connected layers to obtain the classification result.
[0058] The Long Short-Term Memory (LSTM) recurrent neural network model is trained in the following ways:
[0059] During the training process, based on the hardware principle of gyroscope resonance, the gyroscope signal output by the gyroscope after each word sent by the ultrasonic wave is received by the gyroscope is simulated. Based on the gyroscope signal, a synthetic gyroscope signal is generated as training data to train the long short-term memory recurrent neural network model.
[0060] As we can see, the three fully connected layers of the above-mentioned Long Short-Term Memory Recurrent Neural Network model correspond to the classification of 100 words, and in actual implementation, it can be extended to more words as needed.
[0061] The structure of the triplet network model is composed of a long short-term memory recurrent neural network model with three additional fully connected layers, which can generate a 32-dimensional feature vector; the long short-term memory recurrent neural network model has the same structure as the aforementioned long short-term memory recurrent neural network model.
[0062] The triplet network model is trained in the following ways:
[0063] During training, based on the hardware principle of gyroscope resonance, the gyroscope signal output by the gyroscope after each word sent by the ultrasonic wave is received by the gyroscope is simulated. The simulated gyroscope signal is used to generate a synthetic gyroscope signal as training data to train the triplet network model.
[0064] In the above method, based on the hardware principle of gyroscope resonance, the gyroscope signal output by the gyroscope after each word transmitted by ultrasound is received by the gyroscope is simulated in the following manner. A synthetic gyroscope signal is generated based on the simulated gyroscope signal as training data, and a synthetic gyroscope signal is generated based on the simulated signal pattern as training data, including:
[0065] Multiple different gyroscope resonant frequencies and ultrasonic frequencies are randomly set, and multipliers and low-pass filters are simulated according to the different gyroscope resonant frequencies and ultrasonic frequencies to synthesize the gyroscope signal received by the gyroscope and the output of the ultrasonic signal sent for each word.
[0066] Specifically, according to Figure 2 As shown in the signal generation diagram of the gyroscope, an oscillator inside the gyroscope continuously undergoes simple harmonic motion. Let its vibration frequency be fr. When the gyroscope deflects, according to the Coriolis force, it will displace perpendicular to the direction of simple harmonic motion. This displacement distance can be measured using an electromagnetic probe, thus calculating the angular velocity. To eliminate interference from the gyroscope's own simple harmonic vibration, a multiplier is added to the output signal, multiplying it by the cosine of the simple harmonic motion, and then the signal is passed through a low-pass filter to eliminate the influence of the simple harmonic motion. When the frequency of the ultrasonic wave is close to the simple harmonic motion frequency fr, the signal will have an additional low-frequency component after passing through the multiplier. This low-frequency component cannot be eliminated by the low-pass filter, resulting in ultrasonic interference. In this case, by setting the resonant frequency of the gyroscope and the frequency of the ultrasonic wave, and simulating the multiplier and low-pass filter, the mode of the signal received by the gyroscope can be synthesized. This yields the theoretically received gyroscope signal output after each ultrasonic wave is transmitted.
[0067] The above method simulates the differences in data from different devices by randomly setting multiple different gyroscope resonant frequencies and ultrasonic frequencies, thus enabling the machine learning-based decoder to have strong generalization ability.
[0068] The obtained gyroscope signals are processed by transient characteristic processing, signal enhancement processing, gyroscope noise processing, frequency response processing, and data enhancement processing to obtain the simulated generated synthetic gyroscope signals used as training data.
[0069] In the above method, the transient characteristic processing is the energy accumulation process required for gyroscope resonance. When a gyroscope is subjected to ultrasonic interference, there will be a gradual signal enhancement process before it stabilizes. This process will have different lengths and magnitudes for different gyroscopes. To simulate the transient conditions of different gyroscopes, the first a% of the gyroscope signals are randomly selected and multiplied by a decimal number that increases arithmetically from b to 1, where b is a randomly selected decimal number, 0 < b < 1. This simulates the transient characteristics of different gyroscope signals.
[0070] The signal enhancement processing involves multiplying the gyroscope signal by a random number between 0.5 and 4 to simulate the gyroscope signal enhancement characteristics.
[0071] The gyroscope noise processing involves adding Gaussian noise to the gyroscope signal to simulate gyroscope noise.
[0072] The frequency response is simulated by multiplying the frequency axis of the spectrum of the gyroscope signal by a decimal number c that increases arithmetically to 1, where c is a randomly selected decimal number, 0 < c < 1.
[0073] The aforementioned method for synthesizing gyroscope data is based on theoretical simulation, not actual data acquisition. To simulate the effects of different gyroscopes, the generation of each synthesized gyroscope data point is randomized, involving random numbers in the generation process. These various synthesized gyroscope data points form the initial training dataset.
[0074] The data augmentation process involves adding a two-dimensional Gaussian noise matrix directly to the two-dimensional matrix corresponding to the time-frequency plot of each synthetic gyroscope signal on an initial training dataset composed of multiple synthetic gyroscope signals. This simulates gyroscope signals received in reality that are affected by unpredictable factors. Specifically, each sample is a time-frequency plot, which is a two-dimensional matrix. Adding a two-dimensional Gaussian noise matrix directly to this two-dimensional matrix achieves the purpose of data augmentation.
[0075] The final training dataset consists of multiple synthetic gyroscope data points after data augmentation. Since the synthetic gyroscope signals are generated from simulated gyroscope signals as training data, this invention eliminates the need for expensive data collection from gyroscopes. It allows for the training and updating of a machine learning-based decoder based on synthetic data generated from theoretical MEMS resonance analysis.
[0076] In summary, the method of this invention provides a secure, convenient, and real-time word-level IoT device communication solution by establishing a covert and secure channel. Its main idea is to utilize the physical characteristics of IoT devices, using ultrasound as an external excitation to force the microelectromechanical system (MEMS) gyroscope to resonate. The communication information of this invention is specially modulated according to the different hardware characteristics of the gyroscopes, achieving efficient, word-level, and covert communication between the sender's speaker and the receiver's gyroscope, while mitigating eavesdropping attacks. When multiple attackers are involved, this invention solves the challenge of eavesdropping. Extensive evaluation shows that this invention can achieve word-level communication with an accuracy of 95.5%, while the accuracy against non-target targets drops to 1.69%. The communication latency is less than 0.63 seconds.
[0077] The specific advantages are summarized as follows:
[0078] (1) This invention explores the possibility of creating a secret communication channel between a loudspeaker and a target gyroscope, from which it is difficult for an eavesdropper to discern complete information. This invention supports both one-to-one and one-to-many secret communication.
[0079] (2) In order to achieve real-time capability, this invention studies word-level ultrasonic communication to gyroscope, which has not been disclosed in previous studies.
[0080] (3) This invention is highly robust and supports real-time communication with IoT devices equipped with gyroscopes. The decoding module does not require the collection of expensive training data, but is conditioned on synthetic data, which helps to expand the vocabulary.
[0081] (4) The present invention has a good word recognition accuracy, with an average word recognition accuracy of 95.5%. Without retraining the decoding model, the recognition accuracy of new words (outside the training dataset) can reach at least 90%.
[0082] To more clearly demonstrate the technical solution and its effects provided by the present invention, the following detailed description of the side-channel secure communication method based on ultrasound and gyroscope provided by the present invention is given with reference to specific embodiments.
[0083] Example 1
[0084] This embodiment provides a side-channel secure communication method based on ultrasound and gyroscope, which can prevent attackers from stealing data in IoT near-field communication. With the method of this invention, it is difficult for the gyroscope of the non-target receiver to recover complete information from the eavesdropped communication.
[0085] like Figure 2 As shown, this invention utilizes the hardware characteristics of a gyroscope for secure communication. The entire process includes three steps: registration, encoding, and decoding, specifically:
[0086] Step S1, Registration Phase: The sender collects the gyroscope hardware characteristics of each receiver, including resonant frequency and resonant response band. Then, the sender distributes the decoder to each receiver.
[0087] Step S2, Information Encoding: Information encoding mainly includes two parts: audio signal generation, selection of communication frequency band based on the gyroscope hardware characteristics of the target receiver, and ultrasonic modulation, such as... Figure 3 As shown; where,
[0088] (21) Audio signal generation: The word information is converted into an audio signal through TTS, and then the audio signal is encoded into a frequency band of 1000-1010Hz to form an encoded signal;
[0089] (22) Selecting communication frequency bands based on the gyroscope hardware characteristics of the target receiver: Selecting a suitable communication frequency band for each target receiver, so that only the target receiver receives the complete coded signal, thereby enabling one-to-one and one-to-many secure and covert communication.
[0090] (23) Ultrasonic modulation: The encoded signal is modulated from 1000 to 1010 Hz onto the selected communication frequency band and transmitted to the target receiver via ultrasonic waves;
[0091] Step S3, Signal Decoding: Signal decoding consists of two parts: signal preprocessing and signal recognition, such as... Figure 3 As shown; where,
[0092] (31) Signal preprocessing: The received signal is segmented, filtered and STFT is applied according to the variance;
[0093] (32) Signal Recognition: Each signal is identified using a pre-trained Long Short-Term Memory Recurrent Neural Network (LSTM) model, thereby obtaining the word information represented by each signal. A word expansion recognition scheme using a triplet network model for signal recognition is also provided.
[0094] This invention encodes the audio signal corresponding to the word to be sent (its fundamental frequency range is from 20Hz to 20,000Hz) within a complex frequency range (10Hz) and uses machine learning techniques to perform real-time semantic decoding of the received signal. More importantly, this invention does not require the collection of expensive data from gyroscopes; the decoding model can be trained and updated based on synthetic data (generated from theoretical MEMS resonance analysis). This invention addresses the challenge of eavesdropping when multiple attackers are involved. Extensive evaluations show that this invention can achieve word-level communication with an accuracy of 95.5%, while the accuracy against non-target targets drops to 1.69%. The communication latency is less than 0.63 seconds.
[0095] The specific advantages are summarized as follows:
[0096] (1) This invention explores the possibility of creating a secret communication channel between a loudspeaker and a target gyroscope, from which it is difficult for an eavesdropper to discern complete information. This invention supports both one-to-one and one-to-many secret communication.
[0097] (2) In order to achieve real-time capability, this invention studies word-level ultrasonic communication to gyroscope, which has not been well studied before.
[0098] (3) This invention is highly robust and supports real-time communication with IoT devices equipped with gyroscopes. The decoding module does not require the collection of expensive training data, but is conditioned on synthetic data, which helps to expand the vocabulary.
[0099] (4) This invention achieves a good average word recognition accuracy of 95.5%. Without retraining the decoding model, the recognition accuracy for new words (outside the training dataset) can reach at least 90%.
[0100] Example 2
[0101] This embodiment provides a side-channel secure communication method based on ultrasound and gyroscope. The specific stages include the following steps, see below. Figure 2 ,include:
[0102] Step S1, Registration Phase: The sender collects the gyroscope hardware characteristics of each receiver, including resonant frequency and resonant response band. Next, the sender sends random information to the receiver, and the receiver replies with this random information to verify the correctness of the receiver's hardware characteristics. Finally, the sender distributes the decoder to each receiver.
[0103] Step S2, Information Encoding: Information encoding mainly includes two parts: audio signal generation, communication frequency band selection, and ultrasonic modulation, such as... Figure 3 As shown.
[0104] (21) Audio signal generation: In order to convert word information into ultrasonic waves for transmission, the word information is first converted into audio using text-to-speech (TTS) technology, and then the time-frequency ridge of the fundamental tone of each word is extracted from the spectrogram. Next, the time-frequency ridge information is scaled proportionally to the frequency band of 1000-1010Hz. Finally, the spectrogram is converted into a sound signal using the Griffin-Lim algorithm.
[0105] (22) Communication frequency band selection: In order to achieve one-to-one and one-to-many secure and covert communication, this invention needs to select a suitable communication frequency band for each receiver. Since the resonant response frequency band range of each gyroscope is limited, there are three situations: (1) The resonant response frequency bands of the receivers do not overlap, so communication can be carried out directly within the resonant response frequency band of the target receiver; (2) The resonant response frequency bands of the target receiver and the non-target receiver overlap by no more than 50%, so the frequency band not covered by the target receiver is selected for communication; (3) The resonant response frequency bands of the target receiver and the non-target receiver overlap by more than 50%, so communication is carried out on the resonant frequency of the non-target receiver, because according to theoretical analysis, communication on the resonant frequency is extremely low frequency and will be covered by low frequency noise.
[0106] (23) Ultrasonic modulation: After the audio signal is generated, the information is in the range of 1000-1010Hz. In order to communicate with the target gyroscope, the information needs to be modulated from 1000-1010Hz to the selected frequency band. Amplitude modulation is used here. By multiplying the signal by a high-frequency carrier, the ultrasonic signal can be obtained quickly.
[0107] Step S3, Signal Decoding: Signal decoding consists of two parts: signal preprocessing and signal recognition, such as... Figure 3 As shown.
[0108] (31) Signal preprocessing: The received signal is segmented according to the variance, and each segment is subjected to a 10Hz high-pass filter and a short-time Fourier transform (STFT) to obtain a spectrogram. Then the spectrogram is normalized to obtain signal samples of individual words.
[0109] (32) Signal Recognition: This invention uses a pre-trained Long Short-Term Memory Recurrent Neural Network (LSTM) model to recognize each signal, thereby obtaining the word information represented by each signal. It also provides a word expansion recognition scheme, using a triplet network model to extract a 32-dimensional feature vector for each word's signal, forming a word feature library. When a user needs to expand to include new words, the system can directly generate the feature vector of the new word and add it to the word feature library. Upon receiving a signal, the system extracts the signal's feature vector using Triplet and compares it with the vectors in the word feature library; the word with the smallest distance is the recognized word. The generation methods of these two models will be explained in detail later.
[0110] Model Generation: The recognition model is trained using simulated data. By analyzing the hardware principle of gyroscope resonance, the signal pattern of each word received by the gyroscope can be simulated. This Long Short-Term Memory (LSTM) recurrent neural network model trained with simulated data is hardware-independent, exhibits stronger generalization ability, and can achieve a recognition accuracy of over 95%. The Triplet model adds three fully connected layers to the LSTM model, generating a 32-dimensional feature vector. Triplet is also trained using simulated data; without retraining the decoding model, Triplet can achieve at least 90% accuracy in recognizing new words (outside the training dataset).
[0111] Generation of synthetic gyroscope signals: The principle of gyroscope signal generation is as follows... Figure 2 As shown in the diagram, the theoretical signal output by the gyroscope after receiving an ultrasonic wave can be obtained. Then, considering factors such as the transient characteristics of the gyroscope signal, signal strength changes, noise, and frequency response, synthetic data that closely approximates the real signal is generated.
[0112] (41) Transient characteristics: In the resonance phenomenon of gyroscope, the amplitude enhancement effect does not increase to the maximum instantaneously, but has a gradual increase and energy accumulation process. Therefore, multiplying a portion of the early sampling points of the signal by an increasing decimal number can simulate this transient characteristic.
[0113] (42) Signal enhancement: As the distance changes or the strength of the signal source changes, the signal will be amplified or reduced as a whole. Therefore, the entire signal is multiplied by a random number to simulate this signal enhancement characteristic.
[0114] (43) Gyroscope noise: Gyroscopes also output noise signals when they are stationary. Gaussian noise is added to the synthesized gyroscope signal to simulate this characteristic.
[0115] (44) Frequency Response: According to the principle of resonance, the closer to the resonant frequency, the greater the amplitude enhancement. This means that the gyroscope responds differently to sound waves of different frequencies. This invention simulates this characteristic by multiplying the frequency axis of the spectrum by gradually increasing decimals.
[0116] (45) Data augmentation: In reality, received signals may also be affected by some unpredictable factors, resulting in some minor changes. Therefore, Gaussian noise is added to the synthetic training dataset to simulate this characteristic.
[0117] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.
[0118] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims. The information disclosed in the background section is intended only to enhance the understanding of the overall background technology of the present invention and should not be construed as an admission or implication in any way that such information constitutes prior art known to those skilled in the art.
Claims
1. A side-channel secure communication method based on ultrasound and gyroscope, characterized in that, include: Step S1: The sender collects the gyroscope hardware characteristics of each receiver, including the resonant frequency and resonant response band; and distributes the machine learning-based decoder to each receiver. Step S2: The sender converts the word information to be sent into speech audio, encodes it to obtain an encoded signal, selects a communication frequency band based on the gyroscope hardware characteristics of the target receiver, performs ultrasonic modulation, and then sends the encoded signal to the target receiver. In step S2, the sender selects a communication frequency band based on the gyroscope hardware characteristics of the target receiver and performs ultrasonic modulation before sending the encoded signal to the target receiver in the following manner: Based on the resonant frequency and resonant response band of the gyroscope hardware characteristics of the target receiver, the corresponding communication band is selected for the target receiver. Only the device corresponding to the target receiver can receive the complete coded signal sent by the sender. Then, the coded signal is modulated from 1000 to 1010 Hz onto the selected communication band to obtain an ultrasonic signal. The coded signal is then sent to the target receiver through the ultrasonic signal. In step S2, the corresponding communication frequency band for the target receiver is selected based on the resonant frequency and resonant response frequency band of the gyroscope hardware characteristics of the target receiver in the following manner: Assuming the resonant response frequency band of the target receiver's gyroscope is f1 to f2, and the resonant frequency fr is exactly in the middle of the resonant response frequency band f1 to f2, then after removing the range fr-10 to fr+10 from the resonant response frequency band, the corresponding communication frequency band should be selected according to the following conditions: If the resonant response frequency bands of the gyroscope hardware characteristics of the target receiver and the non-target receiver do not overlap, then the resonant response frequency band of the target receiver is selected as the communication frequency band, that is, the communication frequency band in the range of f1~fr-10 and fr+10~f2. If the resonant response frequency bands of the gyroscope hardware characteristics of the target receiver and the non-target receiver overlap by no more than 50%, then the resonant response frequency band not covered by the target receiver is selected as the communication frequency band. That is, if the coverage range is in the range of fr to f2, then the range of f1 to fr-10 is selected as the communication frequency band; if the coverage range is in the range of f1 to fr, then the range of fr+10 to f2 is selected as the communication frequency band. If the resonant response frequency bands of the gyroscope hardware characteristics of the target receiver and the non-target receiver overlap by more than 50%, then the resonant frequency of the non-target receiver is selected as the communication frequency band. Let the resonant frequency of the non-target receiver be fr2, then the communication frequency band is selected as fr2-5 to fr2+5. Step S3: After receiving the encoded signal sent by the sender, the target receiver uses its own machine learning-based decoder to decode the encoded signal through signal preprocessing and signal recognition to obtain the word information corresponding to each signal segment.
2. The side-channel secure communication method based on ultrasound and gyroscope according to claim 1, characterized in that, In step S1, the sender of the communication collects the gyroscope hardware characteristics of each receiver in the following manner, wherein the gyroscope hardware characteristics include: resonant frequency and resonant response frequency band, including: The transmitting direction sends an ultrasonic signal with a linear modulation frequency to each receiver, so that the gyroscope of each receiver can calculate the frequency response and obtain the resonant frequency and resonant response frequency band of each receiver's gyroscope. The resonant frequency is defined as the frequency of the ultrasonic signal received by the gyroscope when the ultrasonic signal has an amplitude that is more than twice the signal strength when the instrument is stationary but less than 0.5 Hz. The resonant response frequency band is the frequency band in which the ultrasonic signal received by the gyroscope has an amplitude increase of more than twice the signal strength when it is stationary.
3. The side-channel secure communication method based on ultrasound and gyroscope according to claim 1 or 2, characterized in that, In step S2, the sender encodes the word information to be sent by generating an audio signal in the following manner: The sender converts the word information to be sent into an audio signal using TTS, extracts the time-frequency ridge of the pitch of each word from the spectrogram of the audio signal, scales the time-frequency ridge proportionally to the frequency band of 1000-1010Hz to obtain a scaled spectrogram, and converts the scaled spectrogram into an audio signal using the Griffin-Lim algorithm as the encoded signal.
4. The side-channel secure communication method based on ultrasound and gyroscope according to claim 1, characterized in that, In step S2, amplitude modulation is used to multiply the encoded signal by a high-frequency carrier wave to modulate the encoded signal from 1000 to 1010 Hz onto the selected communication frequency band to obtain an ultrasonic signal.
5. The side-channel secure communication method based on ultrasound and gyroscope according to claim 1 or 2, characterized in that, In step S3, the target receiver uses its own machine learning-based decoder to perform signal preprocessing in the following manner: The machine learning-based decoder segments the received encoded signal according to the variance to obtain multiple signal segments. Each signal segment corresponds to a single word. Each signal segment is filtered and subjected to short-time Fourier transform to obtain the spectrogram of each signal segment. The spectrogram is then normalized to obtain the sample signal of a single word. The target receiver uses its own machine learning-based decoder to identify the signal in the following ways: If a user uses existing words in the system to communicate, after receiving a sample signal, the system uses a pre-trained long short-term memory recurrent neural network model to identify the sample signal of a single word and obtain the word information corresponding to the sample signal. or, If a user uses new words for communication, after receiving a sample signal, the feature vector of the sample signal is extracted by a pre-trained triplet network model and compared with the feature vectors in the pre-established word feature library. The word corresponding to the feature vector with the smallest distance is taken as the recognized word information. The word feature library is established in the following way: a 32-dimensional feature vector is extracted from the sample signal of each word using a pre-trained triplet network model to form the word feature library, and the feature vectors of newly generated words are added to the word feature library to expand the word feature library.
6. The side-channel secure communication method based on ultrasound and gyroscope according to claim 5, characterized in that, In step S3, the structure of the long short-term memory recurrent neural network model consists of 4 layers of long short-term neural networks and 3 fully connected layers. Each of the 4 layers of long short-term neural networks has 512 long short-term memory neural network units. The neurons in the 3 fully connected layers are 512-1000-500-100 respectively. The 4 layers of long short-term neural networks will output a 512-dimensional feature vector. The 512-dimensional feature vector is input into the three fully connected layers to obtain the classification result. The Long Short-Term Memory (LSTM) recurrent neural network model is trained in the following ways: During the training process, based on the hardware principle of gyroscope resonance, the gyroscope signal output by the gyroscope after each word sent by the ultrasonic wave is received by the gyroscope is simulated. The simulated gyroscope signal is used to generate a synthetic gyroscope signal as training data to train the long short-term memory recurrent neural network model. The structure of the triplet network model is composed of a long short-term memory recurrent neural network model with three additional fully connected layers, which can generate a 32-dimensional feature vector. The triplet network model is trained in the following ways: Based on the hardware principle of gyroscope resonance, the gyroscope signal output by the gyroscope after each word transmitted by ultrasound is simulated. The simulated gyroscope signal is then used to generate a synthetic gyroscope signal as training data to train the triplet network model.
7. The side-channel secure communication method based on ultrasound and gyroscope according to claim 6, characterized in that, In the method described above, based on the hardware principle of gyroscope resonance, the gyroscope signal output by the gyroscope after each word transmitted by ultrasound is received by the gyroscope is simulated in the following manner. A synthetic gyroscope signal is then generated based on the simulated gyroscope signal as training data, including: Multiple different gyroscope resonant frequencies and ultrasonic frequencies are randomly set, and multipliers and low-pass filters are simulated according to the different gyroscope resonant frequencies and ultrasonic frequencies to synthesize the gyroscope signal received by the gyroscope and the output of the ultrasonic signal sent for each word. The obtained gyroscope signals are processed by transient characteristic processing, signal enhancement processing, gyroscope noise processing, frequency response processing, and data enhancement processing to obtain the simulated generated synthetic gyroscope signals used as training data. The transient characteristic processing is achieved by multiplying the first a% of the gyroscope signals randomly selected by b by a decimal number that increases arithmetically from b to 1 to simulate the transient characteristics of different gyroscope signals. b is a randomly selected decimal number, where 0 < b < 1. The signal enhancement processing involves multiplying the gyroscope signal by a random number between 0.5 and 4 to simulate the gyroscope signal enhancement characteristics. The gyroscope noise processing involves adding Gaussian noise to the gyroscope signal to simulate gyroscope noise. The frequency response is simulated by multiplying the frequency axis of the spectrum of the gyroscope signal by a decimal number c that increases arithmetically to 1, where c is a randomly selected decimal number, 0 < c < 1. The data augmentation process involves adding a two-dimensional Gaussian noise matrix directly to the two-dimensional matrix corresponding to the time-frequency plot of each synthetic gyroscope signal on an initial training dataset composed of multiple synthetic gyroscope signals, in order to simulate gyroscope signals received in reality that are affected by unpredictable factors.
